docs: fill in AGENTS.md
Setup, common commands, architecture overview, and the database/testing gotchas actually hit while working in this repo recently: migration files that can drift to the point of being unrunnable (not just stale), inconsistent platform-string casing between Go constants and SQL objects, global (unscoped) mock-platform NOTIFY channels, the require.Eventually-runs-on-a-goroutine hazard, and why domains/reports' fixtures use Etsy rather than Amazon. CLAUDE.md already points here via @AGENTS.md, so no change needed there. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
This commit is contained in:
Claude Sonnet 5
@@ -1 +1,183 @@
|
||||
TODO
|
||||
# AGENTS.md
|
||||
|
||||
Go backend for a multi-platform ecommerce inventory-sync tool. Etsy is the
|
||||
one real, live platform integration; everything else (Amazon, BigCartel,
|
||||
Ebay, Ecwid, Shopify, SquareOnline, Squarespace, Tiktok, WalmartMarketplace,
|
||||
Wix, WooCommerce, Zoho) exists only as a **mock simulation layer** used for
|
||||
development, demos, and testing the sync/reporting logic without needing
|
||||
real store credentials. See `README.md` for the product-level roadmap.
|
||||
|
||||
## Setup
|
||||
|
||||
1. Postgres running locally, with an `app_client` role/database matching
|
||||
`.env.example`'s `DATABASE_URL`.
|
||||
2. `cp .env.example .env` and fill in real values (Auth0, Etsy). For local
|
||||
dev-only work you don't need real Auth0/Etsy secrets if `DEV_AUTH_ENABLED`
|
||||
is `true` (see Auth below) - `changeme` placeholders are fine.
|
||||
3. Create the dedicated test database once: `createdb -O app_client
|
||||
inventory_2_test`. `make test` migrates it automatically after that.
|
||||
|
||||
## Commands
|
||||
|
||||
- `make dev` (default: bare `make`) - applies pending migrations, starts the
|
||||
Tailwind watcher in the background, runs the server in the foreground.
|
||||
Ctrl-C stops both.
|
||||
- `make run` - just `go run .`, no migrations or CSS.
|
||||
- `make test` - runs the full test suite against the dedicated
|
||||
`inventory_2_test` database (migrates it first). Safe to run anytime;
|
||||
never touches real data.
|
||||
- `make test-against-dev-db` - same suite, pointed at the real dev database
|
||||
instead. Useful for checking behavior against real data; tests clean up
|
||||
after themselves via `t.Cleanup`, but it's still touching your dev DB.
|
||||
- `make migrate-up` / `migrate-down` / `migrate-version` - wrap the
|
||||
`migrate` CLI using `DATABASE_URL` from `.env`, so the DSN is never
|
||||
hand-typed. `migrate-test-*` variants target `TEST_DATABASE_URL`.
|
||||
- `go build ./...`, `go vet ./...`, `gofmt -l .` - standard, expected clean
|
||||
before considering work done.
|
||||
- `go test ./... -race` - the test suite is safe to run this way (see
|
||||
Testing below for what made that true).
|
||||
|
||||
## Architecture
|
||||
|
||||
- `domains/` - business logic, one package per bounded concern
|
||||
(`accounts`, `authentication`, `raw_events`, `reports`, `amazon`,
|
||||
`platforms/etsy`). Each domain owns its own DB access; there's no shared
|
||||
ORM/repository layer. Looking to loosely follow CQRS: writes go through
|
||||
domain `Store` methods, reads are mostly separate query methods on the
|
||||
same `Store`.
|
||||
- `domains/accounts/mocks.go` - the mock-platform simulation layer:
|
||||
`CreateMockShop`, `CreateMockListing`, `SaveNewMockSale` /
|
||||
`SaveNewMockRefund` / `SaveNewMockInventoryReset`, etc. These are the
|
||||
*real* entry points production code (and the simulate-sale/refund/
|
||||
inventory UI) uses - prefer them over hand-rolled SQL when writing tests
|
||||
or new features that need mock shop/listing/event data.
|
||||
- `domains/amazon/mock.go` - the one platform with a live *background
|
||||
processor* on top of the mock layer: a Postgres `LISTEN`/`NOTIFY`-driven
|
||||
loop (`(*Mocks).ProcessEvents`) that reacts to newly inserted mock events
|
||||
and dispatches them to a `MockEventListener` (wired to Server-Sent Events
|
||||
in `main.go`, so the UI updates live). No other platform has this yet;
|
||||
if adding one, read this file's history/comments first - it went through
|
||||
several correctness passes (reconnect-on-failure, ack-based retry,
|
||||
configurable poll fallback) worth not re-discovering from scratch.
|
||||
- `server/` - HTTP layer (Gin). `server/api` is the JSON/HTMX API,
|
||||
`server/ui` renders HTML pages, `server/auth` is session/auth middleware,
|
||||
`server/sse` is the Server-Sent Events plumbing.
|
||||
- `config/config.go` - all runtime configuration comes from environment
|
||||
variables (loaded from `.env` via `godotenv`), never hardcoded. If you
|
||||
add a new required external dependency (a new secret, a new service
|
||||
URL), add it here, not as a literal in the code that uses it.
|
||||
- `internal/testdb` - shared test fixtures: `Pool(t)` (connects via
|
||||
`TEST_DATABASE_URL`, skips the test if unset), `Logger()`, `NewUserID(t)`,
|
||||
`SeedOAuthUser`/`SeedOAuthSession`.
|
||||
|
||||
## Database & migrations
|
||||
|
||||
- Migrations live in `database_migrations/`, run via `golang-migrate`
|
||||
(`migrate` CLI). Every migration needs a paired `.up.sql`/`.down.sql`.
|
||||
- **Migration files can drift from the live schema, and drift can mean
|
||||
"unrunnable," not just "stale docs."** This has happened at least twice:
|
||||
an interval-literal syntax error that failed on any fresh database, and a
|
||||
column that existed in the live dev DB but not in what the migration
|
||||
files would produce (fixed by adding a migration that reconciles the two,
|
||||
then `migrate force <version>` on the DB that already matched by hand).
|
||||
When something about the schema seems off, verify against the *live* DB
|
||||
(`psql \d`, `pg_get_viewdef(...)`) before trusting the migration files,
|
||||
and *especially* before assuming a fresh database would come up the same
|
||||
way the long-running dev one does.
|
||||
- The mock-platform schema (`mock.*`) follows a consistent per-platform
|
||||
pattern: `mock.shop_<platform>`, `mock.shop_<platform>_listings`, and for
|
||||
Amazon specifically `mock.shop_<platform>_events` (the background
|
||||
processor's queue). Some platforms also have a recursive
|
||||
`mock.shop_<platform>_listing_counts` view (built on
|
||||
`mock.shop_<platform>_listing_event_sequence`) that computes a running
|
||||
inventory count from `mock.raw_shop_events` - it's real per-platform
|
||||
logic living in SQL, not a passthrough, and worth reading with
|
||||
`pg_get_viewdef(..., true)` rather than assuming it matches another
|
||||
platform's shape.
|
||||
- **Platform-string casing is inconsistent and matters.** Most
|
||||
`accounts.Platform` constants are lowercase/snake_case (`"amazon"`,
|
||||
`"big_cartel"`, ...) matching what SQL triggers/views expect, but
|
||||
`Etsy`/`Tiktok`/`Wix` are capitalized Go-side (`"Etsy"`, not `"etsy"`).
|
||||
Some SQL objects for those platforms expect the capitalized form (e.g.
|
||||
the Etsy listing-counts view filters on `'Etsy'`), others expect
|
||||
lowercase (e.g. the Etsy `raw_shop_events` trigger checks `'etsy'`).
|
||||
Don't assume; check the actual SQL (`pg_get_viewdef`, trigger
|
||||
definitions) for the specific object you're relying on before writing a
|
||||
fixture or a query.
|
||||
- The `mock_shop_<platform>_event_inserted` `NOTIFY` channels are global,
|
||||
not scoped by shop/account. Any insert into `raw_shop_events` for that
|
||||
platform notifies *every* currently-listening connection for that
|
||||
platform, regardless of which test or code path caused it. See Testing
|
||||
below for how this bit two test packages at once.
|
||||
|
||||
## Testing
|
||||
|
||||
- Tests are real integration tests against a real Postgres database
|
||||
(`internal/testdb.Pool(t)`), not mocks/stubs of the DB layer - this
|
||||
codebase is mostly thin `Store` methods wrapping SQL, so testing the Go
|
||||
code in isolation would miss most of the actual risk.
|
||||
- Assertions use `github.com/stretchr/testify`'s `assert`/`require`
|
||||
throughout - `require` for anything the rest of the test depends on
|
||||
(halts immediately), `assert` for independent checks. Use
|
||||
`require.Eventually` for "poll until true or timeout" patterns instead of
|
||||
hand-rolling one.
|
||||
- **`require.Eventually`'s condition function runs on a separate
|
||||
goroutine.** Never call `require.*`/`t.Fatal*` (which invoke
|
||||
`t.FailNow()`) from inside one - Go's testing package requires `FailNow`
|
||||
to only be called from the test's own goroutine; calling it from a
|
||||
spawned one doesn't panic cleanly, it just silently reports the wrong
|
||||
thing. If a helper used inside an `Eventually` closure needs to check an
|
||||
error, give it a `*testing.T`-free variant that returns `(value, error)`
|
||||
and let the closure decide what "not yet satisfied" means.
|
||||
- Every test cleans up its own rows via `t.Cleanup`, in FK-safe order
|
||||
(children before parents). Use unique, randomly-generated IDs
|
||||
(`testdb.NewUserID(t)`, `uuid.NewString()`) so tests are safe to run
|
||||
concurrently with themselves and with real dev data under
|
||||
`test-against-dev-db`.
|
||||
- **Pick a mock platform deliberately when writing new fixtures that touch
|
||||
`mock.raw_shop_events`.** `domains/amazon`'s tests process *every*
|
||||
unprocessed event system-wide (that's correct behavior for its
|
||||
background worker, not a bug) and react to the global Amazon NOTIFY
|
||||
channel - any other package's tests that insert Amazon-platform mock
|
||||
events risk being picked up by `domains/amazon`'s tests when `go test
|
||||
./...` runs both packages' binaries concurrently (Go's default).
|
||||
`domains/reports`, for example, deliberately uses Etsy as its example
|
||||
platform for exactly this reason. If you're not testing
|
||||
`domains/amazon` itself, don't use Amazon as your fixture platform.
|
||||
- Never disable/re-enable a shared DB trigger to simulate a "notify never
|
||||
fires" scenario in a test, even temporarily - it's unsafe to do against
|
||||
a database anything else might be using concurrently (including a real
|
||||
`make dev` session someone has running). If you need to prove a
|
||||
timing/fallback path works, find a way to construct the scenario through
|
||||
normal writes instead (see `domains/amazon/mock_test.go`'s poll-fallback
|
||||
test for an example: it reuses the retry mechanism to guarantee a second
|
||||
dispatch can only come from the poll timer, without touching any
|
||||
trigger).
|
||||
- Killing a real Postgres backend connection (`pg_terminate_backend`,
|
||||
found via `pg_stat_activity`) is a legitimate, safe way to test
|
||||
reconnect/failure-recovery logic when properly scoped - see
|
||||
`domains/amazon/mock_test.go`'s `terminateListenConnection`. Be aware its
|
||||
PID lookup matches on query text globally, not "this test's connection
|
||||
specifically" - check for other live processes (e.g. a running `make
|
||||
dev`) before running a test like this against a shared database.
|
||||
|
||||
## Auth
|
||||
|
||||
- Real auth is Auth0-backed OIDC (`domains/authentication`). For local
|
||||
dev, set `DEV_AUTH_ENABLED=true` and hit `GET
|
||||
/api/auth/dev-login?user_id=whoever` to mint a real session (writes
|
||||
directly to `oauth_users`/`oauth_tokens`, same shape a real login
|
||||
produces) without any Auth0 round-trip. Different `user_id` values let
|
||||
you test multiple identities/accounts side by side. Never enable this
|
||||
outside local development.
|
||||
|
||||
## Commit conventions
|
||||
|
||||
- Commit messages: imperative mood subject line, no period, body explains
|
||||
*why* (what problem existed, what the fix changes) rather than
|
||||
re-describing the diff. One logical change per commit - e.g. a bugfix
|
||||
found while doing unrelated work gets its own commit, not folded into
|
||||
the original task's.
|
||||
- Work summaries: see `work-summaries/` for dated records of past
|
||||
sessions' changes and reasoning - useful context before touching an area
|
||||
someone else (human or agent) recently worked on.
|
||||
|
||||
Reference in New Issue
Block a user