README had drifted: an unresolved merge-conflict marker left over from a
past edit, a directory-structure diagram that no longer matches the repo
(/internal/site, /internal/domains/platforms/tiktok - actual layout is
server/, domains/, etc per AGENTS.md), and a duplicated dev-workflow blurb
that AGENTS.md already documents more accurately.
Splits the roadmap checklist out to ROADMAP.md and the Etsy API compliance
checklist to domains/platforms/etsy/COMPLIANCE.md (next to the code it
governs, where someone touching that integration will actually look for
it) instead of burying both in one large README. README itself becomes a
short front door with a "Where things live" index up top, since scattering
docs across files only helps if there's an obvious map to them.
Moves the CQRS/event-sourcing architecture note into AGENTS.md's
Architecture section (with a new domains/raw_events bullet) rather than
leaving it as prose in README, since AGENTS.md is the maintained
engineering reference and that's where a reader would already be looking
for how the domains are structured.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
.env.dev (used for local dev config, see AGENTS.md's DEV_AUTH_ENABLED
setup) was showing up as untracked instead of being ignored like .env.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Research to inform which mock platform (Amazon, Shopify, etc.) is worth
turning into a real integration next, after Etsy. Adds GMV/seller-count
market-size data for every platform in domains/accounts/platform.go, a
weighted scoring model (GMV 45%, API/inventory-webhook completeness 30%,
growth 15%, integration cost 10%), and the resulting ranked order.
Also fills in Etsy's own row, sourced directly from
domains/platforms/etsy/generated_client rather than external docs since
it's the one platform actually live in this repo: Etsy Open API v3 has no
webhook/push mechanism at all, and only the OAuth connection flow is
built so far - polling-based order/inventory sync doesn't exist yet. Etsy
is included as a reference score (3.85/10) against the same rubric used
for the unbuilt platforms, mainly as a sanity check on the model rather
than a claim that building it first was wrong.
Flags that the existing capability table's BigCommerce row doesn't match
an actual platform in this codebase (Tiktok does, and is now researched
instead), and that Tiktok Shop's API details are unconfirmed pending a
dedicated research pass.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Squeamish about New()'s empty-domain-string sentinel for "dev mode, skip
OIDC discovery" - split into New (always makes a real OIDC discovery
call, all params required) and NewDev (no ctx/domain/credentials at all,
since none are used). main.go now branches on cfg.DevAuthEnabled to pick
the right constructor instead of main.go/config.go coordinating on when
it's safe to pass empty strings.
Also finishes out the dev-auth flow this enables: config.Load reads a
DEV_AUTH_ENABLED-aware env file and only requires Auth0 vars when dev
auth is off; a PORT config var replaces the hardcoded :8082; and the nav
UI (layout/index templates, ui router) points login/logout links at
/api/auth/dev-login and a new /api/auth/dev-logout route when dev auth
is enabled, so the whole login/logout loop works locally without a real
Auth0 app.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Setup, common commands, architecture overview, and the database/testing
gotchas actually hit while working in this repo recently: migration
files that can drift to the point of being unrunnable (not just
stale), inconsistent platform-string casing between Go constants and
SQL objects, global (unscoped) mock-platform NOTIFY channels, the
require.Eventually-runs-on-a-goroutine hazard, and why domains/reports'
fixtures use Etsy rather than Amazon. CLAUDE.md already points here via
@AGENTS.md, so no change needed there.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
domains/reports' fixtures used Amazon as their example platform - same
as domains/amazon's own tests. Both packages' test binaries run
concurrently under go test ./... by default, both wrote to the shared
mock.raw_shop_events table with platform='amazon', and the DB trigger
routed those into mock.shop_amazon_events - the exact table
domains/amazon's background-processing tests poll and assert on.
Confirmed directly: running the two packages together, domains/amazon's
TestProcessUnprocessedEvents_RetriesUnackedNotification picked up 4
events instead of 1, three of them from domains/reports' fixture shop.
Switched to Etsy instead (test-only change, no production code
touched). Verified the casing first since it mattered here:
accounts.Etsy's Go value is "Etsy" (capital), and separately Etsy's
raw_shop_events trigger checks for lowercase 'etsy' - but the view
these tests actually depend on (mock.shop_etsy_listing_event_sequence)
filters on 'Etsy', matching the Go constant, confirmed via
pg_get_viewdef. So the fixtures work correctly and, as a side effect,
never fire the lowercase-gated trigger at all - keeping
mock.shop_etsy_events untouched by these tests regardless.
Combined with the previous commit's goroutine-leak fix, go test ./...
and make test are both reliably green as single commands again.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
listenForNotifications' notification-forwarding goroutine sent on ch
unconditionally after each WaitForNotification. If ProcessEvents' main
loop had already exited (ctx cancelled) at the exact moment this
goroutine had a notification to forward, nobody was left reading from
the unbuffered channel - the send blocked forever, the goroutine never
reached its deferred pc.Release(), and the pooled connection leaked
permanently. Real in production (a shutdown racing an in-flight
NOTIFY), not just a test artifact.
Surfaced by a go test ./... hang inside domains/amazon (a goroutine
stuck in pgxpool.Pool.Close's WaitGroup.Wait) - increased cross-package
NOTIFY traffic from domains/reports' Amazon-platform fixtures made the
race easy to hit, but didn't cause it.
Fixed with a select alongside the send so the goroutine notices
ctx.Done() instead of blocking forever when nobody's listening anymore.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
Replaces raw t.Error/t.Errorf/t.Fatal/t.Fatalf across every test file
that has any (domains/accounts, domains/authentication,
domains/raw_events, domains/amazon, domains/reports x2) with testify's
assert (non-halting) / require (halting) equivalents. The three
Example-based tests (server/ui/svg, server/ui/charts) have no
*testing.T at all - nothing to convert there.
require.Eventually replaces several hand-rolled polling loops in
domains/amazon/mock_test.go. Its condition function runs on a separate
goroutine (confirmed in testify's source), so calling require.* from
inside one - which two of the new Eventually calls initially did, via
the isProcessed helper - is unsafe per Go's testing rules (t.FailNow
must only be called from the test's own goroutine). Fixed by splitting
a *testing.T-free queryIsProcessed(ctx, pool, shopID, eventID) out of
isProcessed for use inside those closures specifically.
github.com/stretchr/testify promoted from an indirect to a direct
dependency (go.mod only - it was already present transitively, so
go.sum is unchanged).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
mock.shop_amazon_listing_counts is a recursive view that computes a
running inventory count: starting from the listing's base count in
mock.shop_amazon_listings, it walks mock.raw_shop_events in order,
applying each event as a delta (sale/refund) or an absolute reset
(inventory-reset), per mock.shop_amazon_listing_event_sequence's
interpretation of each row's JSON payload. This is the first test to
exercise that view directly rather than just the Go code around it.
Fixture uses the real SaveNewMockSale/SaveNewMockRefund/
SaveNewMockInventoryReset methods - the same entry points the
simulate-sale/refund/inventory UI uses - rather than hand-rolling the
JSON payload shape, so the test tracks the real payload contract.
Covers the full running-count sequence (base -> sale -> refund ->
reset), the unknown-listing ErrNotFound case, and
GetListingCountsReport's MaxCount/MinCount over that sequence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
Covers the happy path (ordering by event_timestamp DESC/event_id ASC,
Platform/ShopID fields, RawPayload round-tripping through jsonb), the
empty-shop case, and the unknown-shop ErrNotFound case (via the
accts.GetMockShop check GetRawShopEvents does before querying).
setupAmazonMockShop creates a real account + Amazon mock shop via the
same Store methods the app uses (CreateAccount, CreateMockShop) and
registers cleanup in FK-safe order.
GetListingCountsOverTime/GetListingCountsReport coverage is a separate,
larger task (needs a listing plus count-changing history feeding a DB
view) - not done here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
A dropped connection, a Postgres restart, or any other error on the
dedicated LISTEN connection previously propagated all the way out of
ProcessEvents, and main.go's top-level shutdown logic treats that
error channel firing the same as a fatal server error - taking down
the entire application over a hiccup on one background connection that
has nothing to do with serving HTTP traffic. This matters more with
eleven other platforms already sharing the same trigger+notify shape
in the migrations with no Go processor yet.
Adds (*Mocks).reconnectOrStop: on a real failure (not an ordinary
shutdown), logs a warning and re-establishes LISTEN after a backoff
that starts at 1s, caps at 30s, doubles on repeated immediate
failures, and resets once a reconnect actually succeeds.
ProcessEvents' select no longer returns on a LISTEN error - it loops
back in with fresh channels instead.
Verified against a genuinely killed connection (pg_terminate_backend,
targeting the backend via pg_stat_activity matched on its LISTEN
query text), not a simulated one - both in a manual check and in the
new TestProcessEvents_ReconnectsAfterListenConnectionDrops test.
Closes out all four insights from the domains/amazon design review.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
ProcessEvents' safety-net poll was a bare time.Minute literal inline in
its select statement - no way to verify the fallback path works without
waiting 60+ seconds in a test, no way to tune the cadence without a code
change, and no way to tell afterward whether a given loop iteration was
triggered by a real NOTIFY or by the poll timer.
Adds Mocks.pollInterval (default time.Minute) and a WithPollInterval(d)
builder, mirroring WithNotifyRetryAfter's existing pattern - deploy-time
configurable via construction, not a live/API-adjustable knob, and not
wired through .env, matching how notifyRetryAfter already works.
Adds a Debug log line on each of the two meaningful wake-up branches so
which path fired is now observable.
New test proves the poll branch actually works without waiting or
touching the shared DB trigger (which would be unsafe against the real
dev DB): it reuses the retry mechanism from the previous commit so a
second dispatch can only come from the poll timer, since nothing else
ever notifies again for the rest of the test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
Dispatching an event to the listener previously happened from an
un-awaited goroutine, with only a log line on failure - once
processed_at was set, a dropped or failed notification was permanently
and silently lost, with no way to tell it had happened.
Replaces the processed/processed_successfully booleans with a
notified_at/processed_at pair (migration 000031): the dispatcher sets
notified_at and hands the event to MockEventListener.Notify, which now
also receives an ack callback the listener calls whenever it's truly
done, synchronously or arbitrarily later. Anything still "notified" but
unacked past notifyRetryAfter (a tunable field, not a stored
per-row timestamp) gets notified again on the dispatcher's normal
poll/reactive loop - no new retry mechanism needed. ack is idempotent,
since a late ack from an earlier attempt and one from a retry can both
eventually fire for the same event.
Dispatch is deferred until after the transaction that recorded
notified_at has actually committed, so ack's independent write can't
race a still-open transaction it implicitly depends on being visible.
The real SSE listener (server/sse/db_event_publisher.go) acks inline,
since its work is synchronous.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
Tests (and any other caller) previously had no way to know when the
Postgres LISTEN behind the reactive event-processing loop had actually
registered, forcing a guessed sleep before relying on it. Mocks now
exposes Ready() <-chan struct{}, closed once listenForNotifications
successfully issues LISTEN. Purely additive - ProcessEvents' signature
is unchanged.
Updates the integration tests to wait on Ready() instead of a flat
sleep, which also cut TestProcessEvents_ReactsToNotification's runtime
from ~0.25s to ~0.06s with no flakes across repeated runs.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XEDaCB7C2NEBgyvqEtZuxY
angelandClaude Sonnet 5