From b9dd4fc0e58a62121fb61184dc2e2a55882b947e Mon Sep 17 00:00:00 2001 From: Angel Beltran Date: Thu, 20 Aug 2026 17:34:58 -0600 Subject: [PATCH] docs: split README into distributed docs, fix stale/broken content README had drifted: an unresolved merge-conflict marker left over from a past edit, a directory-structure diagram that no longer matches the repo (/internal/site, /internal/domains/platforms/tiktok - actual layout is server/, domains/, etc per AGENTS.md), and a duplicated dev-workflow blurb that AGENTS.md already documents more accurately. Splits the roadmap checklist out to ROADMAP.md and the Etsy API compliance checklist to domains/platforms/etsy/COMPLIANCE.md (next to the code it governs, where someone touching that integration will actually look for it) instead of burying both in one large README. README itself becomes a short front door with a "Where things live" index up top, since scattering docs across files only helps if there's an obvious map to them. Moves the CQRS/event-sourcing architecture note into AGENTS.md's Architecture section (with a new domains/raw_events bullet) rather than leaving it as prose in README, since AGENTS.md is the maintained engineering reference and that's where a reader would already be looking for how the domains are structured. Co-Authored-By: Claude Sonnet 5 --- AGENTS.md | 6 ++ README.md | 139 ++++----------------------- ROADMAP.md | 38 ++++++++ domains/platforms/etsy/COMPLIANCE.md | 19 ++++ 4 files changed, 84 insertions(+), 118 deletions(-) create mode 100644 ROADMAP.md create mode 100644 domains/platforms/etsy/COMPLIANCE.md diff --git a/AGENTS.md b/AGENTS.md index 2745d3e..ed3b3b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -117,6 +117,12 @@ against the live API doesn't exist yet. ORM/repository layer. Looking to loosely follow CQRS: writes go through domain `Store` methods, reads are mostly separate query methods on the same `Store`. +- `domains/raw_events` - the [event-sourcing](https://martinfowler.com/eaaDev/EventSourcing.html) + store: `Save` appends an `Event`, `LoadEventsForStore` replays a + platform+store's series. Other domains (`reports`, + `domains/accounts/mocks.go`) read from views/queries that project over + this event series rather than mutating their own standalone state. See + also the [CQRS](https://martinfowler.com/bliki/CQRS.html) note above. - `domains/accounts/mocks.go` - the mock-platform simulation layer: `CreateMockShop`, `CreateMockListing`, `SaveNewMockSale` / `SaveNewMockRefund` / `SaveNewMockInventoryReset`, etc. These are the diff --git a/README.md b/README.md index c22e944..84afd7f 100644 --- a/README.md +++ b/README.md @@ -7,12 +7,18 @@ The benefits provided by this application will be to automatically manage shared reducing the amount of time needed to synchronize inventory between stores. -# Research +## Where things live -[Research documents](/research/platforms.md) +Documentation is split up rather than kept in one big doc - start here: + +- **`AGENTS.md`** - the engineering reference: setup, `make` commands, architecture, database/migration gotchas, testing conventions, auth, and the researched platform-integration priority ranking (which platform to build next, and why). +- **`ROADMAP.md`** - in-progress and planned work, as a checklist. +- **`research/platforms.md`** - the underlying API-capability and market-size research the platform-priority ranking is based on. +- **`domains/platforms/etsy/COMPLIANCE.md`** - Etsy API usage obligations to keep satisfied when touching the Etsy integration. +- **`diagrams/`** - architecture and schema diagrams, linked from the Diagrams section below. -# Deploying +## Deploying The application runs locally, from this directory. It is deployed simply by running either from the root of the project, @@ -24,12 +30,7 @@ or go run . ``` - -# Development - -## Testing - -There's a Makefile with a number of operations for running and testing the application +See `AGENTS.md` for the full setup/`make` command reference (migrations, tests, dev auth, etc). ## Technology @@ -37,7 +38,7 @@ There's a Makefile with a number of operations for running and testing the appli ### Languages #### Server side -Golang, Go Templates, +Golang, Go Templates #### Front end HTML, CSS, Javascript @@ -49,7 +50,6 @@ Postgres, in docker Using [migrate](https://github.com/golang-migrate/migrate) to manage build out the database schema, and to run migrations. - ### Tooling #### Server side @@ -62,102 +62,16 @@ migrations. - htmx: for strong hypermedia support - hyperscript: for minimal, inline scripting, with strong integration with htmx - tailwind: for styling the front end, using tried and testing styling paradigms, conventions, and templates. ->>>>>>> 16a3551 (updated readme) - - -# Roadmap - -- [ ] Etsy (WIP) - - [ ] GET ETSY AUTH (WIP) - - [x] move auth state stuff to database (out of cache) - - [x] only generate a sign up link IF they click the link on the accounts page - - [ ] get api key approved - - [ ] automatically clean up access tokens and state when expired - - [ ] access tokens - - [ ] state - - [?] Get new access token using refresh token flow - - [ ] make a FK between the etsy_store_events table and etsy_users table (store_id columns don't match types) -- [ ] Auth0 - - [ ] get off dev api key? - - [x] Get new access token using refresh token flow - - [ ] test - - [x] when token is expired, redirect them to the login page, then redirect them back to where they were heading to. - - [ ] Social connections login - - [x] automatically clean up access tokens and state when expired - - [x] access tokens - - [x] state -- [ ] Complete this design document? -- [ ] Complete defining this roadmap checklist -- [ ] Website displaying an audit of store events - - [ ] Start with just a list of events for a given store (use a static test store) - - [ ] ... -- [ ] Dark mode -- [ ] don't let a listing be in multiple sync groups -- [ ] Next stores on the list (at least hypothetically) - - Shopify - - WooCommerce - - BigCommerce - - Wix - - Squarespace - - Square Online - - Zoho - - Ecwid - - Big Cartel - Bigger marketplaces: - - Amazon - - Walmart Marketplace - - Ebay - -## Nice to haves - -- [ ] Drop in a good logger -- [ ] log all errors caught by the http server - -## Constraints - -- [ ] Etsy - - [ ] API Licensed Uses and Restrictions: - - [ ] Link directly back to the product information and/or image Content on Etsy, where the Application utilizes product information and/or images. - - [ ] Provide a prominently displayed email address on Your Application for third parties to contact You with any questions or issues. You shall respond to such inquiries in a timely manner. - - [ ] Use commercially reasonable efforts to provide a terms of service and privacy policy in a visible location on your Application. - - [ ] Display item Content or product information and/or images which is more than six (6) hours older than such information is on the Website, and other Etsy Content cannot be more than twenty-four (24) hours older than such Content on the Website. - - [ ] Use the API in a manner that exceeds reasonable request volume or constitutes excessive or abusive usage. Users are allocated by default, 10,000 calls per day. - - [ ] You shall not use or alter any text, logos, Etsy's Trademarks, Etsy's signature colors, Etsy's layout, or a confusingly similar layout to Etsy's layout in such a way which may suggest endorsement or affiliation by Etsy. - - [ ] Any use of the Etsy logo or Etsy's Trademarks must be used in its entirety and must not be altered or used in a misleading way. - - [ ] You shall not use a mark which is confusingly similar to Etsy's Trademarks. - - [ ] Any use of the Etsy logo or Etsy's Trademarks in Your Application shall be less prominent than the logo or mark that primarily describes the Application and Your use of the Etsy logo shall not imply any endorsement or affiliation by Etsy. - - [ ] You may publicize, issue press or blog releases of Your Application only if You state that it was created using the Etsy API and that You in no way imply that Your Application is endorsed or certified by Etsy. - - [ ] You must place or display the following notice prominently on Your Application: - "The term 'Etsy' is a trademark of Etsy, Inc. This application uses the Etsy API but is not endorsed or certified by Etsy, Inc." - - [ ] Immediately report any security deficiencies You discover to Etsy by emailing developer@etsy.com. - - -Looking to follow the [CQRS](https://martinfowler.com/bliki/CQRS.html) pattern. - -The database will follow the [event sourcing](https://martinfowler.com/eaaDev/EventSourcing.html) database pattern. -All events (or commands) will be stored in a respective event series, and all database reads will be from views that are projections, reductions, aggregations of those event series. - - -## Application/directory structure -- /internal - - /site: website - - /webhooks: webhooks for platform events - - /domains: packages for each domain - - /store_events: storing and events - - /platforms: ecommerce platform domains - - /tiktok: interface with tiktok - - ... etc - - ... etc - -![Application structure](./diagrams/application_structure.svg) - -## Website hierarchy - -- /site # Architectural and Software Diagrams +## Application structure + +See `AGENTS.md`'s Architecture section for the current, maintained breakdown of packages/directories. + +![Application structure](./diagrams/application_structure.svg) + ## Database schemas **public** @@ -169,39 +83,28 @@ All events (or commands) will be stored in a respective event series, and all da ## Events +`domains/raw_events` is the event-sourcing store behind this - see `AGENTS.md`'s Architecture section. + ### Event Sourcing Architecture ![Event sourcing architecture](./diagrams/event_sourcing.svg) - ### Event Structure - ![Event](./diagrams/event.svg) - ### Store Event Database Tables - ![Event database tables](./diagrams/event_tables.svg) ## Platform: Etsy ### Signing up - -**TODO: need an account page that can create accounts ahead of time - force users to create an account first!** - ![Event database tables](./diagrams/etsy/obtaining_access_token.svg) -***TODO: create a page that will take billing information and include it in this process*** - -### Getting a new refresh token - -***TODO*** - - ### Models - ![Models](./diagrams/etsy/models.svg) +See `ROADMAP.md` for what's still unbuilt in the Etsy flow (an account-creation page ahead of OAuth, billing info collection, refresh-token handling), and `domains/platforms/etsy/COMPLIANCE.md` for API usage obligations. + ## All Diagrams diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..63270a4 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,38 @@ +# Roadmap + +- [ ] Etsy (WIP) + - [ ] GET ETSY AUTH (WIP) + - [x] move auth state stuff to database (out of cache) + - [x] only generate a sign up link IF they click the link on the accounts page + - [ ] get api key approved + - [ ] automatically clean up access tokens and state when expired + - [ ] access tokens + - [ ] state + - [ ] account-creation page ahead of OAuth (force users to create an account before connecting a store) + - [ ] collect billing information as part of that account-creation flow + - [?] Get new access token using refresh token flow + - [ ] make a FK between the etsy_store_events table and etsy_users table (store_id columns don't match types) + - [ ] build polling-based order/inventory sync against the live Etsy API - only the OAuth connection flow is implemented so far (see `research/platforms.md`'s Etsy notes: API v3 has no webhook/push system, so this has to be a poll loop) + - see `domains/platforms/etsy/COMPLIANCE.md` for Etsy API usage obligations to keep satisfied along the way +- [ ] Auth0 + - [ ] get off dev api key? + - [x] Get new access token using refresh token flow + - [ ] test + - [x] when token is expired, redirect them to the login page, then redirect them back to where they were heading to. + - [ ] Social connections login + - [x] automatically clean up access tokens and state when expired + - [x] access tokens + - [x] state +- [ ] Complete this design document? +- [ ] Complete defining this roadmap checklist +- [ ] Website displaying an audit of store events + - [ ] Start with just a list of events for a given store (use a static test store) + - [ ] ... +- [ ] Dark mode +- [ ] don't let a listing be in multiple sync groups +- [ ] Next platform to build out beyond Etsy - see `AGENTS.md`'s "Platform integration priority" section for the researched, weighted ranking (currently: Shopify, then Amazon, then Tiktok Shop) instead of picking from the raw platform list + +## Nice to haves + +- [ ] Drop in a good logger +- [ ] log all errors caught by the http server diff --git a/domains/platforms/etsy/COMPLIANCE.md b/domains/platforms/etsy/COMPLIANCE.md new file mode 100644 index 0000000..b57eacd --- /dev/null +++ b/domains/platforms/etsy/COMPLIANCE.md @@ -0,0 +1,19 @@ +# Etsy API compliance checklist + +Obligations from Etsy's API Licensed Uses and Restrictions. Anything +touching `domains/platforms/etsy` or the Etsy-facing UI should keep these +satisfied before shipping. + +- [ ] Link directly back to the product information and/or image Content on Etsy, where the Application utilizes product information and/or images. +- [ ] Provide a prominently displayed email address on Your Application for third parties to contact You with any questions or issues. You shall respond to such inquiries in a timely manner. +- [ ] Use commercially reasonable efforts to provide a terms of service and privacy policy in a visible location on your Application. +- [ ] Display item Content or product information and/or images which is more than six (6) hours older than such information is on the Website, and other Etsy Content cannot be more than twenty-four (24) hours older than such Content on the Website. +- [ ] Use the API in a manner that exceeds reasonable request volume or constitutes excessive or abusive usage. Users are allocated by default, 10,000 calls per day. +- [ ] You shall not use or alter any text, logos, Etsy's Trademarks, Etsy's signature colors, Etsy's layout, or a confusingly similar layout to Etsy's layout in such a way which may suggest endorsement or affiliation by Etsy. +- [ ] Any use of the Etsy logo or Etsy's Trademarks must be used in its entirety and must not be altered or used in a misleading way. +- [ ] You shall not use a mark which is confusingly similar to Etsy's Trademarks. +- [ ] Any use of the Etsy logo or Etsy's Trademarks in Your Application shall be less prominent than the logo or mark that primarily describes the Application and Your use of the Etsy logo shall not imply any endorsement or affiliation by Etsy. +- [ ] You may publicize, issue press or blog releases of Your Application only if You state that it was created using the Etsy API and that You in no way imply that Your Application is endorsed or certified by Etsy. +- [ ] You must place or display the following notice prominently on Your Application: + "The term 'Etsy' is a trademark of Etsy, Inc. This application uses the Etsy API but is not endorsed or certified by Etsy, Inc." +- [ ] Immediately report any security deficiencies You discover to Etsy by emailing developer@etsy.com.