diff --git a/.gitignore b/.gitignore index 46879f79..bcf36af6 100644 --- a/.gitignore +++ b/.gitignore @@ -82,3 +82,7 @@ session-*.md # Local prod-export scratch: personal annotation/reading data, never commit /export/ + +# Binary media fixtures: never commit (generate with ffmpeg at test time, +# see docs/contributing/development.md → Media fixtures) +internal/services/testdata/ diff --git a/AGENTS.md b/AGENTS.md index f2e62397..36850511 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,6 +85,24 @@ were name-only across ALL libraries. Fixed; do not regress. cleanups** — sync_integration's `DELETE FROM reading_history WHERE true` destroyed the live reader's stats data on every suite run until scoped to the test-sync users (cascade via user delete). +- **RUN ONLY YOUR OWN TESTS while iterating**: integration tests + require the containerized dev DB — a bare host `go test + ./cmd/server/tests` fails on auth on every machine not running the + compose stack. Use the filtered container runner + (`docker compose -f docker-compose.yml -f + docker-compose.dev.yml --profile tests run --rm tests go test + ./cmd/server/tests -run 'TestYourPrefix' -v`) and save full-suite + runs for pre-commit validation; several tests read live shared + state (system_settings, real library folders, the periodic backup + scan's websocket broadcasts) and flap on data/timing your change + never touched. Full recipes: docs/contributing/development.md + → Testing. +- **NEVER COMMIT BINARY MEDIA** (mp3/m4b/flac/... fixtures): + `internal/services/testdata/` is gitignored by policy; generate + media at test runtime with ffmpeg instead (test images ship it). + ffmpeg cannot write Audible freeform atoms — those paths are unit- + tested with synthetic ffprobeReport values. Recipes: + docs/contributing/development.md → Media fixtures. ## Timezones diff --git a/docs/contributing/development.md b/docs/contributing/development.md index 7a5f0eeb..7dbadab9 100644 --- a/docs/contributing/development.md +++ b/docs/contributing/development.md @@ -258,17 +258,82 @@ go tool cover -html=coverage.out ### Integration Tests +Integration tests hit the LIVE dev database and bind-mounted paths, both +of which exist only inside the Docker stack. A bare host `go test +./cmd/server/tests` fails with database-auth errors on every machine +that isn't running the compose stack — the container is the only +supported runner. + ```bash -# Run integration tests (may hit rate limits) -go test ./cmd/server/tests -v +# Run the whole integration suite (pre-commit validation — NOT for +# iterating on a change; some tests are coupled to the shared dev DB's +# live state and this environment) +make test-integration -# Run with test mode (recommended) -TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -v - -# Run specific test -go test ./cmd/server/tests -run TestAuth -v +# Iterate on YOUR change: run only your own tests, in the container +docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile tests run --rm tests \ + go test ./cmd/server/tests -run 'TestYourPrefix' -v -timeout 600s ``` +**Run only your own tests while iterating.** A full-suite run rebuilds +the test image and takes minutes; more importantly, several tests read +live state (the dev DB's `system_settings`, real library folders, a +periodic backup scan broadcasting over websocket) and can fail on +timing or on data your change never touched. Filtered runs are cheap, +deterministic for your scope, and don't fight the live reader for the +database. + +Test-created data rules (harness cleanup depends on them): users use +`@tests.bookhoard.internal` / `@example.com` emails, libraries and +collections carry "test"-prefixed names, and cleanups delete test +libraries (cascades handle the rest) rather than table-wide deletes. + +### Media Fixtures (audio/books) + +**Never commit binary media.** The Go toolchain and Docker prod builds +ignore `internal/services/testdata/` entirely (gitignored by policy — +see `.gitignore`), and the production image never copies `internal/`. +Generate what a test needs at runtime with ffmpeg (shipped in both the +test-runner and runtime images): + +```bash +# The five audio containers (audiobook library accepts all of these) +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libmp3lame -b:a 32k plain.mp3 +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a flac book.flac +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libopus -b:a 24k book.opus +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libvorbis -q:a 1 book.ogg +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a aac -b:a 32k book.aac +ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a pcm_s16le book.wav + +# Tags (album=book title, album_artist=author, composer=narrator — +# Audible's own convention — plus disc/track ordering keys) +ffmpeg -i in.mp3 -codec copy \ + -metadata title="Chapter Three" -metadata album="The Book" \ + -metadata album_artist="Author" -metadata composer="Narrator" \ + -metadata track="3/9" -metadata disc="2/3" \ + -metadata grouping="Series" -metadata genre="Audiobook" -metadata date=2020 out.mp3 + +# Embedded cover art (mp3 APIC / m4b covr): the picture rides as the +# only video stream +ffmpeg -i cover.jpg -i audio-only.mp3 -map 1:a -map 0:v -c:v copy \ + -disposition:v attached_pic tagged.mp3 + +# Embedded m4b chapters: an ffmetadata sidecar file +printf ';FFMETADATA1\n[CHAPTER]\nTIMEBASE=1/1000\nSTART=0\nEND=500\ntitle=Part One\n' > chapters.txt +ffmpeg -i in.m4b -i chapters.txt -map 0:a -map 1 -map_metadata 1 -codec copy -f ipod chaptered.m4b +``` + +CUE sheets and m3u playlists are plain text — write them inline in the +test. **The one thing ffmpeg cannot synthesize: Audible freeform atoms** +(`----:com.apple.iTunes:SERIES`, `:SUBTITLE`, `:ASIN`, `:ABRIDGED`) — +those ingestion paths are covered by unit tests using synthetic +`ffprobeReport` values instead of binary files. + +Manual end-to-end testing against the live system: drop fixture books +into `uploads/Audiobooks/Ingestion Fixtures/` — **one book per +subfolder** (a directory with ≥2 audio files ingests as ONE +folder-of-tracks book) — and run a library scan from the admin UI. + ### API Testing with Bruno OpenCollection YAML ```bash