Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03c3e151dc | ||
|
|
3576a4415a | ||
|
|
3325b64463 | ||
|
|
a9430f3c06 | ||
|
|
96fd2aad42 | ||
|
|
1f30602c23 | ||
|
|
18bed79d3a | ||
|
|
fe7eab91e8 | ||
|
|
295c4b0664 | ||
|
|
db96ea91cd | ||
|
|
408a972665 | ||
|
|
6508abc00f | ||
|
|
3ed852515b | ||
|
|
f62b46fb15 | ||
|
|
35d8ba32e5 | ||
|
|
80bfc1a77b | ||
|
|
970beee4b8 | ||
|
|
fa5b952c54 | ||
|
|
ff17d14694 | ||
|
|
807899b137 | ||
|
|
6791d611f0 | ||
|
|
e5315f1b94 | ||
|
|
86c4940426 | ||
|
|
d7d1495753 | ||
|
|
77cd268f4e | ||
|
|
8aeecaa0ef | ||
|
|
dd4cf1235c | ||
|
|
a42c6d4f75 | ||
|
|
9b5defcb31 | ||
|
|
19b9c7c9b8 | ||
|
|
91b0c45dd4 | ||
|
|
43987262a6 | ||
|
|
68701d6b94 | ||
|
|
ab713e2feb | ||
|
|
9c006a9a2f | ||
|
|
4c03116ec7 | ||
|
|
c747f0a76c | ||
|
|
6f98ca4c22 | ||
|
|
04b36f0563 | ||
|
|
4ee24e7152 | ||
|
|
ebb7f4a5b0 | ||
|
|
f52bb6f073 | ||
|
|
6152137156 | ||
|
|
2c5ad014c0 | ||
|
|
5245f292ca | ||
|
|
6750e9e0dc | ||
|
|
693053a9f5 | ||
|
|
bbedbfb13f | ||
|
|
7e586df48a | ||
|
|
7537063cb7 | ||
|
|
f94e7472b4 | ||
|
|
1dbb0ce406 | ||
|
|
7e4c97f639 | ||
|
|
646477afad | ||
|
|
f7dc8189a1 | ||
|
|
a8641ab343 | ||
|
|
1ce66b2213 | ||
|
|
18f52d240b | ||
|
|
220d97cfe0 | ||
|
|
38efbff7e1 | ||
|
|
f4414aa5b6 | ||
|
|
3daa834a94 | ||
|
|
202a090811 | ||
|
|
ee98c0fb3b | ||
|
|
79ce5ab89d | ||
|
|
6216bd9bcf | ||
|
|
24228e7147 | ||
|
|
e1153b2c4b | ||
|
|
5b81e19f2f | ||
|
|
30ecdd4bf3 | ||
|
|
b2220756e2 | ||
|
|
d74db03cf3 | ||
|
|
9e831085ef | ||
|
|
327fc2c0f9 | ||
|
|
a31da89df5 | ||
|
|
a11b4a9cc4 | ||
|
|
040c10ac76 | ||
|
|
4a68fe7cab | ||
|
|
cb4e892b10 | ||
|
|
506a04b55d | ||
|
|
f1844591b4 | ||
|
|
1fdf306f2d |
No files matched your search
@@ -82,3 +82,7 @@ session-*.md
|
||||
|
||||
# Local prod-export scratch: personal annotation/reading data, never commit
|
||||
/export/
|
||||
|
||||
# Binary media fixtures: never commit (generate with ffmpeg at test time,
|
||||
# see docs/contributing/development.md → Media fixtures)
|
||||
internal/services/testdata/
|
||||
@@ -4,6 +4,43 @@ Standing rules for any agent or developer working in this repo.
|
||||
Project-wide context lives in the sibling repos' AGENTS.md files
|
||||
(../bookhoard-app/AGENTS.md is the Android client's).
|
||||
|
||||
## Rendering ownership (templ vs TypeScript) — ALWAYS apply
|
||||
|
||||
The point of templ + htmx + Alpine is: the server renders the UI, and
|
||||
JavaScript is kept minimal. The rule that keeps it that way:
|
||||
|
||||
> **JavaScript decides what shows. templ decides what it looks like.**
|
||||
|
||||
- Content derived from **server data** (DB rows, search results, match
|
||||
lists, conflict cards, any list/card/badge/table) is rendered by
|
||||
templ. When it changes client-side, fetch a **templ partial** from an
|
||||
SSR route and swap it in (`htmx.ajax(...)`), never rebuild it from a
|
||||
JSON payload with `innerHTML` string templates.
|
||||
- **Pure functions of client-local state** (open/closed panels, tabs,
|
||||
password strength from a typed value, toasts) may render in
|
||||
TS/Alpine — no server round trip needed.
|
||||
- `web/src` is **TypeScript only** — no new `.js` source files.
|
||||
Vendored third-party and vite build output under `web/static` are
|
||||
exempt.
|
||||
- Inline `<script>` in a `.templ` file needs a reason a bundled module
|
||||
cannot do the job (pre-paint execution). Currently sanctioned:
|
||||
`theme_flash_guard.templ` only.
|
||||
|
||||
Sanctioned JS islands (engines, not UI renderers — exempt from the
|
||||
above): the reader (foliate-js + `web/src/reader/`), Chart.js graphs in
|
||||
stats, toasts, password validation. Anything else that renders app UI
|
||||
in TypeScript is a defect; if a templ twin of it exists, one of the two
|
||||
must be deleted.
|
||||
|
||||
**Documented exception — dashboard sections**: `renderDashboardCollections`
|
||||
/ `renderSectionHTML` in `web/src/dashboard.ts` re-render dashboard
|
||||
sections from `/api/dashboard/sections` JSON (library switch, settings
|
||||
save, refresh button) to preserve the switch fade/transition on a page
|
||||
with heavy interactivity. First-paint section markup is still templ
|
||||
(Dashboard handler in `internal/router/frontend.go`). This exception is
|
||||
deliberate; do not clone it to new pages, and do not "fix" the dashboard
|
||||
without restoring a templ-partial fade path first.
|
||||
|
||||
## Library visibility (family-lens rule) — ALWAYS apply
|
||||
|
||||
Per-user library visibility (`library_visibility` table, default-ALLOW:
|
||||
@@ -34,6 +71,16 @@ were name-only across ALL libraries. Fixed; do not regress.
|
||||
|
||||
## Tests
|
||||
|
||||
- **THE DOCTRINE — tests are never run against a real/production
|
||||
server.** The dev DB is the test environment, and the invariant every
|
||||
test must uphold is: **leave the testuser experience alone** — its
|
||||
settings, scans, and reading data must be exactly as the suite found
|
||||
them. The helpers exist to enforce this; if a test needs protection
|
||||
the helpers don't provide, EXPAND THE HELPERS rather than hand-rolling
|
||||
(2026-10-21: `setSystemSettingRestored` — snapshot-and-restore for
|
||||
system_settings — exists because a hardcoded ev restore could clobber
|
||||
a deliberate dev bump, and the live dev server obeys that setting on
|
||||
its next poll).
|
||||
- Use `setupTestServer()` from `cmd/server/tests/test_helpers_test.go`
|
||||
(see the Conventions section below for the pool/cleanup rules).
|
||||
- Integration tests run against the SHARED dev database: never write
|
||||
@@ -57,6 +104,41 @@ were name-only across ALL libraries. Fixed; do not regress.
|
||||
cleanups** — sync_integration's `DELETE FROM reading_history WHERE
|
||||
true` destroyed the live reader's stats data on every suite run
|
||||
until scoped to the test-sync users (cascade via user delete).
|
||||
- **RUN ONLY YOUR OWN TESTS while iterating**: integration tests
|
||||
require the containerized dev DB — a bare host `go test
|
||||
./cmd/server/tests` fails on auth on every machine not running the
|
||||
compose stack. Use the filtered container runner
|
||||
(`docker compose -f docker-compose.yml -f
|
||||
docker-compose.dev.yml --profile tests run --rm tests go test
|
||||
./cmd/server/tests -run 'TestYourPrefix' -v`) and save full-suite
|
||||
runs for pre-commit validation; several tests read live shared
|
||||
state (system_settings, real library folders, the periodic backup
|
||||
scan's websocket broadcasts) and flap on data/timing your change
|
||||
never touched. Full recipes: docs/contributing/development.md
|
||||
→ Testing.
|
||||
- **NEVER COMMIT BINARY MEDIA** (mp3/m4b/flac/... fixtures):
|
||||
`internal/services/testdata/` is gitignored by policy; generate
|
||||
media at test runtime with ffmpeg instead (test images ship it).
|
||||
ffmpeg cannot write Audible freeform atoms — those paths are unit-
|
||||
tested with synthetic ffprobeReport values. Recipes:
|
||||
docs/contributing/development.md → Media fixtures.
|
||||
|
||||
## Web (TypeScript) tests
|
||||
|
||||
- `npm run test` (vp test) runs the vitest suite: colocated `web/src/*.test.ts`
|
||||
files, jsdom environment (configured in vite.config.ts `test`). Do NOT switch
|
||||
to happy-dom — DOMPurify's sanitizer output is mangled under it, which makes
|
||||
the markdown security suite test nothing (verified empirically; jsdom is the
|
||||
documented fallback for exactly this reason).
|
||||
- Static gates: `npx tsc --noEmit`, `npx vp check web/src` (format + lint),
|
||||
`npx vp fmt web/src` to apply formatting.
|
||||
- Tests are never bundled (build entries are main.ts/reader.ts only). When
|
||||
touching build config, re-prove it: back-to-back `npm run build:ts` runs must
|
||||
produce byte-identical hashes, and no test-only strings may appear in
|
||||
web/static.
|
||||
- markdown.test.ts pins the notes XSS defenses (FORBID_TAGS, scheme checks,
|
||||
parser-level html escaping). Loosening any assertion there is a security
|
||||
decision, not cleanup.
|
||||
|
||||
## Timezones
|
||||
|
||||
@@ -96,22 +178,29 @@ of a server bug: stop and flag the user.
|
||||
values** — it emits the literal text. `x-init="f('{ col.ID }')"`
|
||||
shipped a request to `/api/collections/{ collection.ID }` found by
|
||||
the request logs; the whole attribute value must BE an expression:
|
||||
`x-init={ "f('" + col.ID + "')" }`. Note
|
||||
`class="theme-{ user.Theme }"` has always been literal (harmlessly
|
||||
masked by the JS theme engine).
|
||||
`x-init={ "f('" + col.ID + "')" }`. (Every occurrence of this bug —
|
||||
including `class="theme-{ user.Theme }"` on 22 page bodies and 26
|
||||
attribute interpolations — was swept and fixed in 2026-10; when
|
||||
adding attributes with dynamic values, the whole value is an
|
||||
expression, and free text goes through `templates.JSONArg`.)
|
||||
2. **SettingsRegistry caches**: writes must go through the settings
|
||||
API (Reload on save). Direct SQL writes to system_settings leave
|
||||
the cache stale until the next API-path write — never "preview"
|
||||
rule changes via psql and expect them applied (2026-10-14: a live
|
||||
curl test flip looked ignored until the API path was used).
|
||||
3. **User collections are not dashboard sections**: nothing sets
|
||||
collection show_on_dashboard for user rows, and GetDashboardSections
|
||||
skips empty user sections. Collection detail (JSON + SSR) must NOT
|
||||
route user rows through the dashboard-section path — a 7ce7321
|
||||
regression did exactly that (static detail without library_id
|
||||
returned zero books until 0a7fbd6). Only real system query types
|
||||
(continue-reading etc.) route there; `query_type='filter'` is the
|
||||
schema DEFAULT on every user row, not a query type.
|
||||
3. **User collections are not dashboard query types** (amended 2026-10-19,
|
||||
A2c): user collections ARE dashboard sections by default now —
|
||||
GetUserCollectionsForDashboard no longer gates on show_on_dashboard
|
||||
(which remains system-section provisioning state; hiding is
|
||||
dashboard-prefs `hidden_collections` territory). GetDashboardSections
|
||||
still skips genuinely empty user sections, but hide_finished
|
||||
collections emit a Complete section instead of vanishing. Collection
|
||||
detail (JSON + SSR) must still NOT route user rows through the
|
||||
dashboard-section path — a 7ce7321 regression did exactly that
|
||||
(static detail without library_id returned zero books until 0a7fbd6).
|
||||
Only real system query types (continue-reading etc.) route there;
|
||||
`query_type='filter'` is the schema DEFAULT on every user row, not a
|
||||
query type.
|
||||
4. **End-of-day arithmetic**: anchor the END bound to the START of the
|
||||
day, never to NOW(). `now.AddDate(0,0,1) − 1ns` carries this
|
||||
instant's clock time into tomorrow — a range "ending today"
|
||||
@@ -144,3 +233,25 @@ of a server bug: stop and flag the user.
|
||||
test pools use `max_conns=1`; never create a `pgxpool` per subtest
|
||||
(connection exhaustion); use `t.Cleanup()`-based helpers; table-driven
|
||||
subtests over duplicate functions.
|
||||
7. **Scanner re-extraction is write-after-success** — a failed/empty
|
||||
extraction must NEVER clear scan-derived columns. `updateMediaItem`
|
||||
(generic full-row path) preserves `existing.AudioMetadata` when the
|
||||
new extraction carries no audio payload, and `updateAudioBookItem`
|
||||
refuses empty payloads over populated rows (both guarded 2026-10-21
|
||||
after an ev-gated re-extract NULLed every multi-file audiobook's
|
||||
audio_metadata — books displayed as 1 second long and lost their
|
||||
track manifests). New scan-owned column writes need the same guard.
|
||||
8. **templ double-escapes style attribute values** — the codegen wraps
|
||||
style exprs in a sanitizer AND `templ.EscapeString`, so ANY quote
|
||||
character (even inside `templ.SafeCSS`) reaches the DOM as literal
|
||||
`'` text and the CSS never parses (2026-10-21: every book-detail
|
||||
hero backdrop broken + Firefox fetched `/media/&` → 500 spam).
|
||||
Emit quote-free CSS: unquoted `url(...)` tokens via `SafeCSS` +
|
||||
`cssURL()` in book_detail.templ.
|
||||
9. **`vp test` runs the vite config** — plugins with build hooks fire
|
||||
in the test context too. `cleanStaleChunks` guards on
|
||||
`produced.size === 0` precisely because a bare sweep deleted every
|
||||
hashed chunk in `web/static` when run under vitest (2026-10-21:
|
||||
`npm run test` gutted the build; every page 404'd its imports).
|
||||
Any future outDir-mutating plugin needs the same "am I a build?"
|
||||
guard.
|
||||
+4
-1
@@ -59,7 +59,10 @@ RUN cp -r /go/pkg/mod /app/go-module-cache
|
||||
# This stage is ONLY used for running tests, never deployed to production
|
||||
FROM golang:1.26-alpine AS test-runner
|
||||
|
||||
RUN apk --no-cache add ca-certificates curl poppler-utils
|
||||
# ffmpeg/ffprobe match the runtime stage: the audio ingestion (tag
|
||||
# fallbacks, embedded covers, cue probing) must be exercised in tests
|
||||
# exactly as it runs in production.
|
||||
RUN apk --no-cache add ca-certificates curl poppler-utils ffmpeg
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -47,9 +47,34 @@ help:
|
||||
@echo " ./release v0.3.0 - Tag, push, and release (notes auto-generated from commits)"
|
||||
|
||||
# Run unit tests locally (fast, no containers)
|
||||
test:
|
||||
test: lint-rendering
|
||||
go test ./internal/... -v -short
|
||||
|
||||
# Rendering-ownership guard (AGENTS.md → Rendering ownership): templ renders
|
||||
# UI markup, JS orchestrates. Fails when a web/src file builds more HTML
|
||||
# template literals than its recorded baseline (web/src/rendering-allowlist.txt),
|
||||
# or when a .templ file gains an inline <script> block (theme_flash_guard.templ
|
||||
# is the only sanctioned one). Shrink baselines as fragments move to templ
|
||||
# partials; grow one only with a written reason.
|
||||
lint-rendering:
|
||||
@violations=""; \
|
||||
for f in $$(grep -rlE 'innerHTML[[:space:]]*=[[:space:]]*' web/src --include='*.ts' | grep -v 'web/src/reader/'); do \
|
||||
count=$$(grep -E 'innerHTML[[:space:]]*=[[:space:]]*' $$f | grep -cvE 'innerHTML[[:space:]]*=[[:space:]]*""[[:space:]]*$$'); \
|
||||
allowed=$$(grep -F "$$f:" web/src/rendering-allowlist.txt | cut -d: -f2); \
|
||||
allowed=$${allowed:-0}; \
|
||||
if [ $$count -gt $$allowed ]; then \
|
||||
violations="$$violations $$f ($$count > $$allowed)"; \
|
||||
fi; \
|
||||
done; \
|
||||
for f in $$(grep -rln '<script>' templates --include='*.templ'); do \
|
||||
case $$f in *theme_flash_guard*) continue;; esac; \
|
||||
violations="$$violations $$f (inline script)"; \
|
||||
done; \
|
||||
if [ -n "$$violations" ]; then \
|
||||
echo "rendering-ownership violations:$$violations"; exit 1; \
|
||||
fi; \
|
||||
echo "rendering ownership OK"
|
||||
|
||||
# Run integration tests in containers (matches production environment)
|
||||
test-integration:
|
||||
@echo "Building test containers..."
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
info:
|
||||
name: Get Audio Chapters
|
||||
type: http
|
||||
seq: 30
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: "{{base_url}}/api/media-items/{{media_item_id}}/chapters"
|
||||
headers:
|
||||
- name: ""
|
||||
value: application/json
|
||||
auth: inherit
|
||||
|
||||
settings:
|
||||
encodeUrl: true
|
||||
timeout: 0
|
||||
followRedirects: true
|
||||
maxRedirects: 5
|
||||
|
||||
docs: |-
|
||||
## Get Audio Chapters
|
||||
|
||||
The format-abstraction chapter list for audiobooks
|
||||
(media_items.audio_metadata). Single-file books return `layout: ""` and
|
||||
an empty `tracks` array; folder-of-tracks books return `layout:
|
||||
"folder"` and zip books `layout: "zip"`, each with one `tracks` entry
|
||||
per playable file (global-seconds start/end plus the per-track `url`).
|
||||
|
||||
**Method:** GET
|
||||
|
||||
**Endpoint:** /api/media-items/{id}/chapters
|
||||
@@ -0,0 +1,31 @@
|
||||
info:
|
||||
name: Get Audio Track
|
||||
type: http
|
||||
seq: 31
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: "{{base_url}}/api/media-items/{{media_item_id}}/audio/0"
|
||||
headers:
|
||||
- name: Range
|
||||
value: bytes=0-1023
|
||||
auth: inherit
|
||||
|
||||
settings:
|
||||
encodeUrl: true
|
||||
timeout: 0
|
||||
followRedirects: true
|
||||
maxRedirects: 5
|
||||
|
||||
docs: |-
|
||||
## Get Audio Track
|
||||
|
||||
Streams one track of a multi-file audiobook (folder or zip layout).
|
||||
Range requests are honored: folder tracks are served from the real
|
||||
file; zip tracks are extracted in place (backward seeks re-open the
|
||||
entry). Requires the item's library to be visible to the requester
|
||||
(403 otherwise). Single-file books have no track manifest — 404.
|
||||
|
||||
**Method:** GET
|
||||
|
||||
**Endpoint:** /api/media-items/{id}/audio/{trackIndex}
|
||||
+1
-1
@@ -165,7 +165,7 @@ func main() {
|
||||
dashboardHandler := handlers.NewDashboardHandler(queries)
|
||||
seriesHandler := handlers.NewSeriesHandler(queries)
|
||||
filtersHandler := handlers.NewFiltersHandler(queries)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, libraryService, progressService, annotationService)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, dbPool, libraryService, progressService, annotationService)
|
||||
matchingHandler := handlers.NewMatchingHandler(queries, connManager)
|
||||
jobsHandler := handlers.NewJobsHandler(queries, worker)
|
||||
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// The Library Type dropdown on /admin/library (and the setup wizard, which
|
||||
// shares LibraryTypeOptions) must render from the DB-backed library_types
|
||||
// table. Regression: audiobooks shipped backend-complete (constant, allowed
|
||||
// extensions, MIME map, schema seed row, /api/libraries/types) while the
|
||||
// dropdowns hardcoded the original three types, so an audiobook library
|
||||
// could not be created from the UI.
|
||||
func TestAdminLibraryPageDropdownCoversAllLibraryTypes(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
setup := setupTestServer(t)
|
||||
|
||||
code, html := ssrGet(t, setup.Server.URL+"/admin/library", setup.Token)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
|
||||
for _, typ := range []string{"ebooks", "comics", "manga", "audiobooks"} {
|
||||
assert.Contains(t, html, `value="`+typ+`"`,
|
||||
"library type %q missing from the create-library dropdown", typ)
|
||||
}
|
||||
assert.Contains(t, html, "Audiobooks")
|
||||
|
||||
// Options come from the DB loop, not the deleted hardcode: exactly one
|
||||
// audiobooks option per page.
|
||||
assert.Equal(t, 1, strings.Count(html, `value="audiobooks"`))
|
||||
}
|
||||
@@ -0,0 +1,172 @@
|
||||
package main
|
||||
|
||||
// Playback speed preferences (player plan §8): a per-user PER-BOOK
|
||||
// speed plus a user-level default, resolved at the chapters endpoint
|
||||
// (per-book row → users.default_audio_speed → 1.0). All speed writes
|
||||
// ride the fresh per-test regular user — the shared admin account is
|
||||
// never touched (AGENTS doctrine). Fixture books live in test-named
|
||||
// libraries that cascade away with the library delete.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// writeTestTrackTone renders a short sine mp3 at a GIVEN frequency so
|
||||
// multiple fixture books never collide on the scanner's content dedup.
|
||||
// Falls back to dummy bytes (ingest is ffprobe-optional) like
|
||||
// writeTestTrack.
|
||||
func writeTestTrackTone(t *testing.T, path string, hz int) {
|
||||
t.Helper()
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
|
||||
if ffmpeg, err := exec.LookPath("ffmpeg"); err == nil {
|
||||
cmd := exec.Command(ffmpeg, "-y", "-v", "error", "-f", "lavfi",
|
||||
"-i", fmt.Sprintf("sine=frequency=%d:duration=0.3", hz), "-codec:a", "libmp3lame", "-b:a", "32k", path)
|
||||
if out, err := cmd.CombinedOutput(); err == nil {
|
||||
return
|
||||
} else {
|
||||
t.Logf("ffmpeg track render failed (%s), falling back to dummy bytes: %s", err, out)
|
||||
}
|
||||
}
|
||||
require.NoError(t, os.WriteFile(path, []byte(fmt.Sprintf("ID3 dummy audio %d", hz)), 0o644))
|
||||
}
|
||||
|
||||
func TestAudioPlaybackSpeed(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Speed %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrack(t, filepath.Join(folder, "Speed Book", "01 one.mp3"))
|
||||
// 552 Hz (not the shared helper's 440) so the scanner's sha-256
|
||||
// dedup doesn't skip the second book as a byte-duplicate.
|
||||
writeTestTrackTone(t, filepath.Join(folder, "Speed Book Two", "01 one.mp3"), 552)
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
// Grouping depends on track count (single-track folders ingest as
|
||||
// single-file items whose path IS the file; multi-track folders get
|
||||
// the folder path) — find by the library-relative path CONTAINING
|
||||
// the book folder so the test holds either way.
|
||||
var bookPath, book2Path string
|
||||
for _, item := range listLibraryAudioItems(t, setup, libID) {
|
||||
p := filepath.ToSlash(item.FilePath)
|
||||
t.Logf("ingested item path: %s", p)
|
||||
if strings.Contains(p, "Speed Book Two") && book2Path == "" {
|
||||
book2Path = p
|
||||
} else if strings.Contains(p, "Speed Book/") && bookPath == "" {
|
||||
bookPath = p
|
||||
}
|
||||
}
|
||||
require.NotEmpty(t, bookPath, "Speed Book item must exist")
|
||||
require.NotEmpty(t, book2Path, "Speed Book Two item must exist")
|
||||
book, found := findAudioItemByPath(t, setup, libID, bookPath)
|
||||
require.True(t, found)
|
||||
bookID := uuid.UUID(book.ID.Bytes)
|
||||
book2, found := findAudioItemByPath(t, setup, libID, book2Path)
|
||||
require.True(t, found)
|
||||
book2ID := uuid.UUID(book2.ID.Bytes)
|
||||
|
||||
client := &http.Client{}
|
||||
token := setup.RegularToken
|
||||
authed := func(method, url, body string) (*http.Response, func()) {
|
||||
var req *http.Request
|
||||
if body == "" {
|
||||
req, _ = http.NewRequest(method, url, nil)
|
||||
} else {
|
||||
req, _ = http.NewRequest(method, url, strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
return resp, func() { _ = resp.Body.Close() }
|
||||
}
|
||||
getSpeeds := func(itemID uuid.UUID) (playback, def float64) {
|
||||
resp, closer := authed("GET", setup.Server.URL+"/api/media-items/"+itemID.String()+"/chapters", "")
|
||||
defer closer()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var data struct {
|
||||
PlaybackSpeed float64 `json:"playback_speed"`
|
||||
DefaultPlaybackSpeed float64 `json:"default_playback_speed"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&data))
|
||||
return data.PlaybackSpeed, data.DefaultPlaybackSpeed
|
||||
}
|
||||
|
||||
t.Run("unset resolves to 1x", func(t *testing.T) {
|
||||
playback, def := getSpeeds(bookID)
|
||||
require.Equal(t, 1.0, playback)
|
||||
require.Equal(t, 1.0, def)
|
||||
})
|
||||
|
||||
t.Run("put per-book speed upserts and resolves", func(t *testing.T) {
|
||||
resp, closer := authed("PUT", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio-speed", `{"speed":1.5}`)
|
||||
defer closer()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var saved struct {
|
||||
PlaybackSpeed float64 `json:"playback_speed"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&saved))
|
||||
require.Equal(t, 1.5, saved.PlaybackSpeed)
|
||||
|
||||
playback, def := getSpeeds(bookID)
|
||||
require.Equal(t, 1.5, playback)
|
||||
require.Equal(t, 1.0, def) // user default untouched
|
||||
})
|
||||
|
||||
t.Run("out of range speed is rejected", func(t *testing.T) {
|
||||
for _, body := range []string{`{"speed":99}`, `{"speed":0}`, `{"speed":0.1}`} {
|
||||
resp, closer := authed("PUT", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio-speed", body)
|
||||
require.Equal(t, http.StatusBadRequest, resp.StatusCode)
|
||||
closer()
|
||||
}
|
||||
// The stored speed is untouched by the rejects.
|
||||
playback, _ := getSpeeds(bookID)
|
||||
require.Equal(t, 1.5, playback)
|
||||
})
|
||||
|
||||
t.Run("profile default applies to books without their own speed", func(t *testing.T) {
|
||||
resp, closer := authed("PUT", setup.Server.URL+"/api/auth/profile", `{"default_audio_speed":1.25}`)
|
||||
defer closer()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
|
||||
// Per-book row wins over the default on book one...
|
||||
playback, def := getSpeeds(bookID)
|
||||
require.Equal(t, 1.5, playback)
|
||||
require.Equal(t, 1.25, def)
|
||||
// ...book two has no row and picks the default up.
|
||||
playback2, _ := getSpeeds(book2ID)
|
||||
require.Equal(t, 1.25, playback2)
|
||||
|
||||
// GET /profile carries it for the profile form.
|
||||
resp2, closer2 := authed("GET", setup.Server.URL+"/api/auth/profile", "")
|
||||
defer closer2()
|
||||
var profile struct {
|
||||
DefaultAudioSpeed float64 `json:"default_audio_speed"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp2.Body).Decode(&profile))
|
||||
require.Equal(t, 1.25, profile.DefaultAudioSpeed)
|
||||
})
|
||||
|
||||
t.Run("updating the default re-satisfies unset books", func(t *testing.T) {
|
||||
resp, closer := authed("PUT", setup.Server.URL+"/api/auth/profile", `{"default_audio_speed":0.75}`)
|
||||
defer closer()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
playback2, _ := getSpeeds(book2ID)
|
||||
require.Equal(t, 0.75, playback2)
|
||||
playback1, _ := getSpeeds(bookID)
|
||||
require.Equal(t, 1.5, playback1) // per-book pin survives
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,799 @@
|
||||
package main
|
||||
|
||||
// Multi-file audiobook integration tests: folder-of-tracks and zip-of-
|
||||
// audio books must ingest as ONE item each (never per-track items), the
|
||||
// /chapters manifest must expose the track list, the per-track streaming
|
||||
// endpoint must serve bytes behind the library-visibility gate, and
|
||||
// rescans must be idempotent while absorbing pre-grouping one-track
|
||||
// strays. Live dev DB — all fixtures use test-named libraries under
|
||||
// t.TempDir() folders and cascade away with the library delete.
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// writeTestTrack renders a short silent mp3 via ffmpeg when available so
|
||||
// ffprobe can produce real durations; otherwise falls back to dummy bytes
|
||||
// (ingest is ffprobe-optional by design — structure assertions don't need
|
||||
// real audio).
|
||||
func writeTestTrack(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
|
||||
if ffmpeg, err := exec.LookPath("ffmpeg"); err == nil {
|
||||
cmd := exec.Command(ffmpeg, "-y", "-v", "error", "-f", "lavfi",
|
||||
"-i", "sine=frequency=440:duration=0.3", "-codec:a", "libmp3lame", "-b:a", "32k", path)
|
||||
if out, err := cmd.CombinedOutput(); err == nil {
|
||||
return
|
||||
} else {
|
||||
t.Logf("ffmpeg track render failed (%s), falling back to dummy bytes: %s", err, out)
|
||||
}
|
||||
}
|
||||
require.NoError(t, os.WriteFile(path, []byte("ID3 dummy audio"), 0o644))
|
||||
}
|
||||
|
||||
// writeTestTrackM4B is writeTestTrack for the m4b (AAC/ipod muxer)
|
||||
// container — the audiobook format whose rescan route is under guard.
|
||||
// ffmpeg needs -f ipod for the .m4b extension; falls back to dummy bytes.
|
||||
func writeTestTrackM4B(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(path), 0o755))
|
||||
if ffmpeg, err := exec.LookPath("ffmpeg"); err == nil {
|
||||
cmd := exec.Command(ffmpeg, "-y", "-v", "error", "-f", "lavfi",
|
||||
"-i", "sine=frequency=440:duration=0.3", "-codec:a", "aac", "-b:a", "32k", "-f", "ipod", path)
|
||||
if out, err := cmd.CombinedOutput(); err == nil {
|
||||
return
|
||||
} else {
|
||||
t.Logf("ffmpeg m4b render failed (%s), falling back to dummy bytes: %s", err, out)
|
||||
}
|
||||
}
|
||||
require.NoError(t, os.WriteFile(path, []byte("dummy m4b audio"), 0o644))
|
||||
}
|
||||
|
||||
// audioTestLibrary builds an audiobooks-type library with a fresh temp
|
||||
// folder, returning its UUID and the folder path.
|
||||
func audioTestLibrary(t *testing.T, setup *TestServerSetup, name string) (pgtype.UUID, string) {
|
||||
t.Helper()
|
||||
libID, folder := createTypedTestLibrary(t, setup, name, "audiobooks")
|
||||
return libID, folder
|
||||
}
|
||||
|
||||
// listLibraryAudioItems returns every media item row of the library.
|
||||
func listLibraryAudioItems(t *testing.T, setup *TestServerSetup, libID pgtype.UUID) []database.ListMediaItemsByLibraryIncludingArchivedRow {
|
||||
t.Helper()
|
||||
items, err := setup.DB.ListMediaItemsByLibraryIncludingArchived(context.Background(), libID)
|
||||
require.NoError(t, err)
|
||||
return items
|
||||
}
|
||||
|
||||
// findAudioItemByPath returns the item with the given file_path.
|
||||
func findAudioItemByPath(t *testing.T, setup *TestServerSetup, libID pgtype.UUID, filePath string) (database.ListMediaItemsByLibraryIncludingArchivedRow, bool) {
|
||||
t.Helper()
|
||||
for _, item := range listLibraryAudioItems(t, setup, libID) {
|
||||
if item.FilePath == filePath {
|
||||
return item, true
|
||||
}
|
||||
}
|
||||
return database.ListMediaItemsByLibraryIncludingArchivedRow{}, false
|
||||
}
|
||||
|
||||
// decodeAudioPayload parses the audio_metadata JSONB of an item.
|
||||
func decodeAudioPayload(t *testing.T, item database.ListMediaItemsByLibraryIncludingArchivedRow) (layout string, trackCount, chapterCount int, duration float64) {
|
||||
t.Helper()
|
||||
require.NotNil(t, item.AudioMetadata, "audio_metadata must be present")
|
||||
var payload struct {
|
||||
Layout string `json:"layout"`
|
||||
DurationSeconds float64 `json:"duration_seconds"`
|
||||
Tracks []struct {
|
||||
Index int `json:"index"`
|
||||
Path string `json:"path"`
|
||||
} `json:"tracks"`
|
||||
Chapters []json.RawMessage `json:"chapters"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal(item.AudioMetadata, &payload))
|
||||
return payload.Layout, len(payload.Tracks), len(payload.Chapters), payload.DurationSeconds
|
||||
}
|
||||
|
||||
func TestAudiobookFolderOfTracks(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Folder %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Author/Title/01.mp3, 02.mp3 — the canonical folder book.
|
||||
writeTestTrack(t, filepath.Join(folder, "Test Author", "Test Title", "02 Second.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "Test Author", "Test Title", "01 First.mp3"))
|
||||
// A lone single-file book in a sibling folder must stay single-file.
|
||||
writeTestTrack(t, filepath.Join(folder, "Test Author", "Lone Book", "lonely.mp3"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
// The folder is ONE item whose file_path is the directory.
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Test Title"))
|
||||
require.True(t, found, "folder book item must exist at the directory path")
|
||||
layout, tracks, chapters, duration := decodeAudioPayload(t, book)
|
||||
require.Equal(t, "folder", layout)
|
||||
require.Equal(t, 2, tracks)
|
||||
require.Equal(t, 2, chapters)
|
||||
if duration <= 0 {
|
||||
t.Log("no ffprobe durations available in this environment (degraded ingest)")
|
||||
}
|
||||
|
||||
// The lone mp3 keeps the single-file model.
|
||||
_, loneFound := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Lone Book/lonely.mp3"))
|
||||
require.True(t, loneFound, "lone mp3 must stay its own single-file item")
|
||||
require.Len(t, listLibraryAudioItems(t, setup, libID), 2, "exactly two items: one folder book + one single-file book")
|
||||
|
||||
// Rescan idempotency: nothing duplicates, manifest unchanged.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.Len(t, listLibraryAudioItems(t, setup, libID), 2)
|
||||
again, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Test Author/Test Title"))
|
||||
require.True(t, found)
|
||||
require.JSONEq(t, string(book.AudioMetadata), string(again.AudioMetadata))
|
||||
}
|
||||
|
||||
func TestAudiobookAbsorbsOneTrackStrays(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Merge %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// First scan: a single mp3 → single-file item (the pre-grouping model).
|
||||
writeTestTrack(t, filepath.Join(folder, "Merge Book", "01 only.mp3"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
_, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Merge Book/01 only.mp3"))
|
||||
require.True(t, found, "single track ingests as its own item first")
|
||||
|
||||
// Second track arrives; rescan must create the folder book and delete
|
||||
// the stray one-track item.
|
||||
writeTestTrack(t, filepath.Join(folder, "Merge Book", "02 and.mp3"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Merge Book"))
|
||||
require.True(t, found, "folder book must exist after rescan")
|
||||
layout, tracks, _, _ := decodeAudioPayload(t, book)
|
||||
require.Equal(t, "folder", layout)
|
||||
require.Equal(t, 2, tracks)
|
||||
|
||||
items := listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 1, "the one-track stray must be absorbed into the folder book")
|
||||
}
|
||||
|
||||
func TestAudiobookZipBook(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Zip %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Build a zip of mp3s via ffmpeg-rendered tracks.
|
||||
dir := t.TempDir()
|
||||
writeTestTrack(t, filepath.Join(dir, "a.mp3"))
|
||||
writeTestTrack(t, filepath.Join(dir, "b.mp3"))
|
||||
zipPath := filepath.Join(folder, "Zipped Book.zip")
|
||||
zipFiles(t, zipPath, map[string]string{
|
||||
"01 first.mp3": filepath.Join(dir, "a.mp3"),
|
||||
"02 second.mp3": filepath.Join(dir, "b.mp3"),
|
||||
})
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Zipped Book.zip"))
|
||||
require.True(t, found, "zip book item must exist at the zip path")
|
||||
layout, tracks, chapters, _ := decodeAudioPayload(t, book)
|
||||
require.Equal(t, "zip", layout)
|
||||
require.Equal(t, 2, tracks)
|
||||
require.Equal(t, 2, chapters)
|
||||
|
||||
// Rescan idempotency.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.Len(t, listLibraryAudioItems(t, setup, libID), 1)
|
||||
}
|
||||
|
||||
func TestAudiobookTrackEndpoints(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
regularID := getRegularUserID(t, setup.DB)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Serve %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrack(t, filepath.Join(folder, "Serve Book", "01 one.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "Serve Book", "02 two.mp3"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Serve Book"))
|
||||
require.True(t, found)
|
||||
bookID := uuid.UUID(book.ID.Bytes)
|
||||
|
||||
client := &http.Client{}
|
||||
|
||||
t.Run("chapters manifest exposes tracks", func(t *testing.T) {
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/chapters", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
|
||||
var data struct {
|
||||
Layout string `json:"layout"`
|
||||
Tracks []struct {
|
||||
Index int `json:"index"`
|
||||
URL string `json:"url"`
|
||||
Path string `json:"path"`
|
||||
Start float64 `json:"start"`
|
||||
} `json:"tracks"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&data))
|
||||
require.Equal(t, "folder", data.Layout)
|
||||
require.Len(t, data.Tracks, 2)
|
||||
require.Equal(t, "/api/media-items/"+bookID.String()+"/audio/0", data.Tracks[0].URL)
|
||||
})
|
||||
|
||||
t.Run("track streams with range support", func(t *testing.T) {
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio/0", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
req.Header.Set("Range", "bytes=0-99")
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusPartialContent, resp.StatusCode)
|
||||
})
|
||||
|
||||
t.Run("out-of-range track index is 404", func(t *testing.T) {
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/audio/9", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusNotFound, resp.StatusCode)
|
||||
})
|
||||
|
||||
t.Run("hidden library is 403 on chapters and tracks", func(t *testing.T) {
|
||||
_, err := setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
|
||||
LibraryID: libID,
|
||||
IsVisible: false,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
for _, path := range []string{
|
||||
"/api/media-items/" + bookID.String() + "/chapters",
|
||||
"/api/media-items/" + bookID.String() + "/audio/0",
|
||||
} {
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+path, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
_ = resp.Body.Close()
|
||||
require.Equal(t, http.StatusForbidden, resp.StatusCode, "%s must be gated", path)
|
||||
}
|
||||
|
||||
// Restore visibility so harness cleanup ordering never sees a
|
||||
// stale hidden row for a deleted library's user.
|
||||
_, err = setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
|
||||
LibraryID: libID,
|
||||
IsVisible: true,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
})
|
||||
}
|
||||
|
||||
// zipFiles builds a zip archive at zipPath mapping entry names to source
|
||||
// file paths on disk.
|
||||
func zipFiles(t *testing.T, zipPath string, entries map[string]string) {
|
||||
t.Helper()
|
||||
out, err := os.Create(zipPath)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = out.Close() }()
|
||||
w := zip.NewWriter(out)
|
||||
for name, src := range entries {
|
||||
data, err := os.ReadFile(src)
|
||||
require.NoError(t, err)
|
||||
f, err := w.Create(name)
|
||||
require.NoError(t, err)
|
||||
_, err = f.Write(data)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
require.NoError(t, w.Close())
|
||||
}
|
||||
|
||||
// fixtureAudio copies a committed binary fixture (real tagged media —
|
||||
// the test container has no way to generate it) into a library folder.
|
||||
// Path is relative from cmd/server/tests (three deep) to the services
|
||||
// testdata tree.
|
||||
func fixtureAudio(t *testing.T, name, dest string) {
|
||||
t.Helper()
|
||||
src := filepath.Join("..", "..", "..", "internal", "services", "testdata", "audio", name)
|
||||
data, err := os.ReadFile(src)
|
||||
require.NoError(t, err, "fixture %s must exist in the repo", name)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(dest), 0o755))
|
||||
require.NoError(t, os.WriteFile(dest, data, 0o644))
|
||||
}
|
||||
|
||||
// getJSONWithAuth fetches an API endpoint with the admin token and
|
||||
// decodes the body into dst.
|
||||
func getJSONWithAuth(t *testing.T, setup *TestServerSetup, path string, dst interface{}) int {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+path, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if dst != nil {
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(dst))
|
||||
}
|
||||
return resp.StatusCode
|
||||
}
|
||||
|
||||
func TestAudiobookNewContainerFormats(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Formats %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
for _, name := range []string{"book.opus", "book.ogg", "book.oga", "book.aac", "book.wav"} {
|
||||
// one book per folder — a directory with ≥2 audio files is ONE
|
||||
// book by the grouping doctrine
|
||||
fixtureAudio(t, name, filepath.Join(folder, "Formats "+name, name))
|
||||
}
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
items := listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 5, "each new container ingests as its own single-file book")
|
||||
mimes := map[string]string{}
|
||||
for _, item := range items {
|
||||
require.Equal(t, "audiobook", item.FormatGroup, "%s must classify as audiobook", item.FilePath)
|
||||
mimes[filepath.Ext(item.FilePath)] = item.MimeType.String
|
||||
}
|
||||
require.Equal(t, map[string]string{
|
||||
".opus": "audio/ogg", ".ogg": "audio/ogg", ".oga": "audio/ogg",
|
||||
".aac": "audio/aac", ".wav": "audio/wav",
|
||||
}, mimes)
|
||||
|
||||
// The download endpoint must serve the REAL content types — a wrong
|
||||
// audio/mpeg fallback makes browsers refuse the stream.
|
||||
for _, name := range []string{"book.opus", "book.aac", "book.wav"} {
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Formats "+name+"/"+name))
|
||||
require.True(t, found, name)
|
||||
bookID := uuid.UUID(item.ID.Bytes)
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/"+bookID.String()+"/download", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
// Drain but discard: a 0.3s sine is a few KB — nothing to save.
|
||||
_, _ = io.Copy(io.Discard, resp.Body)
|
||||
_ = resp.Body.Close()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
require.Equal(t, item.MimeType.String, resp.Header.Get("Content-Type"), "%s content type", name)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAudiobookTagFallbacks(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Tags %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// No sidecars anywhere: the file's own tags are the only source.
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Tag Author", "The Tagged Album", "tagged.mp3"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Tag Author/The Tagged Album/tagged.mp3"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "The Tagged Album", item.Title, "album tag names the book")
|
||||
require.Equal(t, "Tag Author", item.Author.String, "album_artist tag names the author")
|
||||
require.Len(t, item.Narrators, 1, "composer tag fills narrators (Audible convention)")
|
||||
require.Equal(t, "Tag Narrator", item.Narrators[0])
|
||||
require.Equal(t, "Tag Series", item.Series.String, "grouping tag fills series")
|
||||
require.Equal(t, "Audiobook", item.Genre.String, "genre tag fills genre")
|
||||
require.False(t, item.DatePublished.Time.IsZero(), "date tag fills publish date")
|
||||
require.Equal(t, 2020, item.DatePublished.Time.Year())
|
||||
// ASIN/abridged/subtitle arrive only through Audible freeform atoms,
|
||||
// which ffmpeg cannot write into a fixture — those fallbacks are
|
||||
// pinned by TestApplyAudioTagFallbacks (synthetic reports).
|
||||
}
|
||||
|
||||
func TestAudiobookCueSheetChapters(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Cue %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
fixtureAudio(t, "cuebook.flac", filepath.Join(folder, "Cue Book", "cuebook.flac"))
|
||||
fixtureAudio(t, "cuebook.cue", filepath.Join(folder, "Cue Book", "cuebook.cue"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Cue Book/cuebook.flac"))
|
||||
require.True(t, found)
|
||||
bookID := uuid.UUID(item.ID.Bytes)
|
||||
|
||||
var data struct {
|
||||
Chapters []struct {
|
||||
Index int `json:"index"`
|
||||
Title string `json:"title"`
|
||||
Start float64 `json:"start"`
|
||||
End float64 `json:"end"`
|
||||
} `json:"chapters"`
|
||||
}
|
||||
status := getJSONWithAuth(t, setup, "/api/media-items/"+bookID.String()+"/chapters", &data)
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
require.Len(t, data.Chapters, 3, "cue tracks become chapters")
|
||||
require.Equal(t, "Opening", data.Chapters[0].Title)
|
||||
require.Equal(t, "The Middle Bit", data.Chapters[1].Title)
|
||||
require.InDelta(t, 20.0, data.Chapters[1].Start, 0.01)
|
||||
require.Equal(t, "Closing", data.Chapters[2].Title)
|
||||
}
|
||||
|
||||
func TestAudiobookEmbeddedM4BChapters(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio M4B %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
fixtureAudio(t, "chaptered.m4b", filepath.Join(folder, "Chaptered Book.m4b"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Chaptered Book.m4b"))
|
||||
require.True(t, found)
|
||||
bookID := uuid.UUID(item.ID.Bytes)
|
||||
|
||||
var data struct {
|
||||
Chapters []struct {
|
||||
Title string `json:"title"`
|
||||
} `json:"chapters"`
|
||||
DurationSeconds float64 `json:"duration_seconds"`
|
||||
}
|
||||
status := getJSONWithAuth(t, setup, "/api/media-items/"+bookID.String()+"/chapters", &data)
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
require.Len(t, data.Chapters, 2, "embedded m4b chapters parse")
|
||||
require.Equal(t, "Part One", data.Chapters[0].Title)
|
||||
require.Equal(t, "Part Two", data.Chapters[1].Title)
|
||||
require.Greater(t, data.DurationSeconds, float64(0))
|
||||
}
|
||||
|
||||
func TestAudiobookPlaylistOrdering(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio M3U %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Three tag-less tracks (natural order 01,02,10) plus a playlist
|
||||
// imposing a DIFFERENT order and names.
|
||||
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "01_one.mp3"))
|
||||
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "02_two.mp3"))
|
||||
fixtureAudio(t, "plain.mp3", filepath.Join(folder, "Playlist Book", "10_ten.mp3"))
|
||||
playlist := "#EXTM3U\n#EXTINF:60,Epilogue\n./10_ten.mp3\n#EXTINF:60,Prologue\n01_one.mp3\n"
|
||||
require.NoError(t, os.WriteFile(filepath.Join(folder, "Playlist Book", "book.m3u"), []byte(playlist), 0o644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Playlist Book"))
|
||||
require.True(t, found, "folder book created")
|
||||
require.NotNil(t, item.AudioMetadata)
|
||||
var payload struct {
|
||||
Tracks []struct {
|
||||
Index int `json:"index"`
|
||||
Path string `json:"path"`
|
||||
Title string `json:"title"`
|
||||
} `json:"tracks"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal(item.AudioMetadata, &payload))
|
||||
require.Len(t, payload.Tracks, 3)
|
||||
require.Equal(t, "Playlist Book/10_ten.mp3", payload.Tracks[0].Path, "playlist order beats natural sort")
|
||||
require.Equal(t, "Epilogue", payload.Tracks[0].Title)
|
||||
require.Equal(t, "Playlist Book/01_one.mp3", payload.Tracks[1].Path)
|
||||
require.Equal(t, "Prologue", payload.Tracks[1].Title)
|
||||
require.Equal(t, "Playlist Book/02_two.mp3", payload.Tracks[2].Path, "unlisted track appended in natural order")
|
||||
}
|
||||
|
||||
func TestAudiobookEmbeddedCoverArt(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Cover %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Single file: embedded APIC → "<file>.cover.jpg" sibling.
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Cover Book", "tagged.mp3"))
|
||||
// Folder book: sidecar cover.jpg must WIN over the embedded art.
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "01 track.mp3"))
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "02 track.mp3"))
|
||||
// A real (if tiny) sidecar cover next to the folder book:
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(folder, "Sidecar Book", "cover.jpg"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
single, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Cover Book/tagged.mp3"))
|
||||
require.True(t, found)
|
||||
if !ffmpegAvailableForTest() {
|
||||
t.Skip("ffmpeg unavailable in this environment; embedded covers degrade gracefully")
|
||||
}
|
||||
require.NotEmpty(t, single.CoverImagePath.String, "embedded APIC becomes the cover")
|
||||
coverRel := single.CoverImagePath.String
|
||||
_, err := os.Stat(filepath.Join(folder, coverRel))
|
||||
require.NoError(t, err, "cover sibling file exists on disk")
|
||||
|
||||
folder_, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Sidecar Book"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, filepath.ToSlash("Sidecar Book/cover.jpg"), folder_.CoverImagePath.String,
|
||||
"sidecar cover wins over embedded art")
|
||||
}
|
||||
|
||||
func ffmpegAvailableForTest() bool {
|
||||
_, err := os.Stat("/usr/bin/ffmpeg")
|
||||
if err == nil {
|
||||
return true
|
||||
}
|
||||
_, err = exec.LookPath("ffmpeg")
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func TestRescanKeepsFilenameTitle(t *testing.T) {
|
||||
// Regression pin: extractMetadata's default branch (.mobi/.azw/.pdb
|
||||
// and unknown types) no longer pre-fills the filename as title, so
|
||||
// the rescan path MUST apply the final filename fallback itself —
|
||||
// a sidecar-less, tag-less book at a library root keeps its title
|
||||
// instead of being blanked by a rescan.
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Rescan Title %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Library-root .mobi with no sidecars, no tags, no folder structure.
|
||||
require.NoError(t, os.WriteFile(filepath.Join(folder, "Plain Book.mobi"), []byte("dummy mobi bytes"), 0o644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
item, found := findAudioItemByPath(t, setup, libID, "Plain Book.mobi")
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Plain Book", item.Title, "create path falls back to filename")
|
||||
|
||||
bookID := uuid.UUID(item.ID.Bytes)
|
||||
req, _ := http.NewRequest("POST", setup.Server.URL+"/api/media-items/"+bookID.String()+"/rescan", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
_ = resp.Body.Close()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
|
||||
rescanned, err := setup.DB.GetMediaItem(context.Background(), item.ID)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, "Plain Book", rescanned.Title, "rescan must not blank the filename-derived title")
|
||||
}
|
||||
|
||||
// TestAudiobookFolderSidecarEditDetected is the Phase 6 headline: a
|
||||
// size-stable folder book is changed by EDITING its metadata.json —
|
||||
// the track-count/size fingerprint never saw it, not even on force
|
||||
// rescan. Pins the ladder end to end: seeding at create, pure-skip
|
||||
// steady state, same-size sidecar edit consumed (title + collection
|
||||
// sync), and the refreshed manifest.
|
||||
func TestAudiobookFolderSidecarEditDetected(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Sidecar %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
bookDir := filepath.Join(folder, "Sidecar Author", "Sidecar Dir Name")
|
||||
writeTestTrack(t, filepath.Join(bookDir, "01 first.mp3"))
|
||||
writeTestTrack(t, filepath.Join(bookDir, "02 second.mp3"))
|
||||
sidecar := filepath.Join(bookDir, "metadata.json")
|
||||
// Sidecar title deliberately differs from the directory name so the
|
||||
// assertion proves the SIDECAR won.
|
||||
require.NoError(t, os.WriteFile(sidecar, []byte(`{"title":"Sidecar Real Title"}`), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Sidecar Author/Sidecar Dir Name"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Sidecar Real Title", book.Title, "sidecar title wins at create")
|
||||
require.NotEqual(t, book.UpdatedAt.Time, time.Time{}, "sanity")
|
||||
|
||||
// Manifest seeded at create (the multifile create tail).
|
||||
state, err := setup.DB.GetMediaScanState(context.Background(), book.ID)
|
||||
require.NoError(t, err)
|
||||
require.NotEmpty(t, state.Manifest, "folder create must seed the sidecar manifest")
|
||||
seededManifest := state.Manifest
|
||||
|
||||
// Steady state: the manifest ladder runs and skips — pure skip.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
afterSkip := findItemUpdated(t, setup, libID, book.FilePath)
|
||||
require.Equal(t, book.UpdatedAt.Time, afterSkip, "unchanged rescan with seeded manifest is a pure skip")
|
||||
|
||||
// Same-size sidecar edit — the size-only blind spot.
|
||||
sameSize := []byte(`{"title":"Sidecar Real Titlz"}`)
|
||||
require.Len(t, sameSize, len(`{"title":"Sidecar Real Title"}`), "fixture must be a same-size edit")
|
||||
require.NoError(t, os.WriteFile(sidecar, sameSize, 0o644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
edited := findAudioItemByPathRow(t, setup, libID, book.FilePath)
|
||||
require.Equal(t, "Sidecar Real Titlz", edited.Title, "same-size sidecar edit is detected and consumed")
|
||||
|
||||
state, err = setup.DB.GetMediaScanState(context.Background(), edited.ID)
|
||||
require.NoError(t, err)
|
||||
require.NotEmpty(t, state.Manifest)
|
||||
require.NotEqual(t, string(seededManifest), string(state.Manifest), "change path stores a fresh full manifest")
|
||||
|
||||
// Playlist sync on the change path (plan Correction 2): a reading-
|
||||
// order entry added to an EXISTING book creates its collection, and
|
||||
// a further rescan does not duplicate it.
|
||||
require.NoError(t, os.WriteFile(sidecar, []byte(`{"title":"Sidecar Real Titlz","series":["Sidecar Saga (Reading Order) #1"]}`), 0o644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
var collCount int
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collections WHERE name = 'Sidecar Saga (Reading Order)' AND user_id = $1`, admin,
|
||||
).Scan(&collCount))
|
||||
require.Equal(t, 1, collCount, "reading-order sidecar edit creates the collection for an existing book")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collections WHERE name = 'Sidecar Saga (Reading Order)' AND user_id = $1`, admin,
|
||||
).Scan(&collCount))
|
||||
require.Equal(t, 1, collCount, "rescan is idempotent — no duplicate collection")
|
||||
}
|
||||
|
||||
// TestAudiobookTrackOnlyChangeKeepsEditorial pins the other half of the
|
||||
// contract: a TRACK change (sidecars untouched) keeps the existing
|
||||
// scan-derived-only update — editorial columns stay as the sidecar
|
||||
// wrote them, and no editorial apply fires.
|
||||
func TestAudiobookTrackOnlyChangeKeepsEditorial(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio TrackOnly %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
bookDir := filepath.Join(folder, "Track Only Author", "Track Only Book")
|
||||
writeTestTrack(t, filepath.Join(bookDir, "01 first.mp3"))
|
||||
writeTestTrack(t, filepath.Join(bookDir, "02 second.mp3"))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "metadata.json"), []byte(`{"title":"Editorial Anchor"}`), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Track Only Author/Track Only Book"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Editorial Anchor", book.Title)
|
||||
|
||||
// A third track arrives; sidecars untouched → track-count change →
|
||||
// payload update only.
|
||||
writeTestTrack(t, filepath.Join(bookDir, "03 third.mp3"))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
updated := findAudioItemByPathRow(t, setup, libID, book.FilePath)
|
||||
layout, tracks, _, _ := decodeAudioPayload(t, updated)
|
||||
require.Equal(t, "folder", layout)
|
||||
require.Equal(t, 3, tracks, "track change takes the payload update")
|
||||
require.Equal(t, "Editorial Anchor", updated.Title, "track-only change must not touch editorial columns")
|
||||
}
|
||||
|
||||
// TestAudiobookZipSidecarEditAndRestore covers the zip twin: sibling
|
||||
// metadata.json edits are detected, and the fast path lifts archive
|
||||
// state like the folder path always did (Phase 6 parity fix).
|
||||
func TestAudiobookZipSidecarEditAndRestore(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio ZipSidecar %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
dir := t.TempDir()
|
||||
writeTestTrack(t, filepath.Join(dir, "a.mp3"))
|
||||
writeTestTrack(t, filepath.Join(dir, "b.mp3"))
|
||||
zipPath := filepath.Join(folder, "Zip Sidecar Book.zip")
|
||||
zipFiles(t, zipPath, map[string]string{
|
||||
"01 first.mp3": filepath.Join(dir, "a.mp3"),
|
||||
"02 second.mp3": filepath.Join(dir, "b.mp3"),
|
||||
})
|
||||
sidecar := filepath.Join(folder, "metadata.json")
|
||||
require.NoError(t, os.WriteFile(sidecar, []byte(`{"title":"Zip Sidecar Title"}`), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("Zip Sidecar Book.zip"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Zip Sidecar Title", book.Title, "sibling sidecar feeds the zip book at create")
|
||||
|
||||
state, err := setup.DB.GetMediaScanState(context.Background(), book.ID)
|
||||
require.NoError(t, err)
|
||||
require.NotEmpty(t, state.Manifest, "zip create must seed the sidecar manifest")
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.Equal(t, book.UpdatedAt.Time, findItemUpdated(t, setup, libID, book.FilePath), "unchanged zip rescan is a pure skip")
|
||||
|
||||
require.NoError(t, os.WriteFile(sidecar, []byte(`{"title":"Zip Sidecar Titlz"}`), 0o644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
edited := findAudioItemByPathRow(t, setup, libID, book.FilePath)
|
||||
require.Equal(t, "Zip Sidecar Titlz", edited.Title, "zip sidecar edit is detected and consumed")
|
||||
|
||||
// Archive the row the way the missing-scan does; the file is back,
|
||||
// so the next scan must lift the archive (parity with folders).
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`UPDATE media_items SET archived_at = NOW() WHERE id = $1 RETURNING id`, edited.ID,
|
||||
).Scan(new(pgtype.UUID)))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
restored := findAudioItemByPathRow(t, setup, libID, book.FilePath)
|
||||
require.False(t, restored.ArchivedAt.Valid, "zip fast path must lift archive state when the file is back")
|
||||
require.Equal(t, "Zip Sidecar Titlz", restored.Title)
|
||||
}
|
||||
|
||||
// TestAudiobookSidecarRemovalFallsBackToFilename is the wipe-guard's
|
||||
// testable sibling (faceted plan Phase 6 audit find): a single-file
|
||||
// book whose sidecar is REMOVED must re-derive metadata through the
|
||||
// create-equivalent chain (tags → filename), never blank the title.
|
||||
// The pre-fix change path passed a non-nil empty metadata, which
|
||||
// skipped the filename guard and the tag re-derivation.
|
||||
func TestAudiobookSidecarRemovalFallsBackToFilename(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Removal %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
trackPath := filepath.Join(folder, "Fallback Book.mp3")
|
||||
writeTestTrack(t, trackPath)
|
||||
sidecar := filepath.Join(folder, "metadata.json")
|
||||
require.NoError(t, os.WriteFile(sidecar, []byte(`{"title":"Soon Gone Title"}`), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, found := findAudioItemByPath(t, setup, libID, "Fallback Book.mp3")
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Soon Gone Title", book.Title, "sidecar title wins at create")
|
||||
|
||||
// The sidecar vanishes: every sidecar gone → change → re-extract
|
||||
// finds nothing → title falls back to the filename, not blank.
|
||||
require.NoError(t, os.Remove(sidecar))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
after := findAudioItemByPathRow(t, setup, libID, book.FilePath)
|
||||
require.Equal(t, "Fallback Book", after.Title, "sidecar removal re-derives through the create-equivalent chain")
|
||||
}
|
||||
|
||||
// findItemUpdated reads a media item's updated_at by file_path.
|
||||
func findItemUpdated(t *testing.T, setup *TestServerSetup, libID pgtype.UUID, filePath string) time.Time {
|
||||
t.Helper()
|
||||
for _, item := range listLibraryAudioItems(t, setup, libID) {
|
||||
if item.FilePath == filePath {
|
||||
return item.UpdatedAt.Time
|
||||
}
|
||||
}
|
||||
t.Fatalf("item %q not found in library", filePath)
|
||||
return time.Time{}
|
||||
}
|
||||
|
||||
// findAudioItemByPathRow is findAudioItemByPath with a hard failure.
|
||||
func findAudioItemByPathRow(t *testing.T, setup *TestServerSetup, libID pgtype.UUID, filePath string) database.ListMediaItemsByLibraryIncludingArchivedRow {
|
||||
t.Helper()
|
||||
item, found := findAudioItemByPath(t, setup, libID, filePath)
|
||||
require.True(t, found, "item %q must exist", filePath)
|
||||
return item
|
||||
}
|
||||
@@ -108,7 +108,7 @@ func TestCalibreLibraryScan(t *testing.T) {
|
||||
require.Equal(t, "Test Book", book.Title, "Title should match sidecar")
|
||||
require.Equal(t, "Test Author", book.Author.String, "Author should match sidecar")
|
||||
require.Equal(t, "Test Series", book.Series.String, "Series should match sidecar")
|
||||
require.Equal(t, int32(1), book.SeriesNumber.Int32, "Series number should match sidecar")
|
||||
require.Equal(t, float64(1), book.SeriesNumber.Float64, "Series number should match sidecar")
|
||||
require.Equal(t, "Test Publisher", book.Publisher.String, "Publisher should match sidecar")
|
||||
require.Len(t, book.Tags, 2, "Should have 2 tags from sidecar")
|
||||
require.Contains(t, book.Tags, "Fantasy", "Should have Fantasy tag")
|
||||
|
||||
@@ -382,7 +382,7 @@ func createPreviewTestMediaItems(t *testing.T, db *database.Queries, libraryID s
|
||||
media, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: "Preview Test Book " + string(rune('A'+i)),
|
||||
Author: pgtype.Text{String: "Test Author " + string(rune('A'+i)), Valid: true},
|
||||
Authors: []string{"Test Author " + string(rune('A'+i))},
|
||||
Genre: pgtype.Text{String: genre, Valid: true},
|
||||
FilePath: "/tmp/test" + string(rune('A'+i)) + ".epub",
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
|
||||
@@ -111,7 +111,7 @@ func smartCreateLibraryAndBooks(t *testing.T, setup *TestServerSetup, userID uui
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: title,
|
||||
Author: pgtype.Text{String: "Smart Rule Author", Valid: true},
|
||||
Authors: []string{"Smart Rule Author"},
|
||||
Genre: pgtype.Text{String: genre, Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/%s-%d.epub", title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
|
||||
@@ -123,7 +123,7 @@ func createContractFixture(t *testing.T, setup *TestServerSetup) contractFixture
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: itemTitle,
|
||||
Author: pgtype.Text{String: "Test Contract Author", Valid: true},
|
||||
Authors: []string{"Test Contract Author"},
|
||||
Series: pgtype.Text{String: seriesName, Valid: true},
|
||||
Genre: pgtype.Text{String: "Science Fiction", Valid: true},
|
||||
FilePath: "/tmp/contract-test-item.epub",
|
||||
@@ -490,10 +490,12 @@ func TestContractPin_CollectionsJSON(t *testing.T) {
|
||||
fx := createContractFixture(t, setup)
|
||||
|
||||
// The envelope every detail branch shares.
|
||||
// A2c (2026-10-19): + up_next (the collection's reading-queue strip
|
||||
// payload; docs/contract_pins.md changelog).
|
||||
envelopeKeys := []string{
|
||||
"id", "user_id", "name", "description", "color", "icon",
|
||||
"auto_assign_rules", "view_settings", "books", "book_count",
|
||||
"is_smart", "created_at",
|
||||
"is_smart", "created_at", "up_next",
|
||||
}
|
||||
|
||||
t.Run("user collection with library_id (badged branch)", func(t *testing.T) {
|
||||
@@ -660,6 +662,11 @@ func TestContractPin_SearchJSON(t *testing.T) {
|
||||
// EXPLAIN plan-identical) and the search endpoint serializes the
|
||||
// row directly, so they emit here.
|
||||
"abridged", "audio_metadata",
|
||||
// A2b/A2c (2026-10-19): + subtitle, explicit, authors,
|
||||
// authors_search, publishers, publishers_search, narrators_search
|
||||
// (docs/contract_pins.md changelog).
|
||||
"subtitle", "explicit", "authors", "authors_search",
|
||||
"publishers", "publishers_search", "narrators_search",
|
||||
}
|
||||
|
||||
url := fmt.Sprintf("%s/api/media-items/search?q=%s&library_id=%s", setup.Server.URL, url.QueryEscape(fx.itemTitle), fx.libraryID)
|
||||
@@ -693,6 +700,8 @@ func TestContractPin_DashboardSections(t *testing.T) {
|
||||
sectionKeys := []string{
|
||||
"id", "collection_id", "is_system", "title", "description",
|
||||
"icon", "items", "view_all_url", "priority",
|
||||
// A2c (2026-10-19): + is_playlist, complete (docs/contract_pins.md).
|
||||
"is_playlist", "complete",
|
||||
}
|
||||
sections := getArray(t, body, "sections")
|
||||
require.NotEmpty(t, sections)
|
||||
@@ -711,10 +720,14 @@ func TestContractPin_MediaListBothBranches(t *testing.T) {
|
||||
fx := createContractFixture(t, setup)
|
||||
|
||||
// The sqlc row key sets behind BOTH branches.
|
||||
// A2b (2026-10-19): + subtitle, explicit, authors, authors_search,
|
||||
// publishers, publishers_search (docs/contract_pins.md changelog).
|
||||
rawRowKeys := []string{
|
||||
"id", "library_id", "title", "author", "isbn", "description",
|
||||
"id", "library_id", "title", "subtitle", "author", "isbn", "description",
|
||||
"file_path", "file_size", "mime_type", "cover_image_path",
|
||||
"series", "series_number", "tags", "narrators", "abridged",
|
||||
"explicit", "authors", "authors_search", "publishers",
|
||||
"publishers_search",
|
||||
"asin", "date_published", "publisher", "contributors", "language",
|
||||
"edition", "page_count", "genre", "copyright_year", "goodreads_id",
|
||||
"openlibrary_id", "google_books_id", "added_by_admin_id", "created_at",
|
||||
@@ -731,10 +744,10 @@ func TestContractPin_MediaListBothBranches(t *testing.T) {
|
||||
"library_type_name_2",
|
||||
}
|
||||
resolvedKeys := []string{
|
||||
"id", "library_id", "title", "author", "isbn", "description",
|
||||
"id", "library_id", "title", "subtitle", "author", "authors", "isbn", "description",
|
||||
"file_path", "file_size", "mime_type", "cover_image_path",
|
||||
"series", "series_number", "tags", "asin", "date_published",
|
||||
"publisher", "contributors", "language", "edition", "page_count",
|
||||
"publisher", "publishers", "explicit", "contributors", "language", "edition", "page_count",
|
||||
"genre", "created_at", "updated_at", "format_group",
|
||||
"manga_type", "reading_direction",
|
||||
"series_count", "volume", "imprint", "age_rating", "web_url",
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
package main
|
||||
|
||||
// Extractor-version (ev) mechanism, faceted plan Phase 10: parser fixes
|
||||
// (7e4c97f6, genre model v2) must reach rows ingested under an older
|
||||
// parser without a force rescan. The manifest's ev stamp is the
|
||||
// per-item provenance; scan_extractor_version is the current parser's
|
||||
// claim; a mismatch re-extracts on a later scan, bounded per scan.
|
||||
//
|
||||
// State machine pinned here, driven through REAL scans over a temp
|
||||
// library (the corruption probe makes re-extraction observable: the
|
||||
// only writer that restores the sidecar values is a re-extraction):
|
||||
//
|
||||
// create → stamped ev=1 (current parser, truthful)
|
||||
// equal → no re-extract (corruption persists)
|
||||
// wiped → backfill stores ev UNSET, never stamps, never re-extracts
|
||||
// bump → re-extract ONCE (corruption healed, ev=2)
|
||||
// equal → stable (second corruption persists)
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// storedManifestEv reads the "ev" stamp from an item's stored manifest
|
||||
// (0 = absent — the backfill's v1-or-unknown marker).
|
||||
func storedManifestEv(t *testing.T, setup *TestServerSetup, itemID pgtype.UUID) int {
|
||||
t.Helper()
|
||||
row, err := setup.DB.GetMediaScanState(context.Background(), itemID)
|
||||
require.NoError(t, err, "scan-state row must exist — every book needs an ev home")
|
||||
var m map[string]any
|
||||
require.NoError(t, json.Unmarshal(row.Manifest, &m))
|
||||
ev, _ := m["ev"].(float64)
|
||||
return int(ev)
|
||||
}
|
||||
|
||||
// corruptItemColumn scribbles a scan-owned column on a TEST-CREATED item
|
||||
// so a later re-extraction is observable (the sidecar rewrites it back).
|
||||
// The row is test data cascade-deleted with the library — never real rows.
|
||||
func corruptItemColumn(t *testing.T, setup *TestServerSetup, itemID pgtype.UUID, column, value string) {
|
||||
t.Helper()
|
||||
tag, err := setup.DBPool.Exec(context.Background(),
|
||||
fmt.Sprintf("UPDATE media_items SET %s = $1 WHERE id = $2", column), value, itemID)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, int64(1), tag.RowsAffected(), "corruption must hit exactly the test item")
|
||||
}
|
||||
|
||||
|
||||
func setExtractorVersion(t *testing.T, setup *TestServerSetup, v int) {
|
||||
t.Helper()
|
||||
// Restore-to-prior (not a hardcoded value): the shared dev DB's
|
||||
// setting must survive the suite exactly as the test found it — if a
|
||||
// parser-fix bump was in flight, a hardcoded 1 would silently cancel
|
||||
// it and the live server would stop re-extracting.
|
||||
setSystemSettingRestored(t, setup, "scan_extractor_version", strconv.Itoa(v))
|
||||
}
|
||||
|
||||
func TestExtractorVersionReextract(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
pinExtractorVersionOne(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test EV Library %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// metadata.opf sidecar: the re-extraction's provenance source.
|
||||
bookDir := filepath.Join(folder, "EV Probe Author", "EV Probe Book")
|
||||
require.NoError(t, os.MkdirAll(bookDir, 0o755))
|
||||
opf := `<?xml version='1.0' encoding='utf-8'?>
|
||||
<package xmlns="http://www.idpf.org/2007/opf" version="2.0">
|
||||
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
|
||||
<dc:title>EV Probe Book</dc:title>
|
||||
<dc:creator>EV Probe Author</dc:creator>
|
||||
</metadata>
|
||||
</package>`
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "metadata.opf"), []byte(opf), 0o644))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "EV Probe Book.epub"), []byte("dummy epub content"), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
item, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Probe Author/EV Probe Book/EV Probe Book.epub"))
|
||||
require.True(t, found, "sidecar book must import")
|
||||
|
||||
// Create stamps the current parser version (truthful).
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, item.ID), "create must stamp the default ev=1")
|
||||
|
||||
// Equality → skip: corruption persists (nothing re-extracts at ev=1).
|
||||
// NOTE: `author` is trigger-owned (sync_author_publisher_display
|
||||
// rewrites it from the authors array on every UPDATE) — corrupt a
|
||||
// trigger-free column the re-extraction rewrites: title.
|
||||
corruptItemColumn(t, setup, item.ID, "title", "ZZZ corrupted")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
item, found = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Probe Author/EV Probe Book/EV Probe Book.epub"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "ZZZ corrupted", item.Title, "equal ev must NOT re-extract (corruption persists)")
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, item.ID))
|
||||
|
||||
// Wipe the scan state (test-owned row) to simulate a pre-release
|
||||
// book: backfill re-fingerprints WITHOUT re-extraction and WITHOUT
|
||||
// stamping — the row never claims a version it can't prove.
|
||||
_, err := setup.DBPool.Exec(ctx, "DELETE FROM media_scan_state WHERE media_item_id = $1", item.ID)
|
||||
require.NoError(t, err)
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.Equal(t, 0, storedManifestEv(t, setup, item.ID), "backfill must store ev UNSET (v1-or-unknown)")
|
||||
item, _ = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Probe Author/EV Probe Book/EV Probe Book.epub"))
|
||||
require.Equal(t, "ZZZ corrupted", item.Title, "backfill must NOT re-extract")
|
||||
|
||||
// Bump via the SETTINGS API; the absent ev rides v1 → mismatch →
|
||||
// re-extract once: the corruption heals and the fresh manifest
|
||||
// carries ev=2.
|
||||
setExtractorVersion(t, setup, 2)
|
||||
// Prior-value restore rides inside setExtractorVersion — never a
|
||||
// hardcoded 1, so a deliberate dev bump survives the suite.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
item, found = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Probe Author/EV Probe Book/EV Probe Book.epub"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "EV Probe Book", item.Title, "ev mismatch must re-extract (sidecar value restored)")
|
||||
require.Equal(t, 2, storedManifestEv(t, setup, item.ID), "post-extraction store stamps the new ev")
|
||||
|
||||
// Then stable: the second corruption persists at ev=2 == setting 2.
|
||||
corruptItemColumn(t, setup, item.ID, "title", "ZZZ corrupted again")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
item, _ = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Probe Author/EV Probe Book/EV Probe Book.epub"))
|
||||
require.Equal(t, "ZZZ corrupted again", item.Title, "equal ev must NOT re-extract again (stable)")
|
||||
require.Equal(t, 2, storedManifestEv(t, setup, item.ID))
|
||||
}
|
||||
|
||||
func TestExtractorVersionMultifile(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
pinExtractorVersionOne(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test EV Audio %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// A canonical folder book, sidecar-less: the Phase 6 seed gives it
|
||||
// an entries-less manifest — now also its ev home.
|
||||
writeTestTrack(t, filepath.Join(folder, "EV Audio Author", "EV Audio Title", "01 First.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "EV Audio Author", "EV Audio Title", "02 Second.mp3"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, found := findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Audio Author/EV Audio Title"))
|
||||
require.True(t, found, "folder book must import")
|
||||
require.Equal(t, "EV Audio Title", book.Title)
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, book.ID), "multifile create must stamp the default ev=1")
|
||||
|
||||
// Equality → skip: the multifile ladder's unchanged gate stays shut.
|
||||
corruptItemColumn(t, setup, book.ID, "title", "ZZZ corrupted")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, _ = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Audio Author/EV Audio Title"))
|
||||
require.Equal(t, "ZZZ corrupted", book.Title, "equal ev must NOT re-apply (corruption persists)")
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, book.ID))
|
||||
|
||||
// Bump → the ev gate fires through multifileSidecarChanged → the
|
||||
// editorial apply re-derives the title from the folder basename.
|
||||
setExtractorVersion(t, setup, 2)
|
||||
// Prior-value restore rides inside setExtractorVersion — never a
|
||||
// hardcoded 1, so a deliberate dev bump survives the suite.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, found = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Audio Author/EV Audio Title"))
|
||||
require.True(t, found)
|
||||
require.Equal(t, "EV Audio Title", book.Title, "ev mismatch must re-extract the folder book")
|
||||
require.Equal(t, 2, storedManifestEv(t, setup, book.ID), "change-path store stamps the new ev")
|
||||
|
||||
// Then stable.
|
||||
corruptItemColumn(t, setup, book.ID, "title", "ZZZ corrupted again")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
book, _ = findAudioItemByPath(t, setup, libID, filepath.ToSlash("EV Audio Author/EV Audio Title"))
|
||||
require.Equal(t, "ZZZ corrupted again", book.Title, "equal ev must NOT re-apply again (stable)")
|
||||
require.Equal(t, 2, storedManifestEv(t, setup, book.ID))
|
||||
}
|
||||
@@ -0,0 +1,641 @@
|
||||
package main
|
||||
|
||||
// Facet registry Batch A integration tests (faceted-navigation plan,
|
||||
// Part 1 Phase 1): the nine new SearchMediaItemsUnified narg blocks —
|
||||
// scalar-fuzzy (story_arc, imprint, edition), exact enums (age_rating,
|
||||
// format_group, manga_type, reading_direction), and presence-bound
|
||||
// bools (explicit, abridged) — filter correctly against a real DB.
|
||||
// The registry↔SQL drift guard (unit level) lives in
|
||||
// internal/services/facet_registry_test.go.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// facetSearch runs the unified search endpoint with arbitrary query
|
||||
// params and returns (status, decoded array-or-object).
|
||||
func facetSearch(t *testing.T, setup *TestServerSetup, params url.Values) (int, []map[string]interface{}) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?"+params.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode == http.StatusNotFound {
|
||||
// The endpoint's no-results contract: 404 + error envelope.
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var rows []map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&rows))
|
||||
return resp.StatusCode, rows
|
||||
}
|
||||
|
||||
func facetTitles(t *testing.T, rows []map[string]interface{}) []string {
|
||||
t.Helper()
|
||||
var out []string
|
||||
for _, r := range rows {
|
||||
title, _ := r["title"].(string)
|
||||
out = append(out, title)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func TestFacetRegistryParamsFilter(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
libName := fmt.Sprintf("Test Facet Params Library %d", time.Now().UnixNano())
|
||||
library, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: libName,
|
||||
Description: pgtype.Text{String: "facet params test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
libUUID, err := uuid.FromBytes(library.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, library.ID) })
|
||||
|
||||
createItem := func(title string, mutate func(*database.CreateMediaItemParams)) {
|
||||
t.Helper()
|
||||
p := database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: title,
|
||||
Authors: []string{"Facet Param Author"},
|
||||
FilePath: fmt.Sprintf("/tmp/facet-params/%s-%d.epub", title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
}
|
||||
if mutate != nil {
|
||||
mutate(&p)
|
||||
}
|
||||
_, err := setup.DB.CreateMediaItem(ctx, p)
|
||||
require.NoError(t, err, "create %s", title)
|
||||
}
|
||||
|
||||
createItem("Facet Alpha", func(p *database.CreateMediaItemParams) {
|
||||
p.StoryArc = pgtype.Text{String: "civil war", Valid: true}
|
||||
p.Imprint = pgtype.Text{String: "Vertigo", Valid: true}
|
||||
p.Edition = pgtype.Text{String: "First Edition", Valid: true}
|
||||
p.AgeRating = pgtype.Text{String: "Teen", Valid: true}
|
||||
p.MangaType = pgtype.Text{String: "yes", Valid: true}
|
||||
p.Explicit = true
|
||||
})
|
||||
createItem("Facet Beta", func(p *database.CreateMediaItemParams) {
|
||||
p.StoryArc = pgtype.Text{String: "infinity gauntlet", Valid: true}
|
||||
p.AgeRating = pgtype.Text{String: "Mature", Valid: true}
|
||||
p.ReadingDirection = pgtype.Text{String: "rtl", Valid: true}
|
||||
p.Abridged = true
|
||||
})
|
||||
|
||||
base := url.Values{"library_id": []string{libUUID.String()}}
|
||||
titles := func(params url.Values) []string {
|
||||
status, rows := facetSearch(t, setup, mergeParams(base, params))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
return facetTitles(t, rows)
|
||||
}
|
||||
|
||||
// Unfiltered: both items, the additive params are default-neutral.
|
||||
got := titles(url.Values{})
|
||||
assert.Len(t, got, 2, "unfiltered search returns both items")
|
||||
|
||||
// Scalar-fuzzy: single value...
|
||||
got = titles(url.Values{"story_arc_filter": {"civil war"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "story_arc scalar-fuzzy")
|
||||
got = titles(url.Values{"imprint_filter": {"Vertigo"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "imprint scalar-fuzzy")
|
||||
got = titles(url.Values{"edition_filter": {"First Edition"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "edition scalar-fuzzy")
|
||||
// ...and comma = OR-within (multi-select).
|
||||
got = titles(url.Values{"story_arc_filter": {"civil war,infinity gauntlet"}})
|
||||
assert.Len(t, got, 2, "comma is OR-within the facet")
|
||||
|
||||
// Exact enums: single + multi.
|
||||
got = titles(url.Values{"age_rating_filter": {"Teen"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "age_rating exact")
|
||||
got = titles(url.Values{"age_rating_filter": {"Teen,Mature"}})
|
||||
assert.Len(t, got, 2, "age_rating multi-select")
|
||||
got = titles(url.Values{"format_group_filter": {"reflowable"}})
|
||||
assert.Len(t, got, 2, "format_group exact (create default)")
|
||||
status, _ := facetSearch(t, setup, mergeParams(base, url.Values{"format_group_filter": {"audiobook"}}))
|
||||
assert.Equal(t, http.StatusNotFound, status, "wrong format_group matches nothing")
|
||||
got = titles(url.Values{"reading_direction_filter": {"rtl"}})
|
||||
assert.Equal(t, []string{"Facet Beta"}, got, "reading_direction exact")
|
||||
got = titles(url.Values{"manga_type_filter": {"yes"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "manga_type exact (works cross-type by data)")
|
||||
|
||||
// Presence-bound bools: absent = no filter, value = exact.
|
||||
got = titles(url.Values{"explicit_filter": {"true"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "explicit=true")
|
||||
got = titles(url.Values{"explicit_filter": {"false"}})
|
||||
assert.Equal(t, []string{"Facet Beta"}, got, "explicit=false")
|
||||
got = titles(url.Values{"abridged_filter": {"true"}})
|
||||
assert.Equal(t, []string{"Facet Beta"}, got, "abridged=true")
|
||||
|
||||
// AND-across-facets: story_arc + age_rating intersect to Alpha.
|
||||
got = titles(url.Values{"story_arc_filter": {"civil war"}, "age_rating_filter": {"Teen"}})
|
||||
assert.Equal(t, []string{"Facet Alpha"}, got, "AND-across-facets")
|
||||
|
||||
// Language UX (Batch B): a display name normalizes through the
|
||||
// ingest map — "English" matches the row stored as "en".
|
||||
createItem("Facet Gamma", func(p *database.CreateMediaItemParams) {
|
||||
p.Language = pgtype.Text{String: "en", Valid: true}
|
||||
})
|
||||
got = titles(url.Values{"language_filter": {"English"}})
|
||||
assert.Equal(t, []string{"Facet Gamma"}, got, "display name matches ISO-stored rows")
|
||||
got = titles(url.Values{"language_filter": {"en"}})
|
||||
assert.Equal(t, []string{"Facet Gamma"}, got, "raw code still matches")
|
||||
}
|
||||
|
||||
// TestFacetsEndpoint covers GET /api/media-items/facets (Batch C): the
|
||||
// registry-driven value+count listing, language display names, and the
|
||||
// family-lens gates (hidden library → 403 on BOTH the facets and search
|
||||
// endpoints; bool/range facets → 400 because they render as toggles).
|
||||
func TestFacetsEndpoint(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
libName := fmt.Sprintf("Test Facet Listing Library %d", time.Now().UnixNano())
|
||||
library, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: libName,
|
||||
Description: pgtype.Text{String: "facet listing test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
libUUID, err := uuid.FromBytes(library.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, library.ID) })
|
||||
|
||||
// Two items share the genre; one carries the language + narrator.
|
||||
for _, title := range []string{"Facet List One", "Facet List Two"} {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: title,
|
||||
Authors: []string{"Facet Listing Author"},
|
||||
Genre: pgtype.Text{String: "fantasy", Valid: true},
|
||||
Language: pgtype.Text{String: "en", Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/facet-listing/%s-%d.epub", title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Facet List Three",
|
||||
Authors: []string{"Facet Listing Author"},
|
||||
Genre: pgtype.Text{String: "horror", Valid: true},
|
||||
Narrators: []string{"Facet Listing Narrator"},
|
||||
FilePath: fmt.Sprintf("/tmp/facet-listing/three-%d.epub", time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
getFacets := func(params url.Values) (int, map[string]interface{}) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/facets?"+params.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode >= 400 {
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
var body map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
return resp.StatusCode, body
|
||||
}
|
||||
|
||||
// Counts correct + ordered by count desc.
|
||||
status, body := getFacets(url.Values{
|
||||
"facet": {"genre"},
|
||||
"library_id": {libUUID.String()},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Equal(t, "genre", body["facet"])
|
||||
values, _ := body["values"].([]interface{})
|
||||
require.Len(t, values, 2)
|
||||
first, _ := values[0].(map[string]interface{})
|
||||
second, _ := values[1].(map[string]interface{})
|
||||
assert.Equal(t, "fantasy", first["value"])
|
||||
assert.Equal(t, float64(2), first["count"], "two items share the genre")
|
||||
assert.Equal(t, "horror", second["value"])
|
||||
assert.Equal(t, float64(1), second["count"])
|
||||
|
||||
// Language values carry display names (server owns display).
|
||||
status, body = getFacets(url.Values{
|
||||
"facet": {"language"},
|
||||
"library_id": {libUUID.String()},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
values, _ = body["values"].([]interface{})
|
||||
require.Len(t, values, 1)
|
||||
lang, _ := values[0].(map[string]interface{})
|
||||
assert.Equal(t, "en", lang["value"], "the raw code rides the param")
|
||||
assert.Equal(t, "English", lang["display"], "the name is for display")
|
||||
|
||||
// Bool facets render as toggles, not lists → 400.
|
||||
status, _ = getFacets(url.Values{
|
||||
"facet": {"explicit"},
|
||||
"library_id": {libUUID.String()},
|
||||
})
|
||||
assert.Equal(t, http.StatusBadRequest, status)
|
||||
|
||||
// Unknown facet → 400.
|
||||
status, _ = getFacets(url.Values{
|
||||
"facet": {"woodcut"},
|
||||
"library_id": {libUUID.String()},
|
||||
})
|
||||
assert.Equal(t, http.StatusBadRequest, status)
|
||||
|
||||
// Family-lens: a library explicitly hidden from the regular user is
|
||||
// a 403 on the facets endpoint AND on the search endpoint (an
|
||||
// explicit library_id outside the visible set is never silently
|
||||
// empty — AGENTS.md → Library visibility).
|
||||
_, err = setup.DB.SetLibraryVisibility(ctx, database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
LibraryID: library.ID,
|
||||
IsVisible: false,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
status, _ = getFacets(url.Values{
|
||||
"facet": {"genre"},
|
||||
"library_id": {libUUID.String()},
|
||||
})
|
||||
assert.Equal(t, http.StatusForbidden, status, "hidden library → 403")
|
||||
|
||||
searchReq, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?library_id="+libUUID.String(), nil)
|
||||
searchReq.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
searchResp, err := http.DefaultClient.Do(searchReq)
|
||||
require.NoError(t, err)
|
||||
_, _ = io.Copy(io.Discard, searchResp.Body)
|
||||
_ = searchResp.Body.Close()
|
||||
assert.Equal(t, http.StatusForbidden, searchResp.StatusCode, "search with hidden library_id → 403")
|
||||
}
|
||||
|
||||
func mergeParams(base, extra url.Values) url.Values {
|
||||
out := url.Values{}
|
||||
for k, v := range base {
|
||||
out[k] = v
|
||||
}
|
||||
for k, v := range extra {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// facetFixtureBooks is the under-filter counts fixture: two fantasy
|
||||
// books (Pinecrest / Quartzmill) and one history book (Pinecrest).
|
||||
// Publisher names are trigram-disjoint on purpose — the scalar-fuzzy
|
||||
// filter is word_similarity, and near-identical names ("Ace Books" vs
|
||||
// "Baen Books") cross-match at 0.3+ (session gotcha).
|
||||
func facetFixtureBooks(t *testing.T, setup *TestServerSetup, libraryID pgtype.UUID, prefix string) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
books := []struct {
|
||||
title string
|
||||
genre string
|
||||
publisher string
|
||||
}{
|
||||
{prefix + " Alpha", "fantasy", "Pinecrest"},
|
||||
{prefix + " Beta", "fantasy", "Quartzmill"},
|
||||
{prefix + " Gamma", "history", "Pinecrest"},
|
||||
}
|
||||
for _, b := range books {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: libraryID,
|
||||
Title: b.title,
|
||||
Authors: []string{prefix + " Author"},
|
||||
Genre: pgtype.Text{String: b.genre, Valid: true},
|
||||
Publishers: []string{b.publisher},
|
||||
Language: pgtype.Text{String: "en", Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/facet-counts/%s-%d.epub", b.title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
}
|
||||
|
||||
// facetCountValue is one decoded facet value+count.
|
||||
type facetCountValue struct {
|
||||
Value string `json:"value"`
|
||||
Count float64 `json:"count"`
|
||||
}
|
||||
|
||||
// facetCountsGet runs the all-facets endpoint (no facet param) and
|
||||
// decodes {"facets": {param: [{value, count}, …]}}.
|
||||
func facetCountsGet(t *testing.T, setup *TestServerSetup, params url.Values) (int, map[string][]facetCountValue) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/facets?"+params.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode >= 400 {
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
var body struct {
|
||||
Facets map[string][]facetCountValue `json:"facets"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
return resp.StatusCode, body.Facets
|
||||
}
|
||||
|
||||
// requireFacetCount asserts one value's count within a facet.
|
||||
func requireFacetCount(t *testing.T, facet []facetCountValue, value string, want float64) {
|
||||
t.Helper()
|
||||
for _, v := range facet {
|
||||
if v.Value == value {
|
||||
require.Equal(t, want, v.Count, "facet value %q count", value)
|
||||
return
|
||||
}
|
||||
}
|
||||
require.Failf(t, "facet value missing", "%q not in facet", value)
|
||||
}
|
||||
|
||||
// TestFacetCountsUnderFilters (faceted plan Phase 7): the all-facets
|
||||
// shape computes each facet's counts UNDER the other active filters —
|
||||
// a facet's own block is omitted from its arm (classic faceted
|
||||
// semantics: the genre list stays complete while genre filters the
|
||||
// rest), multi-select is OR-within, and the family lens holds on the
|
||||
// new shape too.
|
||||
func TestFacetCountsUnderFilters(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
libName := fmt.Sprintf("Test Facet Counts %d", time.Now().UnixNano())
|
||||
library, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: libName,
|
||||
Description: pgtype.Text{String: "facet under-filter counts test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
libUUID, err := uuid.FromBytes(library.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, library.ID) })
|
||||
|
||||
hiddenLib, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: fmt.Sprintf("Test Facet Counts Hidden %d", time.Now().UnixNano()),
|
||||
Description: pgtype.Text{String: "hidden for the regular user", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
hiddenUUID, err := uuid.FromBytes(hiddenLib.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, hiddenLib.ID) })
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO library_visibility (library_id, user_id, is_visible) VALUES ($1, $2, false)`,
|
||||
hiddenLib.ID, userID)
|
||||
require.NoError(t, err)
|
||||
|
||||
facetFixtureBooks(t, setup, library.ID, "Counts")
|
||||
// A book in the HIDDEN library: distinctive values that must never
|
||||
// surface for the regular user.
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: hiddenLib.ID,
|
||||
Title: "Counts Hidden Book",
|
||||
Authors: []string{"Counts Secret Author"},
|
||||
Genre: pgtype.Text{String: "cryptobotany", Valid: true},
|
||||
Publishers: []string{"Shadowpress"},
|
||||
FilePath: fmt.Sprintf("/tmp/facet-counts/hidden-%d.epub", time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
// No filters: the v1 global-per-library counts.
|
||||
status, counts := facetCountsGet(t, setup, url.Values{"library_id": {libUUID.String()}})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
requireFacetCount(t, counts["genre"], "fantasy", 2)
|
||||
requireFacetCount(t, counts["genre"], "history", 1)
|
||||
requireFacetCount(t, counts["publisher"], "Pinecrest", 2)
|
||||
requireFacetCount(t, counts["publisher"], "Quartzmill", 1)
|
||||
|
||||
// Under genre=fantasy: the genre facet STILL lists everything with
|
||||
// unfiltered counts (its own block is omitted from its arm), while
|
||||
// the publisher facet counts only within fantasy.
|
||||
status, counts = facetCountsGet(t, setup, url.Values{
|
||||
"library_id": {libUUID.String()},
|
||||
"genre_filter": {"fantasy"},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
requireFacetCount(t, counts["genre"], "fantasy", 2)
|
||||
requireFacetCount(t, counts["genre"], "history", 1)
|
||||
requireFacetCount(t, counts["publisher"], "Pinecrest", 1)
|
||||
requireFacetCount(t, counts["publisher"], "Quartzmill", 1)
|
||||
|
||||
// OR-within: both genres → Pinecrest regains Alpha (fantasy) and
|
||||
// Gamma (history).
|
||||
status, counts = facetCountsGet(t, setup, url.Values{
|
||||
"library_id": {libUUID.String()},
|
||||
"genre_filter": {"fantasy,history"},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
requireFacetCount(t, counts["publisher"], "Pinecrest", 2)
|
||||
requireFacetCount(t, counts["publisher"], "Quartzmill", 1)
|
||||
|
||||
// AND-across: genre=history + publisher=Pinecrest → the genre facet
|
||||
// is unfiltered by ITSELF (its own block is omitted) but still
|
||||
// counts under the active publisher filter: both Pinecrest books
|
||||
// (Alpha fantasy, Gamma history) surface as 1 each, Beta (Quartzmill)
|
||||
// is filtered away. The publisher facet sees only history books, so
|
||||
// Pinecrest counts Gamma alone.
|
||||
status, counts = facetCountsGet(t, setup, url.Values{
|
||||
"library_id": {libUUID.String()},
|
||||
"genre_filter": {"history"},
|
||||
"publisher_filter": {"Pinecrest"},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
requireFacetCount(t, counts["genre"], "fantasy", 1)
|
||||
requireFacetCount(t, counts["publisher"], "Pinecrest", 1)
|
||||
|
||||
// All-Libraries view (no library_id): the visible library's values
|
||||
// are present, the hidden library's are absent everywhere.
|
||||
status, counts = facetCountsGet(t, setup, url.Values{})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
requireFacetCount(t, counts["genre"], "fantasy", 2)
|
||||
for _, v := range counts["genre"] {
|
||||
require.NotEqual(t, "cryptobotany", v.Value, "hidden-library genre must be absent")
|
||||
}
|
||||
for _, v := range counts["publisher"] {
|
||||
require.NotEqual(t, "Shadowpress", v.Value, "hidden-library publisher must be absent")
|
||||
}
|
||||
|
||||
// Family lens: an explicit HIDDEN library_id on the new shape is a
|
||||
// 403 (AGENTS standing rule — never a silent pass).
|
||||
status, _ = facetCountsGet(t, setup, url.Values{"library_id": {hiddenUUID.String()}})
|
||||
require.Equal(t, http.StatusForbidden, status)
|
||||
|
||||
// The pinned single-facet shape is untouched by the extension.
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/facets?facet=genre&library_id="+libUUID.String(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var single struct {
|
||||
Facet string `json:"facet"`
|
||||
Values []facetCountValue `json:"values"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&single))
|
||||
require.Equal(t, "genre", single.Facet)
|
||||
requireFacetCount(t, single.Values, "fantasy", 2)
|
||||
requireFacetCount(t, single.Values, "history", 1)
|
||||
}
|
||||
|
||||
// TestFieldValuesAllLibraries pins the All-Books suggestion path
|
||||
// (filter-plan Phase 8): the field-values autocomplete treats
|
||||
// library_id as OPTIONAL — absent aggregates every library VISIBLE to
|
||||
// the caller (the per-user family lens is enforced inside the query),
|
||||
// and an explicit out-of-set library_id is a 403. This is the
|
||||
// regression net for the hard "library_id is required" 400 that broke
|
||||
// typing in the filter drawer on the All-Books view.
|
||||
func TestFieldValuesAllLibraries(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
|
||||
makeLib := func(name string) database.Libraries {
|
||||
t.Helper()
|
||||
lib, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: fmt.Sprintf("Test Field Values %s %d", name, time.Now().UnixNano()),
|
||||
Description: pgtype.Text{String: "field values all-libraries test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, lib.ID) })
|
||||
return lib
|
||||
}
|
||||
visibleLib := makeLib("Visible")
|
||||
hiddenLib := makeLib("Hidden")
|
||||
hiddenUUID, err := uuid.FromBytes(hiddenLib.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
|
||||
// One book per library with IDENTICAL distinctive values — the
|
||||
// All-Books answer must aggregate across both libraries, and every
|
||||
// value is trigram-disjoint from any real data ("Verdana…" nonsense).
|
||||
for _, lib := range []struct {
|
||||
id pgtype.UUID
|
||||
name string
|
||||
}{{visibleLib.ID, "Visible"}, {hiddenLib.ID, "Hidden"}} {
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: lib.id,
|
||||
Title: fmt.Sprintf("Field Values %s Book", lib.name),
|
||||
Authors: []string{"Verdana Scribe"},
|
||||
Genre: pgtype.Text{String: "verdanafantasy", Valid: true},
|
||||
Series: pgtype.Text{String: "Verdana Saga", Valid: true},
|
||||
Language: pgtype.Text{String: "verdanish", Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/field-values/%s-%d.epub", lib.name, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
fieldValuesGet := func(params url.Values) (int, []map[string]interface{}) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?"+params.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
var body struct {
|
||||
Results []map[string]interface{} `json:"results"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
return resp.StatusCode, body.Results
|
||||
}
|
||||
findValue := func(results []map[string]interface{}, value string) (found bool, count float64) {
|
||||
t.Helper()
|
||||
for _, r := range results {
|
||||
if r["value"] == value {
|
||||
found = true
|
||||
count, _ = r["count"].(float64)
|
||||
}
|
||||
}
|
||||
return found, count
|
||||
}
|
||||
|
||||
// Phase A — NO library_id (the All-Books view): 200, and the same
|
||||
// author surfaces ONCE with count 2 — values aggregate across every
|
||||
// visible library. The old bug answered 400 here.
|
||||
status, results := fieldValuesGet(url.Values{"author": {"Verdana"}})
|
||||
require.Equal(t, http.StatusOK, status, "no library_id must be 200, not the old hard 400")
|
||||
found, count := findValue(results, "Verdana Scribe")
|
||||
require.True(t, found, "author value present on the All-Books path")
|
||||
require.Equal(t, float64(2), count, "value aggregated across both visible libraries")
|
||||
for field, value := range map[string]string{
|
||||
"genre": "verdanafantasy", "series": "Verdana Saga", "language": "verdanish",
|
||||
} {
|
||||
status, results = fieldValuesGet(url.Values{field: {value}})
|
||||
require.Equal(t, http.StatusOK, status, "%s autocomplete with no library_id", field)
|
||||
found, count = findValue(results, value)
|
||||
require.True(t, found, "%s value present on the All-Books path", field)
|
||||
require.Equal(t, float64(2), count, "%s aggregated across both libraries", field)
|
||||
}
|
||||
|
||||
// Phase B — family lens on the same All-Books path: hiding one
|
||||
// library from this user halves the counts without any library_id
|
||||
// in the request at all.
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO library_visibility (library_id, user_id, is_visible) VALUES ($1, $2, false)`,
|
||||
hiddenLib.ID, userID)
|
||||
require.NoError(t, err)
|
||||
|
||||
status, results = fieldValuesGet(url.Values{"author": {"Verdana"}})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
found, count = findValue(results, "Verdana Scribe")
|
||||
require.True(t, found)
|
||||
require.Equal(t, float64(1), count, "hidden library's books leave the All-Books suggestions")
|
||||
|
||||
// Phase C — an explicit out-of-set library_id is a 403, never a
|
||||
// silent pass (AGENTS family-lens standing rule).
|
||||
status, _ = fieldValuesGet(url.Values{"author": {"Verdana"}, "library_id": {hiddenUUID.String()}})
|
||||
require.Equal(t, http.StatusForbidden, status)
|
||||
status, _ = fieldValuesGet(url.Values{"author": {"Verdana"}, "library_id": {uuid.New().String()}})
|
||||
require.Equal(t, http.StatusForbidden, status)
|
||||
}
|
||||
@@ -0,0 +1,790 @@
|
||||
package main
|
||||
|
||||
// Filter rule builder integration tests (faceted plan Phase 9): the
|
||||
// composed engines behind the qf wire param — tree semantics (AND/OR/
|
||||
// negation/nesting), every user-relative field class (the "my data"
|
||||
// weld), the note-text substring pin, family lens, suggestion-under-
|
||||
// rules exactness, qf validation, and qf-over-flat precedence.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// qfSearch runs the search endpoint with arbitrary params (qf trees go
|
||||
// in the qf param as JSON) and returns (status, decoded rows).
|
||||
func qfSearch(t *testing.T, setup *TestServerSetup, token string, params url.Values) (int, []map[string]interface{}) {
|
||||
t.Helper()
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?"+params.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
var rows []map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&rows))
|
||||
return resp.StatusCode, rows
|
||||
}
|
||||
|
||||
// qfRule is a compact rule builder for tests.
|
||||
func qfRule(field, op string, values ...string) map[string]interface{} {
|
||||
return map[string]interface{}{"field": field, "op": op, "values": values}
|
||||
}
|
||||
|
||||
func qfAll(children ...map[string]interface{}) string {
|
||||
return qfGroup("all", children...)
|
||||
}
|
||||
|
||||
func qfAny(children ...map[string]interface{}) string {
|
||||
return qfGroup("any", children...)
|
||||
}
|
||||
|
||||
func qfGroup(mode string, children ...map[string]interface{}) string {
|
||||
b, _ := json.Marshal(map[string]interface{}{"mode": mode, "children": children})
|
||||
return string(b)
|
||||
}
|
||||
|
||||
func qfTitles(t *testing.T, rows []map[string]interface{}) []string {
|
||||
t.Helper()
|
||||
out := make([]string, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
title, _ := r["title"].(string)
|
||||
out = append(out, title)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// builderFixtureLib builds one test library + four books with
|
||||
// trigram-disjoint values. Returns the library UUID and the book titles.
|
||||
func builderFixtureLib(t *testing.T, setup *TestServerSetup, userID uuid.UUID, name string) (uuid.UUID, pgtype.UUID) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
library, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: fmt.Sprintf("Test Builder %s %d", name, time.Now().UnixNano()),
|
||||
Description: pgtype.Text{String: "filter builder test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
libUUID, err := uuid.FromBytes(library.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, library.ID) })
|
||||
|
||||
year := pgtype.Int4{Int32: 1999, Valid: true}
|
||||
datePublished := pgtype.Date{Time: time.Date(1999, 3, 1, 0, 0, 0, 0, time.UTC), Valid: true}
|
||||
books := []database.CreateMediaItemParams{
|
||||
{
|
||||
LibraryID: library.ID, Title: name + " Tide",
|
||||
Authors: []string{"Amara Voss"}, Genre: pgtype.Text{String: "fantasy", Valid: true},
|
||||
Tags: []string{"space", "epic"}, PageCount: pgtype.Int4{Int32: 300, Valid: true},
|
||||
Explicit: true, Isbn: pgtype.Text{String: "1111111111111", Valid: true},
|
||||
Series: pgtype.Text{String: "Voss Saga", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 1, Valid: true},
|
||||
Language: pgtype.Text{String: "en", Valid: true},
|
||||
DatePublished: datePublished, CopyrightYear: year,
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-tide-%d.epub", name, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
},
|
||||
{
|
||||
LibraryID: library.ID, Title: name + " Stone",
|
||||
Authors: []string{"Amara Voss"}, Genre: pgtype.Text{String: "history", Valid: true},
|
||||
Tags: []string{"space"}, PageCount: pgtype.Int4{Int32: 500, Valid: true},
|
||||
Series: pgtype.Text{String: "Voss Saga", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 2, Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-stone-%d.epub", name, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 2048, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
},
|
||||
{
|
||||
LibraryID: library.ID, Title: name + " Fen",
|
||||
Authors: []string{"Benjamin Quill"}, Genre: pgtype.Text{String: "fantasy", Valid: true},
|
||||
Tags: []string{"epic"}, PageCount: pgtype.Int4{Int32: 200, Valid: true},
|
||||
Abridged: true,
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-fen-%d.epub", name, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 512, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
},
|
||||
{
|
||||
LibraryID: library.ID, Title: name + " Wick",
|
||||
Authors: []string{"Coraline Dar"}, Genre: pgtype.Text{String: "history", Valid: true},
|
||||
PageCount: pgtype.Int4{Int32: 700, Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-wick-%d.epub", name, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 4096, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
},
|
||||
}
|
||||
for i := range books {
|
||||
if books[i].Title == "" {
|
||||
books[i].Title = fmt.Sprintf("%s Book %d", name, i)
|
||||
}
|
||||
// created_at binds explicitly — a zero pgtype would insert NULL
|
||||
// and the date fields (added_at within N days) would never match.
|
||||
books[i].CreatedAt = pgtype.Timestamptz{Time: time.Now().UTC(), Valid: true}
|
||||
books[i].ImportedAt = pgtype.Timestamptz{Time: time.Now().UTC(), Valid: true}
|
||||
// Ingest populates the *_search columns (normalized lowercase);
|
||||
// direct-create fixtures must mirror that or the *_search-side
|
||||
// match clauses (tags via tags_search) see nothing.
|
||||
if len(books[i].Authors) > 0 && books[i].AuthorsSearch == nil {
|
||||
lowered := make([]string, len(books[i].Authors))
|
||||
for j, a := range books[i].Authors {
|
||||
lowered[j] = strings.ToLower(a)
|
||||
}
|
||||
books[i].AuthorsSearch = lowered
|
||||
}
|
||||
if len(books[i].Tags) > 0 && books[i].TagsSearch == nil {
|
||||
lowered := make([]string, len(books[i].Tags))
|
||||
for j, tag := range books[i].Tags {
|
||||
lowered[j] = strings.ToLower(tag)
|
||||
}
|
||||
books[i].TagsSearch = lowered
|
||||
}
|
||||
_, err := setup.DB.CreateMediaItem(ctx, books[i])
|
||||
require.NoError(t, err, "create %s", books[i].Title)
|
||||
}
|
||||
return libUUID, library.ID
|
||||
}
|
||||
|
||||
// secondBuilderUser creates a second test user (the "other reader" for
|
||||
// the my-data weld tests) and returns (userID bytes, JWT).
|
||||
func secondBuilderUser(t *testing.T, setup *TestServerSetup) (uuid.UUID, string) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
suffix := time.Now().UnixNano()
|
||||
second, err := setup.DB.CreateUser(ctx, database.CreateUserParams{
|
||||
Email: fmt.Sprintf("testbuilder%d@tests.bookhoard.internal", suffix),
|
||||
Username: fmt.Sprintf("testbuilder%d", suffix),
|
||||
PasswordHash: "$2a$10$JjAtK7PPa1WexQC3AUGe8OXLeuseZ/haN1Mz7emMo6CfOvMiTVXWq",
|
||||
FirstName: pgtype.Text{String: "Second", Valid: true},
|
||||
LastName: pgtype.Text{String: "Reader", Valid: true},
|
||||
Role: "user",
|
||||
Timezone: pgtype.Text{String: "UTC", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteUser(ctx, second.ID) })
|
||||
token := loginWithCredentials(t, setup.Server, second.Email, "Test@Pass123!")
|
||||
secondUUID, err := uuid.FromBytes(second.ID.Bytes[:])
|
||||
require.NoError(t, err)
|
||||
return secondUUID, token
|
||||
}
|
||||
|
||||
func TestFilterComposedSemantics(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
libUUID, _ := builderFixtureLib(t, setup, userID, "Sem")
|
||||
|
||||
titles := func(qf string) []string {
|
||||
p := url.Values{"library_id": {libUUID.String()}, "qf": {qf}}
|
||||
status, rows := qfSearch(t, setup, setup.RegularToken, p)
|
||||
// 404 is the endpoint's pinned no-results contract — an empty set.
|
||||
require.Contains(t, []int{http.StatusOK, http.StatusNotFound}, status)
|
||||
return qfTitles(t, rows)
|
||||
}
|
||||
|
||||
// Plain rule: author contains (fuzzy, per-element).
|
||||
got := titles(qfAll(qfRule("author", "contains", "Voss")))
|
||||
assert.ElementsMatch(t, []string{"Sem Tide", "Sem Stone"}, got, "author contains")
|
||||
|
||||
// Negation as operator: doesn't contain.
|
||||
got = titles(qfAll(qfRule("author", "not_contains", "Voss")))
|
||||
assert.ElementsMatch(t, []string{"Sem Fen", "Sem Wick"}, got, "author not_contains")
|
||||
|
||||
// ALL group = AND.
|
||||
got = titles(qfAll(
|
||||
qfRule("author", "contains", "Voss"),
|
||||
qfRule("genre", "contains", "fantasy"),
|
||||
))
|
||||
assert.Equal(t, []string{"Sem Tide"}, got, "ALL group")
|
||||
|
||||
// ANY group = OR.
|
||||
got = titles(qfAny(
|
||||
qfRule("genre", "is", "history"),
|
||||
qfRule("abridged", "is_true"),
|
||||
))
|
||||
assert.ElementsMatch(t, []string{"Sem Stone", "Sem Fen", "Sem Wick"}, got, "ANY group")
|
||||
|
||||
// Nesting: group ANDed into its parent.
|
||||
got = titles(qfAll(
|
||||
qfRule("author", "contains", "Voss"),
|
||||
map[string]interface{}{"mode": "any", "children": []map[string]interface{}{
|
||||
qfRule("genre", "is", "history"),
|
||||
qfRule("page_count", "at_least", "400"),
|
||||
}},
|
||||
))
|
||||
assert.Equal(t, []string{"Sem Stone"}, got, "nested ANY inside ALL")
|
||||
|
||||
// Negated subgroup inside ALL: author Voss AND NOT(genre fantasy).
|
||||
got = titles(qfAll(
|
||||
qfRule("author", "contains", "Voss"),
|
||||
map[string]interface{}{"mode": "any", "children": []map[string]interface{}{
|
||||
qfRule("genre", "not_contains", "fantasy"),
|
||||
qfRule("title", "contains", "zzz-no-match"),
|
||||
}},
|
||||
))
|
||||
assert.Equal(t, []string{"Sem Stone"}, got, "negation inside nested group")
|
||||
|
||||
// Array exact + negation.
|
||||
got = titles(qfAll(qfRule("tags", "is", "epic")))
|
||||
assert.ElementsMatch(t, []string{"Sem Tide", "Sem Fen"}, got, "tags is")
|
||||
got = titles(qfAll(qfRule("tags", "is_not", "epic")))
|
||||
assert.ElementsMatch(t, []string{"Sem Stone", "Sem Wick"}, got, "tags isn't")
|
||||
|
||||
// Numbers: between / at_most.
|
||||
got = titles(qfAll(qfRule("page_count", "between", "250", "550")))
|
||||
assert.ElementsMatch(t, []string{"Sem Tide", "Sem Stone"}, got, "page_count between")
|
||||
got = titles(qfAll(qfRule("page_count", "at_most", "250")))
|
||||
assert.ElementsMatch(t, []string{"Sem Fen"}, got, "page_count at_most")
|
||||
|
||||
// Empty operators.
|
||||
got = titles(qfAll(qfRule("series", "empty")))
|
||||
assert.ElementsMatch(t, []string{"Sem Fen", "Sem Wick"}, got, "series is empty")
|
||||
got = titles(qfAll(qfRule("isbn", "missing")))
|
||||
assert.ElementsMatch(t, []string{"Sem Stone", "Sem Fen", "Sem Wick"}, got, "isbn missing")
|
||||
|
||||
// Enums exact + isn't.
|
||||
got = titles(qfAll(qfRule("age_rating", "is", "Teen")))
|
||||
assert.Empty(t, got, "age_rating is (none set)")
|
||||
got = titles(qfAll(qfRule("mime_type", "is", "application/epub+zip")))
|
||||
assert.Len(t, got, 4, "mime_type is")
|
||||
|
||||
// Series full membership: the alternate_info series matches too.
|
||||
ctx := context.Background()
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`UPDATE media_items SET alternate_info = '{"alternate_series":"Quill Chronicles"}'::jsonb
|
||||
WHERE title = $1`, "Sem Fen")
|
||||
require.NoError(t, err)
|
||||
got = titles(qfAll(qfRule("series", "contains", "Quill Chron")))
|
||||
assert.Equal(t, []string{"Sem Fen"}, got, "series matches alternate_info membership")
|
||||
got = titles(qfAll(qfRule("series", "is", "Quill Chronicles")))
|
||||
assert.Equal(t, []string{"Sem Fen"}, got, "series exact matches alternate_info membership")
|
||||
|
||||
// Dates: within the last N days (all fixtures created now).
|
||||
got = titles(qfAll(qfRule("added_at", "within_days", "7")))
|
||||
assert.Len(t, got, 4, "added_at within the last 7 days")
|
||||
|
||||
// Identifiers: is / isn't (isn't includes books without one).
|
||||
got = titles(qfAll(qfRule("isbn", "is", "1111111111111")))
|
||||
assert.Equal(t, []string{"Sem Tide"}, got, "isbn is")
|
||||
got = titles(qfAll(qfRule("isbn", "is_not", "1111111111111")))
|
||||
assert.Len(t, got, 3, "isbn isn't includes books without an ISBN")
|
||||
|
||||
// Presence.
|
||||
got = titles(qfAll(qfRule("cover", "missing")))
|
||||
assert.Len(t, got, 4, "cover missing")
|
||||
|
||||
// Published year (flat year-range semantics: date_published falls
|
||||
// back to copyright_year).
|
||||
got = titles(qfAll(qfRule("published_year", "at_least", "1990")))
|
||||
assert.Equal(t, []string{"Sem Tide"}, got, "published_year at_least")
|
||||
}
|
||||
|
||||
func TestFilterUserRelativeFields(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userA := getRegularUserID(t, setup.DB)
|
||||
userBUUID, tokenB := secondBuilderUser(t, setup)
|
||||
ctx := context.Background()
|
||||
|
||||
libUUID, _ := builderFixtureLib(t, setup, userA, "Mine")
|
||||
|
||||
// Resolve the fixture media item ids by title.
|
||||
itemID := func(title string) pgtype.UUID {
|
||||
var id pgtype.UUID
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`SELECT id FROM media_items WHERE title = $1`, title).Scan(&id)
|
||||
require.NoError(t, err, "fixture item %s", title)
|
||||
return id
|
||||
}
|
||||
tideID := itemID("Mine Tide")
|
||||
stoneID := itemID("Mine Stone")
|
||||
fenID := itemID("Mine Fen")
|
||||
wickID := itemID("Mine Wick")
|
||||
|
||||
toPg := func(id uuid.UUID) pgtype.UUID { return pgtype.UUID{Bytes: id, Valid: true} }
|
||||
aID := toPg(userA)
|
||||
bID := toPg(userBUUID)
|
||||
|
||||
// Reader A: reading Tide (50%), finished Stone, rated Tide 8,
|
||||
// reviewed Fen, noted on Tide ("the catharsis scene"), highlighted
|
||||
// Stone, time on Wick (2h), collection holding Tide.
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at) VALUES ($1,$2,0.5, now()), ($3,$4,1.0, now())`,
|
||||
tideID, aID, stoneID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at) VALUES ($1,$2,1.0, now())`,
|
||||
tideID, bID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_ratings (media_item_id, user_id, rating) VALUES ($1,$2,8)`, tideID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_ratings (media_item_id, user_id, rating) VALUES ($1,$2,3)`, tideID, bID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_reviews (media_item_id, user_id, review) VALUES ($1,$2,'splendid')`, fenID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_notes (media_item_id, user_id, content) VALUES ($1,$2,'remember the catharsis scene')`, tideID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_highlights (media_item_id, user_id, selection_text) VALUES ($1,$2,'the dark forest road')`, stoneID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO reading_history (user_id, media_item_id, time_spent_seconds, pages_read) VALUES ($1,$2,7200,120), ($3,$4,600,30)`,
|
||||
aID, wickID, aID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
var collID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO collections (user_id, name) VALUES ($1, $2) RETURNING id`,
|
||||
aID, fmt.Sprintf("Test Builder Coll %d", time.Now().UnixNano())).Scan(&collID)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteCollection(ctx, collID) })
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO collection_items (collection_id, media_item_id) VALUES ($1,$2)`, collID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
titlesFor := func(token string, qf string) []string {
|
||||
p := url.Values{"library_id": {libUUID.String()}, "qf": {qf}}
|
||||
status, rows := qfSearch(t, setup, token, p)
|
||||
// 404 = the pinned no-results contract (an empty set).
|
||||
require.Contains(t, []int{http.StatusOK, http.StatusNotFound}, status)
|
||||
return qfTitles(t, rows)
|
||||
}
|
||||
|
||||
// Status: the "my data" weld — same tree, different readers.
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("status", "is", "reading"))), "A: Tide is in progress")
|
||||
assert.Empty(t, titlesFor(tokenB, qfAll(qfRule("status", "is", "reading"))), "B: nothing in progress")
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(tokenB, qfAll(qfRule("status", "is", "finished"))), "B finished Tide; A did not")
|
||||
assert.ElementsMatch(t, []string{"Mine Fen", "Mine Wick"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("status", "is", "unread"))), "A unread")
|
||||
|
||||
// Progress percent.
|
||||
assert.ElementsMatch(t, []string{"Mine Stone"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("progress_percent", "at_least", "60"))))
|
||||
|
||||
// Last read.
|
||||
assert.Len(t, titlesFor(setup.RegularToken, qfAll(qfRule("last_read", "within_days", "7"))), 2)
|
||||
|
||||
// My rating: two worlds.
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("my_rating", "at_least", "5"))))
|
||||
assert.Empty(t, titlesFor(tokenB, qfAll(qfRule("my_rating", "at_least", "5"))), "B rated 3")
|
||||
assert.ElementsMatch(t, []string{"Mine Fen", "Mine Stone", "Mine Wick"},
|
||||
titlesFor(tokenB, qfAll(qfRule("my_rating", "unrated"))), "B rated only Tide")
|
||||
|
||||
// Review presence.
|
||||
assert.ElementsMatch(t, []string{"Mine Fen"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("review", "has"))))
|
||||
assert.Empty(t, titlesFor(tokenB, qfAll(qfRule("review", "has"))))
|
||||
|
||||
// Notes presence + recency.
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("notes", "has"))))
|
||||
|
||||
// Note text: substring pin + the weld. A's note says "catharsis":
|
||||
// searching "catharsis" matches, "Carthage" does NOT (substring,
|
||||
// not fuzzy), and B — no notes — sees nothing either way.
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("note_text", "contains", "catharsis"))))
|
||||
assert.Empty(t, titlesFor(setup.RegularToken, qfAll(qfRule("note_text", "contains", "Carthage"))),
|
||||
"substring semantics: catharsis ≠ Carthage")
|
||||
assert.Empty(t, titlesFor(tokenB, qfAll(qfRule("note_text", "contains", "catharsis"))),
|
||||
"my data: B's notes are the only notes in B's world")
|
||||
|
||||
// Highlight text.
|
||||
assert.ElementsMatch(t, []string{"Mine Stone"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("highlight_text", "contains", "dark forest"))))
|
||||
|
||||
// Time spent (minutes) + pages read.
|
||||
assert.ElementsMatch(t, []string{"Mine Wick"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("time_spent", "at_least", "60"))))
|
||||
assert.ElementsMatch(t, []string{"Mine Wick"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("pages_read", "at_least", "100"))))
|
||||
|
||||
// Collections: in / in none.
|
||||
assert.ElementsMatch(t, []string{"Mine Tide"},
|
||||
titlesFor(setup.RegularToken, qfAll(qfRule("collection", "in_any"))))
|
||||
assert.Len(t, titlesFor(tokenB, qfAll(qfRule("collection", "in_none"))), 4,
|
||||
"B is in no collections")
|
||||
|
||||
// Library field (holding structure): the fixture library holds all
|
||||
// four books.
|
||||
assert.Len(t, titlesFor(setup.RegularToken, qfAll(qfRule("library", "is", libUUID.String()))), 4)
|
||||
}
|
||||
|
||||
func TestFilterSuggestionsUnderRules(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
libUUID, _ := builderFixtureLib(t, setup, userID, "Sug")
|
||||
|
||||
// Add a second distinct author+genre pair (trigram-disjoint).
|
||||
ctx := context.Background()
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: "Sug Dune",
|
||||
Authors: []string{"Zephyrine Hale"},
|
||||
Genre: pgtype.Text{String: "spacewestern", Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/builder/sug-dune-%d.epub", time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 128, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
suggest := func(field, typed, qf string) (int, []map[string]interface{}) {
|
||||
p := url.Values{"library_id": {libUUID.String()}, field: {typed}, "limit": {"20"}}
|
||||
if qf != "" {
|
||||
p.Set("qf", qf)
|
||||
}
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?"+p.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return resp.StatusCode, nil
|
||||
}
|
||||
var body struct {
|
||||
Results []map[string]interface{} `json:"results"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
return resp.StatusCode, body.Results
|
||||
}
|
||||
valuesOf := func(results []map[string]interface{}) []string {
|
||||
out := make([]string, 0, len(results))
|
||||
for _, r := range results {
|
||||
v, _ := r["value"].(string)
|
||||
out = append(out, v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Typed genre menu: both genres surface for their prefixes.
|
||||
status, results := suggest("genre", "fan", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "fantasy")
|
||||
|
||||
// The two-worlds example: author contains Voss → the genre menu is
|
||||
// HER genres only; author DOESN'T contain → her-only genres absent.
|
||||
status, results = suggest("genre", "fan", qfAll(qfRule("author", "contains", "Voss")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "fantasy")
|
||||
|
||||
// NOTE: "fan" + not_contains STILL lists fantasy — Quill's Fen is
|
||||
// fantasy too; the genre is not author-exclusive. The clean
|
||||
// two-worlds demo is spacewestern (only Hale's book), below.
|
||||
|
||||
// ...and the OTHER world's genre appears only when Voss is excluded.
|
||||
status, results = suggest("genre", "space", qfAll(qfRule("author", "not_contains", "Voss")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "spacewestern", "the other world's genre surfaces under negation")
|
||||
status, results = suggest("genre", "space", qfAll(qfRule("author", "contains", "Voss")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.NotContains(t, valuesOf(results), "spacewestern", "zero-match values never offered")
|
||||
|
||||
// Suggestions are symmetric: the genre rule constrains the AUTHOR
|
||||
// menu just the same.
|
||||
status, results = suggest("author", "zeph", qfAll(qfRule("genre", "is", "spacewestern")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "Zephyrine Hale")
|
||||
status, results = suggest("author", "amara", qfAll(qfRule("genre", "is", "spacewestern")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.NotContains(t, valuesOf(results), "Amara Voss", "zero-match values never offered")
|
||||
|
||||
// BROWSE (the user's click-the-box case): a blank typed query lists
|
||||
// the field's top values UNDER the other rules — "assign an author,
|
||||
// click publisher, see their publishers" without typing anything.
|
||||
// The wire sends a single space as the sentinel.
|
||||
status, results = suggest("genre", " ", qfAll(qfRule("author", "contains", "Quill")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "fantasy", "browse shows the field's values under the rules")
|
||||
assert.NotContains(t, valuesOf(results), "spacewestern", "zero-match values never offered")
|
||||
assert.NotContains(t, valuesOf(results), "history", "values outside the rules never offered")
|
||||
|
||||
// Browse with NO other rules in place: the blank query still lists
|
||||
// the field's values (the static similarity path would answer a
|
||||
// single space with nothing). Regression for the empty-browse bug.
|
||||
status, results = suggest("genre", " ", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "fantasy", "browse with no rules lists the field's values")
|
||||
status, results = suggest("author", " ", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "Amara Voss", "author browse with no rules")
|
||||
|
||||
// Library field menu: visibility-scoped, id values + name display
|
||||
// (this test's fixture libraries are named "Test Builder Sug …").
|
||||
status, results = suggest("library", "Test Builder", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
found := false
|
||||
for _, r := range results {
|
||||
if r["value"] == libUUID.String() {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
assert.True(t, found, "the visible library is offered")
|
||||
|
||||
// BROWSE the library field UNDER rules (the blank-typed click): the
|
||||
// id-source query must not choke on the unused typed parameter and
|
||||
// must list exactly the library holding the surviving books.
|
||||
status, results = suggest("library", " ", qfAll(qfRule("author", "contains", "Voss")))
|
||||
require.Equal(t, http.StatusOK, status, "library browse under rules (blank typed)")
|
||||
require.Len(t, results, 1, "exactly the fixture library survives the rules")
|
||||
assert.Equal(t, libUUID.String(), results[0]["value"])
|
||||
assert.InDelta(t, float64(2), results[0]["count"], 0.001, "both Voss books live there")
|
||||
|
||||
// Series browse (blank typed, under rules): full-membership source.
|
||||
status, results = suggest("series", " ", qfAll(qfRule("author", "contains", "Voss")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Contains(t, valuesOf(results), "Voss Saga")
|
||||
|
||||
// Added-by browse (blank typed): books without added_by → an empty
|
||||
// menu, but NO error (the parameter-discipline regression net).
|
||||
status, results = suggest("added_by", " ", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
|
||||
// Collection browse (blank typed): the container sources ride the
|
||||
// same id-source query — a collection holding a surviving book is
|
||||
// offered with the surviving count; zero-match containers never are.
|
||||
userA := getRegularUserID(t, setup.DB)
|
||||
var collID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO collections (user_id, name) VALUES ($1, $2) RETURNING id`,
|
||||
pgtype.UUID{Bytes: [16]byte(userA), Valid: true},
|
||||
fmt.Sprintf("Test Builder Sug Coll %d", time.Now().UnixNano())).Scan(&collID)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteCollection(ctx, collID) })
|
||||
var tideID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx, `SELECT id FROM media_items WHERE title = $1`, "Sug Tide").Scan(&tideID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO collection_items (collection_id, media_item_id) VALUES ($1, $2)`, collID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
status, results = suggest("collection", " ", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.True(t, func() bool {
|
||||
for _, r := range results {
|
||||
if r["value"] == collID.String() {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}(), "the collection holding a book is offered on browse")
|
||||
|
||||
status, results = suggest("collection", " ", qfAll(qfRule("genre", "is", "spacewestern")))
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.Empty(t, results, "zero-match containers never offered")
|
||||
}
|
||||
|
||||
func TestFilterFamilyLens(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err)
|
||||
|
||||
makeLib := func(name string) (uuid.UUID, pgtype.UUID) {
|
||||
lib, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: fmt.Sprintf("Test Lens %s %d", name, time.Now().UnixNano()),
|
||||
Description: pgtype.Text{String: "family lens test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(userID), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
id, err := uuid.FromBytes(lib.ID.Bytes[0:16])
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, lib.ID) })
|
||||
return id, lib.ID
|
||||
}
|
||||
_, visibleID := makeLib("Visible")
|
||||
hiddenUUID, hiddenID := makeLib("Hidden")
|
||||
|
||||
// One book in each, same author so the All-Books menu would
|
||||
// aggregate both if the lens failed.
|
||||
for _, lib := range []struct {
|
||||
id pgtype.UUID
|
||||
title string
|
||||
}{{visibleID, "Lens Vis Book"}, {hiddenID, "Lens Hid Book"}} {
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: lib.id, Title: lib.title,
|
||||
Authors: []string{"Lens Secret Author"},
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-%d.epub", lib.title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: 64, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO library_visibility (library_id, user_id, is_visible) VALUES ($1, $2, false)`,
|
||||
hiddenID, pgtype.UUID{Bytes: [16]byte(userID), Valid: true})
|
||||
require.NoError(t, err)
|
||||
|
||||
search := func(params url.Values) (int, []map[string]interface{}) {
|
||||
return qfSearch(t, setup, setup.RegularToken, params)
|
||||
}
|
||||
|
||||
// The composed search never sees the hidden library's books —
|
||||
// even when a rule names it explicitly.
|
||||
qfHide := qfAll(qfRule("library", "is", hiddenUUID.String()))
|
||||
status, rows := search(url.Values{"qf": {qfHide}})
|
||||
// 200-with-empty or the 404 no-results contract — either way: nothing.
|
||||
assert.Contains(t, []int{http.StatusOK, http.StatusNotFound}, status)
|
||||
assert.Empty(t, rows, "a rule naming a hidden library matches nothing")
|
||||
|
||||
// Explicit out-of-set scope + a tree: 403 (never silent).
|
||||
status, _ = search(url.Values{"qf": {qfAll(qfRule("author", "contains", "Lens"))}, "library_id": {hiddenUUID.String()}})
|
||||
assert.Equal(t, http.StatusForbidden, status)
|
||||
|
||||
// The library suggestion menu never offers hidden libraries (a
|
||||
// real typed prefix — empty typed values skip the autocomplete
|
||||
// branch entirely).
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/search?"+url.Values{
|
||||
"library": {"Test Lens"}, "limit": {"50"},
|
||||
}.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var body struct {
|
||||
Results []map[string]interface{} `json:"results"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
for _, r := range body.Results {
|
||||
assert.NotEqual(t, hiddenUUID.String(), r["value"], "hidden library never offered")
|
||||
}
|
||||
|
||||
// filter-count: out-of-set scope is a 403 too.
|
||||
req, _ = http.NewRequest("GET", setup.Server.URL+"/api/media-items/filter-count?qf="+url.QueryEscape(qfAll(qfRule("author", "contains", "Lens")))+"&library_id="+hiddenUUID.String(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err = http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
|
||||
}
|
||||
|
||||
func TestFilterQFValidationAndPrecedence(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
libUUID, _ := builderFixtureLib(t, setup, userID, "Val")
|
||||
|
||||
search := func(params url.Values) (int, []map[string]interface{}) {
|
||||
return qfSearch(t, setup, setup.RegularToken, params)
|
||||
}
|
||||
|
||||
// Unknown field → 400.
|
||||
status, _ := search(url.Values{"qf": {qfAll(qfRule("not_a_field", "is", "x"))}})
|
||||
assert.Equal(t, http.StatusBadRequest, status, "unknown field is a 400")
|
||||
|
||||
// Unknown operator → 400.
|
||||
status, _ = search(url.Values{"qf": {qfAll(qfRule("author", "explodes", "x"))}})
|
||||
assert.Equal(t, http.StatusBadRequest, status, "unknown operator is a 400")
|
||||
|
||||
// Operator not allowed on the field → 400.
|
||||
status, _ = search(url.Values{"qf": {qfAll(qfRule("page_count", "contains", "3"))}})
|
||||
assert.Equal(t, http.StatusBadRequest, status, "operator not allowed on field")
|
||||
|
||||
// between with one value → 400.
|
||||
status, _ = search(url.Values{"qf": {qfAll(qfRule("page_count", "between", "3"))}})
|
||||
assert.Equal(t, http.StatusBadRequest, status, "between needs two values")
|
||||
|
||||
// Malformed JSON → 400.
|
||||
status, _ = search(url.Values{"qf": {"{not json"}})
|
||||
assert.Equal(t, http.StatusBadRequest, status)
|
||||
|
||||
// SQL injection in a VALUE is just a value (bound parameter): no
|
||||
// error, no match.
|
||||
inj := qfAll(qfRule("author", "contains", "x' OR 1=1 --"))
|
||||
status, rows := search(url.Values{"qf": {inj}})
|
||||
assert.Contains(t, []int{http.StatusOK, http.StatusNotFound}, status,
|
||||
"injection attempt is a bound value, not SQL")
|
||||
assert.Empty(t, rows)
|
||||
|
||||
// Precedence: a URL carrying BOTH qf and flat params — qf wins.
|
||||
p := url.Values{
|
||||
"library_id": {libUUID.String()},
|
||||
"qf": {qfAll(qfRule("author", "contains", "Voss"))},
|
||||
"author_filter": {"Coraline Dar"},
|
||||
}
|
||||
status, rows = search(p)
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
assert.ElementsMatch(t, []string{"Val Tide", "Val Stone"}, qfTitles(t, rows),
|
||||
"qf wins over flat on a hand-crafted URL")
|
||||
|
||||
// Live count endpoint: rules + scope, a true COUNT.
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items/filter-count?"+url.Values{
|
||||
"qf": {qfAll(qfRule("author", "contains", "Voss"))}, "library_id": {libUUID.String()},
|
||||
}.Encode(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var body struct {
|
||||
Count int `json:"count"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
assert.Equal(t, 2, body.Count, "live count is the true match count")
|
||||
|
||||
// Empty group → 400 (parse validation).
|
||||
status, _ = search(url.Values{"qf": {`{"mode":"all","children":[]}`}})
|
||||
assert.Equal(t, http.StatusBadRequest, status)
|
||||
}
|
||||
|
||||
func TestFilterFlatEquivalence(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
libUUID, _ := builderFixtureLib(t, setup, userID, "Eqv")
|
||||
scope := url.Values{"library_id": {libUUID.String()}}
|
||||
|
||||
// The tree and its flat deep link return the SAME rows (the smart
|
||||
// serialization promise: flat links keep their semantics).
|
||||
statusFlat, flatRows := qfSearch(t, setup, setup.RegularToken,
|
||||
mergeParams(scope, url.Values{"author_filter": {"Voss"}}))
|
||||
require.Equal(t, http.StatusOK, statusFlat)
|
||||
statusTree, treeRows := qfSearch(t, setup, setup.RegularToken,
|
||||
mergeParams(scope, url.Values{"qf": {qfAll(qfRule("author", "contains", "Voss"))}}))
|
||||
require.Equal(t, http.StatusOK, statusTree)
|
||||
assert.ElementsMatch(t, qfTitles(t, flatRows), qfTitles(t, treeRows),
|
||||
"tree and flat link return identical rows")
|
||||
|
||||
// Enum: exact flat vs tree is (both may 404-no-results — the empty
|
||||
// set on both sides is still equivalence).
|
||||
statusFlat, flatRows = qfSearch(t, setup, setup.RegularToken,
|
||||
mergeParams(scope, url.Values{"age_rating_filter": {"Teen"}}))
|
||||
statusTree, treeRows = qfSearch(t, setup, setup.RegularToken,
|
||||
mergeParams(scope, url.Values{"qf": {qfAll(qfRule("age_rating", "is", "Teen"))}}))
|
||||
require.Equal(t, statusFlat, statusTree, "same result status")
|
||||
assert.ElementsMatch(t, qfTitles(t, flatRows), qfTitles(t, treeRows))
|
||||
}
|
||||
@@ -88,7 +88,7 @@ func (s *OwnershipAndAdminTestSuite) createItemViaDB(withFolder bool) (itemID, l
|
||||
item, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Ownership Suite Item",
|
||||
Author: pgtype.Text{String: "Suite Author", Valid: true},
|
||||
Authors: []string{"Suite Author"},
|
||||
FilePath: relPath,
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
@@ -183,8 +183,10 @@ func (s *OwnershipAndAdminTestSuite) TestHighlightDelete_CrossUserNoOp() {
|
||||
// create accounts even with self-service registration disabled, regular
|
||||
// users cannot, and duplicates conflict.
|
||||
func (s *OwnershipAndAdminTestSuite) TestAdminCreateUser() {
|
||||
suffix := fmt.Sprintf("%d", s.setup.DBPool.Config().MaxConns) // stable-ish uniquifier
|
||||
unique := fmt.Sprintf("admincreated%s", suffix)
|
||||
// Real unique suffix: MaxConns is forced to 1 by the harness, so the old
|
||||
// "stable-ish" uniquifier produced the same user every run and only
|
||||
// survived on the setup-time cleanup sweep of the NEXT run.
|
||||
unique := fmt.Sprintf("admincreated-%s", uuid.New().String()[:8])
|
||||
|
||||
// Regular user token → 403 (AdminMiddleware).
|
||||
payload, _ := json.Marshal(map[string]interface{}{
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
package main
|
||||
|
||||
// Phase 13 (faceted plan): force rescan = original-scan semantics, and
|
||||
// the manually_edited filter.
|
||||
//
|
||||
// Force rescan contract: re-review every field on every book from all
|
||||
// sources (sidecars → folder structure → tags → fallbacks) on multi-
|
||||
// file books too — the pre-Phase-13 skip is gone — with
|
||||
// metadata_overrides as the only protection. A force pass also stamps
|
||||
// the fresh manifest with the current parser version (success-only)
|
||||
// and closes the book's open ledger issues.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/services"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// pinExtractorVersionOne pins the SHARED dev DB's extractor version to
|
||||
// the default 1 for the test's duration. Prior interrupted runs can
|
||||
// leave it bumped (an early test failure aborts before its restore
|
||||
// cleanup registers), and every stamp assertion in these tests assumes
|
||||
// the default. Restore-to-prior rides inside setExtractorVersion: the
|
||||
// suite leaves the setting exactly as it found it.
|
||||
func pinExtractorVersionOne(t *testing.T, setup *TestServerSetup) {
|
||||
t.Helper()
|
||||
setExtractorVersion(t, setup, 1)
|
||||
}
|
||||
|
||||
// forceScanTestFolder runs a FORCE scan over one folder.
|
||||
func forceScanTestFolder(t *testing.T, setup *TestServerSetup, adminID pgtype.UUID, folder string) {
|
||||
t.Helper()
|
||||
scanner := services.NewMediaScanner(setup.DB)
|
||||
defer func() { _ = scanner.Close() }()
|
||||
scanner.SetAdminID(adminID)
|
||||
require.NoError(t, scanner.SetFolders([]string{folder}, false))
|
||||
scanner.SetForce(true)
|
||||
require.NoError(t, scanner.ScanFolders(context.Background()))
|
||||
}
|
||||
|
||||
func TestForceRescanMultifileSidecarApply(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
pinExtractorVersionOne(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Force Rescan %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrack(t, filepath.Join(folder, "Force Author", "Force Title", "01 First.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "Force Author", "Force Title", "02 Second.mp3"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
rel := filepath.ToSlash("Force Author/Force Title")
|
||||
book, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found, "folder book must import")
|
||||
require.Equal(t, "Force Title", book.Title)
|
||||
|
||||
// Corrupt, then FORCE: the pre-Phase-13 skip is gone — the full
|
||||
// chain re-derives the title from the folder basename.
|
||||
corruptItemColumn(t, setup, book.ID, "title", "ZZZ")
|
||||
forceScanTestFolder(t, setup, admin, folder)
|
||||
book, found = findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Force Title", book.Title, "force rescan must re-apply folder-book metadata")
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, book.ID), "force rescan stamps the current parser version")
|
||||
|
||||
// metadata_overrides is the ONLY protection: a recorded override
|
||||
// survives the force pass untouched.
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`UPDATE media_items SET metadata_overrides = ARRAY['title'], title = 'Manual Title' WHERE id = $1`,
|
||||
book.ID)
|
||||
require.NoError(t, err)
|
||||
forceScanTestFolder(t, setup, admin, folder)
|
||||
book, _ = findAudioItemByPath(t, setup, libID, rel)
|
||||
require.Equal(t, "Manual Title", book.Title, "metadata_overrides protect manual edits under force rescan")
|
||||
}
|
||||
|
||||
func TestForceRescanSingleFileSidecarReapply(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
pinExtractorVersionOne(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Force Single %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
bookDir := filepath.Join(folder, "Force Probe Author", "Force Probe Book")
|
||||
require.NoError(t, os.MkdirAll(bookDir, 0o755))
|
||||
opf := `<?xml version='1.0' encoding='utf-8'?>
|
||||
<package xmlns="http://www.idpf.org/2007/opf" version="2.0">
|
||||
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
|
||||
<dc:title>Force Probe Book</dc:title>
|
||||
<dc:creator>Force Probe Author</dc:creator>
|
||||
</metadata>
|
||||
</package>`
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "metadata.opf"), []byte(opf), 0o644))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "Force Probe Book.epub"), []byte("dummy epub content"), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
rel := filepath.ToSlash("Force Probe Author/Force Probe Book/Force Probe Book.epub")
|
||||
book, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found)
|
||||
|
||||
// Single-file force rescan already ran the full chain — pin it, and
|
||||
// pin the NEW part: the manifest comes out stamped.
|
||||
corruptItemColumn(t, setup, book.ID, "title", "ZZZ")
|
||||
forceScanTestFolder(t, setup, admin, folder)
|
||||
book, found = findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Force Probe Book", book.Title, "force rescan re-derives sidecar metadata (single-file)")
|
||||
require.Equal(t, 1, storedManifestEv(t, setup, book.ID), "force rescan stamps the manifest")
|
||||
}
|
||||
|
||||
func TestManuallyEditedFilter(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
libID, _ := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Manual Edits %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
createBook := func(path string) pgtype.UUID {
|
||||
created, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: libID,
|
||||
Title: path,
|
||||
Authors: []string{},
|
||||
FilePath: path,
|
||||
FileSize: pgtype.Int8{Int64: 64, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
return created.ID
|
||||
}
|
||||
editedID := createBook("edited-book.epub")
|
||||
cleanID := createBook("clean-book.epub")
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`UPDATE media_items SET metadata_overrides = ARRAY['title'] WHERE id = $1`,
|
||||
editedID)
|
||||
require.NoError(t, err)
|
||||
|
||||
qfIDs := func(op string) []string {
|
||||
status, rows := qfSearch(t, setup, setup.Token, map[string][]string{
|
||||
"qf": {qfAll(qfRule("manually_edited", op))},
|
||||
"library_id": {libID.String()},
|
||||
})
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
ids := make([]string, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
ids = append(ids, r["id"].(string))
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
trueIDs := qfIDs("is_true")
|
||||
require.Len(t, trueIDs, 1)
|
||||
assert.Equal(t, editedID.String(), trueIDs[0], "is_true matches only manually-edited books")
|
||||
|
||||
falseIDs := qfIDs("is_false")
|
||||
require.Len(t, falseIDs, 1)
|
||||
assert.Equal(t, cleanID.String(), falseIDs[0], "is_false matches only untouched books")
|
||||
|
||||
// The deep-linkable JSON shape round-trips through the qf parser
|
||||
// (arity-none rule: no values on the wire).
|
||||
var parsed struct {
|
||||
Mode string `json:"mode"`
|
||||
Children []struct {
|
||||
Field string `json:"field"`
|
||||
} `json:"children"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal([]byte(qfAll(qfRule("manually_edited", "is_true"))), &parsed))
|
||||
require.Len(t, parsed.Children, 1)
|
||||
assert.Equal(t, "manually_edited", parsed.Children[0].Field)
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// A2c end-to-end: an ABS sidecar whose series array mixes a real series and
|
||||
// an ordering construct must (a) land the real series on the item, (b)
|
||||
// auto-create a curated playlist collection for the ordering construct with
|
||||
// provenance + position, (c) survive a rescan without duplicates, (d)
|
||||
// support owner-only reorder, and (e) filter finished books for Up Next /
|
||||
// hide_finished with a completion marker when everything is read.
|
||||
func TestPlaylistCollectionsEndToEnd(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
adminID, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Playlist %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// The user's exact Witcher shape: real series + reading-order construct.
|
||||
itemDir := filepath.Join(folder, "Tag Author", "The Tagged Album")
|
||||
fixtureAudio(t, "tagged.mp3", filepath.Join(itemDir, "tagged.mp3"))
|
||||
sidecar := `{
|
||||
"title": "The Tagged Album",
|
||||
"authors": ["Tag Author"],
|
||||
"series": ["The Tagged Saga #2", "The Tagged Saga (Reading Order) #7"]
|
||||
}`
|
||||
require.NoError(t, os.WriteFile(filepath.Join(itemDir, "metadata.json"), []byte(sidecar), 0644))
|
||||
|
||||
scanTestFolder(t, setup, adminID, folder)
|
||||
|
||||
// (a) The item carries the REAL series only; the reading-order entry
|
||||
// never touches the series columns.
|
||||
var series string
|
||||
var seriesNum float64
|
||||
var foundPath string
|
||||
err := setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT series, series_number, file_path FROM media_items WHERE title = 'The Tagged Album'`,
|
||||
).Scan(&series, &seriesNum, &foundPath)
|
||||
if err != nil {
|
||||
var paths []string
|
||||
rows, qerr := setup.DBPool.Query(context.Background(),
|
||||
`SELECT file_path FROM media_items WHERE library_id = $1`, libID)
|
||||
if qerr == nil {
|
||||
for rows.Next() {
|
||||
var p string
|
||||
_ = rows.Scan(&p)
|
||||
paths = append(paths, p)
|
||||
}
|
||||
rows.Close()
|
||||
}
|
||||
t.Fatalf("item not found by title (paths in library: %v): %v", paths, err)
|
||||
}
|
||||
require.Equal(t, "The Tagged Saga", series)
|
||||
require.Equal(t, 2.0, seriesNum)
|
||||
require.NotEmpty(t, foundPath)
|
||||
|
||||
// (b) The playlist collection exists with provenance, dashboard pin,
|
||||
// and the item at reading position 7.
|
||||
adminUUIDStr := adminID.String()
|
||||
var collID uuid.UUID
|
||||
var collName, collDesc string
|
||||
var showOnDashboard bool
|
||||
var viewSettingsRaw []byte
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT id, name, description, show_on_dashboard, view_settings
|
||||
FROM collections WHERE user_id = $1 AND name = 'The Tagged Saga (Reading Order)'`,
|
||||
adminID).Scan(&collID, &collName, &collDesc, &showOnDashboard, &viewSettingsRaw))
|
||||
require.True(t, showOnDashboard, "playlists are dashboard-eligible by design")
|
||||
require.Contains(t, string(viewSettingsRaw), "imported_from_series")
|
||||
require.Contains(t, string(viewSettingsRaw), "show_up_next")
|
||||
require.Contains(t, collDesc, "safe to edit or delete")
|
||||
_ = adminUUIDStr
|
||||
|
||||
var position *int32
|
||||
var itemCount int
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT ci.position, COUNT(*) OVER () FROM collection_items ci
|
||||
WHERE ci.collection_id = $1`, collID).Scan(&position, &itemCount))
|
||||
require.Equal(t, 1, itemCount)
|
||||
require.NotNil(t, position)
|
||||
require.Equal(t, int32(7), *position)
|
||||
_ = adminUUIDStr
|
||||
|
||||
// (c) Rescan: no duplicate collection, no duplicate membership.
|
||||
scanTestFolder(t, setup, adminID, folder)
|
||||
var collectionCount, membershipCount int
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collections WHERE user_id = $1 AND name = 'The Tagged Saga (Reading Order)'`,
|
||||
adminID).Scan(&collectionCount))
|
||||
require.Equal(t, 1, collectionCount)
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collection_items WHERE collection_id = $1`, collID).Scan(&membershipCount))
|
||||
require.Equal(t, 1, membershipCount)
|
||||
|
||||
// (d) Owner-only reorder: positions become 1..n; a non-owner gets 403.
|
||||
var itemID uuid.UUID
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT id FROM media_items WHERE title = 'The Tagged Album'`).Scan(&itemID))
|
||||
|
||||
reorderURL := fmt.Sprintf("%s/api/collections/%s/items/order", setup.Server.URL, collID.String())
|
||||
body, _ := json.Marshal(map[string][]string{"item_ids": {itemID.String()}})
|
||||
req, _ := http.NewRequest("PUT", reorderURL, bytes.NewBuffer(body))
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
_ = resp.Body.Close()
|
||||
|
||||
regularToken := setup.RegularToken
|
||||
req2, _ := http.NewRequest("PUT", reorderURL, bytes.NewBuffer(body))
|
||||
req2.Header.Set("Authorization", "Bearer "+regularToken)
|
||||
req2.Header.Set("Content-Type", "application/json")
|
||||
resp2, err := http.DefaultClient.Do(req2)
|
||||
require.NoError(t, err)
|
||||
require.Equal(t, http.StatusForbidden, resp2.StatusCode)
|
||||
_ = resp2.Body.Close()
|
||||
|
||||
// (e) hide_finished + Up Next + completion marker.
|
||||
// Flip the collection's flag on, mark the single item finished for the
|
||||
// admin, and the dashboard section becomes Complete with no items.
|
||||
_, err = setup.DBPool.Exec(context.Background(),
|
||||
`UPDATE collections SET view_settings = view_settings || '{"hide_finished": true}'::jsonb WHERE id = $1`, collID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(context.Background(),
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at)
|
||||
VALUES ($1, $2, 1.0, NOW()) ON CONFLICT (media_item_id, user_id) DO UPDATE SET percentage = 1.0`,
|
||||
itemID, adminID)
|
||||
require.NoError(t, err)
|
||||
|
||||
sectionsURL := fmt.Sprintf("%s/api/dashboard/sections?library_id=%s", setup.Server.URL, libID.String())
|
||||
req3, _ := http.NewRequest("GET", sectionsURL, nil)
|
||||
req3.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp3, err := http.DefaultClient.Do(req3)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp3.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp3.StatusCode)
|
||||
|
||||
var payload struct {
|
||||
Sections []struct {
|
||||
CollectionID string `json:"collection_id"`
|
||||
Complete bool `json:"complete"`
|
||||
IsPlaylist bool `json:"is_playlist"`
|
||||
Items []map[string]any `json:"items"`
|
||||
} `json:"sections"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp3.Body).Decode(&payload))
|
||||
|
||||
var found bool
|
||||
for _, s := range payload.Sections {
|
||||
if s.CollectionID == collID.String() {
|
||||
found = true
|
||||
if !s.Complete {
|
||||
b, _ := json.Marshal(s)
|
||||
t.Fatalf("section found but not complete: %s", b)
|
||||
}
|
||||
require.True(t, s.IsPlaylist)
|
||||
require.Empty(t, s.Items, "finished items must not show under hide_finished")
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
b, _ := json.Marshal(payload.Sections)
|
||||
t.Fatalf("playlist section missing; sections = %s", b)
|
||||
}
|
||||
require.True(t, found, "the playlist section must appear on the owner's dashboard")
|
||||
|
||||
// Up Next on the collection detail: the finished queue is empty.
|
||||
detailURL := fmt.Sprintf("%s/api/collections/%s", setup.Server.URL, collID.String())
|
||||
req4, _ := http.NewRequest("GET", detailURL, nil)
|
||||
req4.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp4, err := http.DefaultClient.Do(req4)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp4.Body.Close() }()
|
||||
var detail struct {
|
||||
UpNext []map[string]any `json:"up_next"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp4.Body).Decode(&detail))
|
||||
require.Empty(t, detail.UpNext, "nothing is up next once everything is finished")
|
||||
}
|
||||
@@ -1,12 +1,22 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
@@ -119,8 +129,8 @@ func TestProcessingIssuesListInputValidation(t *testing.T) {
|
||||
{
|
||||
name: "Nonexistent library UUID",
|
||||
libraryID: "00000000-0000-0000-0000-000000000000",
|
||||
expectedStatus: http.StatusOK,
|
||||
description: "Should return empty array for valid but nonexistent UUID",
|
||||
expectedStatus: http.StatusForbidden,
|
||||
description: "Family lens: an id outside the visible set is a 403, never a silent pass (nonexistent is trivially out-of-set)",
|
||||
},
|
||||
{
|
||||
name: "Special characters in UUID",
|
||||
@@ -293,8 +303,8 @@ func TestProcessingIssueStatsInputValidation(t *testing.T) {
|
||||
{
|
||||
name: "Nonexistent library UUID",
|
||||
libraryID: "00000000-0000-0000-0000-000000000000",
|
||||
expectedStatus: http.StatusOK,
|
||||
description: "Should return zero counts for valid but nonexistent UUID",
|
||||
expectedStatus: http.StatusForbidden,
|
||||
description: "Family lens: an id outside the visible set is a 403, never a silent pass (nonexistent is trivially out-of-set)",
|
||||
},
|
||||
{
|
||||
name: "Special characters in UUID",
|
||||
@@ -513,3 +523,166 @@ func TestProcessingIssuesInvalidTokens(t *testing.T) {
|
||||
assert.Equal(t, http.StatusUnauthorized, rec.Code)
|
||||
})
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Phase 11 (faceted plan): the honest ledger — row-less import failures,
|
||||
// truthful upserts, self-healing resolution, family lens.
|
||||
// ============================================================================
|
||||
|
||||
// createTestAdminOnce creates a fresh admin user (never the shared
|
||||
// testuser) with unique credentials and registers its cleanup.
|
||||
func createTestAdminOnce(t *testing.T, db *database.Queries, ts *httptest.Server) (UserTestData, string) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
uniqueID := uuid.New().String()[:8]
|
||||
email := fmt.Sprintf("issuesadmin-%s@tests.bookhoard.internal", uniqueID)
|
||||
username := fmt.Sprintf("issuesadmin-%s", uniqueID)
|
||||
|
||||
passwordHash := "$2a$10$JjAtK7PPa1WexQC3AUGe8OXLeuseZ/haN1Mz7emMo6CfOvMiTVXWq"
|
||||
newUser, err := db.CreateUser(ctx, database.CreateUserParams{
|
||||
Email: email,
|
||||
Username: username,
|
||||
PasswordHash: passwordHash,
|
||||
FirstName: pgtype.Text{String: "Issues", Valid: true},
|
||||
LastName: pgtype.Text{String: "Admin", Valid: true},
|
||||
Role: "admin",
|
||||
Timezone: pgtype.Text{String: "UTC", Valid: true},
|
||||
})
|
||||
require.NoError(t, err, "Should create admin test user")
|
||||
|
||||
userUUID, err := uuid.FromBytes(newUser.ID.Bytes[0:16])
|
||||
require.NoError(t, err, "Should parse admin user UUID")
|
||||
|
||||
t.Cleanup(func() {
|
||||
db.DeleteUser(context.Background(), pgtype.UUID{Bytes: [16]byte(userUUID), Valid: true})
|
||||
})
|
||||
|
||||
user := UserTestData{ID: userUUID, Email: email, Username: username, Password: "Test@Pass123!"}
|
||||
token := loginWithCredentials(t, ts, email, "Test@Pass123!")
|
||||
return user, token
|
||||
}
|
||||
|
||||
func TestProcessingIssuesLedgerImportFailures(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Issues Ledger %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// A book whose library-relative path exceeds media_items.file_path's
|
||||
// VARCHAR(500): CreateMediaItem fails deterministically, and the
|
||||
// relative path itself cannot ever import — the classic row-less
|
||||
// ledger case. Filename and dir components stay within filesystem
|
||||
// limits; only the total path overflows.
|
||||
longRel := ""
|
||||
for i := 0; i < 9; i++ {
|
||||
longRel = filepath.Join(longRel, strings.Repeat("d", 60))
|
||||
}
|
||||
longRel = filepath.Join(longRel, "deepbook.epub")
|
||||
require.Greater(t, len(longRel), 500, "relative path must exceed the column limit")
|
||||
deepDir := filepath.Join(folder, filepath.Dir(longRel))
|
||||
require.NoError(t, os.MkdirAll(deepDir, 0o755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(folder, longRel), []byte("dummy epub content"), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
// Fail → the row-less issue appears, keyed by (library, path).
|
||||
open := func() []database.ListProcessingIssuesByLibraryRow {
|
||||
rows, err := setup.DB.ListProcessingIssuesByLibrary(ctx, libID)
|
||||
require.NoError(t, err)
|
||||
return rows
|
||||
}
|
||||
issues := open()
|
||||
require.Len(t, issues, 1, "the failed import must appear in the ledger")
|
||||
issue := issues[0]
|
||||
require.Equal(t, "import_failed", issue.IssueType)
|
||||
require.Equal(t, "error", issue.Severity)
|
||||
require.False(t, issue.MediaItemID.Valid, "row-less: no media_items row exists")
|
||||
require.Equal(t, filepath.ToSlash(longRel), issue.FilePath, "row-less key is the library-relative path")
|
||||
require.Empty(t, issue.Title, "no item row → no title")
|
||||
|
||||
// The JSON list surfaces the row-less shape: empty item id, title
|
||||
// fallen back to the path (pins the LEFT JOIN — an inner join would
|
||||
// silently drop the ledger's newest kind).
|
||||
req := httptest.NewRequest("GET", "/api/libraries/"+libID.String()+"/issues/list", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
rec := httptest.NewRecorder()
|
||||
setup.Server.Config.Handler.ServeHTTP(rec, req)
|
||||
require.Equal(t, http.StatusOK, rec.Code)
|
||||
var listed []map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(rec.Body).Decode(&listed))
|
||||
require.Len(t, listed, 1)
|
||||
assert.Equal(t, "", listed[0]["media_item_id"], "row-less issues carry no item id")
|
||||
assert.Equal(t, filepath.ToSlash(longRel), listed[0]["title"], "title falls back to the file path")
|
||||
assert.Equal(t, "import_failed", listed[0]["issue_type"])
|
||||
|
||||
// Repeat failure → the SAME row updates (no dups).
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
issues = open()
|
||||
require.Len(t, issues, 1, "repeat failures upsert, never duplicate")
|
||||
require.Equal(t, issue.ID, issues[0].ID)
|
||||
|
||||
// Success resolves by (library, path): a normal book imports at a
|
||||
// path that has an open row-less issue — seeded here to model the
|
||||
// transient failure the long path can't have (its path IS the
|
||||
// failure).
|
||||
goodRel := "Author/ledgerbook.epub"
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO processing_issues (media_item_id, library_id, issue_type, issue_description, severity, file_path)
|
||||
VALUES (NULL, $1, 'import_failed', 'seeded transient failure', 'error', $2)`,
|
||||
libID, goodRel)
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(folder, "Author"), 0o755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(folder, "Author", "ledgerbook.epub"), []byte("dummy epub content"), 0o644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
var resolved int
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT count(*) FROM processing_issues WHERE library_id = $1 AND file_path = $2 AND resolved AND resolved_at IS NOT NULL`,
|
||||
libID, goodRel).Scan(&resolved))
|
||||
require.Equal(t, 1, resolved, "the file's successful import must resolve its open issue")
|
||||
require.Len(t, open(), 1, "only the long-path issue stays open")
|
||||
}
|
||||
|
||||
func TestProcessingIssuesFamilyLens(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
libID, _ := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Issues Lens %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// A fresh admin (never the shared testuser) + a hidden-library row:
|
||||
// default-ALLOW visibility with one explicit hide.
|
||||
lensAdmin, lensToken := createTestAdminOnce(t, setup.DB, setup.Server)
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO library_visibility (library_id, user_id, is_visible) VALUES ($1, $2, false)`,
|
||||
libID, pgtype.UUID{Bytes: [16]byte(lensAdmin.ID), Valid: true})
|
||||
require.NoError(t, err)
|
||||
|
||||
// An open issue in that library (row-less, no item to cascade).
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO processing_issues (media_item_id, library_id, issue_type, issue_description, severity, file_path)
|
||||
VALUES (NULL, $1, 'import_failed', 'lens probe', 'error', 'hidden/book.epub')`,
|
||||
libID)
|
||||
require.NoError(t, err)
|
||||
|
||||
get := func(token, suffix string) *httptest.ResponseRecorder {
|
||||
req := httptest.NewRequest("GET", "/api/libraries/"+libID.String()+"/issues/"+suffix, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
rec := httptest.NewRecorder()
|
||||
setup.Server.Config.Handler.ServeHTTP(rec, req)
|
||||
return rec
|
||||
}
|
||||
|
||||
// Hidden from the fresh admin → 403 on both endpoints (never silent).
|
||||
assert.Equal(t, http.StatusForbidden, get(lensToken, "list").Code, "hidden library list is a 403")
|
||||
assert.Equal(t, http.StatusForbidden, get(lensToken, "stats").Code, "hidden library stats are a 403")
|
||||
|
||||
// Visible to the shared admin (no hide row) → the rows show.
|
||||
assert.Equal(t, http.StatusOK, get(setup.Token, "list").Code)
|
||||
assert.Equal(t, http.StatusOK, get(setup.Token, "stats").Code)
|
||||
}
|
||||
@@ -51,7 +51,7 @@ func createTestQueueItem(t *testing.T, db *database.Queries, userID uuid.UUID) (
|
||||
mediaItem, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Test Book for Queue",
|
||||
Author: pgtype.Text{String: "Test Author", Valid: true},
|
||||
Authors: []string{"Test Author"},
|
||||
FilePath: "/tmp/test.epub",
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
package main
|
||||
|
||||
// Bulk scan-state tests (faceted plan Part 2 Phase 3): the scanner's
|
||||
// one-query-per-library bulk load replaces the per-file
|
||||
// GetMediaItemByFilePath flood. Behavioral equivalence asserted at the
|
||||
// container level: unchanged rescans skip (updated_at stable), items
|
||||
// created OUTSIDE the scanner (API create path) are seen by the bulk
|
||||
// load, and size changes still take the update path.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestScanStateBulkLoad(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Scan State %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// Two scanner-created fixture books.
|
||||
epubA := filepath.Join(folder, "Author A", "State Book A", "book.epub")
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(epubA), 0755))
|
||||
require.NoError(t, os.WriteFile(epubA, []byte("state book A content"), 0644))
|
||||
epubB := filepath.Join(folder, "Author B", "State Book B", "book.epub")
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(epubB), 0755))
|
||||
require.NoError(t, os.WriteFile(epubB, []byte("state book B content"), 0644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
items := listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 2, "both books ingested")
|
||||
var updatedA time.Time
|
||||
for _, item := range items {
|
||||
switch item.Title {
|
||||
case "State Book A":
|
||||
updatedA = item.UpdatedAt.Time
|
||||
case "State Book B":
|
||||
default:
|
||||
t.Errorf("unexpected item %q", item.Title)
|
||||
}
|
||||
}
|
||||
require.False(t, updatedA.IsZero())
|
||||
|
||||
// Unchanged rescan: the skip path must not touch the rows.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
items = listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 2, "no duplicates on rescan")
|
||||
for _, item := range items {
|
||||
if item.Title == "State Book A" {
|
||||
assert.Equal(t, updatedA, item.UpdatedAt.Time, "unchanged rescan leaves updated_at stable")
|
||||
}
|
||||
}
|
||||
|
||||
// A row created by the API create path (NOT by a scan) must be seen
|
||||
// by the bulk load: the rescan skips it instead of re-creating. The
|
||||
// file must really exist (the archive pass missing-checks rows
|
||||
// whose files are absent).
|
||||
apiFile := filepath.Join(folder, "Author C", "API Made Book", "book.epub")
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(apiFile), 0755))
|
||||
require.NoError(t, os.WriteFile(apiFile, []byte("api made book content"), 0644))
|
||||
apiStat, err := os.Stat(apiFile)
|
||||
require.NoError(t, err)
|
||||
created, err := setup.DB.CreateMediaItem(context.Background(), database.CreateMediaItemParams{
|
||||
LibraryID: libID,
|
||||
Title: "API Made Book",
|
||||
Authors: []string{"API Author"},
|
||||
FilePath: "Author C/API Made Book/book.epub",
|
||||
FileSize: pgtype.Int8{Int64: apiStat.Size(), Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
// Stamp the file's real sha the way the scanner does — otherwise the
|
||||
// skip path's one-time hash self-heal fires and bumps updated_at.
|
||||
// The identifiers UPDATE also bumps updated_at, so read the row
|
||||
// AFTER the stamp for the baseline.
|
||||
apiSha := sha256.Sum256([]byte("api made book content"))
|
||||
stamped, err := setup.DB.UpdateMediaItemIdentifiers(context.Background(), database.UpdateMediaItemIdentifiersParams{
|
||||
ID: created.ID,
|
||||
FileSha256: pgtype.Text{String: hex.EncodeToString(apiSha[:]), Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
apiUpdated := stamped.UpdatedAt.Time
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
items = listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 3, "API-created item survives the rescan as-is")
|
||||
for _, item := range items {
|
||||
if item.Title == "API Made Book" {
|
||||
assert.Equal(t, apiUpdated, item.UpdatedAt.Time, "bulk-loaded state sees scanner-external rows; skip path leaves them alone")
|
||||
}
|
||||
}
|
||||
|
||||
// Size change: the update path still fires (with a fresh full-row
|
||||
// read + hash recompute, then a write-through of the new size).
|
||||
require.NoError(t, os.WriteFile(epubA, []byte("state book A content — now longer"), 0644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
items = listLibraryAudioItems(t, setup, libID)
|
||||
require.Len(t, items, 3, "size change updates in place, never duplicates")
|
||||
for _, item := range items {
|
||||
if item.Title == "State Book A" {
|
||||
assert.NotEqual(t, updatedA, item.UpdatedAt.Time, "size change takes the update path")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestScanManifestLifecycle exercises the manifest-gated change ladder
|
||||
// end to end (faceted plan Part 2 Phase 4): seeding at create, repoint
|
||||
// reconciliation (mtime drift, identical content → no re-extraction),
|
||||
// same-size sidecar edits (the size-only detection blind spot),
|
||||
// playlist sync on the change path, and the no-leak guard.
|
||||
func TestScanManifestLifecycle(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Manifest Lifecycle %d", time.Now().UnixNano()), "ebooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
// The book ships WITH an ABS sidecar: the create path seeds the
|
||||
// manifest so the next scan compares instead of backfilling.
|
||||
bookDir := filepath.Join(folder, "Manifest Author", "Manifest Book")
|
||||
require.NoError(t, os.MkdirAll(bookDir, 0755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(bookDir, "book.epub"), []byte("manifest book content"), 0644))
|
||||
opfPath := filepath.Join(bookDir, "metadata.json")
|
||||
require.NoError(t, os.WriteFile(opfPath, []byte(`{"title":"Manifest Book"}`), 0644))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
|
||||
var itemID pgtype.UUID
|
||||
var itemIDStr, updatedStr string
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT id::text, updated_at::text FROM media_items WHERE title = 'Manifest Book' AND library_id = $1`,
|
||||
libID,
|
||||
).Scan(&itemIDStr, &updatedStr))
|
||||
require.NoError(t, itemID.Scan(itemIDStr))
|
||||
|
||||
// Seeded manifest exists.
|
||||
seeded, err := setup.DB.GetMediaScanState(context.Background(), itemID)
|
||||
require.NoError(t, err, "create path must seed the manifest")
|
||||
require.NotEmpty(t, seeded.Manifest)
|
||||
|
||||
// Unchanged rescan: skip path, nothing moves.
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
var updated2 string
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT updated_at::text FROM media_items WHERE id = $1`, itemID).Scan(&updated2))
|
||||
assert.Equal(t, updatedStr, updated2, "unchanged rescan is a pure skip")
|
||||
|
||||
// Repoint: mtime drift with identical content → reconciled, ZERO
|
||||
// re-extraction; the stored stat pair refreshes.
|
||||
past := time.Now().Add(-2 * time.Hour)
|
||||
require.NoError(t, os.Chtimes(opfPath, past, past))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
var updated3 string
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT updated_at::text FROM media_items WHERE id = $1`, itemID).Scan(&updated3))
|
||||
assert.Equal(t, updatedStr, updated3, "repoint reconciliation must not re-extract")
|
||||
reconciled, err := setup.DB.GetMediaScanState(context.Background(), itemID)
|
||||
require.NoError(t, err)
|
||||
var storedManifest struct {
|
||||
Entries []struct {
|
||||
Name string `json:"n"`
|
||||
MTime int64 `json:"m"`
|
||||
} `json:"entries"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal(reconciled.Manifest, &storedManifest))
|
||||
var sawRefreshed bool
|
||||
for _, e := range storedManifest.Entries {
|
||||
if e.Name == "metadata.json" && e.MTime == past.UnixNano() {
|
||||
sawRefreshed = true
|
||||
}
|
||||
}
|
||||
assert.True(t, sawRefreshed, "repoint refreshes the stored stat pair")
|
||||
|
||||
// Same-size sidecar edit — the size-only detection blind spot: the
|
||||
// stat pair moves, the hash differs → re-extract consumes the edit.
|
||||
sameLen := []byte(`{"title":"Manifesu Book"}`)
|
||||
require.Len(t, sameLen, len(`{"title":"Manifest Book"}`), "fixture must be a same-size edit")
|
||||
require.NoError(t, os.WriteFile(opfPath, sameLen, 0644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
var title4 string
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT title FROM media_items WHERE id = $1`, itemID).Scan(&title4))
|
||||
assert.Equal(t, "Manifesu Book", title4, "same-size sidecar edit is detected and consumed")
|
||||
|
||||
// Playlist sync on the change path (Correction 2): the sidecar gains
|
||||
// a reading-order series → the collection is created for an
|
||||
// EXISTING book, and a rescan does not duplicate it.
|
||||
require.NoError(t, os.WriteFile(opfPath, []byte(`{"title":"Manifesu Book","series":["Lifecycle Saga (Reading Order) #1"]}`), 0644))
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
var collCount int
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collections WHERE name = 'Lifecycle Saga (Reading Order)' AND user_id = $1`, admin,
|
||||
).Scan(&collCount))
|
||||
assert.Equal(t, 1, collCount, "reading-order sidecar added to an existing book creates its collection")
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
require.NoError(t, setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*) FROM collections WHERE name = 'Lifecycle Saga (Reading Order)' AND user_id = $1`, admin,
|
||||
).Scan(&collCount))
|
||||
assert.Equal(t, 1, collCount, "rescan is idempotent — no duplicate collection")
|
||||
|
||||
// No-leak guard (plan Part 2 test 7): the library-scoped list emits
|
||||
// raw rows — scanner state must never ride along (structural by
|
||||
// design; this assertion keeps it honest).
|
||||
req, _ := http.NewRequest("GET", setup.Server.URL+"/api/media-items?library_id="+libID.String(), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
require.NoError(t, err)
|
||||
assert.NotContains(t, string(body), `"manifest":`, "scanner state must not leak into media list JSON")
|
||||
assert.NotContains(t, string(body), `"sidecar`, "scanner state must not leak into media list JSON")
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/services"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestRescanMediaItemPreservesAudioMetadata pins the 2026-10-21 incident:
|
||||
// a per-item rescan of a multi-file audiobook routes book-level metadata
|
||||
// (sidecars/tags — AudioMeta nil) through the generic row update, whose
|
||||
// unguarded write NULLed audio_metadata. A NULL payload drops every
|
||||
// track/chapter and the duration, so the book displays as 1 second long
|
||||
// and the audio reader loses its track manifest. Re-extraction must be
|
||||
// write-after-success: no fresh audio payload from the audio chain means
|
||||
// the existing column survives byte-for-byte.
|
||||
func TestRescanMediaItemPreservesAudioMetadata(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Audio Guard %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrack(t, filepath.Join(folder, "Guard Author", "Guard Book", "01 One.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "Guard Author", "Guard Book", "02 Two.mp3"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
rel := filepath.ToSlash("Guard Author/Guard Book")
|
||||
book, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found, "folder book must import")
|
||||
|
||||
audioBytes := func(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
var raw []byte
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT audio_metadata FROM media_items WHERE id = $1`, book.ID).Scan(&raw))
|
||||
return raw
|
||||
}
|
||||
|
||||
before := audioBytes(t)
|
||||
require.NotNil(t, before, "initial scan must populate audio_metadata")
|
||||
var payload services.AudioMetadataPayload
|
||||
require.NoError(t, json.Unmarshal(before, &payload), "payload must be valid JSON")
|
||||
require.Greater(t, payload.DurationSeconds, 0.0, "initial scan must carry a duration")
|
||||
require.Len(t, payload.Tracks, 2, "initial scan must carry the track manifest")
|
||||
|
||||
scanner := services.NewMediaScanner(setup.DB)
|
||||
defer func() { _ = scanner.Close() }()
|
||||
scanner.SetAdminID(admin)
|
||||
require.NoError(t, scanner.RescanMediaItem(ctx, book.ID, false), "per-item rescan must succeed")
|
||||
|
||||
after := audioBytes(t)
|
||||
require.NotNil(t, after, "rescan must never NULL audio_metadata (2026-10-21 incident)")
|
||||
var afterPayload services.AudioMetadataPayload
|
||||
require.NoError(t, json.Unmarshal(after, &afterPayload), "surviving payload must stay valid JSON")
|
||||
require.Greater(t, afterPayload.DurationSeconds, 0.0, "duration must survive the rescan")
|
||||
require.Len(t, afterPayload.Tracks, 2, "track manifest must survive the rescan")
|
||||
}
|
||||
|
||||
// TestMetadataEditorSavePreservesAudioMetadata pins the editor-side half
|
||||
// of the incident: PUT /api/media-items/:id (the metadata editor) writes
|
||||
// the FULL row, and audio_metadata has no editor field — an unguarded
|
||||
// save NULLed every audiobook's payload on a plain metadata edit.
|
||||
func TestMetadataEditorSavePreservesAudioMetadata(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test Editor Guard %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrack(t, filepath.Join(folder, "Editor Author", "Editor Book", "01 One.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folder, "Editor Author", "Editor Book", "02 Two.mp3"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
rel := filepath.ToSlash("Editor Author/Editor Book")
|
||||
book, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found, "folder book must import")
|
||||
|
||||
audioBytes := func(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
var raw []byte
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT audio_metadata FROM media_items WHERE id = $1`, book.ID).Scan(&raw))
|
||||
return raw
|
||||
}
|
||||
before := audioBytes(t)
|
||||
require.NotNil(t, before, "initial scan must populate audio_metadata")
|
||||
|
||||
// A plain metadata edit through the editor endpoint (admin token —
|
||||
// the editor is admin-only), no audio fields involved.
|
||||
editorSave(t, setup, book.ID, map[string]interface{}{
|
||||
"title": "Editor Book Renamed",
|
||||
"cover_action": "keep",
|
||||
})
|
||||
|
||||
after := audioBytes(t)
|
||||
require.NotNil(t, after, "metadata edit must never NULL audio_metadata")
|
||||
var afterPayload services.AudioMetadataPayload
|
||||
require.NoError(t, json.Unmarshal(after, &afterPayload))
|
||||
require.Greater(t, afterPayload.DurationSeconds, 0.0, "duration must survive the edit")
|
||||
require.Len(t, afterPayload.Tracks, 2, "track manifest must survive the edit")
|
||||
|
||||
renamed, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found)
|
||||
require.Equal(t, "Editor Book Renamed", renamed.Title, "the edit itself must still apply")
|
||||
}
|
||||
|
||||
// TestRescanMediaItemPreservesAudioMetadataSingleFileM4B is the incident
|
||||
// class verbatim: the failing books were SINGLE-FILE m4b audiobooks
|
||||
// (audio_metadata without a layout key), which RescanMediaItem routes
|
||||
// through the generic per-file updater — the same writer whose unguarded
|
||||
// pass NULLed their payloads. The whole flow must keep the m4b's
|
||||
// duration and chapters intact.
|
||||
func TestRescanMediaItemPreservesAudioMetadataSingleFileM4B(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
ctx := context.Background()
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
libID, folder := audioTestLibrary(t, setup, fmt.Sprintf("Test M4B Guard %d", time.Now().UnixNano()))
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
writeTestTrackM4B(t, filepath.Join(folder, "M4B Guard Author", "M4B Guard Book.m4b"))
|
||||
|
||||
scanTestFolder(t, setup, admin, folder)
|
||||
rel := filepath.ToSlash("M4B Guard Author/M4B Guard Book.m4b")
|
||||
book, found := findAudioItemByPath(t, setup, libID, rel)
|
||||
require.True(t, found, "single-file m4b must import")
|
||||
|
||||
audioBytes := func(t *testing.T) []byte {
|
||||
t.Helper()
|
||||
var raw []byte
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT audio_metadata FROM media_items WHERE id = $1`, book.ID).Scan(&raw))
|
||||
return raw
|
||||
}
|
||||
before := audioBytes(t)
|
||||
require.NotNil(t, before, "initial scan must populate audio_metadata")
|
||||
var beforePayload services.AudioMetadataPayload
|
||||
require.NoError(t, json.Unmarshal(before, &beforePayload))
|
||||
require.Greater(t, beforePayload.DurationSeconds, 0.0, "m4b probe must carry a real duration")
|
||||
|
||||
scanner := services.NewMediaScanner(setup.DB)
|
||||
defer func() { _ = scanner.Close() }()
|
||||
scanner.SetAdminID(admin)
|
||||
require.NoError(t, scanner.RescanMediaItem(ctx, book.ID, false))
|
||||
|
||||
after := audioBytes(t)
|
||||
require.NotNil(t, after, "m4b rescan must never NULL audio_metadata")
|
||||
var afterPayload services.AudioMetadataPayload
|
||||
require.NoError(t, json.Unmarshal(after, &afterPayload))
|
||||
require.Greater(t, afterPayload.DurationSeconds, 0.0, "m4b duration must survive the rescan")
|
||||
}
|
||||
|
||||
// editorSave drives PUT /api/media-items/:id as the admin (the metadata
|
||||
// editor's endpoint; admin-only per the handler).
|
||||
func editorSave(t *testing.T, setup *TestServerSetup, itemID pgtype.UUID, fields map[string]interface{}) {
|
||||
t.Helper()
|
||||
body, _ := json.Marshal(fields)
|
||||
req, _ := http.NewRequest("PUT", setup.Server.URL+"/api/media-items/"+uuid.UUID(itemID.Bytes).String(), bytes.NewBuffer(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode, "metadata save must succeed")
|
||||
}
|
||||
@@ -0,0 +1,182 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// POST /api/scanner/cancel/:jobId is admin-only and must answer with
|
||||
// distinct codes for unknown (404) and already-terminal (409) jobs.
|
||||
func TestScanCancelEndpointSemantics(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
|
||||
client := &http.Client{}
|
||||
|
||||
t.Run("cancel without admin role is forbidden", func(t *testing.T) {
|
||||
url := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, uuid.New().String())
|
||||
req, _ := http.NewRequest("POST", url, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
assert.Equal(t, http.StatusForbidden, resp.StatusCode)
|
||||
})
|
||||
|
||||
t.Run("cancel unknown job is not found", func(t *testing.T) {
|
||||
url := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, uuid.New().String())
|
||||
req, _ := http.NewRequest("POST", url, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
assert.Equal(t, http.StatusNotFound, resp.StatusCode)
|
||||
})
|
||||
}
|
||||
|
||||
// Cancelling a manual scan mid-run must never corrupt library state:
|
||||
// the abort skips the archive pass, so unvisited (but on-disk) files
|
||||
// are not treated as missing. The canary item points at a file that is
|
||||
// lexically LAST in the folder, so a mid-walk abort that (incorrectly)
|
||||
// ran the archive pass would mark it missing and fail the assertion.
|
||||
// The assertions hold under every interleaving — whether the cancel
|
||||
// lands mid-scan or the scan already finished — so the test cannot
|
||||
// flap against timing.
|
||||
func TestScanCancelMidScanSafety(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
client := &http.Client{}
|
||||
token := setup.Token
|
||||
|
||||
// Test library (helper registers API-delete cleanup so the run
|
||||
// leaves no residue in the dev database).
|
||||
libraryID := createTestLibraryWithFolder(t, setup.Server, token, "Test Scan Cancel Library", false)
|
||||
libUUID, err := uuid.Parse(libraryID)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Folder on disk: many junk .epub files (zero bytes cannot parse as
|
||||
// a zip/epub, so processMediaFile rejects each one and creates no
|
||||
// rows) plus the canary file, lexically last.
|
||||
dir := t.TempDir()
|
||||
const fileCount = 1200
|
||||
junk := bytes.Repeat([]byte{0}, 1024)
|
||||
for i := 0; i < fileCount; i++ {
|
||||
name := filepath.Join(dir, fmt.Sprintf("book-%04d.epub", i))
|
||||
require.NoError(t, os.WriteFile(name, junk, 0o644))
|
||||
}
|
||||
canaryRel := "zzz-last.epub"
|
||||
require.NoError(t, os.WriteFile(filepath.Join(dir, canaryRel), junk, 0o644))
|
||||
|
||||
// Register the folder with the library.
|
||||
folderURL := fmt.Sprintf("%s/api/libraries/%s/folders", setup.Server.URL, libraryID)
|
||||
body, _ := json.Marshal(map[string]interface{}{"folder_path": dir})
|
||||
req, _ := http.NewRequest("POST", folderURL, bytes.NewBuffer(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
_ = resp.Body.Close()
|
||||
require.Equal(t, http.StatusCreated, resp.StatusCode, "folder creation should succeed")
|
||||
|
||||
// Pre-existing item whose file exists on disk. A normal completed
|
||||
// scan never marks it missing; neither may a cancelled one.
|
||||
_, err = setup.DBPool.Exec(context.Background(),
|
||||
"INSERT INTO media_items (library_id, title, file_path) VALUES ($1, $2, $3)",
|
||||
pgtype.UUID{Bytes: libUUID, Valid: true}, "Test Cancel Canary", canaryRel)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Start the scan and cancel it as soon as it is accepted.
|
||||
jobID := startScan(t, client, setup.Server.URL, token, libraryID)
|
||||
|
||||
cancelURL := fmt.Sprintf("%s/api/scanner/cancel/%s", setup.Server.URL, jobID)
|
||||
cancelReq, _ := http.NewRequest("POST", cancelURL, nil)
|
||||
cancelReq.Header.Set("Authorization", "Bearer "+token)
|
||||
cancelResp, err := client.Do(cancelReq)
|
||||
require.NoError(t, err)
|
||||
_ = cancelResp.Body.Close()
|
||||
// 200 = cancelled mid-run; 409 = the scan already finished before
|
||||
// the cancel landed. Both are acceptable; the assertions below hold
|
||||
// either way.
|
||||
assert.Contains(t, []int{http.StatusOK, http.StatusConflict}, cancelResp.StatusCode)
|
||||
|
||||
// Poll to a terminal state.
|
||||
final := pollJobToTerminal(t, client, setup.Server.URL, token, jobID)
|
||||
assert.Contains(t, []string{"completed", "cancelled"}, final["status"],
|
||||
"scan must finish cleanly or be cancelled, got %v (error: %v)", final["status"], final["error"])
|
||||
|
||||
// The safety invariant: no item was marked missing or archived, and
|
||||
// the junk files created no phantom rows.
|
||||
var items, missingSum, archived int
|
||||
err = setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT COUNT(*),
|
||||
COALESCE(SUM(missing_scan_count), 0),
|
||||
COUNT(*) FILTER (WHERE archived_at IS NOT NULL)
|
||||
FROM media_items WHERE library_id = $1`,
|
||||
pgtype.UUID{Bytes: libUUID, Valid: true}).Scan(&items, &missingSum, &archived)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, 1, items, "junk files must not create media items")
|
||||
assert.Zero(t, missingSum, "cancelled/completed scan must not mark on-disk files missing")
|
||||
assert.Zero(t, archived, "no item may be archived by this scan")
|
||||
|
||||
// Cancelling a terminal job is a conflict, whichever terminal state
|
||||
// the job reached.
|
||||
cancelReq2, _ := http.NewRequest("POST", cancelURL, nil)
|
||||
cancelReq2.Header.Set("Authorization", "Bearer "+token)
|
||||
cancelResp2, err := client.Do(cancelReq2)
|
||||
require.NoError(t, err)
|
||||
_ = cancelResp2.Body.Close()
|
||||
assert.Equal(t, http.StatusConflict, cancelResp2.StatusCode)
|
||||
}
|
||||
|
||||
func startScan(t *testing.T, client *http.Client, baseURL, token, libraryID string) string {
|
||||
url := fmt.Sprintf("%s/api/libraries/%s/scan", baseURL, libraryID)
|
||||
req, _ := http.NewRequest("POST", url, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
require.Equal(t, http.StatusAccepted, resp.StatusCode)
|
||||
|
||||
var decoded map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&decoded))
|
||||
jobID, ok := decoded["job_id"].(string)
|
||||
require.True(t, ok, "job_id should be a string")
|
||||
require.NotEmpty(t, jobID)
|
||||
return jobID
|
||||
}
|
||||
|
||||
func pollJobToTerminal(t *testing.T, client *http.Client, baseURL, token, jobID string) map[string]interface{} {
|
||||
url := fmt.Sprintf("%s/api/scanner/status/%s", baseURL, jobID)
|
||||
for i := 0; i < 30; i++ {
|
||||
req, _ := http.NewRequest("GET", url, nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
var status map[string]interface{}
|
||||
err = json.NewDecoder(resp.Body).Decode(&status)
|
||||
_ = resp.Body.Close()
|
||||
require.NoError(t, err)
|
||||
|
||||
switch status["status"] {
|
||||
case "completed", "failed", "cancelled":
|
||||
return status
|
||||
}
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
}
|
||||
t.Fatalf("job %s did not reach a terminal state within 15s", jobID)
|
||||
return nil
|
||||
}
|
||||
@@ -413,10 +413,15 @@ func TestCollectionSearchLibraryFilter(t *testing.T) {
|
||||
shouldContain: book2ID,
|
||||
},
|
||||
{
|
||||
name: "invalid library_id",
|
||||
query: "Harry",
|
||||
libraryID: "00000000-0000-0000-0000-000000000000",
|
||||
expectedStatus: http.StatusNotFound,
|
||||
name: "invalid library_id",
|
||||
query: "Harry",
|
||||
libraryID: "00000000-0000-0000-0000-000000000000",
|
||||
// Family-lens standing rule (AGENTS.md): an explicit
|
||||
// library_id outside the visible set is a 403, never a
|
||||
// silent pass — and a nonexistent id is out-of-set too,
|
||||
// so existence is not enumerable. Behavior shipped in
|
||||
// 7e586df4 (Phase 1); this pin update was missed then.
|
||||
expectedStatus: http.StatusForbidden,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestSeriesPagination pins the SSR series-page pagination contract
|
||||
// (2026-10-19 paginator fix). The /series page renders its grid and
|
||||
// paginator entirely server-side (templates/series.templ); page state
|
||||
// lives in the ?page= query param. The fixture mirrors a prod bulk
|
||||
// import: 30 series whose items share ONE identical created_at (a single
|
||||
// transaction's NOW()), which historically left the ORDER BY tied — the
|
||||
// tiebreaker (library_id, series) makes pages deterministic.
|
||||
func TestSeriesPagination(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
setup := setupTestServer(t)
|
||||
|
||||
libID := createTestLibraryWithFolder(t, setup.Server, setup.Token, "Test Pagination Library", false)
|
||||
|
||||
ctx := context.Background()
|
||||
const totalSeries = 30
|
||||
for i := 1; i <= totalSeries; i++ {
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, series, created_at, imported_at, updated_at)
|
||||
VALUES ($1, $2, $3, $4, '2026-01-01 00:00:00+00', '2026-01-01 00:00:00+00', '2026-01-01 00:00:00+00')`,
|
||||
libID, fmt.Sprintf("Pagination Probe Book %02d", i),
|
||||
fmt.Sprintf("/tmp/test-pagination/book-%02d.epub", i),
|
||||
fmt.Sprintf("Pagination Probe Series %02d", i),
|
||||
)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
_, _ = setup.DBPool.Exec(ctx, "DELETE FROM libraries WHERE id = $1", libID)
|
||||
})
|
||||
|
||||
// templ separates dynamic text segments with HTML comments; strip them
|
||||
// so plain regexes work on names and the page chip.
|
||||
stripComments := func(html string) string {
|
||||
return regexp.MustCompile(`<!--.*?-->`).ReplaceAllString(html, "")
|
||||
}
|
||||
seriesNames := func(html string) map[string]bool {
|
||||
names := map[string]bool{}
|
||||
for _, m := range regexp.MustCompile(`<h3[^>]*>([^<]+)</h3>`).FindAllStringSubmatch(stripComments(html), -1) {
|
||||
names[strings.TrimSpace(m[1])] = true
|
||||
}
|
||||
return names
|
||||
}
|
||||
pageChip := func(html string) string {
|
||||
m := regexp.MustCompile(`Page\s+(\d+)\s+of\s+(\d+)`).FindStringSubmatch(stripComments(html))
|
||||
if m == nil {
|
||||
return ""
|
||||
}
|
||||
return fmt.Sprintf("Page %s of %s", m[1], m[2])
|
||||
}
|
||||
|
||||
perPage := 24 // internal/router/frontend.go perSeriesPage
|
||||
|
||||
t.Run("page1 defaults and next link", func(t *testing.T) {
|
||||
code, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID, setup.Token)
|
||||
require.Equal(t, 200, code)
|
||||
|
||||
assert.Equal(t, fmt.Sprintf("Page 1 of 2"), pageChip(body))
|
||||
names := seriesNames(body)
|
||||
assert.Len(t, names, perPage, "page 1 renders exactly 24 cards")
|
||||
assert.Contains(t, names, "Pagination Probe Series 01", "deterministic order: page 1 starts at the first series")
|
||||
assert.Contains(t, names, fmt.Sprintf("Pagination Probe Series %02d", perPage), "page 1 ends at the 24th series")
|
||||
assert.NotContains(t, names, fmt.Sprintf("Pagination Probe Series %02d", perPage+1), "page 1 must not leak page-2 rows")
|
||||
|
||||
assert.NotContains(t, body, "Previous", "no Previous link on page 1")
|
||||
// templ HTML-escapes & as & inside attribute values.
|
||||
assert.Contains(t, body, fmt.Sprintf(`href="/series?library_id=%s&page=2"`, libID), "Next link carries library_id and page")
|
||||
})
|
||||
|
||||
t.Run("page2 is disjoint and terminal", func(t *testing.T) {
|
||||
code, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID+"&page=2", setup.Token)
|
||||
require.Equal(t, 200, code)
|
||||
|
||||
assert.Equal(t, "Page 2 of 2", pageChip(body))
|
||||
names := seriesNames(body)
|
||||
assert.Len(t, names, totalSeries-perPage, "page 2 renders the remainder")
|
||||
|
||||
code1, body1 := ssrGet(t, setup.Server.URL+"/series?library_id="+libID+"&page=1", setup.Token)
|
||||
require.Equal(t, 200, code1)
|
||||
page1 := seriesNames(body1)
|
||||
|
||||
for name := range names {
|
||||
assert.NotContains(t, page1, name, "pages must be disjoint even with fully tied created_at")
|
||||
}
|
||||
assert.NotContains(t, body, "Next", "no Next link on the last page")
|
||||
assert.Contains(t, body, fmt.Sprintf(`href="/series?library_id=%s&page=1"`, libID), "Previous link carries library_id and page")
|
||||
})
|
||||
|
||||
t.Run("missing page param defaults to page 1", func(t *testing.T) {
|
||||
code, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID, setup.Token)
|
||||
require.Equal(t, 200, code)
|
||||
assert.Equal(t, "Page 1 of 2", pageChip(body))
|
||||
})
|
||||
|
||||
t.Run("invalid page param falls back to page 1", func(t *testing.T) {
|
||||
for _, bad := range []string{"0", "-3", "two"} {
|
||||
code, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID+"&page="+bad, setup.Token)
|
||||
require.Equal(t, 200, code, "page=%s", bad)
|
||||
assert.Equal(t, "Page 1 of 2", pageChip(body), "page=%s must clamp to page 1", bad)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -544,7 +544,7 @@ func TestLibraryUsageStats(t *testing.T) {
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: [16]byte(lib), Valid: true},
|
||||
Title: title,
|
||||
Author: pgtype.Text{String: author, Valid: true},
|
||||
Authors: []string{author},
|
||||
Genre: pgtype.Text{String: genre, Valid: true},
|
||||
FilePath: fmt.Sprintf("/tmp/usage-%s-%d.bin", title, time.Now().UnixNano()),
|
||||
FileSize: pgtype.Int8{Int64: size, Valid: true},
|
||||
|
||||
@@ -95,7 +95,7 @@ func createSyncTestMedia(t *testing.T, db *database.Queries, libraryID pgtype.UU
|
||||
media, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
LibraryID: libraryID,
|
||||
Title: "Sync Test Book",
|
||||
Author: pgtype.Text{String: "Test Author", Valid: true},
|
||||
Authors: []string{"Test Author"},
|
||||
FilePath: "/tmp/test.epub",
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
|
||||
@@ -25,6 +25,7 @@ import (
|
||||
|
||||
"github.com/go-playground/validator/v10"
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
"github.com/labstack/echo/v5"
|
||||
@@ -181,6 +182,54 @@ func getUploadPath() string {
|
||||
return "./uploads" // Host path (left side of volume mount)
|
||||
}
|
||||
|
||||
// setSystemSettingRestored writes a system setting through the sanctioned
|
||||
// API path and registers a cleanup that restores the value the row had
|
||||
// BEFORE the write. The testuser-experience rule: the suite must leave
|
||||
// shared settings exactly as it found them — a hardcoded restore would
|
||||
// silently clobber a deliberate dev bump (e.g. scan_extractor_version
|
||||
// pinned mid-deploy), and the live dev server obeys this setting on its
|
||||
// next poll, so what tests write here reaches real scans.
|
||||
//
|
||||
// Writes ride the settings API (never raw SQL) because the registry
|
||||
// caches values and only the API path reloads it. If the row did not
|
||||
// exist before the write, the test value is left in place and the test
|
||||
// log says so — there is no sanctioned delete; settings rows that
|
||||
// matter (scan_extractor_version) always exist.
|
||||
func setSystemSettingRestored(t *testing.T, setup *TestServerSetup, key, value string) {
|
||||
t.Helper()
|
||||
var prior *string
|
||||
err := setup.DBPool.QueryRow(context.Background(),
|
||||
`SELECT setting_value FROM system_settings WHERE setting_key = $1`, key).Scan(&prior)
|
||||
if err != nil && err != pgx.ErrNoRows {
|
||||
require.NoError(t, err, "failed to read prior value of %s", key)
|
||||
}
|
||||
|
||||
setSettingViaAPI(t, setup, key, value)
|
||||
|
||||
if prior == nil {
|
||||
t.Logf("setSystemSettingRestored: %s had no prior row; leaving %q in place after the test", key, value)
|
||||
return
|
||||
}
|
||||
priorValue := *prior
|
||||
t.Cleanup(func() {
|
||||
setSettingViaAPI(t, setup, key, priorValue)
|
||||
})
|
||||
}
|
||||
|
||||
// setSettingViaAPI PUTs one setting through /api/system/settings — the
|
||||
// only sanctioned write path (the settings registry reloads here).
|
||||
func setSettingViaAPI(t *testing.T, setup *TestServerSetup, key, value string) {
|
||||
t.Helper()
|
||||
body, _ := json.Marshal(map[string]string{"key": key, "value": value})
|
||||
req, _ := http.NewRequest("PUT", setup.Server.URL+"/api/system/settings", bytes.NewBuffer(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode, "settings API is the only sanctioned write path")
|
||||
}
|
||||
|
||||
// getCachePath returns the appropriate cache path based on runtime environment
|
||||
func getCachePath() string {
|
||||
// Check for explicit override first
|
||||
@@ -467,6 +516,17 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
|
||||
queries := database.New(dbPool)
|
||||
|
||||
// Wire the tunable settings registry exactly like main.go: Load
|
||||
// populates the cache (missing rows fall back to compiled defaults)
|
||||
// and the SystemSettingsHandler must hold it or every
|
||||
// /api/system/settings call 503s ("settings registry not
|
||||
// initialized") — tests that write settings through the API (the
|
||||
// only sanctioned path) need this wired.
|
||||
settingsRegistry := database.NewSettingsRegistry(queries)
|
||||
if err := settingsRegistry.Load(context.Background()); err != nil {
|
||||
t.Logf("settings registry Load failed (defaults apply): %v", err)
|
||||
}
|
||||
|
||||
// Create login attempt tracker
|
||||
loginAttemptTracker := ratelimit.NewLoginAttemptTracker(5, 15*time.Minute, 5*time.Minute)
|
||||
|
||||
@@ -492,7 +552,12 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
conflictHandler := handlers.NewConflictHandler(queries, connManager, progressService)
|
||||
analyticsHandler := handlers.NewAnalyticsHandler(queries)
|
||||
queueHandler := handlers.NewQueueHandler(queries, queueProcessor)
|
||||
// main.go:112 wires the registry into the auth handler too — without it
|
||||
// defaultTimezone() short-circuits to UTC and every new user (Register +
|
||||
// AdminCreateUser) ignores the system default_timezone setting in tests.
|
||||
authHandler.SetSettings(settingsRegistry)
|
||||
systemSettingsHandler := handlers.NewSystemSettingsHandler(queries)
|
||||
systemSettingsHandler.SetSettings(settingsRegistry)
|
||||
processingIssuesHandler := handlers.NewProcessingIssuesHandler(queries)
|
||||
|
||||
// Create refactored handlers (matching main.go)
|
||||
@@ -507,7 +572,7 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
statisticsService := services.NewStatisticsService(queries)
|
||||
statsHandler := handlers.NewStatsHandler(statisticsService)
|
||||
seriesHandler := handlers.NewSeriesHandler(queries)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, libraryService, progressService, annotationService, worker)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, dbPool, libraryService, progressService, annotationService, worker)
|
||||
matchingHandler := handlers.NewMatchingHandler(queries, connManager)
|
||||
|
||||
// Create conversion service for OPDS
|
||||
@@ -571,7 +636,7 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
// RegisterRoutes dereferences Settings for the auth rate limit;
|
||||
// the main binary wires database.NewSettingsRegistry(queries) —
|
||||
// the test harness must too or route registration nil-panics.
|
||||
Settings: database.NewSettingsRegistry(queries),
|
||||
Settings: settingsRegistry,
|
||||
// main.go:246 passes the shared LibraryService here; omitting it
|
||||
// hands the kobo/kepub CFI converter a typed-nil (kobo markup
|
||||
// panic-recovered as 500s in TestProgressWeb_KoboIntegration).
|
||||
|
||||
@@ -157,37 +157,14 @@ func TestWebSocketProgressBroadcast(t *testing.T) {
|
||||
assert.Equal(t, "web", sourceDevice["type"])
|
||||
}
|
||||
|
||||
// TestWebSocketPingPong tests that ping/pong messages work correctly
|
||||
func TestWebSocketPingPong(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
|
||||
token := setup.Token
|
||||
|
||||
wsURL := strings.Replace(setup.Server.URL, "http", "ws", 1) + "/ws/sync?token=" + token
|
||||
ws, _, err := websocket.DefaultDialer.Dial(wsURL, nil)
|
||||
require.NoError(t, err)
|
||||
defer ws.Close()
|
||||
|
||||
// Read initial message
|
||||
ws.SetReadDeadline(time.Now().Add(5 * time.Second))
|
||||
_, _, _ = ws.ReadMessage()
|
||||
|
||||
// Send a ping message (as a text message for testing)
|
||||
err = ws.WriteMessage(websocket.TextMessage, []byte(`{"type":"ping"}`))
|
||||
require.NoError(t, err)
|
||||
|
||||
// Server should respond with pong
|
||||
ws.SetReadDeadline(time.Now().Add(2 * time.Second))
|
||||
_, msg, err := ws.ReadMessage()
|
||||
if err == nil {
|
||||
var pongMsg map[string]interface{}
|
||||
err = json.Unmarshal(msg, &pongMsg)
|
||||
if err == nil {
|
||||
// Server might respond with pong
|
||||
assert.Equal(t, "pong", pongMsg["type"])
|
||||
}
|
||||
}
|
||||
}
|
||||
// TestWebSocketPingPong was deleted (2026-10-20, Phase 6 full-suite
|
||||
// repair): it asserted a TEXT-level {"type":"ping"} → {"type":"pong"}
|
||||
// exchange that no server code has ever implemented and no client
|
||||
// (web/src or Android) has ever sent — it passed vacuously through its
|
||||
// read-timeout path until interleaved scan_progress broadcasts started
|
||||
// failing the assert. Keepalive is the standard protocol-level ping the
|
||||
// hub already sends (websocket.go's ticker); if a text-level app ping
|
||||
// is ever wanted, build it server-side first, then re-pin it.
|
||||
|
||||
// TestWebSocketConnectionLimit tests that the server handles multiple connections
|
||||
func TestWebSocketConnectionLimit(t *testing.T) {
|
||||
|
||||
+165
-5
@@ -37,6 +37,11 @@ CREATE TABLE IF NOT EXISTS users (
|
||||
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
|
||||
);
|
||||
|
||||
-- User-level default audiobook playback speed (player plan §8): books
|
||||
-- without a per-book audio_playback_speed row pick this up. REAL
|
||||
-- mirrors the per-book table's column.
|
||||
ALTER TABLE users ADD COLUMN IF NOT EXISTS default_audio_speed REAL NOT NULL DEFAULT 1.0;
|
||||
|
||||
-- Create system_settings table
|
||||
CREATE TABLE IF NOT EXISTS system_settings (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
@@ -61,6 +66,8 @@ ALTER TABLE system_settings ADD COLUMN IF NOT EXISTS category VARCHAR(40);
|
||||
INSERT INTO system_settings (setting_key, setting_value, description, setting_type, min_value, max_value, requires_restart, category) VALUES
|
||||
('scan_poll_interval_seconds', '60', 'How often to scan all libraries (seconds)', 'int', '1', '3600', FALSE, 'scanner'),
|
||||
('auto_scan_enabled', 'true', 'Whether auto-scanning is enabled system-wide', 'bool', NULL, NULL, FALSE, 'scanner'),
|
||||
('scan_extractor_version', '1', 'Parser version stamped on each book scan manifest; raising it re-extracts metadata from books scanned under an older parser (bounded batch per scan)', 'int', '1', '1000000', FALSE, 'scanner'),
|
||||
('auto_playlist_collections', 'true', 'Auto-create curated playlist collections from sidecar series marked as reading orders', 'bool', NULL, NULL, FALSE, 'scanner'),
|
||||
('default_timezone', 'UTC', 'System default timezone', 'string', NULL, NULL, FALSE, 'general'),
|
||||
-- security / auth (live)
|
||||
('session_duration_seconds', '604800', 'How long a login session stays valid', 'int', '300', '31536000', FALSE, 'security'),
|
||||
@@ -136,7 +143,8 @@ CREATE TABLE IF NOT EXISTS media_items (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
library_id UUID NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
|
||||
title VARCHAR(255) NOT NULL,
|
||||
author VARCHAR(255),
|
||||
subtitle TEXT, -- Series-y/edition subtitle from sidecars; display field, never part of `title`
|
||||
author VARCHAR(255), -- DEPRECATED for writes: display string regenerated from `authors` by trigger
|
||||
isbn VARCHAR(13), -- Still relevant for ebooks
|
||||
description TEXT,
|
||||
file_path VARCHAR(500) NOT NULL,
|
||||
@@ -144,10 +152,15 @@ CREATE TABLE IF NOT EXISTS media_items (
|
||||
mime_type VARCHAR(100),
|
||||
cover_image_path VARCHAR(500),
|
||||
series VARCHAR(255),
|
||||
series_number INTEGER,
|
||||
series_number DOUBLE PRECISION, -- Real ordinals are decimals (0.5, 1.5 novellas); was INTEGER
|
||||
tags TEXT[],
|
||||
narrators TEXT[], -- Audiobook narrators (sidecar first; user-editable)
|
||||
authors TEXT[] NOT NULL DEFAULT '{}', -- Write truth for authorship; `author` above is trigger-regenerated display
|
||||
authors_search TEXT[] NOT NULL DEFAULT '{}',
|
||||
publishers TEXT[] NOT NULL DEFAULT '{}', -- Write truth; `publisher` below is trigger-regenerated display
|
||||
publishers_search TEXT[] NOT NULL DEFAULT '{}',
|
||||
abridged BOOLEAN NOT NULL DEFAULT FALSE, -- Libation/Audible abridged flag
|
||||
explicit BOOLEAN NOT NULL DEFAULT FALSE, -- Adult flag from sidecars (ABS explicit / Audible isAdult)
|
||||
asin VARCHAR(20), -- Still relevant for ebooks
|
||||
date_published DATE,
|
||||
publisher VARCHAR(255),
|
||||
@@ -244,8 +257,12 @@ CREATE TABLE IF NOT EXISTS media_items (
|
||||
-- { "narrators": [], "duration_seconds": N, "codec_profile": "LC",
|
||||
-- "bit_rate": N, "asin": "...", "chapters": [
|
||||
-- { "index": 0, "title": "...", "start": 0.0, "end": 168.0 } ] }
|
||||
-- The chapters array IS the format abstraction: embedded m4b chapters
|
||||
-- and (later) folder-as-book files both write this exact shape.
|
||||
-- The chapters array IS the format abstraction: embedded m4b chapters,
|
||||
-- folder-of-tracks books and zip-of-audio books all write this exact
|
||||
-- shape. Multi-file books add additive keys: "layout"
|
||||
-- ("folder" | "zip"; absent = single file) and "tracks" (one entry
|
||||
-- per playable file with global-seconds start/end plus the path the
|
||||
-- /audio/:trackIndex endpoint resolves).
|
||||
audio_metadata JSONB
|
||||
);
|
||||
|
||||
@@ -263,6 +280,30 @@ ALTER TABLE media_items ADD COLUMN IF NOT EXISTS contributors_search TEXT[];
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS narrators TEXT[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS narrators_search TEXT[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS abridged BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
|
||||
-- A2b: subtitle + explicit flags, authors/publishers arrays (narrators
|
||||
-- precedent — arrays are write truth, varchar display columns regenerate).
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS subtitle TEXT;
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS explicit BOOLEAN NOT NULL DEFAULT FALSE;
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS authors TEXT[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS authors_search TEXT[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS publishers TEXT[] NOT NULL DEFAULT '{}';
|
||||
ALTER TABLE media_items ADD COLUMN IF NOT EXISTS publishers_search TEXT[] NOT NULL DEFAULT '{}';
|
||||
|
||||
-- Series ordinals are real-world decimals (0.5, 1.5 novellas); INTEGER
|
||||
-- truncated them to the wrong shelf order. In-place widening; no-op on a
|
||||
-- fresh install (column is already double precision from CREATE TABLE).
|
||||
ALTER TABLE media_items ALTER COLUMN series_number TYPE double precision
|
||||
USING series_number::double precision;
|
||||
|
||||
-- Backfill the arrays from the legacy single-value columns. Deliberately NO
|
||||
-- comma-splitting ("Tolkien, J.R.R." would corrupt); every existing row is
|
||||
-- effectively single-valued because the scanner historically kept only
|
||||
-- authors[0]. Guarded so the pass is free after the first startup.
|
||||
UPDATE media_items SET authors = ARRAY[author]
|
||||
WHERE author IS NOT NULL AND author <> '' AND authors = '{}';
|
||||
UPDATE media_items SET publishers = ARRAY[publisher]
|
||||
WHERE publisher IS NOT NULL AND publisher <> '' AND publishers = '{}';
|
||||
-- audio_metadata is declared in the CREATE TABLE above, which only fresh
|
||||
-- installs ever see — this ALTER is the upgrade path for pre-A2 installs
|
||||
-- (its absence 42703'd every scan and archive pass on upgraded deployments).
|
||||
@@ -312,6 +353,20 @@ BEGIN
|
||||
IF NEW.narrators_search IS NULL THEN
|
||||
NEW.narrators_search := '{}';
|
||||
END IF;
|
||||
-- A2b: authors/publishers arrays are NOT NULL — Go nil slices become
|
||||
-- explicit NULLs in params and would 23502 (the narrators lesson).
|
||||
IF NEW.authors IS NULL THEN
|
||||
NEW.authors := '{}';
|
||||
END IF;
|
||||
IF NEW.authors_search IS NULL THEN
|
||||
NEW.authors_search := '{}';
|
||||
END IF;
|
||||
IF NEW.publishers IS NULL THEN
|
||||
NEW.publishers := '{}';
|
||||
END IF;
|
||||
IF NEW.publishers_search IS NULL THEN
|
||||
NEW.publishers_search := '{}';
|
||||
END IF;
|
||||
RETURN NEW;
|
||||
END;
|
||||
$$ LANGUAGE plpgsql;
|
||||
@@ -329,6 +384,35 @@ CREATE TRIGGER set_library_type_name_on_library_change
|
||||
FOR EACH ROW
|
||||
EXECUTE FUNCTION set_library_type_name();
|
||||
|
||||
-- Display sync: `author`/`publisher` are denormalized display strings
|
||||
-- regenerated from the authors/publishers arrays on every write. The varchar
|
||||
-- columns face search (trgm), facets and API responses; the arrays are the
|
||||
-- write truth. array_to_string is STABLE on PG15 (not IMMUTABLE), so this
|
||||
-- must be a trigger — a generated column would reject it. Absent arrays
|
||||
-- yield NULL display (matches the legacy no-author row shape).
|
||||
DROP TRIGGER IF EXISTS sync_author_publisher_display_trg ON media_items;
|
||||
DROP FUNCTION IF EXISTS sync_author_publisher_display();
|
||||
CREATE OR REPLACE FUNCTION sync_author_publisher_display() RETURNS TRIGGER AS $$
|
||||
BEGIN
|
||||
-- Defense in depth: normalize NULL arrays first (this trigger may fire
|
||||
-- before set_library_type_name depending on name-ordered execution).
|
||||
IF NEW.authors IS NULL THEN
|
||||
NEW.authors := '{}';
|
||||
END IF;
|
||||
IF NEW.publishers IS NULL THEN
|
||||
NEW.publishers := '{}';
|
||||
END IF;
|
||||
NEW.author := CASE WHEN NEW.authors = '{}' THEN NULL
|
||||
ELSE array_to_string(NEW.authors, ', ') END;
|
||||
NEW.publisher := CASE WHEN NEW.publishers = '{}' THEN NULL
|
||||
ELSE array_to_string(NEW.publishers, ', ') END;
|
||||
RETURN NEW;
|
||||
END $$ LANGUAGE plpgsql;
|
||||
CREATE TRIGGER sync_author_publisher_display_trg
|
||||
BEFORE INSERT OR UPDATE ON media_items
|
||||
FOR EACH ROW
|
||||
EXECUTE FUNCTION sync_author_publisher_display();
|
||||
|
||||
-- Add GIN indexes for pg_trgm fuzzy search performance
|
||||
CREATE INDEX IF NOT EXISTS idx_media_items_author_trgm
|
||||
ON media_items USING GIN (author gin_trgm_ops);
|
||||
@@ -542,8 +626,33 @@ CREATE INDEX IF NOT EXISTS idx_processing_issues_library
|
||||
CREATE INDEX IF NOT EXISTS idx_processing_issues_severity
|
||||
ON processing_issues(severity, resolved);
|
||||
|
||||
-- Phase 11 (faceted plan): the ledger records ROW-LESS import failures
|
||||
-- too — a book whose media_items row was never created has no item to
|
||||
-- reference. media_item_id becomes nullable; file_path carries the
|
||||
-- failing file's library-relative path for those rows. The table-level
|
||||
-- UNIQUE stays (item-backed safety; row-less rows have NULL item ids,
|
||||
-- which unique constraints treat as pairwise-distinct). The widened
|
||||
-- dedup key is a COALESCE expression index covering BOTH classes in one
|
||||
-- shape: item-backed = (item, library, type, ''), row-less =
|
||||
-- (zero-uuid, library, type, path). An expression index (not NULLS NOT
|
||||
-- DISTINCT) because the two classes key on different columns.
|
||||
ALTER TABLE processing_issues ALTER COLUMN media_item_id DROP NOT NULL;
|
||||
-- TEXT, deliberately NOT VARCHAR(500): the ledger must be able to
|
||||
-- record paths LONGER than media_items' limit — an over-long path is
|
||||
-- itself a failure mode, and a 500-char cap would make the issue row
|
||||
-- die on the exact error it exists to report.
|
||||
ALTER TABLE processing_issues ADD COLUMN IF NOT EXISTS file_path TEXT;
|
||||
ALTER TABLE processing_issues ALTER COLUMN file_path TYPE TEXT;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_processing_issues_dedup
|
||||
ON processing_issues (
|
||||
COALESCE(media_item_id, '00000000-0000-0000-0000-000000000000'::uuid),
|
||||
library_id,
|
||||
issue_type,
|
||||
COALESCE(file_path, '')
|
||||
);
|
||||
|
||||
-- Add comment for documentation
|
||||
COMMENT ON TABLE processing_issues IS 'Tracks media items that cannot be properly processed in their assigned library due to format mismatches or other issues';
|
||||
COMMENT ON TABLE processing_issues IS 'Tracks media that failed processing in a library: item-backed issues (library-type mismatches, metadata extraction failures) and row-less import failures (no media_items row exists; keyed by library + file_path). Auto-resolved by later successful scans.';
|
||||
|
||||
|
||||
-- ============================================
|
||||
@@ -1169,6 +1278,28 @@ CREATE TABLE IF NOT EXISTS media_item_formats (
|
||||
CREATE INDEX IF NOT EXISTS idx_media_item_formats_media ON media_item_formats(media_item_id, format_type);
|
||||
CREATE INDEX IF NOT EXISTS idx_media_item_formats_sha256 ON media_item_formats(file_sha256);
|
||||
|
||||
-- media_scan_state: the scanner's per-item fingerprint cache (faceted
|
||||
-- plan Part 2). Deliberately NOT a media_items column: the library-
|
||||
-- scoped media list emits raw sqlc rows as JSON, and scanner
|
||||
-- bookkeeping must never leak into API payloads (nor churn the model
|
||||
-- or the contract pins). One row per item; cascade on delete.
|
||||
-- manifest shape v1:
|
||||
-- {"v":1,"entries":[{"n":"metadata.opf","s":3013013,"m":1697500000123456789,"h":"<sha256>"}, ...]}
|
||||
-- n basename, s size, m mtime as unix NANOS (second granularity
|
||||
-- would hide edits landing in the backfill scan's own second), h
|
||||
-- sha256 of the content — present for ALL entries, covers included:
|
||||
-- the repoint reconciliation needs a baseline hash for any stat-
|
||||
-- drifted sidecar. "ev" (extractor version, default 1): the parser
|
||||
-- version that produced the item's row. ABSENT means "v1-or-unknown"
|
||||
-- — backfill stamps are unset so a row never claims a version its
|
||||
-- data can't prove; every post-extraction store stamps the setting's
|
||||
-- current value. A stored ev != scan_extractor_version → re-extract
|
||||
-- (bounded per scan, resumable across polls).
|
||||
CREATE TABLE IF NOT EXISTS media_scan_state (
|
||||
media_item_id UUID PRIMARY KEY REFERENCES media_items(id) ON DELETE CASCADE,
|
||||
manifest JSONB
|
||||
);
|
||||
|
||||
-- Create device_file_aliases table (track file paths per device for cross-device matching)
|
||||
CREATE TABLE IF NOT EXISTS device_file_aliases (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
@@ -1252,12 +1383,21 @@ CREATE TABLE IF NOT EXISTS collection_items (
|
||||
added_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
|
||||
added_by_user_id UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
excluded BOOLEAN DEFAULT false,
|
||||
-- Manual playlist ordering (reading-order collections). NULL = un-ordered
|
||||
-- legacy item; ordering falls back to added_at ASC.
|
||||
position INTEGER,
|
||||
UNIQUE(collection_id, media_item_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_collection_items_collection ON collection_items(collection_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_collection_items_media ON collection_items(media_item_id);
|
||||
|
||||
-- Playlist ordering (A2c): position-first ordering for collection listings;
|
||||
-- NULLs fall back to added_at ASC.
|
||||
ALTER TABLE collection_items ADD COLUMN IF NOT EXISTS position INTEGER;
|
||||
CREATE INDEX IF NOT EXISTS idx_collection_items_order
|
||||
ON collection_items(collection_id, position ASC NULLS LAST);
|
||||
|
||||
-- Index for excluding auto-assigned items
|
||||
CREATE INDEX IF NOT EXISTS idx_collection_items_excluded ON collection_items(collection_id, excluded)
|
||||
WHERE excluded = true;
|
||||
@@ -1353,6 +1493,8 @@ CREATE TABLE IF NOT EXISTS unlinked_books (
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_unlinked_books_device ON unlinked_books(device_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_unlinked_books_content_id ON unlinked_books(content_id);
|
||||
-- One queue row per (device, document): the sync ingest upserts on this pair.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_unlinked_books_device_content ON unlinked_books(device_id, content_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_unlinked_books_resolved ON unlinked_books(resolved);
|
||||
|
||||
-- Create saved_filters for users to be able to filter different parts of the application
|
||||
@@ -1426,6 +1568,24 @@ CREATE TABLE IF NOT EXISTS reading_speed (
|
||||
CREATE INDEX IF NOT EXISTS idx_reading_speed_user ON reading_speed(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_reading_speed_item ON reading_speed(media_item_id);
|
||||
|
||||
-- Per-user PER-BOOK playback speed preference (player plan §8): a
|
||||
-- preference, NOT a measurement (reading_speed holds measured rates),
|
||||
-- and NOT a reading_progress column (a speed tap must not create a
|
||||
-- "currently reading" row). Resolution lives at the chapters endpoint:
|
||||
-- per-book row → users.default_audio_speed → 1.0.
|
||||
CREATE TABLE IF NOT EXISTS audio_playback_speed (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
media_item_id UUID NOT NULL REFERENCES media_items(id) ON DELETE CASCADE,
|
||||
speed REAL NOT NULL DEFAULT 1.0,
|
||||
created_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ DEFAULT NOW(),
|
||||
UNIQUE(user_id, media_item_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_audio_playback_speed_user ON audio_playback_speed(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_audio_playback_speed_item ON audio_playback_speed(media_item_id);
|
||||
|
||||
-- Dictionary cache (for offline use)
|
||||
CREATE TABLE IF NOT EXISTS dictionary_cache (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
|
||||
@@ -102,3 +102,315 @@ cover_image_path, has_conflict, format_group, badge`.
|
||||
origin — its rows are unchanged. Request-parse widening documented per
|
||||
rule 2; response shapes untouched; locked by
|
||||
TestOriginWriterAttribution.
|
||||
|
||||
## Changelog entry — multi-file audiobooks (2026-10-18)
|
||||
|
||||
- `GET /api/media-items/:id/chapters` gained ADDITIVE response keys
|
||||
`layout` ("" for single-file books, `"folder"` or `"zip"` for
|
||||
multi-file books) and `tracks[]` (`index, url, title, path, start,
|
||||
end, duration_seconds, size_bytes`; empty for single-file books).
|
||||
Existing keys (`media_item_id, chapters, duration_seconds,
|
||||
codec_profile`) unchanged — no pin existed on this endpoint, so no
|
||||
pin widened; recorded here per rule 2's spirit.
|
||||
- New sibling endpoint `GET /api/media-items/:id/audio/:trackIndex`
|
||||
streams one track of a multi-file book (Range-capable). New surface,
|
||||
not a modification; gated by library visibility like `ServeFile`
|
||||
(403 on hidden libraries).
|
||||
- `media_items.audio_metadata` payload gained additive JSON keys
|
||||
`layout` and `tracks` (see schema.sql comment). Consumers parse
|
||||
additively; stored single-file payloads are unchanged.
|
||||
|
||||
## 2026-10-19 — A2b: metadata array/model overhaul (phase 1)
|
||||
|
||||
- `media_items.series_number` widened INTEGER → DOUBLE PRECISION across all
|
||||
surfaces. JSON serializers emit the same key; values are JSON numbers, so
|
||||
integer-valued ordinals are byte-identical ("3") and decimal ordinals
|
||||
(0.5/1.5) now serialize as decimals instead of truncating. Clients reading
|
||||
JSON numbers are unaffected; the Android client parses either form.
|
||||
- `GET /api/media-items*` / detail responses gain ADDITIVE keys:
|
||||
`subtitle` (string), `explicit` (bool), `authors` ([]string, write truth —
|
||||
legacy `author` display string still emitted, now trigger-regenerated),
|
||||
`publishers` ([]string; legacy `publisher` likewise).
|
||||
- `POST/PUT /api/media-items*` accept ADDITIVE request keys `subtitle`,
|
||||
`explicit`, `authors`, `publishers`; legacy `author`/`publisher` singles
|
||||
still accepted (arrays win when both present).
|
||||
- Override keys unchanged: `author`/`publisher` override names are preserved
|
||||
for existing rows and now protect the arrays.
|
||||
|
||||
## 2026-10-19 — A2c: collections/playlists phase
|
||||
|
||||
- `GET /api/collections/:id` gained ADDITIVE `up_next` (array of
|
||||
`{id, title, author, cover, format, progress}` — the static collection's
|
||||
reading queue head in position order, viewer-progress-aware; empty array
|
||||
for smart collections).
|
||||
- `GET /api/dashboard/sections` rows gained ADDITIVE `is_playlist` (bool —
|
||||
collection was imported from a sidecar ordering construct) and
|
||||
`complete` (bool — hide_finished filtered every item out of a non-empty
|
||||
collection; render "playlist complete").
|
||||
- `PUT /api/collections/:id/items/order` is a NEW endpoint (owner-only,
|
||||
full ordered id list → positions 1..n).
|
||||
- User collections are dashboard-eligible by default (the
|
||||
show_on_dashboard gate is gone from GetUserCollectionsForDashboard);
|
||||
hiding stays dashboard-prefs (`hidden_collections`) territory.
|
||||
- `GET /api/media-items` (unscoped resolved-map branch) and
|
||||
`GET /api/media-items/:id` gained the same ADDITIVE keys as the raw
|
||||
branch: `subtitle`, `explicit`, `authors`, `publishers`.
|
||||
- `GET /api/media-items/search` rows gained the same ADDITIVE keys plus
|
||||
the search pairs `authors_search`, `publishers_search`,
|
||||
`narrators_search` (the unified SELECT now carries the new columns).
|
||||
- `author_filter` / `publisher_filter` are now per-element against the
|
||||
arrays; a comma-joined value is OR-within-facet multi-select. The
|
||||
param NAMES and single-value behavior are unchanged.
|
||||
|
||||
## 2026-10-19 — Faceted navigation Batch A: new filter facets
|
||||
|
||||
- `GET /api/media-items/search` accepts ADDITIVE REQUEST params (query
|
||||
string, response shapes unchanged): `story_arc_filter`,
|
||||
`imprint_filter`, `edition_filter` (scalar-fuzzy),
|
||||
`age_rating_filter`, `format_group_filter`, `manga_type_filter`,
|
||||
`reading_direction_filter` (exact), `explicit_filter`,
|
||||
`abridged_filter` (bool: absent = unset, "true"/"false" = bound).
|
||||
A comma-joined value is OR-within-facet multi-select on every new
|
||||
TEXT facet (the SQL splits via string_to_array). `services/facet_registry.go`
|
||||
is the Go-side source of truth; the drift-guard unit test
|
||||
(`TestFacetRegistrySQLDrift`) pins registry ↔ SQL agreement.
|
||||
- Genre model v2 changes SCANNER OUTPUT (not a wire contract): sidecar
|
||||
genre selection now stores only top-level shelves in `genre`
|
||||
(subgenres become tags mapped to their parent shelf; unknown-only
|
||||
input leaves `genre` empty). Existing rows are unaffected until a
|
||||
rescan/re-extract.
|
||||
|
||||
## 2026-10-19 — Faceted navigation Batch B/C: chips, FacetLink, facets endpoint
|
||||
|
||||
- `GET /api/media-items/facets?facet=X&library_id=...` is a NEW pinned
|
||||
surface (registry-driven value+count listing for the filter panel):
|
||||
200 → `{facet, values[]: {value, display?, count}}` (display present
|
||||
only for language codes → names); 400 for unknown facets and for
|
||||
bool/range facets (they render as toggles, not lists); 403 for an
|
||||
explicit library_id outside the requester's visible set.
|
||||
- `GET /api/media-items/search` (JSON) and its HTMX grid partial now
|
||||
share ONE param parser (`handlers.ParseMediaSearchParams`). Behavior
|
||||
notes: the HTMX branch previously dropped `publisher_filter` and
|
||||
`narrator_filter` silently — it now honors every facet param; and an
|
||||
explicit hidden-library `library_id` is now a 403 on BOTH branches
|
||||
(was: silent empty set on JSON). Response shapes are untouched; the
|
||||
search row pin (47 keys) is unchanged.
|
||||
- `language_filter` now normalizes comma elements through the ingest
|
||||
language map, so display names ("English") match rows stored as ISO
|
||||
codes ("en"). The param name is unchanged; single-code values behave
|
||||
identically.
|
||||
- Web: the bookshelf gains an active-filter chips bar (client-local
|
||||
form state rendered by Alpine; templ owns the markup) and
|
||||
`applyURLFilters` accepts the new facet params.
|
||||
|
||||
## 2026-10-19 — Scanner Phase 4: manifest-gated sidecar detection
|
||||
|
||||
- No wire change: `media_scan_state` lives in its own table, so the
|
||||
raw-row list JSON cannot contain scanner state (guarded by an
|
||||
assertion in TestScanManifestLifecycle).
|
||||
- Scanner BEHAVIOR: size-stable books are now checked against a stored
|
||||
sidecar fingerprint (`{"v":1,"entries":[{n,s,m-nanos,h-sha256},...]}`).
|
||||
Stat-pair-identical → skip with zero reads; stat drift → hash
|
||||
reconciliation (equal hash = host repoint, stat pair refreshed, no
|
||||
re-extraction; differing hash = metadata re-extract + fresh manifest,
|
||||
metadata_overrides protected, playlist collections synced on the
|
||||
change path — a "(Reading Order)" entry added to an EXISTING book now
|
||||
creates its collection). NULL stored manifest → backfill only, no
|
||||
re-extraction (deploy-day rule). Multifile folder/zip books keep the
|
||||
Phase 3 behavior (their change path does not consume sidecar
|
||||
extractors yet — gating them would flip books "changed" for nothing).
|
||||
|
||||
## 2026-10-19 — Scanner Phase 5: quiet scans, coalescing, watcher probe
|
||||
|
||||
- `POST /api/scanner/scan` now answers **409** (with `active_job_id`)
|
||||
when a scan for the same library is already pending/running —
|
||||
coalescing instead of stacking a second full scan. The endpoint was
|
||||
never pinned; no existing response shape changed. Poll ticks and
|
||||
watcher events coalesce silently (no second job enqueued).
|
||||
- Scan jobs carry a `trigger` label (poll/watcher/admin/initial); the
|
||||
per-scan log is now ONE summary line:
|
||||
`Scan of <folders> (trigger: <t>): N files, N changed, N new, Xs, N errors`.
|
||||
Per-file "processing/success/skipping" chatter is gone; inserts,
|
||||
changes, and errors still log individually.
|
||||
- `JobResult` gains additive `job_type` (coalescing needs it; the
|
||||
scan-progress surfaces tolerate unknown keys).
|
||||
- Watcher: a failed capability probe (one `Add` on a root folder)
|
||||
disables the watcher with ONE startup line instead of per-directory
|
||||
warning spam.
|
||||
|
||||
## 2026-10-20 — Stale pin repair (behavior shipped in 7e586df4)
|
||||
|
||||
- `GET /api/media-items/search` with an explicit `library_id` outside
|
||||
the requesting user's visible set — INCLUDING a nonexistent id —
|
||||
answers **403** (`library not accessible`), not 404. The family-lens
|
||||
standing rule (AGENTS.md) mandates 403 for every out-of-set id so
|
||||
library existence is not enumerable; Phase 1 (7e586df4) shipped it
|
||||
and documented it here, but the `invalid library_id` subtest in
|
||||
search_test.go still pinned the pre-family-lens 404. Pin widened to
|
||||
the shipped 403; no server change (discovered during the Phase 6
|
||||
full-suite run; the Phase 6 diff touches no handler code).
|
||||
- `TestWebSocketPingPong` DELETED: it asserted a text-level
|
||||
`{"type":"ping"}` → `{"type":"pong"}` exchange that no server code
|
||||
has ever implemented and no client (web/src, Android) has ever sent.
|
||||
It passed vacuously through its read-timeout path (deadline error
|
||||
skipped the asserts) until interleaved `scan_progress` broadcasts
|
||||
from the live server's scans started reaching the socket first.
|
||||
Keepalive is the standard protocol-level ping the hub already sends.
|
||||
Never-built behavior is not a pinnable contract.
|
||||
|
||||
## 2026-10-20 — Scanner Phase 7: all-facets shape + SSR facet panel
|
||||
|
||||
- `GET /api/media-items/facets` gains a SECOND shape: with NO `facet`
|
||||
param it answers `{"facets": {<param>: [{"value","display?","count"},
|
||||
…]}}` — every listable facet in one response, counts computed under
|
||||
the ACTIVE FILTERS (the same query params the search endpoint takes;
|
||||
each facet is counted under the others — its own filter does not
|
||||
restrict its own list). `library_id` is optional on this shape:
|
||||
absent = the All-Libraries view. Family-lens unchanged: an explicit
|
||||
out-of-set `library_id` is 403 on both shapes.
|
||||
- The PINNED single-facet shape (`?facet=X&library_id=…` →
|
||||
`{facet, values[]}`) is byte-identical in semantics; its
|
||||
implementation now routes through the same UNION-ALL query with
|
||||
neutral filters (the 14 per-column SQL queries were deleted — no
|
||||
wire change).
|
||||
- NEW SSR partial: `GET /media-items/facet-panel` (session-cookie
|
||||
auth, bookshelf-protected group) renders the drawer's value lists —
|
||||
templ-rendered, registry order, library-type-scoped sections, active
|
||||
values marked. An out-of-set explicit `library_id` is 403 HTML.
|
||||
- The 14 per-column facet SQL queries are replaced by one
|
||||
`FacetCountsUnderFilters` UNION-ALL statement; the registry drift
|
||||
guard now reflects over both generated param structs.
|
||||
|
||||
## 2026-10-20 — Filter plumbing: facet-panel partial removed; field-values `library_id` optional
|
||||
|
||||
- `GET /media-items/facet-panel` (the SSR partial documented above,
|
||||
same day) is REMOVED — the panel UX was rejected on scale grounds
|
||||
and the filter drawer becomes a rule builder with no other mode.
|
||||
Never contract-pinned while it lived (same-day entry, same-day
|
||||
removal); `TestFacetPanelPartial` was deleted with the behavior.
|
||||
The all-facets JSON endpoint and its two shapes are UNCHANGED and
|
||||
stay pinned; `FacetCountsUnderFilters` and the registry drift guard
|
||||
keep their roles (the engine outlives the panel).
|
||||
- `GET /api/media-items/search?field=value` autocomplete
|
||||
(author/genre/tags/narrator/series/language): `library_id` is now
|
||||
OPTIONAL — absent means every library VISIBLE to the caller (the
|
||||
per-user family lens applies inside the query), fixing the All-Books
|
||||
view's "library_id is required" 400. An explicit out-of-set
|
||||
`library_id` is now a 403 on this endpoint too (was: silent empty
|
||||
result for hidden libraries — the silent pass is the anti-pattern
|
||||
the standing rule forbids). The single-facet
|
||||
`/api/media-items/facets?facet=X` shape still REQUIRES `library_id`
|
||||
— that pin is unchanged.
|
||||
|
||||
## 2026-10-20 — Filter rule builder: the `qf` wire param + composed engines (faceted plan Phase 9)
|
||||
|
||||
- NEW wire param on the search endpoint family: `qf` — a JSON rule tree
|
||||
`{mode: all|any, children: [{field, op, values} | {mode, children}]}`.
|
||||
Fields/operators are registry-typed constants
|
||||
(`services/filter_registry.go` — ~55 fields incl. every book column,
|
||||
file/lifecycle facts, the three HOLDING STRUCTURES — library,
|
||||
collections (playlists included, `excluded IS NOT TRUE` membership),
|
||||
series (primary + `alternate_info` full membership) — and the whole
|
||||
user-relative class reading the VIEWER's rows). Unknown fields or
|
||||
operators, disallowed operator/field pairs, malformed values, empty
|
||||
groups, and nesting beyond 8 are a **400** (`ErrInvalidFilterTree`),
|
||||
never a silently-ignored token. Values are always bound pgx
|
||||
parameters — an injection-shaped value is just a value.
|
||||
- **Precedence (pinned)**: when a request carries BOTH `qf` and flat
|
||||
facet params, **qf wins** — the flat params are ignored by the engine.
|
||||
Flat links keep working unchanged server-side (and a tree equivalent
|
||||
to a flat link returns identical rows — pinned by test).
|
||||
- The composed engines (`services/filter_query.go`) run the mandatory
|
||||
WHERE head (visibility join + archived + missing-scan — family lens)
|
||||
for search, suggestions, and count alike; row shape is identical to
|
||||
`SearchMediaItemsUnifiedRow`. A rule naming a hidden library matches
|
||||
nothing; an explicit out-of-set `library_id` stays a 403 on every
|
||||
entry point.
|
||||
- NEW endpoint `GET /api/media-items/filter-count?qf=…[&library_id=…]`
|
||||
— the builder's live count: a true COUNT(*) of the rules + scope
|
||||
(not the page length). Family-lens 403 applies.
|
||||
- Field-values autocomplete UPGRADED in place (same route): ANY registry
|
||||
field key in its bare form (`?note_text=…`, `?collection=…`,
|
||||
`?library=…`) routes to the field-values path; with a `qf` param the
|
||||
OTHER rules bind as filters (symmetric under-filter suggestions,
|
||||
zero-match values never offered). Without a tree, the classic six
|
||||
(author/genre/tags/narrator/series/language) keep their static
|
||||
queries and response shape — pinned contract, unchanged. ID-valued
|
||||
suggestions (library/collection/added_by) add an OPTIONAL
|
||||
`display` field to result items (`value` stays the id; `omitempty`
|
||||
elsewhere).
|
||||
- The Phase-7 facet-panel partial stays removed; the all-facets JSON
|
||||
endpoint and its two shapes are unchanged (they serve flat params —
|
||||
the builder's UI does not use them).
|
||||
|
||||
## 2026-10-20 — Scanner Phase 10: extractor version (ev) mechanism
|
||||
- `media_scan_state.manifest` JSONB gains the reserved `ev` key:
|
||||
the parser version that produced the item's CURRENT row. ABSENT
|
||||
means "v1-or-unknown" — decode normalizes it to 1.
|
||||
- NEW setting `scan_extractor_version` (int, default 1, min 1,
|
||||
category scanner): the unified `GET/PUT /api/system/settings` list
|
||||
now includes this key (additive; the list was not a pinned closed
|
||||
set). Writes stay settings-API-only — direct SQL leaves the registry
|
||||
cache stale (known gotcha).
|
||||
- Scanner behavior: a stored manifest whose ev ≠ the setting is
|
||||
treated as CHANGED → full re-extract + the fresh manifest stamps
|
||||
the new ev. Stamping is TRUTHFUL-ONLY: create paths and
|
||||
post-extraction stores stamp; deploy-day BACKFILL and the
|
||||
extract-failure guard store UNSET (the row never claims a version
|
||||
its data can't prove — a mid-backfill bump self-heals next poll).
|
||||
Repoint reconciliation preserves the stored ev (stat-only refresh,
|
||||
nothing re-extracted).
|
||||
- Catch-up is BOUNDED per scan pass (`evReextractBudgetPerScan = 100`,
|
||||
atomic, reset at each ScanFolders): resumable across polls, no
|
||||
force rescan, no flood. Budget exhaustion leaves ev stale → later
|
||||
polls resume.
|
||||
- Deploy-day rule (unchanged, re-verified live): manifests without ev
|
||||
ride v1; at default setting 1 the first post-release scan re-extracts
|
||||
NOTHING (0 changed on 1198 files).
|
||||
|
||||
## 2026-10-20 — Scanner Phase 11: the processing-issues ledger
|
||||
- `processing_issues.media_item_id` is now NULLABLE and the table gains
|
||||
`file_path` (TEXT, deliberately unbounded — an over-long path is
|
||||
itself a failure mode the ledger must survive): ROW-LESS import
|
||||
failures (no media_items row exists) key on (library, file_path);
|
||||
item-backed rows keep (item, type). The widened dedup is the
|
||||
COALESCE-expression unique index idx_processing_issues_dedup; the old
|
||||
table-level UNIQUE stays (row-less rows have NULL items, pairwise
|
||||
distinct under unique constraints).
|
||||
- `GET /api/libraries/:id/issues/list` — the media_items join is now
|
||||
LEFT: row-less issues APPEAR (they were unrepresentable before).
|
||||
Response shape unchanged for item-backed rows; row-less rows carry
|
||||
`media_item_id: ""` and a `title` fallen back to the file path.
|
||||
- FAMILY LENS now applies to ALL issues endpoints (list, stats, resolve,
|
||||
delete) and the SSR page — they leak media metadata. An explicit
|
||||
library id outside the requester's visible set is a **403** (this
|
||||
tightens one pre-lens behavior: a nonexistent library id used to
|
||||
return 200-empty; it is trivially out-of-set → 403 now, pinned).
|
||||
- NEW scanner wirings: CreateMediaItem failure → row-less
|
||||
`import_failed` (error); extraction hard-fail with fallback import →
|
||||
item-backed `metadata_extraction_failed` (warning); change-path
|
||||
re-extraction failure → item-backed `metadata_extraction_failed`.
|
||||
- Auto-resolution: a successful import resolves row-less issues by
|
||||
(library, file_path); a successful (re)extraction resolves the item's
|
||||
open issues. Unchanged-skip scans do NOT resolve (extraction never
|
||||
re-ran — the degraded metadata is still degraded). A repeat failure
|
||||
reopens a resolved row (upsert sets resolved=false).
|
||||
|
||||
## 2026-10-21 — Phase 13: force rescan = original scan; the manual-edits filter
|
||||
- **Force rescan contract change (owner-directed)**: force rescan now
|
||||
behaves like an original import — the full metadata chain (sidecars →
|
||||
folder structure → tags → fallbacks) re-lands on MULTI-FILE books
|
||||
too (the pre-Phase-13 editorial-keep skip is retired; the owner's
|
||||
metadata_overrides count in the single digits, and overrides are the
|
||||
only protection, honored inside the apply). A force pass also STAMPS
|
||||
each re-derived book's manifest with the current parser version
|
||||
(success-only) and closes its open processing issues. Single-file
|
||||
force rescan already behaved this way — now pinned by test.
|
||||
- **NEW filter field** `manually_edited` ("Manually edited", boolean):
|
||||
is_true = `cardinality(mi.metadata_overrides) > 0`, is_false = `= 0`.
|
||||
Composed-engine (qf) only — no flat param, no value menu, no
|
||||
suggestion surface; family lens inherited.
|
||||
- Deployment ordering note (owner Q&A): set scan_extractor_version
|
||||
BEFORE Rescan All — force rescan stamps books with the current
|
||||
setting, so bump-first + force = one pass of work; force-first +
|
||||
bump = the trickle re-derives everything again (wasted, not harmful).
|
||||
@@ -258,17 +258,82 @@ go tool cover -html=coverage.out
|
||||
|
||||
### Integration Tests
|
||||
|
||||
Integration tests hit the LIVE dev database and bind-mounted paths, both
|
||||
of which exist only inside the Docker stack. A bare host `go test
|
||||
./cmd/server/tests` fails with database-auth errors on every machine
|
||||
that isn't running the compose stack — the container is the only
|
||||
supported runner.
|
||||
|
||||
```bash
|
||||
# Run integration tests (may hit rate limits)
|
||||
go test ./cmd/server/tests -v
|
||||
# Run the whole integration suite (pre-commit validation — NOT for
|
||||
# iterating on a change; some tests are coupled to the shared dev DB's
|
||||
# live state and this environment)
|
||||
make test-integration
|
||||
|
||||
# Run with test mode (recommended)
|
||||
TEST_MODE=true RATE_LIMIT_ENABLED=false go test ./cmd/server/tests -v
|
||||
|
||||
# Run specific test
|
||||
go test ./cmd/server/tests -run TestAuth -v
|
||||
# Iterate on YOUR change: run only your own tests, in the container
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml --profile tests run --rm tests \
|
||||
go test ./cmd/server/tests -run 'TestYourPrefix' -v -timeout 600s
|
||||
```
|
||||
|
||||
**Run only your own tests while iterating.** A full-suite run rebuilds
|
||||
the test image and takes minutes; more importantly, several tests read
|
||||
live state (the dev DB's `system_settings`, real library folders, a
|
||||
periodic backup scan broadcasting over websocket) and can fail on
|
||||
timing or on data your change never touched. Filtered runs are cheap,
|
||||
deterministic for your scope, and don't fight the live reader for the
|
||||
database.
|
||||
|
||||
Test-created data rules (harness cleanup depends on them): users use
|
||||
`@tests.bookhoard.internal` / `@example.com` emails, libraries and
|
||||
collections carry "test"-prefixed names, and cleanups delete test
|
||||
libraries (cascades handle the rest) rather than table-wide deletes.
|
||||
|
||||
### Media Fixtures (audio/books)
|
||||
|
||||
**Never commit binary media.** The Go toolchain and Docker prod builds
|
||||
ignore `internal/services/testdata/` entirely (gitignored by policy —
|
||||
see `.gitignore`), and the production image never copies `internal/`.
|
||||
Generate what a test needs at runtime with ffmpeg (shipped in both the
|
||||
test-runner and runtime images):
|
||||
|
||||
```bash
|
||||
# The five audio containers (audiobook library accepts all of these)
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libmp3lame -b:a 32k plain.mp3
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a flac book.flac
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libopus -b:a 24k book.opus
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a libvorbis -q:a 1 book.ogg
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a aac -b:a 32k book.aac
|
||||
ffmpeg -f lavfi -i sine=frequency=440:duration=0.3 -codec:a pcm_s16le book.wav
|
||||
|
||||
# Tags (album=book title, album_artist=author, composer=narrator —
|
||||
# Audible's own convention — plus disc/track ordering keys)
|
||||
ffmpeg -i in.mp3 -codec copy \
|
||||
-metadata title="Chapter Three" -metadata album="The Book" \
|
||||
-metadata album_artist="Author" -metadata composer="Narrator" \
|
||||
-metadata track="3/9" -metadata disc="2/3" \
|
||||
-metadata grouping="Series" -metadata genre="Audiobook" -metadata date=2020 out.mp3
|
||||
|
||||
# Embedded cover art (mp3 APIC / m4b covr): the picture rides as the
|
||||
# only video stream
|
||||
ffmpeg -i cover.jpg -i audio-only.mp3 -map 1:a -map 0:v -c:v copy \
|
||||
-disposition:v attached_pic tagged.mp3
|
||||
|
||||
# Embedded m4b chapters: an ffmetadata sidecar file
|
||||
printf ';FFMETADATA1\n[CHAPTER]\nTIMEBASE=1/1000\nSTART=0\nEND=500\ntitle=Part One\n' > chapters.txt
|
||||
ffmpeg -i in.m4b -i chapters.txt -map 0:a -map 1 -map_metadata 1 -codec copy -f ipod chaptered.m4b
|
||||
```
|
||||
|
||||
CUE sheets and m3u playlists are plain text — write them inline in the
|
||||
test. **The one thing ffmpeg cannot synthesize: Audible freeform atoms**
|
||||
(`----:com.apple.iTunes:SERIES`, `:SUBTITLE`, `:ASIN`, `:ABRIDGED`) —
|
||||
those ingestion paths are covered by unit tests using synthetic
|
||||
`ffprobeReport` values instead of binary files.
|
||||
|
||||
Manual end-to-end testing against the live system: drop fixture books
|
||||
into `uploads/Audiobooks/Ingestion Fixtures/` — **one book per
|
||||
subfolder** (a directory with ≥2 audio files ingests as ONE
|
||||
folder-of-tracks book) — and run a library scan from the admin UI.
|
||||
|
||||
### API Testing with Bruno OpenCollection YAML
|
||||
|
||||
```bash
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
# Get Audio Chapters
|
||||
|
||||
The format-abstraction chapter list for audiobooks, sourced from
|
||||
`media_items.audio_metadata`. Embedded m4b chapters, folder-of-tracks
|
||||
books and zip-of-audio books all write the same chapter shape; the
|
||||
locator `{chapter, audio_offset_seconds}` and this endpoint never learn
|
||||
which file model produced them.
|
||||
|
||||
**Endpoint**: `GET /api/media-items/{id}/chapters`
|
||||
**Auth**: Required (JWT)
|
||||
|
||||
## Response
|
||||
|
||||
```json
|
||||
{
|
||||
"media_item_id": "…",
|
||||
"chapters": [
|
||||
{ "index": 0, "title": "Chapter One", "start": 0.0, "end": 168.0 }
|
||||
],
|
||||
"duration_seconds": 36000.0,
|
||||
"codec_profile": "mp3",
|
||||
"layout": "folder",
|
||||
"tracks": [
|
||||
{
|
||||
"index": 0,
|
||||
"url": "/api/media-items/{id}/audio/0",
|
||||
"title": "01 First",
|
||||
"path": "Author/Title/01 First.mp3",
|
||||
"start": 0.0,
|
||||
"end": 168.0,
|
||||
"duration_seconds": 168.0,
|
||||
"size_bytes": 2700000
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
- `layout` is `""` for single-file books (the Batch A model — the whole
|
||||
file streams through the `/uploads` path), `"folder"` for a directory
|
||||
of track files and `"zip"` for a zip-of-audio book.
|
||||
- `tracks` is empty for single-file books and carries one entry per
|
||||
playable file otherwise. `start`/`end` are cumulative GLOBAL seconds —
|
||||
the same coordinate system chapters, bookmarks and
|
||||
`audio_offset_seconds` use.
|
||||
|
||||
## Error Responses
|
||||
|
||||
| Code | Description |
|
||||
| ---- | ------------------------------- |
|
||||
| 400 | Invalid media item id |
|
||||
| 401 | Missing/invalid token |
|
||||
| 404 | Book not found |
|
||||
| 500 | audio_metadata failed to parse |
|
||||
@@ -0,0 +1,44 @@
|
||||
# Get Audio Track
|
||||
|
||||
Streams one playable track of a multi-file audiobook (folder-of-tracks
|
||||
or zip-of-audio book). Track indexes and bounds come from the track
|
||||
manifest on [Get Audio Chapters](get_audio_chapters.md).
|
||||
|
||||
**Endpoint**: `GET /api/media-items/{id}/audio/{trackIndex}`
|
||||
**Auth**: Required (JWT — Bearer header or session cookie; the web
|
||||
player's `<audio>` element authenticates by cookie)
|
||||
|
||||
## Serving
|
||||
|
||||
- **Folder books**: the track is served straight from its real file
|
||||
(`http.ServeContent`) — full `Range` support.
|
||||
- **Zip books**: the entry is extracted in place. Forward seeks discard
|
||||
bytes in the open stream; backward seeks re-open the entry and skip
|
||||
(deflate has no random access — cheap at audio bitrates). `Range`
|
||||
requests are honored.
|
||||
|
||||
Single-file books have no track manifest — they stream through the
|
||||
`/uploads` path exactly as before and this endpoint answers 404.
|
||||
|
||||
## Library visibility
|
||||
|
||||
The item's library must be visible to the requester; a hidden library
|
||||
answers **403** on every track (same gate as the download endpoint).
|
||||
|
||||
## Error Responses
|
||||
|
||||
| Code | Description |
|
||||
| ---- | -------------------------------------------- |
|
||||
| 400 | Invalid media item id or track index |
|
||||
| 401 | Missing/invalid token |
|
||||
| 403 | Library not visible to the requester |
|
||||
| 404 | Book/track not found, or single-file book |
|
||||
| 500 | audio_metadata failed to parse |
|
||||
|
||||
## Example
|
||||
|
||||
```bash
|
||||
curl -H "Authorization: Bearer $TOKEN" \
|
||||
-H "Range: bytes=0-1023" \
|
||||
"http://localhost:8765/api/media-items/550e8400-.../audio/0"
|
||||
```
|
||||
@@ -90,8 +90,11 @@ Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
|
||||
### Absent data
|
||||
|
||||
`Audio Tracks` (Audiobookshelf's per-file count) is deliberately absent:
|
||||
Bookhoard audiobooks are one file per book until multi-file support
|
||||
lands, so a file count would just equal `items`. See HANDOFF item 9.
|
||||
multi-file audiobooks (folder-of-tracks / zip books) now ingest as ONE
|
||||
item whose track list lives in `audio_metadata.tracks`, so a file count
|
||||
would still not equal a per-book "tracks" metric without a new query.
|
||||
Add it by aggregating `jsonb_array_length(audio_metadata->'tracks')` if
|
||||
the panel is ever wanted.
|
||||
|
||||
## Error Responses
|
||||
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# Audiobook metadata: sources & precedence
|
||||
|
||||
Every audiobook ingestion path (single-file, folder-of-tracks, zip-of-
|
||||
audio) fills the same fields from the same ranked sources. Each source
|
||||
**only fills fields the earlier ones left empty** — curated data always
|
||||
outranks guesses — and every source is optional: a book still ingests
|
||||
(title from the filename) when all of them are absent. Admin edits made
|
||||
after ingest are protected by the metadata-override mechanism on
|
||||
rescans, regardless of source quality.
|
||||
|
||||
## Book-level metadata (title, author, narrators, series, …)
|
||||
|
||||
| Rank | Source | Feeds |
|
||||
|---|---|---|
|
||||
| 1 | **Sidecars** — `metadata.opf` (Calibre), `metadata.json` (Audiobookshelf/Libation, array-or-string tolerant) | title (+subtitle join), author, series + index, narrators, abridged, ASIN, tags/genres, dates, publisher, description |
|
||||
| 2 | **Folder structure** — `Author/Title`, `Author/Series, Book #N - Title`, `Author/Series/Title` | title, author, series + index |
|
||||
| 3 | **Embedded tags** (via ffprobe) | `album` → title; `album_artist` → author (`artist` fallback); `composer` → **narrators** (Audible's own convention); `grouping` → series; `date` → publish date; `genre` → genre; Audible freeform atoms `----:com.apple.iTunes:SERIES`/`:SERIES-PART`/`:SUBTITLE`/`:ASIN`/`:ABRIDGED` |
|
||||
| 4 | **Filename/folder name** | title last of all (created only — never lets a rescan blank a title) |
|
||||
|
||||
Abridged is the one guarded field: an explicit sidecar `false` wins even
|
||||
when a tag says `true` (the sidecar's presence is tracked, not just its
|
||||
value). Narrators from the composer tag stay ONE string — composer
|
||||
values have no reliable multi-narrator delimiter, and splitting
|
||||
"Vance, Simon" would manufacture fake people.
|
||||
|
||||
## Chapters (the `{index, title, start, end}` array)
|
||||
|
||||
| Rank | Source |
|
||||
|---|---|
|
||||
| 1 | Sidecar `metadata.json` `chapters[]` (curated, global seconds) |
|
||||
| 2 | Embedded m4b chapter atoms (`chpl` / QuickTime text track, read by ffprobe) |
|
||||
| 3 | **CUE sheet** — `.cue` next to the audio file (FILE-directive matched, `INDEX 01 MM:SS:FF` at 75 fps); the classic one-big-flac + cue rip shape |
|
||||
| 4 | Track boundaries — for folder/zip books, each file is a chapter |
|
||||
|
||||
## Track ordering (multi-file books)
|
||||
|
||||
| Rank | Source |
|
||||
|---|---|
|
||||
| 1 | **Playlist** — exactly one `.m3u`/`.m3u8` among the book root's direct children; its entry order wins and `#EXTINF` names the tracks |
|
||||
| 2 | **Tags** — `(disc, track)` sort when *every* track carries a track number (fixes flat `01.mp3` dumps from multi-disc rips) |
|
||||
| 3 | **Natural sort** of relative paths (`02` before `10`), recursive — the default |
|
||||
|
||||
Track display titles, highest first: `movement_name` → `title` tag →
|
||||
playlist `#EXTINF` → cleaned filename (leading ordinals/separators
|
||||
stripped).
|
||||
|
||||
## Book grouping (which files form one book)
|
||||
|
||||
- The book root is the nearest ancestor holding audio files DIRECTLY,
|
||||
merged upward through disc-part directories (`CD1`, `Disc 2`, `Part 3`,
|
||||
`05`) when the parent has no audio of its own.
|
||||
- Sibling directories are NEVER merged (two books under one author stay
|
||||
two books); a root with a single track falls back to the single-file
|
||||
model.
|
||||
- A lone `.zip` of audio files is its own book (layout "zip"); a folder
|
||||
of tracks is one book with `file_path` = the directory (layout
|
||||
"folder"); a single audio file is the Batch A model (no layout key).
|
||||
- Rescans take a fast path (track count + total size + known durations
|
||||
unchanged → no re-probe) and absorb pre-grouping one-track strays.
|
||||
|
||||
## Covers
|
||||
|
||||
| Rank | Source |
|
||||
|---|---|
|
||||
| 1 | Sidecar `cover.jpg` / `folder.jpg` / … next to the book |
|
||||
| 2 | **Embedded art** (`APIC` mp3, `covr` m4b, flac picture) — one optional ffmpeg call, written as a `<book>.cover.jpg` sibling following the EPUB convention |
|
||||
|
||||
## Accepted containers (audiobook libraries)
|
||||
|
||||
`.m4b` `.m4a` `.mp3` `.flac` `.zip` (book container) `.opus` `.ogg`
|
||||
`.oga` `.aac` `.wav` — all browser-playable; served with their real
|
||||
media types (`audio/ogg` per RFC 7845 for Opus-in-Ogg, etc.).
|
||||
|
||||
## Documented non-goals
|
||||
|
||||
- **ID3v2 CHAP/CTOC chapter frames** — the written MP3 standard for
|
||||
embedded chapters, near-zero audiobook adoption; would need a
|
||||
hand-rolled ID3v2.4 parser. Revisit only if real files show up.
|
||||
- **NFO sidecars** — no genuine audiobook NFO standard exists (Jellyfin's
|
||||
Bookshelf uses Calibre OPF, which we read).
|
||||
- **ReplayGain / EBU R128 loudness tags** — a playback feature, not
|
||||
ingestion; separate plan if ever wanted.
|
||||
- **`.wma`** — browsers cannot play it; excluded deliberately.
|
||||
+55
-32
@@ -8,6 +8,15 @@ import (
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
type AudioPlaybackSpeed struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
UserID pgtype.UUID `db:"user_id" json:"user_id"`
|
||||
MediaItemID pgtype.UUID `db:"media_item_id" json:"media_item_id"`
|
||||
Speed float32 `db:"speed" json:"speed"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
type CollectionItems struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
CollectionID pgtype.UUID `db:"collection_id" json:"collection_id"`
|
||||
@@ -15,6 +24,7 @@ type CollectionItems struct {
|
||||
AddedAt pgtype.Timestamptz `db:"added_at" json:"added_at"`
|
||||
AddedByUserID pgtype.UUID `db:"added_by_user_id" json:"added_by_user_id"`
|
||||
Excluded pgtype.Bool `db:"excluded" json:"excluded"`
|
||||
Position pgtype.Int4 `db:"position" json:"position"`
|
||||
}
|
||||
|
||||
type Collections struct {
|
||||
@@ -232,25 +242,31 @@ type MediaItemFormats struct {
|
||||
}
|
||||
|
||||
type MediaItems struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
LibraryID pgtype.UUID `db:"library_id" json:"library_id"`
|
||||
Title string `db:"title" json:"title"`
|
||||
Author pgtype.Text `db:"author" json:"author"`
|
||||
Isbn pgtype.Text `db:"isbn" json:"isbn"`
|
||||
Description pgtype.Text `db:"description" json:"description"`
|
||||
FilePath string `db:"file_path" json:"file_path"`
|
||||
FileSize pgtype.Int8 `db:"file_size" json:"file_size"`
|
||||
MimeType pgtype.Text `db:"mime_type" json:"mime_type"`
|
||||
CoverImagePath pgtype.Text `db:"cover_image_path" json:"cover_image_path"`
|
||||
Series pgtype.Text `db:"series" json:"series"`
|
||||
SeriesNumber pgtype.Int4 `db:"series_number" json:"series_number"`
|
||||
Tags []string `db:"tags" json:"tags"`
|
||||
Narrators []string `db:"narrators" json:"narrators"`
|
||||
Abridged bool `db:"abridged" json:"abridged"`
|
||||
Asin pgtype.Text `db:"asin" json:"asin"`
|
||||
DatePublished pgtype.Date `db:"date_published" json:"date_published"`
|
||||
Publisher pgtype.Text `db:"publisher" json:"publisher"`
|
||||
Contributors []string `db:"contributors" json:"contributors"`
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
LibraryID pgtype.UUID `db:"library_id" json:"library_id"`
|
||||
Title string `db:"title" json:"title"`
|
||||
Subtitle pgtype.Text `db:"subtitle" json:"subtitle"`
|
||||
Author pgtype.Text `db:"author" json:"author"`
|
||||
Isbn pgtype.Text `db:"isbn" json:"isbn"`
|
||||
Description pgtype.Text `db:"description" json:"description"`
|
||||
FilePath string `db:"file_path" json:"file_path"`
|
||||
FileSize pgtype.Int8 `db:"file_size" json:"file_size"`
|
||||
MimeType pgtype.Text `db:"mime_type" json:"mime_type"`
|
||||
CoverImagePath pgtype.Text `db:"cover_image_path" json:"cover_image_path"`
|
||||
Series pgtype.Text `db:"series" json:"series"`
|
||||
SeriesNumber pgtype.Float8 `db:"series_number" json:"series_number"`
|
||||
Tags []string `db:"tags" json:"tags"`
|
||||
Narrators []string `db:"narrators" json:"narrators"`
|
||||
Authors []string `db:"authors" json:"authors"`
|
||||
AuthorsSearch []string `db:"authors_search" json:"authors_search"`
|
||||
Publishers []string `db:"publishers" json:"publishers"`
|
||||
PublishersSearch []string `db:"publishers_search" json:"publishers_search"`
|
||||
Abridged bool `db:"abridged" json:"abridged"`
|
||||
Explicit bool `db:"explicit" json:"explicit"`
|
||||
Asin pgtype.Text `db:"asin" json:"asin"`
|
||||
DatePublished pgtype.Date `db:"date_published" json:"date_published"`
|
||||
Publisher pgtype.Text `db:"publisher" json:"publisher"`
|
||||
Contributors []string `db:"contributors" json:"contributors"`
|
||||
// ISO 639-1 language code (e.g., en, es, fr)
|
||||
Language pgtype.Text `db:"language" json:"language"`
|
||||
// Edition information (e.g., "First Edition", "Revised Edition")
|
||||
@@ -365,6 +381,11 @@ type MediaReviews struct {
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
type MediaScanState struct {
|
||||
MediaItemID pgtype.UUID `db:"media_item_id" json:"media_item_id"`
|
||||
Manifest []byte `db:"manifest" json:"manifest"`
|
||||
}
|
||||
|
||||
type OpdsTokens struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
DeviceID pgtype.UUID `db:"device_id" json:"device_id"`
|
||||
@@ -384,7 +405,7 @@ type PanelData struct {
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
// Tracks media items that cannot be properly processed in their assigned library due to format mismatches or other issues
|
||||
// Tracks media that failed processing in a library: item-backed issues (library-type mismatches, metadata extraction failures) and row-less import failures (no media_items row exists; keyed by library + file_path). Auto-resolved by later successful scans.
|
||||
type ProcessingIssues struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
MediaItemID pgtype.UUID `db:"media_item_id" json:"media_item_id"`
|
||||
@@ -395,6 +416,7 @@ type ProcessingIssues struct {
|
||||
Resolved pgtype.Bool `db:"resolved" json:"resolved"`
|
||||
ResolvedAt pgtype.Timestamptz `db:"resolved_at" json:"resolved_at"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
FilePath pgtype.Text `db:"file_path" json:"file_path"`
|
||||
}
|
||||
|
||||
type ReaderSettings struct {
|
||||
@@ -562,16 +584,17 @@ type UserDashboardPreferencesGlobal struct {
|
||||
}
|
||||
|
||||
type Users struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
Email string `db:"email" json:"email"`
|
||||
Username string `db:"username" json:"username"`
|
||||
PasswordHash string `db:"password_hash" json:"password_hash"`
|
||||
FirstName pgtype.Text `db:"first_name" json:"first_name"`
|
||||
LastName pgtype.Text `db:"last_name" json:"last_name"`
|
||||
Role string `db:"role" json:"role"`
|
||||
Theme pgtype.Text `db:"theme" json:"theme"`
|
||||
MaxDevices pgtype.Int4 `db:"max_devices" json:"max_devices"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
Timezone pgtype.Text `db:"timezone" json:"timezone"`
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
Email string `db:"email" json:"email"`
|
||||
Username string `db:"username" json:"username"`
|
||||
PasswordHash string `db:"password_hash" json:"password_hash"`
|
||||
FirstName pgtype.Text `db:"first_name" json:"first_name"`
|
||||
LastName pgtype.Text `db:"last_name" json:"last_name"`
|
||||
Role string `db:"role" json:"role"`
|
||||
Theme pgtype.Text `db:"theme" json:"theme"`
|
||||
MaxDevices pgtype.Int4 `db:"max_devices" json:"max_devices"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
Timezone pgtype.Text `db:"timezone" json:"timezone"`
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
DefaultAudioSpeed float32 `db:"default_audio_speed" json:"default_audio_speed"`
|
||||
}
|
||||
@@ -14,6 +14,9 @@ type Querier interface {
|
||||
// COLLECTION ITEMS QUERIES
|
||||
// Add book to collection
|
||||
AddBookToCollection(ctx context.Context, arg AddBookToCollectionParams) (CollectionItems, error)
|
||||
// A2c: playlist upsert — position set ONLY on insert, so rescans and
|
||||
// re-adds never clobber the user's manual drag ordering.
|
||||
AddBookToCollectionOrdered(ctx context.Context, arg AddBookToCollectionOrderedParams) (CollectionItems, error)
|
||||
// ============================================
|
||||
// KOBO SHELF MANAGEMENT QUERIES
|
||||
// ============================================
|
||||
@@ -42,12 +45,18 @@ type Querier interface {
|
||||
ClearMediaItemMetadataOverrides(ctx context.Context, id pgtype.UUID) error
|
||||
CountAdmins(ctx context.Context) (int64, error)
|
||||
CountArchivedMediaItems(ctx context.Context) (int64, error)
|
||||
// A2c §3.5: distinguish "playlist complete" from "genuinely empty" — the
|
||||
// dashboard renders a completion marker when hide_finished filtered
|
||||
// everything out of a non-empty collection.
|
||||
CountCollectionItems(ctx context.Context, collectionID pgtype.UUID) (int64, error)
|
||||
// Live item count for a STATIC collection (rule-free cards on the
|
||||
// collections page); same live filter as GetCollectionItems, plus the
|
||||
// future-hybrid excluded exclusion and an optional library scope.
|
||||
CountCollectionLiveItems(ctx context.Context, arg CountCollectionLiveItemsParams) (int32, error)
|
||||
// Count unlinked books for a device
|
||||
CountUnlinkedBooks(ctx context.Context, deviceID pgtype.UUID) (int64, error)
|
||||
// Banner count: unresolved queue rows across the caller's own devices.
|
||||
CountUnresolvedUnlinkedBooksForUser(ctx context.Context, userID pgtype.UUID) (int64, error)
|
||||
CountUserDevices(ctx context.Context, userID pgtype.UUID) (int64, error)
|
||||
CreateAutoResolvedSyncConflict(ctx context.Context, arg CreateAutoResolvedSyncConflictParams) (SyncConflicts, error)
|
||||
// COLLECTIONS QUERIES
|
||||
@@ -103,6 +112,11 @@ type Querier interface {
|
||||
// ============================================================================
|
||||
// PROCESSING ISSUES QUERIES
|
||||
// ============================================================================
|
||||
// Widened upsert (faceted plan Phase 11): media_item_id and file_path
|
||||
// are nullable — item-backed rows key on (item, library, type, ''),
|
||||
// row-less import failures on (zero-uuid, library, type, path). The
|
||||
// conflict target matches idx_processing_issues_dedup's expressions.
|
||||
// A repeat failure REOPENS a resolved row (resolved=false).
|
||||
CreateProcessingIssue(ctx context.Context, arg CreateProcessingIssueParams) (ProcessingIssues, error)
|
||||
// Create reading history entry
|
||||
CreateReadingHistory(ctx context.Context, arg CreateReadingHistoryParams) (ReadingHistory, error)
|
||||
@@ -155,6 +169,27 @@ type Querier interface {
|
||||
// historically stuff the user id into DeviceID (it only fed a broadcast
|
||||
// label before stats); a capture row must only reference a REAL device.
|
||||
DeviceExistsForUser(ctx context.Context, arg DeviceExistsForUserParams) (bool, error)
|
||||
// Facet-value listings (faceted-navigation Batch C v2, faceted plan
|
||||
// Phase 7): ALL listable facets' top values + counts in ONE statement —
|
||||
// one round trip serves the filter drawer's panel, with classic faceted
|
||||
// semantics: each arm OMITS its own facet's filter block and carries
|
||||
// every OTHER registered facet's block (copied mechanically from
|
||||
// SearchMediaItemsUnified — keep the two in lockstep), so with filters
|
||||
// active each facet counts under the others, and with no filters the
|
||||
// blocks are neutral and the counts are the v1 global-per-library ones.
|
||||
// One query serves both cases.
|
||||
//
|
||||
// Visibility: same family-lens join as the search query — a user's
|
||||
// hidden-library rows are absent; the handler 403s an out-of-set
|
||||
// explicit library_id. library_id is nullable: NULL = the All-Libraries
|
||||
// view (no scope beyond the lens). Bool/range facets (has_cover,
|
||||
// explicit, abridged, year) filter every arm but list no values — they
|
||||
// render as toggles/inputs, not lists.
|
||||
//
|
||||
// Per-arm LIMIT (parens required: ORDER BY/LIMIT inside a compound
|
||||
// select bind per-arm only in parenthesized subqueries). The same
|
||||
// sqlc.narg names repeat across arms; sqlc binds them once.
|
||||
FacetCountsUnderFilters(ctx context.Context, arg FacetCountsUnderFiltersParams) ([]FacetCountsUnderFiltersRow, error)
|
||||
// Find content-duplicate groups (same library + SHA-256, more than one row)
|
||||
FindHashConflictGroups(ctx context.Context) ([]FindHashConflictGroupsRow, error)
|
||||
GenerateKoboEntitlementId(ctx context.Context) (interface{}, error)
|
||||
@@ -167,17 +202,37 @@ type Querier interface {
|
||||
GetAllSystemSettings(ctx context.Context) ([]GetAllSystemSettingsRow, error)
|
||||
GetAllSystemSettingsFull(ctx context.Context) ([]SystemSettings, error)
|
||||
GetAnnotationsForBook(ctx context.Context, arg GetAnnotationsForBookParams) ([]GetAnnotationsForBookRow, error)
|
||||
// Per-book playback speed preference (player plan §8).
|
||||
GetAudioPlaybackSpeed(ctx context.Context, arg GetAudioPlaybackSpeedParams) (float32, error)
|
||||
// Lifetime session rows for ONE book (the book-detail stats card).
|
||||
GetBookReadingSessions(ctx context.Context, arg GetBookReadingSessionsParams) ([]GetBookReadingSessionsRow, error)
|
||||
GetBooksByTag(ctx context.Context, arg GetBooksByTagParams) ([]MediaItems, error)
|
||||
// Get collection
|
||||
GetCollection(ctx context.Context, id pgtype.UUID) (Collections, error)
|
||||
// A2c §2.4: playlist-collection discovery — provenance key first so a user
|
||||
// rename can't spawn duplicates on the next scan.
|
||||
GetCollectionByImportedSeries(ctx context.Context, arg GetCollectionByImportedSeriesParams) (Collections, error)
|
||||
GetCollectionByUserAndName(ctx context.Context, arg GetCollectionByUserAndNameParams) (Collections, error)
|
||||
// Get collection items
|
||||
// A2c: position-first ordering (playlist reading order); un-positioned
|
||||
// legacy items keep added_at ASC after the positioned tail — NOT DESC,
|
||||
// which walked a playlist backwards.
|
||||
GetCollectionItems(ctx context.Context, collectionID pgtype.UUID) ([]GetCollectionItemsRow, error)
|
||||
// The excluded filter lives here (was: callers filtered ci.excluded in
|
||||
// Go over a 69-field row + a 45-field converter to MediaItems; embed
|
||||
// returns the real row so neither exists).
|
||||
// A2c: position-first ordering; optional hide_finished — when the flag is
|
||||
// set AND a viewer is supplied, items the viewer finished (percentage
|
||||
// >= 0.999) drop out, turning the section into the reading queue head.
|
||||
// viewer_user_id NULL = no progress filter (never-finished semantics for
|
||||
// anonymous/legacy callers is NOT implied: hide_finished without a viewer
|
||||
// is a caller bug and shows everything).
|
||||
GetCollectionItemsForDashboard(ctx context.Context, arg GetCollectionItemsForDashboardParams) ([]GetCollectionItemsForDashboardRow, error)
|
||||
// A2c §3.4: the collection's reading queue — first N un-finished items in
|
||||
// position order. Strict playlist semantics: finished items drop out as
|
||||
// they complete, so the head IS "up next". Library visibility is enforced
|
||||
// by the family-lens array param (never NULL for user-facing callers).
|
||||
GetCollectionUpNext(ctx context.Context, arg GetCollectionUpNextParams) ([]GetCollectionUpNextRow, error)
|
||||
// Get collection with book count
|
||||
GetCollectionWithBookCount(ctx context.Context, id pgtype.UUID) (GetCollectionWithBookCountRow, error)
|
||||
// Get collections by user
|
||||
@@ -219,6 +274,11 @@ type Querier interface {
|
||||
// Series are grouped PER LIBRARY: the same series name in two libraries
|
||||
// (novels vs manga adaptations, LN splits) stays two entries. Visible
|
||||
// filter enforces per-user library visibility at the source.
|
||||
// Tiebreaker REQUIRED for LIMIT/OFFSET pagination: bulk imports in one
|
||||
// transaction share a single NOW() timestamp, leaving many groups tied on
|
||||
// MAX(created_at). Postgres does not guarantee tie order between
|
||||
// executions, so page boundaries could repeat or drop series. The GROUP BY
|
||||
// keys make the sort total and deterministic.
|
||||
GetDistinctSeries(ctx context.Context, arg GetDistinctSeriesParams) ([]GetDistinctSeriesRow, error)
|
||||
GetDistinctSeriesCount(ctx context.Context, arg GetDistinctSeriesCountParams) (int32, error)
|
||||
GetFailedSyncQueueItems(ctx context.Context, limit int32) ([]SyncQueue, error)
|
||||
@@ -327,6 +387,9 @@ type Querier interface {
|
||||
GetMediaRating(ctx context.Context, arg GetMediaRatingParams) (MediaRatings, error)
|
||||
GetMediaRatings(ctx context.Context, mediaItemID pgtype.UUID) ([]GetMediaRatingsRow, error)
|
||||
GetMediaReview(ctx context.Context, arg GetMediaReviewParams) (MediaReviews, error)
|
||||
// GetMediaScanState reads one item's manifest (per-item rescan paths
|
||||
// and the manifest integration tests).
|
||||
GetMediaScanState(ctx context.Context, mediaItemID pgtype.UUID) (MediaScanState, error)
|
||||
GetNextRetryTime(ctx context.Context) (interface{}, error)
|
||||
GetNotStartedItems(ctx context.Context, arg GetNotStartedItemsParams) ([]MediaItems, error)
|
||||
// Get OPDS token
|
||||
@@ -336,6 +399,8 @@ type Querier interface {
|
||||
GetPanelData(ctx context.Context, arg GetPanelDataParams) (PanelData, error)
|
||||
// Get most popular books for a user
|
||||
GetPopularBooks(ctx context.Context, arg GetPopularBooksParams) ([]GetPopularBooksRow, error)
|
||||
// Single-issue read for the resolve/delete endpoints' family-lens gate.
|
||||
GetProcessingIssue(ctx context.Context, id pgtype.UUID) (ProcessingIssues, error)
|
||||
GetProcessingIssueStats(ctx context.Context, libraryID pgtype.UUID) (GetProcessingIssueStatsRow, error)
|
||||
GetReaderSettings(ctx context.Context, userID pgtype.UUID) ([]byte, error)
|
||||
// Get reading history for a user and book
|
||||
@@ -347,6 +412,17 @@ type Querier interface {
|
||||
GetRefreshToken(ctx context.Context, token pgtype.UUID) (GetRefreshTokenRow, error)
|
||||
GetSavedFilterByID(ctx context.Context, arg GetSavedFilterByIDParams) (SavedFilters, error)
|
||||
GetSavedFilters(ctx context.Context, arg GetSavedFiltersParams) ([]SavedFilters, error)
|
||||
// GetScanStateByLibrary: the scanner's ONE bulk query per library scan
|
||||
// (faceted plan Part 2 §2) — every item's existence + scanner-owned
|
||||
// facts + sidecar manifest in a single pass. Replaces the per-file
|
||||
// GetMediaItemByFilePath flood (~3,094 queries per scan in prod).
|
||||
// Lean projection: metadata blobs (audio_metadata, chapter_metadata,
|
||||
// authors, description...) are NEVER loaded — the skip path never
|
||||
// needs them, and the change path re-reads the full row.
|
||||
// archived_at + missing_scan_count + file_sha256 ride along so the
|
||||
// skip path's archive-restore and hash self-heal checks stay
|
||||
// DB-free too.
|
||||
GetScanStateByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]GetScanStateByLibraryRow, error)
|
||||
// Scoped to ONE library: same-name series in different libraries are
|
||||
// different stories (manga adaptations, LN splits). Callers must pass a
|
||||
// library_id and verify the requester can see that library.
|
||||
@@ -386,6 +462,10 @@ type Querier interface {
|
||||
GetUserByEmail(ctx context.Context, email string) (GetUserByEmailRow, error)
|
||||
GetUserByEmailOrUsername(ctx context.Context, email string) (GetUserByEmailOrUsernameRow, error)
|
||||
GetUserByUsername(ctx context.Context, username string) (GetUserByUsernameRow, error)
|
||||
// A2c (2026-10-19): user collections are dashboard-ELIGIBLE BY DEFAULT —
|
||||
// show_on_dashboard gated them forever-false (no writer ever set it on
|
||||
// user rows), and hiding is the dashboard-prefs job (hidden_collections).
|
||||
// The flag remains system-section provisioning state only.
|
||||
GetUserCollectionsForDashboard(ctx context.Context, userID pgtype.UUID) ([]Collections, error)
|
||||
// Per-device reading breakdown for a range. LEFT JOIN so pushes that
|
||||
// carry no device (web reader until it identifies) still appear as one
|
||||
@@ -464,11 +544,17 @@ type Querier interface {
|
||||
// enumerator passes NULL; a bare `device_id = NULL` never matches, which
|
||||
// silently disabled the offline flush until 2026-10-17).
|
||||
ListPendingSyncQueueItems(ctx context.Context, arg ListPendingSyncQueueItemsParams) ([]SyncQueue, error)
|
||||
// LEFT JOIN (Phase 11): row-less import failures have NO media_items
|
||||
// row — an inner join would silently drop the ledger's newest kind.
|
||||
// Item-side projections COALESCE to keep them non-nullable strings;
|
||||
// the display file path falls back to the issue's own for row-less rows.
|
||||
ListProcessingIssuesByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]ListProcessingIssuesByLibraryRow, error)
|
||||
ListSyncConflictsByMediaItem(ctx context.Context, arg ListSyncConflictsByMediaItemParams) ([]SyncConflicts, error)
|
||||
ListSyncConflictsByUser(ctx context.Context, userID pgtype.UUID) ([]ListSyncConflictsByUserRow, error)
|
||||
// List unresolved unlinked books with pagination
|
||||
ListUnresolvedUnlinkedBooks(ctx context.Context, arg ListUnresolvedUnlinkedBooksParams) ([]ListUnresolvedUnlinkedBooksRow, error)
|
||||
// Page list: unresolved queue rows for the caller's own devices.
|
||||
ListUnresolvedUnlinkedBooksForUser(ctx context.Context, userID pgtype.UUID) ([]ListUnresolvedUnlinkedBooksForUserRow, error)
|
||||
ListUsers(ctx context.Context) ([]ListUsersRow, error)
|
||||
// First consecutive scan that cannot find the file on disk.
|
||||
MarkMediaItemMissing(ctx context.Context, id pgtype.UUID) error
|
||||
@@ -501,10 +587,19 @@ type Querier interface {
|
||||
// Remove book from collection
|
||||
RemoveBookFromCollection(ctx context.Context, arg RemoveBookFromCollectionParams) error
|
||||
RemoveBookFromKoboShelf(ctx context.Context, arg RemoveBookFromKoboShelfParams) error
|
||||
// A2c: full-order rewrite for the drag-to-reorder endpoint. The payload is
|
||||
// the complete ordered id list; positions become 1..n in payload order.
|
||||
ReorderCollectionItems(ctx context.Context, arg ReorderCollectionItemsParams) error
|
||||
// Re-parent all child rows of p_source onto p_target (defined in schema.sql)
|
||||
ReparentMediaItemChildren(ctx context.Context, arg ReparentMediaItemChildrenParams) error
|
||||
ResetSystemCollectionMetadata(ctx context.Context, arg ResetSystemCollectionMetadataParams) error
|
||||
ResolveHashConflict(ctx context.Context, arg ResolveHashConflictParams) error
|
||||
// Ledger self-heal (faceted plan Phase 11): a scan that processes the
|
||||
// item successfully closes its open issues (extraction failures heal).
|
||||
ResolveOpenProcessingIssuesForItem(ctx context.Context, mediaItemID pgtype.UUID) (int64, error)
|
||||
// Row-less issues resolve by (library, file_path) when the file finally
|
||||
// imports — the page heals itself without admin action.
|
||||
ResolveOpenProcessingIssuesForPath(ctx context.Context, arg ResolveOpenProcessingIssuesForPathParams) (int64, error)
|
||||
ResolveProcessingIssue(ctx context.Context, arg ResolveProcessingIssueParams) (ProcessingIssues, error)
|
||||
ResolveSyncConflict(ctx context.Context, arg ResolveSyncConflictParams) (SyncConflicts, error)
|
||||
// Resolve unlinked book
|
||||
@@ -547,6 +642,9 @@ type Querier interface {
|
||||
// Replace ONLY the auto_assign_rules document (rule CRUD: read-modify-write
|
||||
// of the JSONB without touching metadata stale clients did not send).
|
||||
UpdateCollectionRulesJSON(ctx context.Context, arg UpdateCollectionRulesJSONParams) (Collections, error)
|
||||
// A2c §2.4: provenance + playlist flags on an existing collection without
|
||||
// touching the rest of the row.
|
||||
UpdateCollectionViewSettings(ctx context.Context, arg UpdateCollectionViewSettingsParams) error
|
||||
UpdateDashboardPreferences(ctx context.Context, arg UpdateDashboardPreferencesParams) (UserDashboardPreferences, error)
|
||||
UpdateDevice(ctx context.Context, arg UpdateDeviceParams) (Devices, error)
|
||||
UpdateDeviceAuthToken(ctx context.Context, arg UpdateDeviceAuthTokenParams) (Devices, error)
|
||||
@@ -570,6 +668,11 @@ type Querier interface {
|
||||
UpdateMediaBookmarkForSync(ctx context.Context, arg UpdateMediaBookmarkForSyncParams) (MediaBookmarks, error)
|
||||
UpdateMediaHighlightForSync(ctx context.Context, arg UpdateMediaHighlightForSyncParams) (MediaHighlights, error)
|
||||
UpdateMediaItem(ctx context.Context, arg UpdateMediaItemParams) (MediaItems, error)
|
||||
// Focused refresh for multi-file audiobooks (folder-of-tracks / zip
|
||||
// books): rescans recompute the track manifest and the aggregate facts
|
||||
// that follow from it. Editorial fields stay untouched — admins may have
|
||||
// edited title/narrators after ingest.
|
||||
UpdateMediaItemAudioBookPayload(ctx context.Context, arg UpdateMediaItemAudioBookPayloadParams) (UpdateMediaItemAudioBookPayloadRow, error)
|
||||
UpdateMediaItemChapterMetadata(ctx context.Context, arg UpdateMediaItemChapterMetadataParams) (MediaItems, error)
|
||||
// Update media item format
|
||||
UpdateMediaItemFormat(ctx context.Context, arg UpdateMediaItemFormatParams) (MediaItemFormats, error)
|
||||
@@ -595,18 +698,28 @@ type Querier interface {
|
||||
UpdateSystemSetting(ctx context.Context, arg UpdateSystemSettingParams) error
|
||||
// Update universal progress
|
||||
UpdateUniversalProgress(ctx context.Context, arg UpdateUniversalProgressParams) (ReadingProgress, error)
|
||||
UpdateUserDefaultAudioSpeed(ctx context.Context, arg UpdateUserDefaultAudioSpeedParams) error
|
||||
UpdateUserMaxDevices(ctx context.Context, arg UpdateUserMaxDevicesParams) (Users, error)
|
||||
UpdateUserProfile(ctx context.Context, arg UpdateUserProfileParams) error
|
||||
UpdateUserRole(ctx context.Context, arg UpdateUserRoleParams) (UpdateUserRoleRow, error)
|
||||
UpdateUserTheme(ctx context.Context, arg UpdateUserThemeParams) error
|
||||
UpdateUserTimezone(ctx context.Context, arg UpdateUserTimezoneParams) error
|
||||
UpdateUsername(ctx context.Context, arg UpdateUsernameParams) error
|
||||
UpsertAudioPlaybackSpeed(ctx context.Context, arg UpsertAudioPlaybackSpeedParams) (float32, error)
|
||||
UpsertDashboardPreferences(ctx context.Context, arg UpsertDashboardPreferencesParams) (UserDashboardPreferences, error)
|
||||
UpsertGlobalDashboardPreferences(ctx context.Context, arg UpsertGlobalDashboardPreferencesParams) (UserDashboardPreferencesGlobal, error)
|
||||
UpsertMediaReview(ctx context.Context, arg UpsertMediaReviewParams) (MediaReviews, error)
|
||||
// UpsertMediaScanState stores a book's sidecar manifest fingerprint
|
||||
// (faceted plan Part 2 §3). Written on backfill (first sighting —
|
||||
// no re-extraction), on repoint reconciliation (stat pairs refreshed,
|
||||
// content identical), and after a manifest-gated re-extract.
|
||||
UpsertMediaScanState(ctx context.Context, arg UpsertMediaScanStateParams) error
|
||||
UpsertPanelData(ctx context.Context, arg UpsertPanelDataParams) (PanelData, error)
|
||||
UpsertReaderSettings(ctx context.Context, arg UpsertReaderSettingsParams) (ReaderSettings, error)
|
||||
UpsertSystemSetting(ctx context.Context, arg UpsertSystemSettingParams) (SystemSettings, error)
|
||||
// Ingest: insert or refresh the queue row for a device file that could not
|
||||
// be identified. One row per (device, content_id); re-syncs bump last_seen.
|
||||
UpsertUnlinkedBook(ctx context.Context, arg UpsertUnlinkedBookParams) (UnlinkedBooks, error)
|
||||
}
|
||||
|
||||
var _ Querier = (*Queries)(nil)
|
||||
+2494
-204
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -58,6 +58,8 @@ type SettingDefault struct {
|
||||
var SettingDefaults = []SettingDefault{
|
||||
{Key: "scan_poll_interval_seconds", Value: "60", Type: SettingTypeInt, Min: "1", Max: "3600", Category: "scanner", Group: "Scanning", Description: "How often to scan all libraries (seconds)"},
|
||||
{Key: "auto_scan_enabled", Value: "true", Type: SettingTypeBool, Category: "scanner", Group: "Scanning", Description: "Whether auto-scanning is enabled system-wide"},
|
||||
{Key: "auto_playlist_collections", Value: "true", Type: SettingTypeBool, Category: "scanner", Group: "Scanning", Description: "Auto-create curated playlist collections from sidecar series marked as reading orders"},
|
||||
{Key: "scan_extractor_version", Value: "1", Type: SettingTypeInt, Min: "1", Max: "1000000", Category: "scanner", Group: "Scanning", Description: "Parser version stamped on each book's scan manifest. Raising it re-extracts metadata from books scanned under an older parser, a bounded batch per scan until all are caught up"},
|
||||
{Key: "default_timezone", Value: "UTC", Type: SettingTypeString, Category: "general", Group: "System Defaults", Description: "System default timezone"},
|
||||
{Key: "session_duration_seconds", Value: "604800", Type: SettingTypeInt, Min: "300", Max: "31536000", Category: "security", Group: "Session", Description: "How long a login session stays valid"},
|
||||
{Key: "password_min_length", Value: "8", Type: SettingTypeInt, Min: "1", Max: "128", Category: "security", Group: "Password Quality", Description: "Minimum password length"},
|
||||
@@ -189,6 +191,17 @@ func (r *SettingsRegistry) ScanPollInterval() time.Duration {
|
||||
// AutoScanEnabled reports whether auto-scanning is on.
|
||||
func (r *SettingsRegistry) AutoScanEnabled() bool { return r.getBool("auto_scan_enabled") }
|
||||
|
||||
// AutoPlaylistCollections reports whether the scanner should turn
|
||||
// obviously-playlist sidecar series ("The Witcher (Reading Order)") into
|
||||
// curated static collections (A2c §2.4).
|
||||
func (r *SettingsRegistry) AutoPlaylistCollections() bool { return r.getBool("auto_playlist_collections") }
|
||||
|
||||
// ScanExtractorVersion is the parser version stamped on each book's scan
|
||||
// manifest. Books whose stored stamp differs re-extract their metadata on
|
||||
// later scans (bounded per scan) — the mechanism that lets parser fixes
|
||||
// reach rows ingested under an older parser without a force rescan.
|
||||
func (r *SettingsRegistry) ScanExtractorVersion() int { return r.getInt("scan_extractor_version") }
|
||||
|
||||
// RegistrationEnabled reports whether public signup is allowed. Admin user
|
||||
// creation is never affected — only self-service registration rides this
|
||||
// flag; handlers additionally grant the first-user exception (see
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"html/template"
|
||||
"math"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -120,6 +121,8 @@ type UserProfile struct {
|
||||
LastName string `json:"last_name"`
|
||||
Role string `json:"role"`
|
||||
Theme string `json:"theme,omitempty"`
|
||||
// User-level default audiobook playback speed (plan §8); 0 omits.
|
||||
DefaultAudioSpeed float64 `json:"default_audio_speed,omitempty"`
|
||||
}
|
||||
|
||||
type UpdateProfileRequest struct {
|
||||
@@ -129,6 +132,9 @@ type UpdateProfileRequest struct {
|
||||
LastName string `json:"last_name,omitempty" form:"last_name" validate:"omitempty,max=100"`
|
||||
Theme string `json:"theme,omitempty" form:"theme" validate:"omitempty"`
|
||||
Timezone string `json:"timezone,omitempty" form:"timezone" validate:"omitempty"`
|
||||
// Per the SPEEDS list bounds; the profile select always sends it,
|
||||
// JSON clients omit it (0 = untouched).
|
||||
DefaultAudioSpeed float64 `json:"default_audio_speed,omitempty" form:"default_audio_speed" validate:"omitempty,min=0.25,max=4"`
|
||||
}
|
||||
|
||||
type AdminUpdateUserRequest struct {
|
||||
@@ -139,6 +145,8 @@ type AdminUpdateUserRequest struct {
|
||||
Theme string `json:"theme,omitempty" form:"theme" validate:"omitempty"`
|
||||
Timezone string `json:"timezone,omitempty" form:"timezone" validate:"omitempty"`
|
||||
Role string `json:"role,omitempty" form:"role" validate:"omitempty,oneof=user admin"`
|
||||
// Same semantics as UpdateProfileRequest.DefaultAudioSpeed.
|
||||
DefaultAudioSpeed float64 `json:"default_audio_speed,omitempty" form:"default_audio_speed" validate:"omitempty,min=0.25,max=4"`
|
||||
}
|
||||
|
||||
// Register handles POST /api/auth/register
|
||||
@@ -538,13 +546,14 @@ func (h *AuthHandler) GetProfile(c *echo.Context) error {
|
||||
lastName = user.LastName.String
|
||||
}
|
||||
return c.JSON(http.StatusOK, UserProfile{
|
||||
ID: uuid.UUID(user.ID.Bytes).String(),
|
||||
Email: user.Email,
|
||||
Username: user.Username,
|
||||
FirstName: firstName,
|
||||
LastName: lastName,
|
||||
Role: user.Role,
|
||||
Theme: user.Theme.String,
|
||||
ID: uuid.UUID(user.ID.Bytes).String(),
|
||||
Email: user.Email,
|
||||
Username: user.Username,
|
||||
FirstName: firstName,
|
||||
LastName: lastName,
|
||||
Role: user.Role,
|
||||
Theme: user.Theme.String,
|
||||
DefaultAudioSpeed: float64(user.DefaultAudioSpeed),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -693,6 +702,22 @@ func (h *AuthHandler) UpdateProfile(c *echo.Context) error {
|
||||
}
|
||||
}
|
||||
|
||||
// Update the default audiobook playback speed (plan §8). The form
|
||||
// select always sends it; JSON clients that omit it decode as 0 and
|
||||
// leave the column untouched.
|
||||
if req.DefaultAudioSpeed != 0 {
|
||||
if req.DefaultAudioSpeed < 0.25 || req.DefaultAudioSpeed > 4 || math.IsNaN(req.DefaultAudioSpeed) || math.IsInf(req.DefaultAudioSpeed, 0) {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "default playback speed out of range"})
|
||||
}
|
||||
err := h.db.UpdateUserDefaultAudioSpeed(c.Request().Context(), database.UpdateUserDefaultAudioSpeedParams{
|
||||
ID: targetUserUUID,
|
||||
DefaultAudioSpeed: float32(req.DefaultAudioSpeed),
|
||||
})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
|
||||
}
|
||||
}
|
||||
|
||||
// Browser path (htmx modal + profile page): refresh so every rendered
|
||||
// value (timezone select, table rows, header) reflects the save and the
|
||||
// modal closes. A bare JSON body swapped into #profile-result read as
|
||||
|
||||
@@ -80,6 +80,48 @@ func (h *Handler) LinkBook(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
// Ownership: the device must belong to the caller.
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{
|
||||
"error": "user not authenticated",
|
||||
})
|
||||
}
|
||||
deviceOwned, err := h.db.DeviceExistsForUser(c.Request().Context(), database.DeviceExistsForUserParams{
|
||||
DeviceID: pgtype.UUID{Bytes: deviceID, Valid: true},
|
||||
UserID: user.ID,
|
||||
})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
"error": "Failed to verify device",
|
||||
})
|
||||
}
|
||||
if !deviceOwned {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{
|
||||
"error": "Device not found",
|
||||
})
|
||||
}
|
||||
|
||||
// Family-lens: the link target must sit in a library the caller can
|
||||
// see — a hidden library's book can never be linked.
|
||||
visibleSet, visErr := h.visibleLibrarySetFor(c)
|
||||
if visErr != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
"error": "Failed to resolve visible libraries",
|
||||
})
|
||||
}
|
||||
item, itemErr := h.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: req.MediaItemID, Valid: true})
|
||||
if itemErr != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{
|
||||
"error": "Media item not found",
|
||||
})
|
||||
}
|
||||
if !item.LibraryID.Valid || !visibleSet[uuid.UUID(item.LibraryID.Bytes).String()] {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{
|
||||
"error": "Book is not available to you",
|
||||
})
|
||||
}
|
||||
|
||||
alias, err := matchingService.LinkBook(c.Request().Context(), deviceID, &req)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
@@ -112,6 +154,27 @@ func (h *Handler) GetUnlinkedBooks(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{
|
||||
"error": "user not authenticated",
|
||||
})
|
||||
}
|
||||
deviceOwned, devErr := h.db.DeviceExistsForUser(c.Request().Context(), database.DeviceExistsForUserParams{
|
||||
DeviceID: pgtype.UUID{Bytes: deviceID, Valid: true},
|
||||
UserID: user.ID,
|
||||
})
|
||||
if devErr != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
"error": "Failed to verify device",
|
||||
})
|
||||
}
|
||||
if !deviceOwned {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{
|
||||
"error": "Device not found",
|
||||
})
|
||||
}
|
||||
|
||||
unlinked, err := matchingService.GetUnlinkedBooks(c.Request().Context(), deviceID)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
@@ -509,10 +572,35 @@ func (h *Handler) DeleteDeviceFileAlias(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
// visibleLibrarySetFor resolves the requesting user's visible-library set
|
||||
// (family-lens) from the request context identity.
|
||||
func (h *Handler) visibleLibrarySetFor(c *echo.Context) (map[string]bool, error) {
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return nil, echo.NewHTTPError(http.StatusUnauthorized, "user not authenticated")
|
||||
}
|
||||
set, _, err := services.NewLibraryService(h.db).VisibleLibrarySet(
|
||||
c.Request().Context(), user.ID)
|
||||
return set, err
|
||||
}
|
||||
|
||||
// GetBookMatches handles GET /api/books/match
|
||||
func (h *Handler) GetBookMatches(c *echo.Context) error {
|
||||
matchingService := h.getMatchingService()
|
||||
|
||||
// Family-lens: restrict matches to libraries the requester can see
|
||||
// (absent-from-results — a hidden library's book simply never matches).
|
||||
visibleSet, visErr := h.visibleLibrarySetFor(c)
|
||||
if visErr != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{
|
||||
"error": "Failed to resolve visible libraries",
|
||||
})
|
||||
}
|
||||
visibleIDs := make([]string, 0, len(visibleSet))
|
||||
for id := range visibleSet {
|
||||
visibleIDs = append(visibleIDs, id)
|
||||
}
|
||||
|
||||
// Get query parameters
|
||||
identifiers := c.QueryParams()["identifier"]
|
||||
sha256 := c.QueryParam("sha256")
|
||||
@@ -532,11 +620,12 @@ func (h *Handler) GetBookMatches(c *echo.Context) error {
|
||||
}
|
||||
|
||||
req := &services.BookQueryRequest{
|
||||
Identifiers: identifiers,
|
||||
SHA256: sha256,
|
||||
Title: title,
|
||||
Author: author,
|
||||
FileSize: fileSize,
|
||||
Identifiers: identifiers,
|
||||
SHA256: sha256,
|
||||
Title: title,
|
||||
Author: author,
|
||||
FileSize: fileSize,
|
||||
VisibleLibraryIDs: visibleIDs,
|
||||
}
|
||||
|
||||
response, err := matchingService.QueryBooks(c.Request().Context(), req)
|
||||
|
||||
@@ -52,6 +52,12 @@ type UpdateCollectionRequest struct {
|
||||
Icon string `form:"icon" json:"icon"`
|
||||
AutoAssignRules []services.Rule `form:"auto_assign_rules" json:"auto_assign_rules"`
|
||||
ViewSettings map[string]interface{} `form:"view_settings" json:"view_settings"`
|
||||
// A2c §3.3: playlist toggles — merged into view_settings server-side
|
||||
// (nil = untouched). []string because the form posts a hidden "false"
|
||||
// sibling next to the checkbox; the LAST value wins. JSON clients may
|
||||
// send a single-element array (or omit — untouched).
|
||||
HideFinished []string `form:"hide_finished" json:"hide_finished"`
|
||||
ShowUpNext []string `form:"show_up_next" json:"show_up_next"`
|
||||
}
|
||||
|
||||
type AddBooksRequest struct {
|
||||
@@ -79,6 +85,10 @@ type SectionData struct {
|
||||
Items []BookInfo `json:"items"`
|
||||
ViewAllURL string `json:"view_all_url"`
|
||||
Priority int `json:"priority"`
|
||||
// A2c: curated playlist marker (imported reading-order collection) and
|
||||
// the hide_finished completion state — dashboard.ts renders both.
|
||||
IsPlaylist bool `json:"is_playlist"`
|
||||
Complete bool `json:"complete"`
|
||||
}
|
||||
|
||||
func (h *CollectionHandler) CreateCollection(c *echo.Context) error {
|
||||
@@ -284,10 +294,28 @@ func (h *CollectionHandler) GetCollection(c *echo.Context) error {
|
||||
}
|
||||
}
|
||||
} else if libUUID.Valid {
|
||||
// A2c §3.3: the detail page honors the same per-collection
|
||||
// hide_finished as the dashboard section, against the viewer's
|
||||
// progress. The viewer_percentage rides along for card graying.
|
||||
hideFinished := false
|
||||
if len(collection.ViewSettings) > 0 {
|
||||
var vs map[string]interface{}
|
||||
if json.Unmarshal(collection.ViewSettings, &vs) == nil {
|
||||
if b, ok := vs["hide_finished"].(bool); ok {
|
||||
hideFinished = b
|
||||
}
|
||||
}
|
||||
}
|
||||
var viewerID pgtype.UUID
|
||||
if u, ok := c.Get("user").(database.Users); ok {
|
||||
viewerID = u.ID
|
||||
}
|
||||
collItems, collErr := h.db.GetCollectionItemsForDashboard(c.Request().Context(),
|
||||
database.GetCollectionItemsForDashboardParams{
|
||||
CollectionID: pgtype.UUID{Bytes: collectionID, Valid: true},
|
||||
LibraryID: libUUID,
|
||||
ViewerUserID: viewerID,
|
||||
HideFinished: pgtype.Bool{Bool: hideFinished, Valid: hideFinished},
|
||||
Limit: pgtype.Int4{Int32: 10000, Valid: true},
|
||||
})
|
||||
if collErr != nil {
|
||||
@@ -317,6 +345,40 @@ func (h *CollectionHandler) GetCollection(c *echo.Context) error {
|
||||
}
|
||||
}
|
||||
|
||||
// A2c §3.4: the collection's reading queue for the Up Next strip.
|
||||
// Library visibility enforced — a hidden library's books never surface
|
||||
// here even if they sit in a visible collection.
|
||||
upNext := []map[string]interface{}{}
|
||||
if !isSmart {
|
||||
hideFinished := false
|
||||
if b, ok := viewSettings["hide_finished"].(bool); ok {
|
||||
hideFinished = b
|
||||
}
|
||||
_ = hideFinished
|
||||
visibleIDs, visErr := h.userVisibleLibraryIDs(c)
|
||||
if visErr == nil {
|
||||
user := c.Get("user").(database.Users)
|
||||
rows, upErr := h.db.GetCollectionUpNext(c.Request().Context(), database.GetCollectionUpNextParams{
|
||||
CollectionID: pgtype.UUID{Bytes: collectionID, Valid: true},
|
||||
ViewerUserID: pgtype.UUID{Bytes: user.ID.Bytes, Valid: true},
|
||||
VisibleLibraryIds: visibleIDs,
|
||||
Limit: pgtype.Int4{Int32: 10, Valid: true},
|
||||
})
|
||||
if upErr == nil {
|
||||
for _, row := range rows {
|
||||
upNext = append(upNext, map[string]interface{}{
|
||||
"id": uuid.UUID(row.MediaItems.ID.Bytes).String(),
|
||||
"title": row.MediaItems.Title,
|
||||
"author": utils.PgTextString(row.MediaItems.Author),
|
||||
"cover": utils.PgTextString(row.MediaItems.CoverImagePath),
|
||||
"format": row.MediaItems.FormatGroup,
|
||||
"progress": row.ViewerPercentage,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"id": uuid.UUID(collection.ID.Bytes).String(),
|
||||
"user_id": uuid.UUID(collection.UserID.Bytes).String(),
|
||||
@@ -329,6 +391,7 @@ func (h *CollectionHandler) GetCollection(c *echo.Context) error {
|
||||
"books": bookList,
|
||||
"book_count": len(bookList),
|
||||
"is_smart": isSmart,
|
||||
"up_next": upNext,
|
||||
"created_at": collection.CreatedAt.Time.String(),
|
||||
})
|
||||
}
|
||||
@@ -347,6 +410,23 @@ func (h *CollectionHandler) UpdateCollection(c *echo.Context) error {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
|
||||
}
|
||||
|
||||
// A2c §3.3: playlist toggles ride the edit form as top-level fields and
|
||||
// merge into view_settings server-side (the form can't bind a map).
|
||||
// The form posts a hidden "false" sibling before the checkbox, so the
|
||||
// LAST value is the checkbox state.
|
||||
if v := optionalFlag(req.HideFinished); v != nil {
|
||||
if req.ViewSettings == nil {
|
||||
req.ViewSettings = map[string]interface{}{}
|
||||
}
|
||||
req.ViewSettings["hide_finished"] = *v
|
||||
}
|
||||
if v := optionalFlag(req.ShowUpNext); v != nil {
|
||||
if req.ViewSettings == nil {
|
||||
req.ViewSettings = map[string]interface{}{}
|
||||
}
|
||||
req.ViewSettings["show_up_next"] = *v
|
||||
}
|
||||
|
||||
collection, err := h.collectionService.UpdateCollection(
|
||||
c.Request().Context(),
|
||||
collectionID,
|
||||
@@ -1238,3 +1318,54 @@ func (h *CollectionHandler) PreviewCollection(c *echo.Context) error {
|
||||
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{"items": bookCards})
|
||||
}
|
||||
|
||||
// ReorderItems applies a static collection's manual playlist order
|
||||
// (A2c §3.2). Owner-only; the payload is the COMPLETE ordered id list —
|
||||
// ids omitted keep their relative order appended after (the service never
|
||||
// deletes membership).
|
||||
func (h *CollectionHandler) ReorderItems(c *echo.Context) error {
|
||||
collectionID, err := uuid.Parse(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid collection id"})
|
||||
}
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{"error": "unauthorized"})
|
||||
}
|
||||
|
||||
var req struct {
|
||||
ItemIDs []string `json:"item_ids"`
|
||||
}
|
||||
if err := c.Bind(&req); err != nil || len(req.ItemIDs) == 0 {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "item_ids must be a non-empty ordered id list"})
|
||||
}
|
||||
|
||||
owns, err := h.collectionService.UserOwnsCollection(c.Request().Context(), collectionID, uuid.UUID(user.ID.Bytes))
|
||||
if err != nil || !owns {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "only the collection owner can reorder it"})
|
||||
}
|
||||
|
||||
ordered := make([]uuid.UUID, 0, len(req.ItemIDs))
|
||||
for _, raw := range req.ItemIDs {
|
||||
id, parseErr := uuid.Parse(raw)
|
||||
if parseErr != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id: " + raw})
|
||||
}
|
||||
ordered = append(ordered, id)
|
||||
}
|
||||
|
||||
if err := h.collectionService.ReorderItems(c.Request().Context(), collectionID, ordered); err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]string{"status": "reordered"})
|
||||
}
|
||||
|
||||
// optionalFlag interprets a multi-valued form flag: last value wins
|
||||
// (hidden-false sibling precedes the checkbox), absent → nil (untouched).
|
||||
func optionalFlag(values []string) *bool {
|
||||
if len(values) == 0 {
|
||||
return nil
|
||||
}
|
||||
v := strings.EqualFold(values[len(values)-1], "true")
|
||||
return &v
|
||||
}
|
||||
@@ -211,6 +211,8 @@ func BuildSections(sections []services.DashboardSection, currentLibraryID, libra
|
||||
Items: bookCards,
|
||||
ViewAllURL: getViewAllURL(ds.CollectionID.String(), currentLibraryID),
|
||||
Priority: ds.Priority,
|
||||
IsPlaylist: ds.IsPlaylist,
|
||||
Complete: ds.Complete,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -451,6 +451,10 @@ func (h *KOReaderHandler) SyncProgress(c *echo.Context) error {
|
||||
for _, book := range req.Books {
|
||||
mediaItemID, _ := h.resolveBookToMediaItem(c, device.ID, pgUserID, book)
|
||||
if !mediaItemID.Valid {
|
||||
// Unidentified file: queue it for manual identification
|
||||
// (unlinked-books page) instead of silently dropping the
|
||||
// file's progress. One row per (device, content id).
|
||||
h.queueUnlinkedBook(c, device.ID, book)
|
||||
bookResults = append(bookResults, KOReaderBookSyncResult{
|
||||
SHA256: book.SHA256,
|
||||
Synced: false,
|
||||
@@ -498,9 +502,47 @@ func (h *KOReaderHandler) SyncProgress(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
func (h *KOReaderHandler) resolveBookToMediaItem(c *echo.Context, deviceID pgtype.UUID, userID pgtype.UUID, book KOReaderBookProgress) (pgtype.UUID, float64) {
|
||||
// queueUnlinkedBook records a device file that could not be identified, so
|
||||
// the user can resolve it on the unlinked-books page. One row per
|
||||
// (device, content id); re-syncs refresh the row instead of duplicating it.
|
||||
// Best-effort: queueing failures never fail the sync response.
|
||||
func (h *KOReaderHandler) queueUnlinkedBook(c *echo.Context, deviceID pgtype.UUID, book KOReaderBookProgress) {
|
||||
ctx := c.Request().Context()
|
||||
|
||||
contentID := book.UUID
|
||||
if contentID == "" {
|
||||
contentID = book.SHA256
|
||||
}
|
||||
if contentID == "" {
|
||||
// Nothing stable to identify the file by — it could never resolve.
|
||||
return
|
||||
}
|
||||
|
||||
var title pgtype.Text
|
||||
if book.Title != "" {
|
||||
title = pgtype.Text{String: book.Title, Valid: true}
|
||||
}
|
||||
var author pgtype.Text
|
||||
if len(book.Authors) > 0 && book.Authors[0] != "" {
|
||||
author = pgtype.Text{String: book.Authors[0], Valid: true}
|
||||
}
|
||||
var filePath pgtype.Text
|
||||
if book.FilePath != "" {
|
||||
filePath = pgtype.Text{String: book.FilePath, Valid: true}
|
||||
}
|
||||
|
||||
_, _ = h.db.UpsertUnlinkedBook(ctx, database.UpsertUnlinkedBookParams{
|
||||
DeviceID: deviceID,
|
||||
ContentID: contentID,
|
||||
FilePath: filePath,
|
||||
Title: title,
|
||||
Author: author,
|
||||
ConfidenceScore: pgtype.Float8{Float64: 0, Valid: true},
|
||||
})
|
||||
}
|
||||
|
||||
func (h *KOReaderHandler) resolveBookToMediaItem(c *echo.Context, deviceID pgtype.UUID, userID pgtype.UUID, book KOReaderBookProgress) (pgtype.UUID, float64) { ctx := c.Request().Context()
|
||||
|
||||
// Priority 1: UUID provided (highest confidence - 1.0)
|
||||
if book.UUID != "" {
|
||||
mediaUUID, err := uuid.Parse(book.UUID)
|
||||
|
||||
+708
-119
File diff suppressed because it is too large.
Load diff
@@ -916,9 +916,16 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
|
||||
}
|
||||
|
||||
// Check if file exists
|
||||
if _, err := os.Stat(filePath); os.IsNotExist(err) {
|
||||
fileInfo, err := os.Stat(filePath)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "file not found"})
|
||||
}
|
||||
// Folder-of-tracks audiobooks point at a book root DIRECTORY: there
|
||||
// is no single file to stream for OPDS clients (a directory fd reads
|
||||
// EISDIR). Zip books are real files and keep working.
|
||||
if fileInfo.IsDir() {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "multi-file books have no single download"})
|
||||
}
|
||||
|
||||
// Open file
|
||||
file, err := os.Open(filePath)
|
||||
@@ -927,11 +934,7 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
// Get file info
|
||||
fileInfo, err := file.Stat()
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get file info"})
|
||||
}
|
||||
// Get file info (fileInfo already statted above)
|
||||
|
||||
// Set headers
|
||||
filename := filepath.Base(filePath)
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
|
||||
"bookhoard/internal/services"
|
||||
)
|
||||
|
||||
type ProcessingIssuesHandler struct {
|
||||
@@ -19,6 +21,45 @@ func NewProcessingIssuesHandler(db *database.Queries) *ProcessingIssuesHandler {
|
||||
return &ProcessingIssuesHandler{db: db}
|
||||
}
|
||||
|
||||
// visibleLibrarySetFor resolves the requesting user's visible-library
|
||||
// set (family-lens) from the request context identity. The issues
|
||||
// endpoints leak media metadata (titles, paths) — a hidden library's
|
||||
// rows must never reach a user it's hidden from, same rule as every
|
||||
// media surface. Identity follows the router's visibleLibraryIDsFor
|
||||
// (the JWT middleware sets "user_id"; AdminMiddleware only reads the
|
||||
// role — there is no "user" object on this chain).
|
||||
func (h *ProcessingIssuesHandler) visibleLibrarySetFor(c *echo.Context) (map[string]bool, error) {
|
||||
userUUIDStr, ok := c.Get("user_id").(string)
|
||||
if !ok {
|
||||
return nil, echo.NewHTTPError(http.StatusUnauthorized, "invalid user")
|
||||
}
|
||||
userU, err := uuid.Parse(userUUIDStr)
|
||||
if err != nil {
|
||||
return nil, echo.NewHTTPError(http.StatusUnauthorized, "invalid user")
|
||||
}
|
||||
set, _, err := services.NewLibraryService(h.db).VisibleLibrarySet(
|
||||
c.Request().Context(), pgtype.UUID{Bytes: userU, Valid: true})
|
||||
return set, err
|
||||
}
|
||||
|
||||
// gateLibrary enforces the family lens on a library-scoped request:
|
||||
// an explicit library id outside the requester's visible set is a 403,
|
||||
// never a silent pass (faceted plan family-lens rule).
|
||||
func (h *ProcessingIssuesHandler) gateLibrary(c *echo.Context, libraryIDStr string) error {
|
||||
libUUID, err := uuid.Parse(libraryIDStr)
|
||||
if err != nil {
|
||||
return echo.NewHTTPError(http.StatusBadRequest, "Invalid library ID")
|
||||
}
|
||||
visibleSet, visErr := h.visibleLibrarySetFor(c)
|
||||
if visErr != nil {
|
||||
return echo.NewHTTPError(http.StatusInternalServerError, "Failed to check library visibility")
|
||||
}
|
||||
if !visibleSet[libUUID.String()] {
|
||||
return echo.NewHTTPError(http.StatusForbidden, "Library not accessible")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type ProcessingIssueResponse struct {
|
||||
ID string `json:"id"`
|
||||
MediaItemID string `json:"media_item_id"`
|
||||
@@ -40,10 +81,10 @@ type ProcessingIssueStats struct {
|
||||
|
||||
// ListProcessingIssues returns all processing issues for a library
|
||||
func (h *ProcessingIssuesHandler) ListProcessingIssues(c *echo.Context) error {
|
||||
libraryID, err := uuid.Parse(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "Invalid library ID"})
|
||||
if err := h.gateLibrary(c, c.Param("id")); err != nil {
|
||||
return err
|
||||
}
|
||||
libraryID, _ := uuid.Parse(c.Param("id"))
|
||||
|
||||
issues, err := h.db.ListProcessingIssuesByLibrary(
|
||||
c.Request().Context(),
|
||||
@@ -55,10 +96,21 @@ func (h *ProcessingIssuesHandler) ListProcessingIssues(c *echo.Context) error {
|
||||
|
||||
response := make([]ProcessingIssueResponse, len(issues))
|
||||
for i, issue := range issues {
|
||||
// Row-less import failures have no media item — emit an empty id
|
||||
// rather than the zero UUID, and fall back the display title to
|
||||
// the file path so the ledger reads like itself.
|
||||
itemID := ""
|
||||
if issue.MediaItemID.Valid {
|
||||
itemID = issue.MediaItemID.String()
|
||||
}
|
||||
title := issue.Title
|
||||
if title == "" {
|
||||
title = issue.FilePath
|
||||
}
|
||||
response[i] = ProcessingIssueResponse{
|
||||
ID: issue.ID.String(),
|
||||
MediaItemID: issue.MediaItemID.String(),
|
||||
Title: issue.Title,
|
||||
MediaItemID: itemID,
|
||||
Title: title,
|
||||
FilePath: issue.FilePath,
|
||||
FormatGroup: issue.FormatGroup,
|
||||
LibraryTypeName: issue.LibraryTypeName,
|
||||
@@ -74,10 +126,10 @@ func (h *ProcessingIssuesHandler) ListProcessingIssues(c *echo.Context) error {
|
||||
|
||||
// GetProcessingIssueStats returns statistics about processing issues
|
||||
func (h *ProcessingIssuesHandler) GetProcessingIssueStats(c *echo.Context) error {
|
||||
libraryID, err := uuid.Parse(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "Invalid library ID"})
|
||||
if err := h.gateLibrary(c, c.Param("id")); err != nil {
|
||||
return err
|
||||
}
|
||||
libraryID, _ := uuid.Parse(c.Param("id"))
|
||||
|
||||
stats, err := h.db.GetProcessingIssueStats(
|
||||
c.Request().Context(),
|
||||
@@ -106,6 +158,10 @@ func (h *ProcessingIssuesHandler) ResolveProcessingIssue(c *echo.Context) error
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "Invalid media item ID"})
|
||||
}
|
||||
|
||||
if err := h.gateIssue(c, issueID); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = h.db.ResolveProcessingIssue(
|
||||
c.Request().Context(),
|
||||
database.ResolveProcessingIssueParams{
|
||||
@@ -130,6 +186,10 @@ func (h *ProcessingIssuesHandler) DeleteProcessingIssue(c *echo.Context) error {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "Invalid issue ID"})
|
||||
}
|
||||
|
||||
if err := h.gateIssue(c, issueID); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
_, err = h.db.DeleteProcessingIssue(c.Request().Context(), pgtype.UUID{Bytes: issueID, Valid: true})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "Failed to delete issue"})
|
||||
@@ -141,6 +201,23 @@ func (h *ProcessingIssuesHandler) DeleteProcessingIssue(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
// gateIssue enforces the family lens on an issue-addressed request: the
|
||||
// issue's library must be in the requester's visible set (403 otherwise).
|
||||
func (h *ProcessingIssuesHandler) gateIssue(c *echo.Context, issueID uuid.UUID) error {
|
||||
issue, err := h.db.GetProcessingIssue(c.Request().Context(), pgtype.UUID{Bytes: issueID, Valid: true})
|
||||
if err != nil {
|
||||
return echo.NewHTTPError(http.StatusNotFound, "Issue not found")
|
||||
}
|
||||
visibleSet, visErr := h.visibleLibrarySetFor(c)
|
||||
if visErr != nil {
|
||||
return echo.NewHTTPError(http.StatusInternalServerError, "Failed to check library visibility")
|
||||
}
|
||||
if !issue.LibraryID.Valid || !visibleSet[uuid.UUID(issue.LibraryID.Bytes).String()] {
|
||||
return echo.NewHTTPError(http.StatusForbidden, "Library not accessible")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetProcessingIssueStatsData returns stats for SSR (not JSON response)
|
||||
func (h *ProcessingIssuesHandler) GetProcessingIssueStatsData(ctx context.Context, libraryID pgtype.UUID) (database.GetProcessingIssueStatsRow, error) {
|
||||
return h.db.GetProcessingIssueStats(ctx, libraryID)
|
||||
|
||||
@@ -3,6 +3,7 @@ package handlers
|
||||
import (
|
||||
"bookhoard/internal/services"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
@@ -83,6 +84,23 @@ func (h *Handler) ScanLibrary(c *echo.Context) error {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "either library_id or folder_paths required for scanning"})
|
||||
}
|
||||
|
||||
// Coalescing (plan Part 2 §5): a scan for this library is already
|
||||
// pending or running — 409 with the active job so the UI can show
|
||||
// progress instead of stacking a second full scan.
|
||||
if req.LibraryID != "" && h.worker.HasActiveScanForLibrary(req.LibraryID) {
|
||||
var activeJobID string
|
||||
for _, active := range h.worker.GetActiveScanJobs() {
|
||||
if active.JobType == services.JobTypeScan && active.LibraryID == req.LibraryID {
|
||||
activeJobID = active.JobID
|
||||
break
|
||||
}
|
||||
}
|
||||
return c.JSON(http.StatusConflict, map[string]string{
|
||||
"error": "a scan for this library is already pending or running",
|
||||
"active_job_id": activeJobID,
|
||||
})
|
||||
}
|
||||
|
||||
jobID := uuid.New().String()
|
||||
|
||||
job := &services.Job{
|
||||
@@ -95,6 +113,7 @@ func (h *Handler) ScanLibrary(c *echo.Context) error {
|
||||
"admin_id": userUUID.String(),
|
||||
"db": h.db,
|
||||
"force": req.Force,
|
||||
"trigger": "admin",
|
||||
},
|
||||
Status: services.JobStatusPending,
|
||||
Context: h.ctx,
|
||||
@@ -209,6 +228,28 @@ func (h *Handler) GetScanSummary(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
// CancelScan handles POST /api/scanner/cancel/:jobId (admin): requests
|
||||
// cancellation of a pending or running scan job. Cancellation is
|
||||
// cooperative — the file currently being processed finishes, the walk
|
||||
// stops at its next context check, and the archive pass is skipped so
|
||||
// unvisited files are never treated as missing.
|
||||
func (h *Handler) CancelScan(c *echo.Context) error {
|
||||
jobID := c.Param("jobId")
|
||||
|
||||
if err := h.worker.CancelJob(jobID); err != nil {
|
||||
if errors.Is(err, services.ErrJobNotFound) {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "job not found"})
|
||||
}
|
||||
return c.JSON(http.StatusConflict, map[string]string{"error": err.Error()})
|
||||
}
|
||||
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"message": "scan cancellation requested",
|
||||
"job_id": jobID,
|
||||
"status": "cancelled",
|
||||
})
|
||||
}
|
||||
|
||||
// StartWatchModeForLibrary starts watching a specific library's folders
|
||||
func (h *Handler) StartWatchModeForLibrary(ctx context.Context, libraryID pgtype.UUID, adminID pgtype.UUID) error {
|
||||
h.mu.Lock()
|
||||
|
||||
@@ -16,6 +16,7 @@ func registerCollectionsRoutes(cfg *Config) {
|
||||
collections.DELETE("/:id", cfg.CollectionHandler.DeleteCollection)
|
||||
collections.GET("/:id/books", cfg.CollectionHandler.GetBookCollections)
|
||||
collections.POST("/:id/books", cfg.CollectionHandler.AddBooks)
|
||||
collections.PUT("/:id/items/order", cfg.CollectionHandler.ReorderItems)
|
||||
collections.DELETE("/:id/books/:bookId", cfg.CollectionHandler.RemoveBook)
|
||||
collections.POST("/:id/books/bulk-remove", cfg.CollectionHandler.BulkRemoveBooks)
|
||||
collections.POST("/bulk-add-books", cfg.CollectionHandler.HandleBulkAddBooks)
|
||||
|
||||
+200
-94
@@ -475,8 +475,18 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
}
|
||||
}
|
||||
|
||||
userUUID, userUUIDErr := uuid.Parse(user.ID)
|
||||
if userUUIDErr != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
unlinkedCount, countErr := cfg.Queries.CountUnresolvedUnlinkedBooksForUser(c.Request().Context(), pgtype.UUID{Bytes: userUUID, Valid: true})
|
||||
if countErr != nil {
|
||||
log.Printf("CountUnresolvedUnlinkedBooksForUser failed: %v", countErr)
|
||||
unlinkedCount = 0
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = templates.Dashboard(user, sectionData, allSectionsData, libRes.Libraries, libraryID, prefs.HiddenCollections, limit, week, activeLibType, errorMsg).Render(c.Request().Context(), &buf)
|
||||
err = templates.Dashboard(user, sectionData, allSectionsData, libRes.Libraries, libraryID, prefs.HiddenCollections, limit, week, activeLibType, errorMsg, int(unlinkedCount)).Render(c.Request().Context(), &buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -595,6 +605,19 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
Icon: utils.PgTextString(collection.Icon),
|
||||
}
|
||||
|
||||
// A2c §3.3: playlist toggles from view_settings.
|
||||
if len(collection.ViewSettings) > 0 {
|
||||
var vs map[string]interface{}
|
||||
if json.Unmarshal(collection.ViewSettings, &vs) == nil {
|
||||
if b, ok := vs["hide_finished"].(bool); ok {
|
||||
colData.HideFinished = b
|
||||
}
|
||||
if b, ok := vs["show_up_next"].(bool); ok {
|
||||
colData.ShowUpNext = b
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure default color if empty
|
||||
if colData.Color == "" {
|
||||
colData.Color = "blue"
|
||||
@@ -660,18 +683,19 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
})
|
||||
|
||||
// Collection detail page (works for both system and user collections)
|
||||
// Collection detail page. Book loading lives in loadCollectionBooks —
|
||||
// shared verbatim with the books-grid partial below so the SSR page and
|
||||
// the partial can never drift (rendering-ownership rule).
|
||||
frontendProtected.GET("/collections/:id", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
// Parse collection ID from URL
|
||||
collectionID := c.Param("id")
|
||||
collUUID, err := uuid.Parse(collectionID)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Invalid collection ID", "invalid_id")
|
||||
}
|
||||
// Fetch collection details
|
||||
collection, err := cfg.Queries.GetCollection(c.Request().Context(), pgtype.UUID{Bytes: collUUID, Valid: true})
|
||||
if err != nil {
|
||||
if err.Error() == "no rows in result set" {
|
||||
@@ -679,106 +703,146 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
}
|
||||
return renderErrorPage(c, "Error loading collection", "collection_load_error")
|
||||
}
|
||||
// Fetch books in collection
|
||||
userUUID, _ := uuid.Parse(user.ID)
|
||||
var books []handlers.BookInfo
|
||||
|
||||
libRes := resolveLibrary(c, cfg, user.ID)
|
||||
libraryID := libRes.LibraryID
|
||||
|
||||
var libScope pgtype.UUID
|
||||
if !libRes.IsAll {
|
||||
if parsed, parseErr := uuid.Parse(libraryID); parseErr == nil {
|
||||
libScope = pgtype.UUID{Bytes: parsed, Valid: true}
|
||||
}
|
||||
}
|
||||
colService := cfg.CollectionHandler.CollectionService()
|
||||
isSmart := colService.IsSmartCollection(collection.AutoAssignRules)
|
||||
|
||||
visibleIDs, visErr := cfg.CollectionHandler.UserVisibleLibraryIDs(c)
|
||||
if visErr != nil {
|
||||
log.Printf("UserVisibleLibraryIDs failed: %v", visErr)
|
||||
visibleIDs = nil
|
||||
}
|
||||
|
||||
libTypes := handlers.LibraryTypeMap(c.Request().Context(), cfg.Queries)
|
||||
|
||||
if isSmart {
|
||||
// Smart (dynamic) collection: membership is evaluated live —
|
||||
// visible libraries, live items only, any enabled rule matches.
|
||||
matches, evalErr := colService.EvaluateSmartCollection(c.Request().Context(),
|
||||
colService.ParseRules(collection.AutoAssignRules), services.SmartEvalOptions{
|
||||
LibraryID: libScope,
|
||||
VisibleIDs: visibleIDs,
|
||||
})
|
||||
if evalErr != nil {
|
||||
return renderErrorPage(c, "Error loading books", "books_load_error")
|
||||
}
|
||||
books = handlers.BookCardsFromMediaItems(matches, libTypes)
|
||||
} else if collection.IsSystemCollection.Bool && collection.QueryType.Valid && collection.QueryType.String != "" && collection.QueryType.String != "filter" {
|
||||
dashboardSvc := services.NewDashboardService(cfg.Queries)
|
||||
sections, secErr := dashboardSvc.GetDashboardSections(c.Request().Context(), userUUID, libRes.LibUUID, 1000, []string{}, []string{})
|
||||
if secErr != nil {
|
||||
return renderErrorPage(c, "Error loading books", "books_load_error")
|
||||
}
|
||||
|
||||
for _, section := range sections {
|
||||
if section.CollectionID.String() == collectionID {
|
||||
books = handlers.BookCardsFromMediaItems(section.Items, libTypes)
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if libraryID != "" && !libRes.IsAll {
|
||||
libUUID, parseErr := uuid.Parse(libraryID)
|
||||
if parseErr != nil {
|
||||
return renderErrorPage(c, "Invalid library ID", "invalid_library_id")
|
||||
}
|
||||
|
||||
collItems, collErr := cfg.Queries.GetCollectionItemsForDashboard(c.Request().Context(),
|
||||
database.GetCollectionItemsForDashboardParams{
|
||||
CollectionID: pgtype.UUID{Bytes: collUUID, Valid: true},
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Limit: pgtype.Int4{Int32: 1000, Valid: true},
|
||||
})
|
||||
if collErr != nil {
|
||||
books = handlers.BookCardsFromCollectionDashRows(nil, libTypes)
|
||||
} else {
|
||||
// The query filters excluded rows in SQL.
|
||||
books = handlers.BookCardsFromCollectionDashRows(collItems, libTypes)
|
||||
}
|
||||
} else {
|
||||
collItems, collErr := cfg.Queries.GetCollectionItems(c.Request().Context(), pgtype.UUID{Bytes: collUUID, Valid: true})
|
||||
if collErr != nil {
|
||||
collItems = nil
|
||||
}
|
||||
books = handlers.BookCardsFromCollectionItemRows(collItems, libTypes)
|
||||
}
|
||||
}
|
||||
// Build collection data
|
||||
colData := templates.CollectionData{
|
||||
ID: collectionID,
|
||||
Name: collection.Name,
|
||||
Description: collection.Description.String,
|
||||
Color: collection.Color.String,
|
||||
Icon: collection.Icon.String,
|
||||
IsSystem: collection.IsSystemCollection.Bool,
|
||||
IsSmart: isSmart,
|
||||
BookCount: len(books),
|
||||
books, libRes, isSmart, errCode := cfg.loadCollectionBooks(c, user, collection, collectionID)
|
||||
if errCode != "" {
|
||||
msg := map[string]string{
|
||||
"books_load_error": "Error loading books",
|
||||
"invalid_library_id": "Invalid library ID",
|
||||
"invalid_id": "Invalid collection ID",
|
||||
}[errCode]
|
||||
return renderErrorPage(c, msg, errCode)
|
||||
}
|
||||
colData := collectionDataFromRow(collection, collectionID, isSmart, len(books))
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = templates.CollectionDetail(user, colData, books, libraryID, libRes.Libraries).Render(c.Request().Context(), &buf)
|
||||
err = templates.CollectionDetail(user, colData, books, libRes.LibraryID, libRes.Libraries).Render(c.Request().Context(), &buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
// Books-grid partial: the collection detail page's #books-container
|
||||
// content. The library switcher swaps this in (htmx.ajax) instead of
|
||||
// re-rendering the grid client-side from JSON. Family-lens scoped via
|
||||
// loadCollectionBooksLite — the requested library must be in the
|
||||
// caller's visible set, without the page-only library resolution.
|
||||
frontendProtected.GET("/collections/:id/books-grid", func(c *echo.Context) error {
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return c.HTML(http.StatusUnauthorized, "<div>Unauthorized</div>")
|
||||
}
|
||||
userUUID, uuidErr := uuid.Parse(user.ID.String())
|
||||
if uuidErr != nil {
|
||||
return c.HTML(http.StatusUnauthorized, "<div>Unauthorized</div>")
|
||||
}
|
||||
collectionID := c.Param("id")
|
||||
collUUID, parseErr := uuid.Parse(collectionID)
|
||||
if parseErr != nil {
|
||||
return c.HTML(http.StatusBadRequest, "<div>Invalid collection ID</div>")
|
||||
}
|
||||
collection, err := cfg.Queries.GetCollection(c.Request().Context(), pgtype.UUID{Bytes: collUUID, Valid: true})
|
||||
if err != nil {
|
||||
return c.HTML(http.StatusNotFound, "<div>Collection not found</div>")
|
||||
}
|
||||
books, isSmart, errCode := cfg.loadCollectionBooksLite(c, userUUID, collection, collectionID)
|
||||
if errCode == "library_forbidden" {
|
||||
return c.HTML(http.StatusForbidden, "<div>Library not available</div>")
|
||||
}
|
||||
if errCode != "" {
|
||||
return c.HTML(http.StatusBadRequest, "<div>Error loading books</div>")
|
||||
}
|
||||
colData := collectionDataFromRow(collection, collectionID, isSmart, len(books))
|
||||
|
||||
var buf bytes.Buffer
|
||||
if renderErr := templates.CollectionBooksGrid(colData, books).Render(c.Request().Context(), &buf); renderErr != nil {
|
||||
return renderErr
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
// Week-widget fragment: the collapsible "This Week" strip re-fetches
|
||||
// itself here (htmx) whenever the library switcher dispatches
|
||||
// week-refresh, carrying the current #library-select value. Same
|
||||
// resolution rules as the dashboard page (family-lens safe).
|
||||
// Unlinked-books page: the identification queue for device files the
|
||||
// sync could not match (personal data — the caller's own devices only).
|
||||
frontendProtected.GET("/unlinked-books", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
userUUID, userUUIDErr := uuid.Parse(user.ID)
|
||||
if userUUIDErr != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
rows, listErr := cfg.Queries.ListUnresolvedUnlinkedBooksForUser(c.Request().Context(), pgtype.UUID{Bytes: userUUID, Valid: true})
|
||||
if listErr != nil {
|
||||
log.Printf("ListUnresolvedUnlinkedBooksForUser failed: %v", listErr)
|
||||
rows = []database.ListUnresolvedUnlinkedBooksForUserRow{}
|
||||
}
|
||||
|
||||
books := make([]templates.UnlinkedBookData, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
books = append(books, templates.UnlinkedBookData{
|
||||
ProgressID: uuid.UUID(r.ID.Bytes).String(),
|
||||
DeviceID: uuid.UUID(r.DeviceID.Bytes).String(),
|
||||
DeviceType: r.DeviceType,
|
||||
DeviceName: r.DeviceName,
|
||||
FilePath: utils.PgTextString(r.FilePath),
|
||||
SHA256: r.ContentID,
|
||||
TitleFromDevice: utils.PgTextString(r.Title),
|
||||
LastSyncTime: utils.FormatTimeInTimezone(r.LastSeenAt.Time, user.Timezone),
|
||||
ConfidenceScore: r.ConfidenceScore.Float64,
|
||||
})
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if renderErr := templates.UnlinkedBooks(user, books).Render(c.Request().Context(), &buf); renderErr != nil {
|
||||
return renderErr
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
// Unlinked-book match results partial: the match cascade for one device
|
||||
// file, rendered server-side and scoped to the requester's visible
|
||||
// libraries (absent-from-results) — rendering-ownership: templ renders,
|
||||
// JS only swaps. NOTE: nothing routes to the unlinked-books page yet;
|
||||
// this endpoint is ready for whichever UI wires it up.
|
||||
frontendProtected.GET("/unlinked-books/matches", func(c *echo.Context) error {
|
||||
user, ok := c.Get("user").(database.Users)
|
||||
if !ok {
|
||||
return c.HTML(http.StatusUnauthorized, "<div>Unauthorized</div>")
|
||||
}
|
||||
userUUID, uuidErr := uuid.Parse(user.ID.String())
|
||||
if uuidErr != nil {
|
||||
return c.HTML(http.StatusUnauthorized, "<div>Unauthorized</div>")
|
||||
}
|
||||
set, _, setErr := cfg.LibraryService.VisibleLibrarySet(c.Request().Context(), pgtype.UUID{Bytes: userUUID, Valid: true})
|
||||
if setErr != nil {
|
||||
return c.HTML(http.StatusInternalServerError, "<div>Failed to resolve visible libraries</div>")
|
||||
}
|
||||
visibleIDs := make([]string, 0, len(set))
|
||||
for id := range set {
|
||||
visibleIDs = append(visibleIDs, id)
|
||||
}
|
||||
req := &services.BookQueryRequest{
|
||||
SHA256: c.QueryParam("sha256"),
|
||||
Title: c.QueryParam("title"),
|
||||
VisibleLibraryIDs: visibleIDs,
|
||||
}
|
||||
resp, matchErr := services.NewBookMatchingService(cfg.Queries).QueryBooks(c.Request().Context(), req)
|
||||
if matchErr != nil {
|
||||
return c.HTML(http.StatusInternalServerError, "<div>Failed to search</div>")
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if renderErr := templates.UnlinkedMatchCards(c.QueryParam("progress_id"), resp.Matches).Render(c.Request().Context(), &buf); renderErr != nil {
|
||||
return renderErr
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
frontendProtected.GET("/dashboard/week-widget", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
@@ -945,6 +1009,29 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
// Conflicts list partial: fresh list plus the stats bar as an
|
||||
// out-of-band swap. The client refreshes through this instead of
|
||||
// re-rendering conflict cards from JSON (rendering-ownership rule:
|
||||
// templ renders, JS only swaps).
|
||||
frontendProtected.GET("/conflicts/list", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return c.HTML(http.StatusUnauthorized, "<div>Unauthorized</div>")
|
||||
}
|
||||
conflictsData, total, unresolved, dataErr := cfg.ConflictHandler.GetConflictsData(c)
|
||||
if dataErr != nil {
|
||||
log.Printf("GetConflictsData failed: %v", dataErr)
|
||||
conflictsData = []handlers.ConflictDetailResponse{}
|
||||
total = 0
|
||||
unresolved = 0
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
if renderErr := templates.ConflictsRefreshFragment(user, conflictsData, total, unresolved).Render(c.Request().Context(), &buf); renderErr != nil {
|
||||
return renderErr
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
// Analytics page → /stats (the rebuilt reading-statistics page;
|
||||
// nobody bookmarks orphan pages twice).
|
||||
@@ -1118,7 +1205,8 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
|
||||
var buf bytes.Buffer
|
||||
archivedCount, _ := cfg.Queries.CountArchivedMediaItems(c.Request().Context())
|
||||
err = templates.AdminLibrary(user, libData, userData, archivedCount).Render(c.Request().Context(), &buf)
|
||||
libraryTypes := libraryTypeOptionsFor(c.Request().Context(), cfg)
|
||||
err = templates.AdminLibrary(user, libData, userData, archivedCount, libraryTypes).Render(c.Request().Context(), &buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -1135,6 +1223,15 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Invalid library ID", "invalid_id")
|
||||
}
|
||||
// Family lens: issues leak media metadata (titles, file paths) —
|
||||
// a library outside the requester's visible set renders nothing.
|
||||
visibleSet, _, visErr := visibleLibraryIDsFor(c, cfg)
|
||||
if visErr != nil {
|
||||
return renderErrorPage(c, "Error checking library access", "visibility_error")
|
||||
}
|
||||
if !visibleSet[libraryID] {
|
||||
return renderErrorPage(c, "Library not accessible", "library_not_accessible")
|
||||
}
|
||||
// Fetch stats
|
||||
stats, err := cfg.Queries.GetProcessingIssueStats(
|
||||
c.Request().Context(),
|
||||
@@ -1151,14 +1248,23 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading issues", "issues_load_error")
|
||||
}
|
||||
// Convert to template data
|
||||
// Convert to template data (row-less import failures have no
|
||||
// media item: empty id, and the title falls back to the path).
|
||||
issues := make([]templates.ProcessingIssueData, len(dbIssues))
|
||||
for i, issue := range dbIssues {
|
||||
issueUUID, _ := uuid.FromBytes(issue.ID.Bytes[0:16])
|
||||
itemID := ""
|
||||
if issue.MediaItemID.Valid {
|
||||
itemID = uuid.UUID(issue.MediaItemID.Bytes).String()
|
||||
}
|
||||
title := issue.Title
|
||||
if title == "" {
|
||||
title = issue.FilePath
|
||||
}
|
||||
issues[i] = templates.ProcessingIssueData{
|
||||
ID: issueUUID.String(),
|
||||
MediaItemID: uuid.UUID(issue.MediaItemID.Bytes).String(),
|
||||
Title: issue.Title,
|
||||
MediaItemID: itemID,
|
||||
Title: title,
|
||||
FilePath: issue.FilePath,
|
||||
FormatGroup: issue.FormatGroup,
|
||||
LibraryTypeName: issue.LibraryTypeName,
|
||||
|
||||
+180
-10
@@ -10,6 +10,8 @@ import (
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/services"
|
||||
"bookhoard/internal/utils"
|
||||
"bookhoard/templates"
|
||||
|
||||
@@ -54,16 +56,17 @@ func getTemplateUserWithTheme(c *echo.Context, cfg *Config) (templates.User, err
|
||||
}
|
||||
|
||||
return templates.User{
|
||||
ID: userID,
|
||||
Email: userEmail,
|
||||
Username: userUsername,
|
||||
Role: userRole,
|
||||
Theme: userTheme,
|
||||
Token: token,
|
||||
Timezone: userTimezone,
|
||||
FirstName: utils.PgTextString(userDB.FirstName),
|
||||
LastName: utils.PgTextString(userDB.LastName),
|
||||
CreatedAt: userDB.CreatedAt.Time,
|
||||
ID: userID,
|
||||
Email: userEmail,
|
||||
Username: userUsername,
|
||||
Role: userRole,
|
||||
Theme: userTheme,
|
||||
Token: token,
|
||||
Timezone: userTimezone,
|
||||
FirstName: utils.PgTextString(userDB.FirstName),
|
||||
LastName: utils.PgTextString(userDB.LastName),
|
||||
CreatedAt: userDB.CreatedAt.Time,
|
||||
DefaultAudioSpeed: float64(userDB.DefaultAudioSpeed),
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -257,3 +260,170 @@ func visibleLibraryIDsFor(c *echo.Context, cfg *Config) (map[string]bool, []pgty
|
||||
}
|
||||
return cfg.LibraryService.VisibleLibrarySet(c.Request().Context(), uuidToPGType(userU))
|
||||
}
|
||||
|
||||
// collectionDataFromRow builds the template collection payload shared by
|
||||
// the collection detail page and its books-grid partial.
|
||||
func collectionDataFromRow(collection database.Collections, collectionID string, isSmart bool, bookCount int) templates.CollectionData {
|
||||
return templates.CollectionData{
|
||||
ID: collectionID,
|
||||
Name: collection.Name,
|
||||
Description: collection.Description.String,
|
||||
Color: collection.Color.String,
|
||||
Icon: collection.Icon.String,
|
||||
IsSystem: collection.IsSystemCollection.Bool,
|
||||
IsSmart: isSmart,
|
||||
BookCount: bookCount,
|
||||
}
|
||||
}
|
||||
|
||||
// collectionBooksForScope runs the per-scope book loading shared by the
|
||||
// detail page and the books-grid partial: smart evaluation, a dashboard
|
||||
// query section, or manual membership. Callers resolve the scope:
|
||||
// libUUID/libScope/isAll/scoped per their own resolution rules.
|
||||
func (cfg *Config) collectionBooksForScope(c *echo.Context, userUUID uuid.UUID, collUUID pgtype.UUID, collection database.Collections, collectionID string, libUUID pgtype.UUID, libScope pgtype.UUID, scoped bool, visibleIDs []pgtype.UUID) (books []handlers.BookInfo, isSmart bool, errCode string) {
|
||||
colService := cfg.CollectionHandler.CollectionService()
|
||||
isSmart = colService.IsSmartCollection(collection.AutoAssignRules)
|
||||
libTypes := handlers.LibraryTypeMap(c.Request().Context(), cfg.Queries)
|
||||
|
||||
if isSmart {
|
||||
// Smart (dynamic) collection: membership is evaluated live —
|
||||
// visible libraries, live items only, any enabled rule matches.
|
||||
matches, evalErr := colService.EvaluateSmartCollection(c.Request().Context(),
|
||||
colService.ParseRules(collection.AutoAssignRules), services.SmartEvalOptions{
|
||||
LibraryID: libScope,
|
||||
VisibleIDs: visibleIDs,
|
||||
})
|
||||
if evalErr != nil {
|
||||
return nil, isSmart, "books_load_error"
|
||||
}
|
||||
books = handlers.BookCardsFromMediaItems(matches, libTypes)
|
||||
} else if collection.IsSystemCollection.Bool && collection.QueryType.Valid && collection.QueryType.String != "" && collection.QueryType.String != "filter" {
|
||||
dashboardSvc := services.NewDashboardService(cfg.Queries)
|
||||
sections, secErr := dashboardSvc.GetDashboardSections(c.Request().Context(), userUUID, libUUID, 1000, []string{}, []string{})
|
||||
if secErr != nil {
|
||||
return nil, isSmart, "books_load_error"
|
||||
}
|
||||
|
||||
for _, section := range sections {
|
||||
if section.CollectionID.String() == collectionID {
|
||||
books = handlers.BookCardsFromMediaItems(section.Items, libTypes)
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if scoped && libUUID.Valid {
|
||||
collItems, collErr := cfg.Queries.GetCollectionItemsForDashboard(c.Request().Context(),
|
||||
database.GetCollectionItemsForDashboardParams{
|
||||
CollectionID: collUUID,
|
||||
LibraryID: libUUID,
|
||||
Limit: pgtype.Int4{Int32: 1000, Valid: true},
|
||||
})
|
||||
if collErr != nil {
|
||||
books = handlers.BookCardsFromCollectionDashRows(nil, libTypes)
|
||||
} else {
|
||||
// The query filters excluded rows in SQL.
|
||||
books = handlers.BookCardsFromCollectionDashRows(collItems, libTypes)
|
||||
}
|
||||
} else {
|
||||
collItems, collErr := cfg.Queries.GetCollectionItems(c.Request().Context(), collUUID)
|
||||
if collErr != nil {
|
||||
collItems = nil
|
||||
}
|
||||
books = handlers.BookCardsFromCollectionItemRows(collItems, libTypes)
|
||||
}
|
||||
}
|
||||
return books, isSmart, ""
|
||||
}
|
||||
|
||||
// loadCollectionBooks resolves the books shown for a collection detail
|
||||
// page: full library resolution (visible-library payload, folder
|
||||
// reachability, counts) because the page render needs it. The books-grid
|
||||
// partial uses the lighter loadCollectionBooksLite instead.
|
||||
// errCode is "" on success, otherwise a renderErrorPage code.
|
||||
func (cfg *Config) loadCollectionBooks(c *echo.Context, user templates.User, collection database.Collections, collectionID string) (books []handlers.BookInfo, libRes LibraryResolution, isSmart bool, errCode string) {
|
||||
collUUID, parseErr := uuid.Parse(collectionID)
|
||||
if parseErr != nil {
|
||||
return nil, libRes, isSmart, "invalid_id"
|
||||
}
|
||||
userUUID, _ := uuid.Parse(user.ID)
|
||||
|
||||
libRes = resolveLibrary(c, cfg, user.ID)
|
||||
libraryID := libRes.LibraryID
|
||||
|
||||
var libScope pgtype.UUID
|
||||
if !libRes.IsAll {
|
||||
if parsed, parseErr := uuid.Parse(libraryID); parseErr == nil {
|
||||
libScope = pgtype.UUID{Bytes: parsed, Valid: true}
|
||||
}
|
||||
}
|
||||
|
||||
visibleIDs, visErr := cfg.CollectionHandler.UserVisibleLibraryIDs(c)
|
||||
if visErr != nil {
|
||||
log.Printf("UserVisibleLibraryIDs failed: %v", visErr)
|
||||
visibleIDs = nil
|
||||
}
|
||||
|
||||
scoped := libraryID != "" && !libRes.IsAll
|
||||
books, isSmart, errCode = cfg.collectionBooksForScope(c, userUUID, pgtype.UUID{Bytes: collUUID, Valid: true}, collection, collectionID, libRes.LibUUID, libScope, scoped, visibleIDs)
|
||||
return books, libRes, isSmart, errCode
|
||||
}
|
||||
|
||||
// loadCollectionBooksLite is the books-grid partial's loader. The switcher
|
||||
// refresh only needs "is the requested library visible, and its books" —
|
||||
// so it validates the requested library against the caller's visible set
|
||||
// directly (family-lens: out-of-set is rejected) and skips the page-only
|
||||
// library resolution (folder reachability, counts, dropdown payload).
|
||||
func (cfg *Config) loadCollectionBooksLite(c *echo.Context, userUUID uuid.UUID, collection database.Collections, collectionID string) (books []handlers.BookInfo, isSmart bool, errCode string) {
|
||||
collUUID, parseErr := uuid.Parse(collectionID)
|
||||
if parseErr != nil {
|
||||
return nil, isSmart, "invalid_id"
|
||||
}
|
||||
libraryID := c.QueryParam("library_id")
|
||||
|
||||
var libUUID pgtype.UUID
|
||||
if libraryID != "" {
|
||||
parsed, parseErr := uuid.Parse(libraryID)
|
||||
if parseErr != nil {
|
||||
return nil, isSmart, "invalid_library_id"
|
||||
}
|
||||
set, _, setErr := cfg.LibraryService.VisibleLibrarySet(
|
||||
c.Request().Context(), pgtype.UUID{Bytes: userUUID, Valid: true})
|
||||
if setErr != nil {
|
||||
return nil, isSmart, "books_load_error"
|
||||
}
|
||||
if !set[libraryID] {
|
||||
return nil, isSmart, "library_forbidden"
|
||||
}
|
||||
libUUID = pgtype.UUID{Bytes: parsed, Valid: true}
|
||||
}
|
||||
|
||||
visibleIDs, visErr := cfg.CollectionHandler.UserVisibleLibraryIDs(c)
|
||||
if visErr != nil {
|
||||
log.Printf("UserVisibleLibraryIDs failed: %v", visErr)
|
||||
visibleIDs = nil
|
||||
}
|
||||
|
||||
return cfg.collectionBooksForScope(c, userUUID, pgtype.UUID{Bytes: collUUID, Valid: true}, collection, collectionID,
|
||||
libUUID, libUUID, libraryID != "", visibleIDs)
|
||||
}
|
||||
|
||||
// libraryTypeOptionsFor loads the library_types rows for the create-library
|
||||
// dropdowns (admin page + setup wizard), ordered for display. On DB failure
|
||||
// it falls back to the built-in type names so the form stays usable —
|
||||
// CreateLibrary resolves the name against the DB, so this is never
|
||||
// authoritative.
|
||||
func libraryTypeOptionsFor(ctx context.Context, cfg *Config) []database.LibraryTypes {
|
||||
types, err := cfg.LibraryService.GetLibraryTypes(ctx)
|
||||
if err != nil {
|
||||
log.Printf("libraryTypeOptionsFor: GetLibraryTypes failed, using built-in type names: %v", err)
|
||||
for _, name := range []string{
|
||||
services.LibraryTypeEbooks,
|
||||
services.LibraryTypeComics,
|
||||
services.LibraryTypeManga,
|
||||
services.LibraryTypeAudiobooks,
|
||||
} {
|
||||
types = append(types, database.LibraryTypes{Name: name})
|
||||
}
|
||||
}
|
||||
return templates.OrderedLibraryTypes(types)
|
||||
}
|
||||
@@ -18,10 +18,12 @@ func registerMediaRoutes(cfg *Config) {
|
||||
// Book download endpoint - same ServeFile flow as /uploads/library-:id/*
|
||||
// (JWT + library-visibility gated), addressed by media item ID.
|
||||
protected.GET("/media-items/:id/download", cfg.MediaHandler.ServeFile)
|
||||
// The format-abstraction chapter list (audio_metadata). Streaming
|
||||
// itself rides the existing /uploads Range-serving path with cookie
|
||||
// auth — no separate endpoint.
|
||||
// The format-abstraction chapter list (audio_metadata). Single-file
|
||||
// books stream through the existing /uploads Range-serving path;
|
||||
// multi-file books additionally expose per-track streaming.
|
||||
protected.GET("/media-items/:id/chapters", cfg.MediaHandler.GetAudioChapters)
|
||||
protected.PUT("/media-items/:id/audio-speed", cfg.MediaHandler.SetAudioSpeed)
|
||||
protected.GET("/media-items/:id/audio/:trackIndex", cfg.MediaHandler.GetAudioTrack)
|
||||
|
||||
// Media rating routes (all authenticated users)
|
||||
protected.POST("/media-items/:id/rating", cfg.MediaHandler.CreateMediaRating)
|
||||
|
||||
@@ -217,6 +217,11 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
|
||||
mime.AddExtensionType(".m4a", "audio/mp4")
|
||||
mime.AddExtensionType(".mp3", "audio/mpeg")
|
||||
mime.AddExtensionType(".flac", "audio/flac")
|
||||
mime.AddExtensionType(".opus", "audio/ogg")
|
||||
mime.AddExtensionType(".ogg", "audio/ogg")
|
||||
mime.AddExtensionType(".oga", "audio/ogg")
|
||||
mime.AddExtensionType(".aac", "audio/aac")
|
||||
mime.AddExtensionType(".wav", "audio/wav")
|
||||
e := cfg.Echo
|
||||
|
||||
// Set up validator
|
||||
|
||||
@@ -17,6 +17,7 @@ func registerScannerRoutes(protected, admin *echo.Group, h *handlers.Handler) {
|
||||
admin.POST("/scanner/scan", h.ScanLibrary)
|
||||
admin.POST("/scanner/start", h.StartScanner)
|
||||
admin.POST("/scanner/stop", h.StopScanner)
|
||||
admin.POST("/scanner/cancel/:jobId", h.CancelScan)
|
||||
admin.GET("/scanner/status/:jobId", h.GetScanStatus)
|
||||
admin.GET("/scanner/active", h.GetActiveScans)
|
||||
|
||||
|
||||
+39
-54
@@ -3,15 +3,11 @@ package router
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/services"
|
||||
"bookhoard/templates"
|
||||
"bytes"
|
||||
"log"
|
||||
"net/http"
|
||||
"strconv"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
@@ -30,6 +26,12 @@ func registerSearchRoutes(cfg *Config) {
|
||||
// Otherwise return JSON (API clients)
|
||||
return cfg.MediaHandler.SearchMediaItems(c)
|
||||
})
|
||||
// Facet-value listings for the filter panel (faceted-navigation
|
||||
// Batch C): /api/media-items/facets?facet=X&library_id=...
|
||||
protected.GET("/media-items/facets", cfg.MediaHandler.GetFacets)
|
||||
// Rule-builder live count (faceted plan Phase 9): the composed
|
||||
// tree's true match count (COUNT(*)), rules + library scope.
|
||||
protected.GET("/media-items/filter-count", cfg.MediaHandler.FilterCount)
|
||||
protected.POST("/sync/books/query", cfg.MatchingHandler.QueryBooks)
|
||||
}
|
||||
|
||||
@@ -39,60 +41,43 @@ func handleSearchHTML(c *echo.Context, cfg *Config) error {
|
||||
if !ok {
|
||||
return c.String(http.StatusUnauthorized, "Unauthorized")
|
||||
}
|
||||
// Extract query parameters
|
||||
query := c.QueryParam("q")
|
||||
// Extract filters via the SHARED parser (handlers.ParseMediaSearchParams)
|
||||
// — the HTMX grid partial and the JSON endpoint can never disagree on
|
||||
// which facet params exist or how they parse. (This branch previously
|
||||
// dropped publisher_filter and narrator_filter entirely — the shared
|
||||
// parser fixes that too.)
|
||||
params := handlers.ParseMediaSearchParams(c)
|
||||
params.UserID = user.ID
|
||||
|
||||
// Filter rule builder: a malformed qf is a 400, never a silently
|
||||
// ignored token.
|
||||
if params.FilterParseErr != "" {
|
||||
return c.HTML(http.StatusBadRequest, `<div style="color: red;">Invalid filter</div>`)
|
||||
}
|
||||
limit := params.Limit
|
||||
offset := params.Offset
|
||||
libraryID := c.QueryParam("library_id")
|
||||
limit, _ := strconv.Atoi(c.QueryParam("limit"))
|
||||
if limit == 0 {
|
||||
limit = 50
|
||||
}
|
||||
offset, _ := strconv.Atoi(c.QueryParam("offset"))
|
||||
// Validate library_id
|
||||
var libUUID pgtype.UUID
|
||||
if libraryID != "" {
|
||||
lib, err := uuid.Parse(libraryID)
|
||||
|
||||
// Family-lens gate: an explicit library_id outside the requester's
|
||||
// visible set is a 403 (same rule as the JSON branch).
|
||||
if params.LibraryID.Valid {
|
||||
_, ids, err := cfg.LibraryService.VisibleLibrarySet(c.Request().Context(), user.ID)
|
||||
if err != nil {
|
||||
return c.String(http.StatusBadRequest, "invalid library_id")
|
||||
log.Printf("VisibleLibrarySet failed: %v", err)
|
||||
return c.HTML(http.StatusInternalServerError, `<div style="color: red;">Search error</div>`)
|
||||
}
|
||||
visible := false
|
||||
for _, id := range ids {
|
||||
if id == params.LibraryID {
|
||||
visible = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !visible {
|
||||
return c.HTML(http.StatusForbidden, `<div style="color: red;">Library not visible</div>`)
|
||||
}
|
||||
libUUID = pgtype.UUID{Bytes: lib, Valid: true}
|
||||
} else {
|
||||
libUUID = pgtype.UUID{Valid: false}
|
||||
}
|
||||
// Extract filters
|
||||
authorFilter := c.QueryParam("author_filter")
|
||||
seriesFilter := c.QueryParam("series_filter")
|
||||
genreFilter := c.QueryParam("genre_filter")
|
||||
tagsFilter := c.QueryParam("tags_filter")
|
||||
languageFilter := c.QueryParam("language_filter")
|
||||
yearMin, _ := strconv.Atoi(c.QueryParam("year_min"))
|
||||
yearMax, _ := strconv.Atoi(c.QueryParam("year_max"))
|
||||
hasCover := false
|
||||
hasCoverValid := false
|
||||
if _, exists := c.QueryParams()["has_cover"]; exists {
|
||||
hasCover = c.QueryParam("has_cover") == "true"
|
||||
hasCoverValid = true
|
||||
}
|
||||
sortParam := c.QueryParam("sort")
|
||||
if sortParam == "" {
|
||||
sortParam = "title ASC"
|
||||
}
|
||||
// Build search params
|
||||
params := services.SearchParams{
|
||||
UserID: user.ID,
|
||||
LibraryID: libUUID,
|
||||
SearchQuery: query,
|
||||
AuthorFilter: authorFilter,
|
||||
SeriesFilter: seriesFilter,
|
||||
GenreFilter: genreFilter,
|
||||
TagsFilter: tagsFilter,
|
||||
LanguageFilter: languageFilter,
|
||||
YearMin: yearMin,
|
||||
YearMax: yearMax,
|
||||
HasCover: pgtype.Bool{Bool: hasCover, Valid: hasCoverValid},
|
||||
Sort: sortParam,
|
||||
Limit: limit,
|
||||
Offset: offset,
|
||||
}
|
||||
|
||||
// Execute search using handler method
|
||||
results, totalCount, err := cfg.MediaHandler.ExecuteSearch(c.Request().Context(), params)
|
||||
if err != nil {
|
||||
|
||||
@@ -81,7 +81,8 @@ func registerSetupRoutes(cfg *Config) {
|
||||
return c.Redirect(http.StatusFound, "/")
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := templates.Setup().Render(c.Request().Context(), &buf); err != nil {
|
||||
libraryTypes := libraryTypeOptionsFor(c.Request().Context(), cfg)
|
||||
if err := templates.Setup(libraryTypes).Render(c.Request().Context(), &buf); err != nil {
|
||||
log.Printf("Failed to render setup template: %v", err)
|
||||
return c.HTML(http.StatusInternalServerError, "Failed to render setup page")
|
||||
}
|
||||
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
"fmt"
|
||||
"math"
|
||||
|
||||
"bookhoard/internal/utils"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
@@ -16,6 +18,10 @@ type BookMatch struct {
|
||||
BookhoardUUID uuid.UUID `json:"bookhoard_uuid"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
MatchMethod string `json:"match_method"`
|
||||
// Display fields, filled during the family-lens pass in QueryBooks.
|
||||
Title string `json:"title"`
|
||||
Author string `json:"author"`
|
||||
CoverImagePath string `json:"cover_image_path"`
|
||||
}
|
||||
|
||||
// BookQueryRequest represents a book query from a device
|
||||
@@ -24,7 +30,11 @@ type BookQueryRequest struct {
|
||||
SHA256 string `json:"sha256"`
|
||||
Title string `json:"title"`
|
||||
Author string `json:"author"`
|
||||
FileSize int64 `json:"file_size"`
|
||||
// VisibleLibraryIDs scopes results to the requesting user's visible
|
||||
// libraries (family-lens, absent-from-results). Set server-side by the
|
||||
// handler — never from client input. nil keeps legacy callers unfiltered.
|
||||
VisibleLibraryIDs []string `json:"-"`
|
||||
FileSize int64 `json:"file_size"`
|
||||
}
|
||||
|
||||
// BookQueryResponse represents the response to a book query
|
||||
@@ -106,6 +116,36 @@ func (s *BookMatchingService) QueryBooks(ctx context.Context, req *BookQueryRequ
|
||||
}
|
||||
}
|
||||
|
||||
// Family-lens: matches may only surface items from libraries the
|
||||
// requesting user can see — a hidden library's book simply never
|
||||
// matches (absent-from-results). Runs before determineAction so the
|
||||
// suggested action reflects the filtered set. nil (unset) keeps
|
||||
// legacy callers unfiltered.
|
||||
// Enrich + scope in one pass over the cascade hits: display fields come
|
||||
// from the resolved item, and when scoped the family-lens filter drops
|
||||
// matches from libraries the requester cannot see
|
||||
// (absent-from-results).
|
||||
scoped := req.VisibleLibraryIDs != nil
|
||||
allowed := make(map[string]bool, len(req.VisibleLibraryIDs))
|
||||
for _, id := range req.VisibleLibraryIDs {
|
||||
allowed[id] = true
|
||||
}
|
||||
filtered := matches[:0]
|
||||
for _, m := range matches {
|
||||
item, err := s.db.GetMediaItem(ctx, pgtype.UUID{Bytes: m.MediaItemID, Valid: true})
|
||||
if err != nil || !item.LibraryID.Valid {
|
||||
continue // vanished between cascade and filter — drop
|
||||
}
|
||||
m.Title = item.Title
|
||||
m.Author = utils.PgTextString(item.Author)
|
||||
m.CoverImagePath = utils.PgTextString(item.CoverImagePath)
|
||||
if scoped && !allowed[uuid.UUID(item.LibraryID.Bytes).String()] {
|
||||
continue
|
||||
}
|
||||
filtered = append(filtered, m)
|
||||
}
|
||||
matches = filtered
|
||||
|
||||
// Determine action
|
||||
action := s.determineAction(matches)
|
||||
|
||||
|
||||
@@ -155,6 +155,23 @@ func (s *CollectionService) GetUserCollections(ctx context.Context, userID uuid.
|
||||
|
||||
// UpdateCollection updates a collection
|
||||
func (s *CollectionService) UpdateCollection(ctx context.Context, collectionID uuid.UUID, name, description, color, icon string, autoAssignRules []Rule, viewSettings map[string]any) (database.Collections, error) {
|
||||
// A2c §3.3: the edit form carries only the toggles it renders — merge
|
||||
// them OVER the stored view_settings so provenance keys
|
||||
// (imported_from_series) and anything else on the row survive.
|
||||
if len(viewSettings) > 0 {
|
||||
existing, err := s.db.GetCollection(ctx, pgtype.UUID{Bytes: collectionID, Valid: true})
|
||||
if err == nil && len(existing.ViewSettings) > 0 {
|
||||
var stored map[string]any
|
||||
if json.Unmarshal(existing.ViewSettings, &stored) == nil {
|
||||
merged := stored
|
||||
for k, v := range viewSettings {
|
||||
merged[k] = v
|
||||
}
|
||||
viewSettings = merged
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Convert rules to JSONB ([]byte)
|
||||
var rulesJSON []byte
|
||||
if len(autoAssignRules) > 0 {
|
||||
@@ -785,3 +802,35 @@ func parseFloat(s string) (float64, error) {
|
||||
|
||||
return sign * result / divisor, nil
|
||||
}
|
||||
|
||||
// UserOwnsCollection gates playlist-ordering writes (A2c §3.2): the
|
||||
// collection row's owner is the only user whose drag actions rewrite
|
||||
// positions.
|
||||
func (s *CollectionService) UserOwnsCollection(ctx context.Context, collectionID, userID uuid.UUID) (bool, error) {
|
||||
collection, err := s.db.GetCollection(ctx, pgtype.UUID{Bytes: collectionID, Valid: true})
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return uuid.UUID(collection.UserID.Bytes) == userID, nil
|
||||
}
|
||||
|
||||
// ReorderItems applies a full playlist order (A2c §3.2): the payload is the
|
||||
// complete ordered id list; positions become 1..n in payload order. Ids
|
||||
// not in the payload keep their relative order after — the SQL only
|
||||
// touches the given ids, and NULL positions sort last. Never deletes.
|
||||
func (s *CollectionService) ReorderItems(ctx context.Context, collectionID uuid.UUID, orderedIDs []uuid.UUID) error {
|
||||
if len(orderedIDs) == 0 {
|
||||
return nil
|
||||
}
|
||||
ids := make([]pgtype.UUID, 0, len(orderedIDs))
|
||||
for _, id := range orderedIDs {
|
||||
ids = append(ids, pgtype.UUID{Bytes: id, Valid: true})
|
||||
}
|
||||
if err := s.db.ReorderCollectionItems(ctx, database.ReorderCollectionItemsParams{
|
||||
CollectionID: pgtype.UUID{Bytes: collectionID, Valid: true},
|
||||
MediaItemIds: ids,
|
||||
}); err != nil {
|
||||
return fmt.Errorf("failed to reorder collection items: %v", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -24,6 +24,34 @@ func NewDashboardService(db *database.Queries) *DashboardService {
|
||||
}
|
||||
}
|
||||
|
||||
// viewSettingsFlag reads a boolean key from a collection's view_settings
|
||||
// JSONB. Malformed or absent → false.
|
||||
func viewSettingsFlag(raw []byte, key string) bool {
|
||||
if len(raw) == 0 {
|
||||
return false
|
||||
}
|
||||
var settings map[string]interface{}
|
||||
if err := json.Unmarshal(raw, &settings); err != nil {
|
||||
return false
|
||||
}
|
||||
b, ok := settings[key].(bool)
|
||||
return ok && b
|
||||
}
|
||||
|
||||
// viewSettingsHasKey reports whether the key exists at all (any value) —
|
||||
// used for provenance markers like imported_from_series.
|
||||
func viewSettingsHasKey(raw []byte, key string) bool {
|
||||
if len(raw) == 0 {
|
||||
return false
|
||||
}
|
||||
var settings map[string]interface{}
|
||||
if err := json.Unmarshal(raw, &settings); err != nil {
|
||||
return false
|
||||
}
|
||||
_, ok := settings[key]
|
||||
return ok
|
||||
}
|
||||
|
||||
type DashboardSection struct {
|
||||
CollectionID uuid.UUID
|
||||
CollectionName string
|
||||
@@ -34,6 +62,13 @@ type DashboardSection struct {
|
||||
Title string
|
||||
Description string
|
||||
Icon string
|
||||
// A2c: the collection's own ordering-construct marker — the section is
|
||||
// a curated playlist ("The Witcher (Reading Order)"), rendered with a
|
||||
// badge by the frontend.
|
||||
IsPlaylist bool
|
||||
// A2c §3.5: hide_finished filtered every item out of a NON-empty
|
||||
// collection — render "playlist complete" instead of dropping silently.
|
||||
Complete bool
|
||||
}
|
||||
|
||||
func (s *DashboardService) GetDashboardSections(
|
||||
@@ -81,7 +116,29 @@ func (s *DashboardService) GetDashboardSections(
|
||||
continue
|
||||
}
|
||||
|
||||
// A2c §3.5: hide_finished filtered everything out — but the
|
||||
// collection is NOT empty. That's a finished playlist: render a
|
||||
// completion marker instead of silently vanishing.
|
||||
if len(items) == 0 {
|
||||
hideFinished := viewSettingsFlag(coll.ViewSettings, "hide_finished")
|
||||
if hideFinished {
|
||||
collUUIDBytes, _ := uuid.FromBytes(coll.ID.Bytes[0:16])
|
||||
if count, countErr := s.db.CountCollectionItems(ctx, pgtype.UUID{Bytes: collUUIDBytes, Valid: true}); countErr == nil && count > 0 {
|
||||
results = append(results, DashboardSection{
|
||||
CollectionID: coll.ID.Bytes,
|
||||
CollectionName: coll.Name,
|
||||
Items: []database.MediaItems{},
|
||||
QueryType: coll.QueryType.String,
|
||||
Priority: int(coll.Priority.Int32),
|
||||
IsSystem: false,
|
||||
Title: coll.Name,
|
||||
Description: coll.Description.String,
|
||||
Icon: coll.Icon.String,
|
||||
IsPlaylist: viewSettingsHasKey(coll.ViewSettings, "imported_from_series"),
|
||||
Complete: true,
|
||||
})
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -95,6 +152,7 @@ func (s *DashboardService) GetDashboardSections(
|
||||
Title: coll.Name,
|
||||
Description: coll.Description.String,
|
||||
Icon: coll.Icon.String,
|
||||
IsPlaylist: viewSettingsHasKey(coll.ViewSettings, "imported_from_series"),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -218,9 +276,15 @@ func (s *DashboardService) getCollectionItemsByQueryType(ctx context.Context, co
|
||||
func (s *DashboardService) getUserCollectionItems(ctx context.Context, coll database.Collections, userID uuid.UUID, libraryID pgtype.UUID, limit int) ([]database.MediaItems, error) {
|
||||
collUUID, _ := uuid.FromBytes(coll.ID.Bytes[0:16])
|
||||
|
||||
// A2c §3.3: per-collection hide_finished (view_settings), evaluated
|
||||
// against the VIEWER's reading progress. The section then shows the
|
||||
// head of the reading queue in playlist order.
|
||||
hideFinished := viewSettingsFlag(coll.ViewSettings, "hide_finished")
|
||||
manualItems, err := s.db.GetCollectionItemsForDashboard(ctx, database.GetCollectionItemsForDashboardParams{
|
||||
CollectionID: pgtype.UUID{Bytes: collUUID, Valid: true},
|
||||
LibraryID: libraryID,
|
||||
ViewerUserID: pgtype.UUID{Bytes: userID, Valid: true},
|
||||
HideFinished: pgtype.Bool{Bool: hideFinished, Valid: true},
|
||||
Limit: pgtype.Int4{Int32: int32(limit), Valid: true},
|
||||
})
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
package services
|
||||
|
||||
// Facet registry (faceted-navigation plan, Part 1 Batch A).
|
||||
//
|
||||
// The single declarative source of truth for bookshelf filter facets.
|
||||
// sqlc has NO dynamic WHERE — each registry entry corresponds to an
|
||||
// explicit `sqlc.narg('<param>_filter')` block in
|
||||
// `SearchMediaItemsUnified` (internal/database/queries/queries.sql);
|
||||
// the drift-guard test (facet_registry_test.go) fails the build when an
|
||||
// entry's block is missing. Adding a facet = one registry entry + one
|
||||
// narg block + one handler line (+ one templ call in Batch B).
|
||||
//
|
||||
// Wire semantics: the query param is `<Param>_filter`; a comma-joined
|
||||
// value is multi-select (OR within the facet, AND across facets) for
|
||||
// every text kind — the SQL splits via string_to_array. Bools are
|
||||
// presence-bound: absent → NULL (no filtering), "true"/"false" → bound.
|
||||
|
||||
import "strings"
|
||||
|
||||
// FacetKind classifies the media_items column shape a facet filters on.
|
||||
type FacetKind string
|
||||
|
||||
const (
|
||||
FacetArray FacetKind = "array" // TEXT[] column, per-element match
|
||||
FacetScalar FacetKind = "scalar" // varchar column
|
||||
FacetBool FacetKind = "bool" // bool column
|
||||
FacetRange FacetKind = "range" // int min/max pair (year_min/year_max)
|
||||
)
|
||||
|
||||
// FacetMatch classifies how values are compared within the kind's SQL
|
||||
// shape.
|
||||
type FacetMatch string
|
||||
|
||||
const (
|
||||
MatchPerElementFuzzy FacetMatch = "per_element_fuzzy" // word_similarity per array element
|
||||
MatchScalarFuzzy FacetMatch = "scalar_fuzzy" // word_similarity against the column
|
||||
MatchExact FacetMatch = "exact" // = / ANY() / bool equality
|
||||
)
|
||||
|
||||
// Facet is one registry entry.
|
||||
type Facet struct {
|
||||
Param string // "publisher" → ?publisher_filter= (see WireParam)
|
||||
// Wire is an optional full query-param override for facets whose
|
||||
// wire name predates the _filter convention (has_cover). Existing
|
||||
// facets migrate in with their frozen wire names — behavior does
|
||||
// not change.
|
||||
Wire string
|
||||
Kind FacetKind
|
||||
Match FacetMatch
|
||||
// Column is the media_items column(s) the facet filters (documentation).
|
||||
Column string
|
||||
// LibraryTypes limits the facet to those library_types.name values
|
||||
// (e.g. manga_type only on "manga"). Empty/nil = every library type.
|
||||
// The All-Libraries view shows the UNION of all facets — matching is
|
||||
// type-partitioned by the data, so cross-type filtering is correct
|
||||
// by construction (plan Batch D decision).
|
||||
LibraryTypes []string
|
||||
}
|
||||
|
||||
// facetRegistry — the inventory. Order is display order for the filter
|
||||
// panel (Batch B); the existing facets come first (they shipped
|
||||
// 646477af..7537063c and their wire behavior is frozen).
|
||||
var facetRegistry = []Facet{
|
||||
{Param: "author", Kind: FacetArray, Match: MatchPerElementFuzzy, Column: "authors"},
|
||||
{Param: "publisher", Kind: FacetArray, Match: MatchPerElementFuzzy, Column: "publishers"},
|
||||
{Param: "narrator", Kind: FacetArray, Match: MatchPerElementFuzzy, Column: "narrators", LibraryTypes: []string{"audiobooks"}},
|
||||
{Param: "tags", Kind: FacetArray, Match: MatchPerElementFuzzy, Column: "tags"},
|
||||
{Param: "series", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "series"},
|
||||
{Param: "genre", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "genre"},
|
||||
{Param: "language", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "language"},
|
||||
{Param: "year", Kind: FacetRange, Match: MatchExact, Column: "date_published / copyright_year"},
|
||||
{Param: "has_cover", Wire: "has_cover", Kind: FacetBool, Match: MatchExact, Column: "cover_image_path"},
|
||||
// Batch A additions:
|
||||
{Param: "story_arc", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "story_arc"},
|
||||
{Param: "imprint", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "imprint"},
|
||||
{Param: "edition", Kind: FacetScalar, Match: MatchScalarFuzzy, Column: "edition"},
|
||||
{Param: "age_rating", Kind: FacetScalar, Match: MatchExact, Column: "age_rating"},
|
||||
{Param: "format_group", Kind: FacetScalar, Match: MatchExact, Column: "format_group"},
|
||||
{Param: "manga_type", Kind: FacetScalar, Match: MatchExact, Column: "manga_type", LibraryTypes: []string{"manga"}},
|
||||
{Param: "reading_direction", Kind: FacetScalar, Match: MatchExact, Column: "reading_direction", LibraryTypes: []string{"manga", "comics"}},
|
||||
{Param: "explicit", Kind: FacetBool, Match: MatchExact, Column: "explicit"},
|
||||
{Param: "abridged", Kind: FacetBool, Match: MatchExact, Column: "abridged"},
|
||||
}
|
||||
|
||||
// Facets returns a copy of the registry.
|
||||
func Facets() []Facet {
|
||||
out := make([]Facet, len(facetRegistry))
|
||||
copy(out, facetRegistry)
|
||||
return out
|
||||
}
|
||||
|
||||
// WireParam is the query-param name for the facet: the Wire override
|
||||
// when set, else `<Param>_filter`.
|
||||
func (f Facet) WireParam() string {
|
||||
if f.Wire != "" {
|
||||
return f.Wire
|
||||
}
|
||||
return f.Param + "_filter"
|
||||
}
|
||||
|
||||
// SplitFacetValues splits a wire value into its OR-within alternatives.
|
||||
// Shared by the SQL semantics (string_to_array) and later by the
|
||||
// facet-count endpoint so display and matching can never disagree.
|
||||
func SplitFacetValues(wireValue string) []string {
|
||||
if wireValue == "" {
|
||||
return nil
|
||||
}
|
||||
parts := strings.Split(wireValue, ",")
|
||||
out := make([]string, 0, len(parts))
|
||||
for _, p := range parts {
|
||||
if t := strings.TrimSpace(p); t != "" {
|
||||
out = append(out, t)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// FacetByParam finds a registry entry by its base param name (nil when
|
||||
// absent).
|
||||
func FacetByParam(param string) *Facet {
|
||||
for i := range facetRegistry {
|
||||
if facetRegistry[i].Param == param {
|
||||
f := facetRegistry[i]
|
||||
return &f
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// FacetsForLibraryType returns the facets that RENDER for a view. An
|
||||
// empty libraryTypeName is the All-Libraries view → the UNION of every
|
||||
// registered facet (plan Batch D: no special SQL — matching is
|
||||
// type-partitioned by the data). A specific type gets facets with no
|
||||
// type restriction plus those listing it.
|
||||
func FacetsForLibraryType(libraryTypeName string) []Facet {
|
||||
if libraryTypeName == "" {
|
||||
return Facets()
|
||||
}
|
||||
var out []Facet
|
||||
for _, f := range facetRegistry {
|
||||
if len(f.LibraryTypes) == 0 {
|
||||
out = append(out, f)
|
||||
continue
|
||||
}
|
||||
for _, t := range f.LibraryTypes {
|
||||
if t == libraryTypeName {
|
||||
out = append(out, f)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// The generated SQL constant for the unified search query. The
|
||||
// drift-guard test asserts every registry entry's narg block exists in
|
||||
// it — sqlc has no dynamic WHERE, so registry ↔ SQL agreement is
|
||||
// enforced here instead (plan: "one registry entry + one narg block").
|
||||
|
||||
func TestFacetRegistryShape(t *testing.T) {
|
||||
seen := map[string]bool{}
|
||||
knownLibraryTypes := map[string]bool{
|
||||
"ebooks": true, "comics": true, "manga": true, "audiobooks": true,
|
||||
}
|
||||
for _, f := range facetRegistry {
|
||||
if f.Param == "" {
|
||||
t.Fatalf("facet with empty Param: %+v", f)
|
||||
}
|
||||
wire := f.WireParam()
|
||||
if seen[wire] {
|
||||
t.Errorf("duplicate wire param %q", wire)
|
||||
}
|
||||
seen[wire] = true
|
||||
if f.Column == "" {
|
||||
t.Errorf("%s: empty Column", wire)
|
||||
}
|
||||
// Kind ↔ Match consistency.
|
||||
switch f.Kind {
|
||||
case FacetArray:
|
||||
if f.Match != MatchPerElementFuzzy {
|
||||
t.Errorf("%s: array facets are per-element fuzzy, got %s", wire, f.Match)
|
||||
}
|
||||
case FacetScalar:
|
||||
if f.Match != MatchScalarFuzzy && f.Match != MatchExact {
|
||||
t.Errorf("%s: scalar facets are scalar-fuzzy or exact, got %s", wire, f.Match)
|
||||
}
|
||||
case FacetBool:
|
||||
if f.Match != MatchExact {
|
||||
t.Errorf("%s: bool facets are exact, got %s", wire, f.Match)
|
||||
}
|
||||
case FacetRange:
|
||||
if f.Match != MatchExact {
|
||||
t.Errorf("%s: range facets are exact, got %s", wire, f.Match)
|
||||
}
|
||||
default:
|
||||
t.Errorf("%s: unknown kind %q", wire, f.Kind)
|
||||
}
|
||||
for _, lt := range f.LibraryTypes {
|
||||
if !knownLibraryTypes[lt] {
|
||||
t.Errorf("%s: unknown library type %q (seed names: ebooks, comics, manga, audiobooks)", wire, lt)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacetRegistrySQLDrift(t *testing.T) {
|
||||
// sqlc substitutes sqlc.narg(...) with ? placeholders in the emitted
|
||||
// const, so the drift guard reflects over the GENERATED params
|
||||
// struct instead: a narg block exists in the query iff the struct
|
||||
// has its field. This checks the artifact the binary actually runs.
|
||||
paramsType := reflect.TypeOf(database.SearchMediaItemsUnifiedParams{})
|
||||
dbTags := map[string]reflect.Type{}
|
||||
for i := 0; i < paramsType.NumField(); i++ {
|
||||
tag := strings.Split(paramsType.Field(i).Tag.Get("db"), ",")[0]
|
||||
dbTags[tag] = paramsType.Field(i).Type
|
||||
}
|
||||
boolType := reflect.TypeOf(pgtype.Bool{})
|
||||
for _, f := range facetRegistry {
|
||||
wire := f.WireParam()
|
||||
switch f.Kind {
|
||||
case FacetRange:
|
||||
// The year range binds the year_min/year_max pair.
|
||||
if _, ok := dbTags["year_min"]; !ok {
|
||||
t.Error("year range facets must bind year_min in SearchMediaItemsUnified")
|
||||
}
|
||||
if _, ok := dbTags["year_max"]; !ok {
|
||||
t.Error("year range facets must bind year_max in SearchMediaItemsUnified")
|
||||
}
|
||||
case FacetBool:
|
||||
ft, ok := dbTags[wire]
|
||||
if !ok {
|
||||
t.Errorf("registry facet %q missing from SearchMediaItemsUnified (want a %s param)", wire, wire)
|
||||
continue
|
||||
}
|
||||
if ft != boolType {
|
||||
t.Errorf("%s: bool facet must generate pgtype.Bool (the ::bool cast), got %s", wire, ft)
|
||||
}
|
||||
default:
|
||||
if _, ok := dbTags[wire]; !ok {
|
||||
t.Errorf("registry facet %q missing from SearchMediaItemsUnified (want a %s param)", wire, wire)
|
||||
}
|
||||
}
|
||||
}
|
||||
// The Batch A additions specifically (pin the param names).
|
||||
for _, wire := range []string{
|
||||
"story_arc_filter", "imprint_filter", "edition_filter",
|
||||
"age_rating_filter", "format_group_filter", "manga_type_filter",
|
||||
"reading_direction_filter", "explicit_filter", "abridged_filter",
|
||||
} {
|
||||
if _, ok := dbTags[wire]; !ok {
|
||||
t.Errorf("Batch A facet %s missing from SearchMediaItemsUnified", wire)
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 7: the counts query (FacetCountsUnderFilters) carries the
|
||||
// same filter blocks — copied mechanically from the search query —
|
||||
// so the panel's under-filter counts and the grid's rows can never
|
||||
// disagree about what a filter means. Reflect the generated params
|
||||
// struct the same way.
|
||||
countParamsType := reflect.TypeOf(database.FacetCountsUnderFiltersParams{})
|
||||
countTags := map[string]bool{}
|
||||
for i := 0; i < countParamsType.NumField(); i++ {
|
||||
tag := strings.Split(countParamsType.Field(i).Tag.Get("db"), ",")[0]
|
||||
countTags[tag] = true
|
||||
}
|
||||
for _, want := range []string{"user_id", "library_id", "limit"} {
|
||||
if !countTags[want] {
|
||||
t.Errorf("FacetCountsUnderFilters missing %s param", want)
|
||||
}
|
||||
}
|
||||
for _, f := range facetRegistry {
|
||||
wire := f.WireParam()
|
||||
if f.Kind == FacetRange {
|
||||
if !countTags["year_min"] || !countTags["year_max"] {
|
||||
t.Error("year range must bind year_min/year_max in FacetCountsUnderFilters")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !countTags[wire] {
|
||||
t.Errorf("registry facet %q missing from FacetCountsUnderFilters (each arm binds every OTHER facet's block, so every facet's param must exist)", wire)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFacetsForLibraryType(t *testing.T) {
|
||||
// Empty = All Libraries → the UNION of every facet (Batch D decision).
|
||||
all := FacetsForLibraryType("")
|
||||
if len(all) != len(facetRegistry) {
|
||||
t.Fatalf("all-libraries union = %d facets, want %d", len(all), len(facetRegistry))
|
||||
}
|
||||
|
||||
byWire := func(fs []Facet) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
for _, f := range fs {
|
||||
m[f.WireParam()] = true
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
manga := byWire(FacetsForLibraryType("manga"))
|
||||
if !manga["manga_type_filter"] || !manga["reading_direction_filter"] {
|
||||
t.Error("manga view must include manga_type and reading_direction")
|
||||
}
|
||||
if manga["narrator_filter"] {
|
||||
t.Error("narrator facet is audiobooks-only, must not render on manga")
|
||||
}
|
||||
if !manga["author_filter"] || !manga["genre_filter"] {
|
||||
t.Error("unrestricted facets render on every library type")
|
||||
}
|
||||
|
||||
audio := byWire(FacetsForLibraryType("audiobooks"))
|
||||
if !audio["narrator_filter"] {
|
||||
t.Error("audiobooks view must include narrator")
|
||||
}
|
||||
if audio["manga_type_filter"] || audio["reading_direction_filter"] {
|
||||
t.Error("manga facets must not render on audiobooks")
|
||||
}
|
||||
|
||||
comics := byWire(FacetsForLibraryType("comics"))
|
||||
if !comics["reading_direction_filter"] {
|
||||
t.Error("comics view must include reading_direction")
|
||||
}
|
||||
if comics["manga_type_filter"] {
|
||||
t.Error("manga_type is manga-only, must not render on comics")
|
||||
}
|
||||
|
||||
unknown := byWire(FacetsForLibraryType("woodcuts"))
|
||||
if unknown["manga_type_filter"] || unknown["narrator_filter"] {
|
||||
t.Error("restricted facets must not render for an unknown type")
|
||||
}
|
||||
if !unknown["author_filter"] {
|
||||
t.Error("unrestricted facets still render for an unknown type")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitFacetValues(t *testing.T) {
|
||||
if got := SplitFacetValues(""); got != nil {
|
||||
t.Errorf("empty → nil, got %v", got)
|
||||
}
|
||||
got := SplitFacetValues("Vertigo, DC Black Label ,Vertigo")
|
||||
if len(got) != 3 || got[0] != "Vertigo" || got[1] != "DC Black Label" || got[2] != "Vertigo" {
|
||||
t.Errorf("SplitFacetValues = %v, want [Vertigo DC Black Label Vertigo]", got)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,560 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strconv"
|
||||
|
||||
"github.com/google/uuid"
|
||||
)
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════
|
||||
// Filter rule builder — the field registry (faceted plan Phase 9).
|
||||
//
|
||||
// The mandate: EVERYTHING is filterable — every attribute on a book and
|
||||
// everything that pertains to one. This registry is the completeness
|
||||
// check: fields and operators are typed constants (the allow-list — no
|
||||
// user string ever reaches a field/operator position), every value is a
|
||||
// bound pgx parameter, and each entry carries its own match-clause
|
||||
// closure so the composed engines (search + suggestions + count) grow
|
||||
// by adding entries, not SQL.
|
||||
//
|
||||
// User-relative fields read the VIEWER's rows: the user id is
|
||||
// server-injected (never a client param) — two users get different
|
||||
// results for the same tree (the "my data" weld).
|
||||
//
|
||||
// Holding structures (library, collections, series — primary AND
|
||||
// alternate) are fields like any other; any future structure that
|
||||
// "holds" a book must join this registry.
|
||||
// ════════════════════════════════════════════════════════════════════
|
||||
|
||||
// Filter operators (the allow-list).
|
||||
const (
|
||||
OpContains = "contains"
|
||||
OpNotContains = "not_contains"
|
||||
OpIs = "is"
|
||||
OpIsNot = "is_not"
|
||||
OpEmpty = "empty"
|
||||
OpIsTrue = "is_true"
|
||||
OpIsFalse = "is_false"
|
||||
OpBetween = "between"
|
||||
OpAtLeast = "at_least"
|
||||
OpAtMost = "at_most"
|
||||
OpWithinDays = "within_days"
|
||||
OpHas = "has"
|
||||
OpMissing = "missing"
|
||||
OpUnrated = "unrated"
|
||||
OpIn = "in"
|
||||
OpNotIn = "not_in"
|
||||
OpInAny = "in_any"
|
||||
OpInNone = "in_none"
|
||||
)
|
||||
|
||||
// FilterKind — how a field's values match and what the value cell renders.
|
||||
type FilterKind string
|
||||
|
||||
const (
|
||||
KindScalarFuzzy FilterKind = "scalar_fuzzy" // word_similarity on a text column
|
||||
KindArrayFuzzy FilterKind = "array_fuzzy" // word_similarity per element of a text[]
|
||||
KindText FilterKind = "text" // fuzzy contains + exact is + empty (title-like)
|
||||
KindEnum FilterKind = "enum" // exact scalar, = ANY(values)
|
||||
KindBool FilterKind = "bool"
|
||||
KindNumber FilterKind = "number"
|
||||
KindDate FilterKind = "date"
|
||||
KindPresence FilterKind = "presence" // has / missing
|
||||
KindIdentifier FilterKind = "identifier" // exact is/is_not + has/missing
|
||||
KindNoteText FilterKind = "note_text" // joined-table ILIKE substring (viewer's rows)
|
||||
KindStatus FilterKind = "status" // unread / reading / finished
|
||||
KindLibrary FilterKind = "library"
|
||||
KindCollection FilterKind = "collection"
|
||||
)
|
||||
|
||||
// Suggestion sources — where a field's value menu comes from. A source
|
||||
// of suggestNone means the value cell is free input (numbers, dates,
|
||||
// bools, fixed enums).
|
||||
type SuggestSource string
|
||||
|
||||
const (
|
||||
suggestNone SuggestSource = ""
|
||||
suggestAuthor SuggestSource = "author"
|
||||
suggestNarrator SuggestSource = "narrator"
|
||||
suggestContrib SuggestSource = "contributor"
|
||||
suggestPublisher SuggestSource = "publisher"
|
||||
suggestTags SuggestSource = "tags"
|
||||
suggestColumn SuggestSource = "column" // the field's own scalar column
|
||||
suggestSeries SuggestSource = "series" // primary + alternate_info
|
||||
suggestNoteText SuggestSource = "note_text" // the viewer's note snippets
|
||||
suggestHighlight SuggestSource = "highlight_text"
|
||||
suggestLibrary SuggestSource = "library" // visibility-scoped libraries
|
||||
suggestCollection SuggestSource = "collection" // the viewer's collections
|
||||
suggestAddedBy SuggestSource = "added_by" // admins who added books in scope
|
||||
)
|
||||
|
||||
// Filter menu groups (display order for the field menu).
|
||||
const (
|
||||
GroupPeople = "People"
|
||||
GroupPublish = "Publishing"
|
||||
GroupTopics = "Topics"
|
||||
GroupBookFacts = "Book facts"
|
||||
GroupComics = "Comics & manga"
|
||||
GroupMyReading = "My reading"
|
||||
GroupLibrary = "Library"
|
||||
)
|
||||
|
||||
// FilterRule is one leaf: field + operator + values (all bound params).
|
||||
type FilterRule struct {
|
||||
Field string
|
||||
Op string
|
||||
Values []string
|
||||
}
|
||||
|
||||
// FilterTree is the qf wire shape: a group (mode + children) whose
|
||||
// children are rules or nested groups. Groups AND/OR their children
|
||||
// together; a group ANDs into its parent.
|
||||
type FilterTree struct {
|
||||
Mode string `json:"mode"`
|
||||
Children []FilterChild `json:"children"`
|
||||
}
|
||||
|
||||
// FilterChild is one node: a rule (Field set) or a group (Mode set).
|
||||
type FilterChild struct {
|
||||
Field string `json:"field,omitempty"`
|
||||
Op string `json:"op,omitempty"`
|
||||
Values []string `json:"values,omitempty"`
|
||||
Mode string `json:"mode,omitempty"`
|
||||
Children []FilterChild `json:"children,omitempty"`
|
||||
}
|
||||
|
||||
// FilterField is one registry entry.
|
||||
type FilterField struct {
|
||||
Key string // wire name (the qf "field" value)
|
||||
Label string // menu label
|
||||
Group string // menu group
|
||||
Kind FilterKind
|
||||
Ops []string // allowed operators, display order
|
||||
DefaultOp string // operator a new rule starts on (field-switch reset)
|
||||
Clause func(r FilterRule, b *filterBuilder) string
|
||||
Suggest SuggestSource
|
||||
}
|
||||
|
||||
// opsWithValue / opsWithoutValue drive validation.
|
||||
var opsNoValue = map[string]bool{
|
||||
OpEmpty: true, OpIsTrue: true, OpIsFalse: true, OpUnrated: true,
|
||||
OpInAny: true, OpInNone: true, OpHas: true, OpMissing: true,
|
||||
}
|
||||
|
||||
// fixedDomains are the operator value domains that are STATIC (validated
|
||||
// here) rather than data-driven suggestions.
|
||||
var fixedDomains = map[string][]string{
|
||||
"status": {"unread", "reading", "finished"},
|
||||
"file_availability": {"available", "missing"},
|
||||
}
|
||||
|
||||
// filterFields is THE registry. Order within a group = menu order.
|
||||
var filterFields = []FilterField{
|
||||
// ── People ────────────────────────────────────────────────────
|
||||
{Key: "author", Label: "Author", Group: GroupPeople, Kind: KindArrayFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseAuthorArray, Suggest: suggestAuthor},
|
||||
{Key: "narrator", Label: "Narrator", Group: GroupPeople, Kind: KindArrayFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseArrayFuzzyFactory("mi.narrators"), Suggest: suggestNarrator},
|
||||
{Key: "contributor", Label: "Contributor", Group: GroupPeople, Kind: KindArrayFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseArrayFuzzyFactory("mi.contributors"), Suggest: suggestContrib},
|
||||
|
||||
// ── Publishing ────────────────────────────────────────────────
|
||||
{Key: "publisher", Label: "Publisher", Group: GroupPublish, Kind: KindArrayFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseArrayFuzzyFactory("mi.publishers"), Suggest: suggestPublisher},
|
||||
{Key: "imprint", Label: "Imprint", Group: GroupPublish, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.imprint"), Suggest: suggestColumn},
|
||||
{Key: "edition", Label: "Edition", Group: GroupPublish, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.edition"), Suggest: suggestColumn},
|
||||
{Key: "published_year", Label: "Published year", Group: GroupPublish, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clausePublishedYear, Suggest: suggestNone},
|
||||
|
||||
// ── Topics ────────────────────────────────────────────────────
|
||||
{Key: "genre", Label: "Genre", Group: GroupTopics, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.genre"), Suggest: suggestColumn},
|
||||
{Key: "tags", Label: "Tags", Group: GroupTopics, Kind: KindArrayFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseArrayFuzzyFactory("mi.tags_search"), Suggest: suggestTags},
|
||||
{Key: "story_arc", Label: "Story arc", Group: GroupTopics, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.story_arc"), Suggest: suggestColumn},
|
||||
{Key: "age_rating", Label: "Age rating", Group: GroupTopics, Kind: KindEnum,
|
||||
Ops: []string{OpIs, OpIsNot, OpEmpty}, DefaultOp: OpIs,
|
||||
Clause: clauseEnumFactory("mi.age_rating"), Suggest: suggestColumn},
|
||||
|
||||
// ── Book facts ────────────────────────────────────────────────
|
||||
{Key: "title", Label: "Title", Group: GroupBookFacts, Kind: KindText,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.title"), Suggest: suggestColumn},
|
||||
{Key: "subtitle", Label: "Subtitle", Group: GroupBookFacts, Kind: KindText,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.subtitle"), Suggest: suggestColumn},
|
||||
{Key: "description", Label: "Description", Group: GroupBookFacts, Kind: KindText,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseScalarFuzzyFactory("mi.description"), Suggest: suggestNone},
|
||||
{Key: "summary", Label: "Summary", Group: GroupBookFacts, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clausePresenceFactory("mi.summary"), Suggest: suggestNone},
|
||||
{Key: "language", Label: "Language", Group: GroupBookFacts, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseLanguage, Suggest: suggestColumn},
|
||||
{Key: "page_count", Label: "Page count", Group: GroupBookFacts, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.page_count", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "chapter_count", Label: "Chapter count", Group: GroupBookFacts, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.chapter_count", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "total_characters", Label: "Total characters", Group: GroupBookFacts, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.total_characters", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "community_rating", Label: "Community rating", Group: GroupBookFacts, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.community_rating", parseNumberFloat), Suggest: suggestNone},
|
||||
{Key: "added_at", Label: "Date added", Group: GroupBookFacts, Kind: KindDate,
|
||||
Ops: []string{OpWithinDays, OpBetween, OpEmpty}, DefaultOp: OpWithinDays,
|
||||
Clause: clauseDateFactory("mi.created_at"), Suggest: suggestNone},
|
||||
{Key: "imported_at", Label: "Date imported", Group: GroupBookFacts, Kind: KindDate,
|
||||
Ops: []string{OpWithinDays, OpBetween, OpEmpty}, DefaultOp: OpWithinDays,
|
||||
Clause: clauseDateFactory("mi.imported_at"), Suggest: suggestNone},
|
||||
{Key: "web_url", Label: "Web URL", Group: GroupBookFacts, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clausePresenceFactory("mi.web_url"), Suggest: suggestNone},
|
||||
{Key: "metadata_notes", Label: "Metadata notes", Group: GroupBookFacts, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clausePresenceFactory("mi.metadata_notes"), Suggest: suggestNone},
|
||||
{Key: "isbn", Label: "ISBN", Group: GroupBookFacts, Kind: KindIdentifier,
|
||||
Ops: []string{OpIs, OpIsNot, OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clauseIdentifierFactory("mi.isbn"), Suggest: suggestNone},
|
||||
{Key: "asin", Label: "ASIN", Group: GroupBookFacts, Kind: KindIdentifier,
|
||||
Ops: []string{OpIs, OpIsNot, OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clauseIdentifierFactory("mi.asin"), Suggest: suggestNone},
|
||||
{Key: "file_size", Label: "File size", Group: GroupBookFacts, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.file_size", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "file_availability", Label: "File availability", Group: GroupBookFacts, Kind: KindStatus,
|
||||
Ops: []string{OpIs, OpIsNot}, DefaultOp: OpIs,
|
||||
Clause: clauseFileAvailability, Suggest: suggestNone},
|
||||
{Key: "explicit", Label: "Explicit", Group: GroupBookFacts, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
Clause: clauseBoolFactory("mi.explicit"), Suggest: suggestNone},
|
||||
{Key: "abridged", Label: "Abridged", Group: GroupBookFacts, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
Clause: clauseBoolFactory("mi.abridged"), Suggest: suggestNone},
|
||||
{Key: "manually_edited", Label: "Manually edited", Group: GroupBookFacts, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
// metadata_overrides is NOT NULL TEXT[] — '{}' means untouched.
|
||||
Clause: func(r FilterRule, b *filterBuilder) string {
|
||||
switch r.Op {
|
||||
case OpIsTrue:
|
||||
return "(cardinality(mi.metadata_overrides) > 0)"
|
||||
case OpIsFalse:
|
||||
return "(cardinality(mi.metadata_overrides) = 0)"
|
||||
}
|
||||
return "TRUE"
|
||||
}, Suggest: suggestNone},
|
||||
{Key: "cover", Label: "Cover", Group: GroupBookFacts, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clauseCover, Suggest: suggestNone},
|
||||
{Key: "added_by", Label: "Added by", Group: GroupBookFacts, Kind: KindIdentifier,
|
||||
Ops: []string{OpIs, OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clauseAddedBy, Suggest: suggestAddedBy},
|
||||
|
||||
// ── Comics & manga ────────────────────────────────────────────
|
||||
{Key: "series", Label: "Series", Group: GroupComics, Kind: KindScalarFuzzy,
|
||||
Ops: []string{OpContains, OpNotContains, OpIs, OpIsNot, OpEmpty}, DefaultOp: OpContains,
|
||||
// Full series membership: the primary series column AND the
|
||||
// additional series held in alternate_info — a book whose
|
||||
// membership of X is secondary must not hide.
|
||||
Clause: clauseSeriesMembership, Suggest: suggestSeries},
|
||||
{Key: "series_number", Label: "Series number", Group: GroupComics, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.series_number", parseNumberFloat), Suggest: suggestNone},
|
||||
{Key: "series_count", Label: "Series count", Group: GroupComics, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.series_count", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "volume", Label: "Volume", Group: GroupComics, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseNumberFactory("mi.volume", parseNumberInt), Suggest: suggestNone},
|
||||
{Key: "format_group", Label: "Format group", Group: GroupComics, Kind: KindEnum,
|
||||
Ops: []string{OpIs, OpIsNot}, DefaultOp: OpIs,
|
||||
Clause: clauseEnumFactory("mi.format_group"), Suggest: suggestColumn},
|
||||
{Key: "mime_type", Label: "File type (mime)", Group: GroupComics, Kind: KindEnum,
|
||||
Ops: []string{OpIs, OpIsNot}, DefaultOp: OpIs,
|
||||
Clause: clauseEnumFactory("mi.mime_type"), Suggest: suggestColumn},
|
||||
{Key: "is_reflowable", Label: "Reflowable", Group: GroupComics, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
Clause: clauseBoolFactory("mi.is_reflowable"), Suggest: suggestNone},
|
||||
{Key: "has_fixed_layout", Label: "Fixed layout", Group: GroupComics, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
Clause: clauseBoolFactory("mi.has_fixed_layout"), Suggest: suggestNone},
|
||||
{Key: "is_black_and_white", Label: "Black & white", Group: GroupComics, Kind: KindBool,
|
||||
Ops: []string{OpIsTrue, OpIsFalse}, DefaultOp: OpIsTrue,
|
||||
Clause: clauseBoolFactory("mi.is_black_and_white"), Suggest: suggestNone},
|
||||
{Key: "manga_type", Label: "Manga type", Group: GroupComics, Kind: KindEnum,
|
||||
Ops: []string{OpIs, OpIsNot, OpEmpty}, DefaultOp: OpIs,
|
||||
Clause: clauseEnumFactory("mi.manga_type"), Suggest: suggestColumn},
|
||||
{Key: "reading_direction", Label: "Reading direction", Group: GroupComics, Kind: KindEnum,
|
||||
Ops: []string{OpIs, OpIsNot, OpEmpty}, DefaultOp: OpIs,
|
||||
Clause: clauseEnumFactory("mi.reading_direction"), Suggest: suggestColumn},
|
||||
|
||||
// ── My reading (user-relative — the VIEWER's rows) ────────────
|
||||
{Key: "status", Label: "Reading status", Group: GroupMyReading, Kind: KindStatus,
|
||||
Ops: []string{OpIs}, DefaultOp: OpIs,
|
||||
Clause: clauseStatus, Suggest: suggestNone},
|
||||
{Key: "progress_percent", Label: "Progress percent", Group: GroupMyReading, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost, OpEmpty}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseProgressPercent, Suggest: suggestNone},
|
||||
{Key: "last_read", Label: "Last read", Group: GroupMyReading, Kind: KindDate,
|
||||
Ops: []string{OpWithinDays, OpBetween, OpEmpty}, DefaultOp: OpWithinDays,
|
||||
Clause: clauseLastRead, Suggest: suggestNone},
|
||||
{Key: "my_rating", Label: "My rating", Group: GroupMyReading, Kind: KindNumber,
|
||||
Ops: []string{OpIs, OpAtLeast, OpAtMost, OpUnrated}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseMyRating, Suggest: suggestNone},
|
||||
{Key: "review", Label: "Has review", Group: GroupMyReading, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing}, DefaultOp: OpHas,
|
||||
Clause: clauseReviewPresence, Suggest: suggestNone},
|
||||
{Key: "notes", Label: "Notes", Group: GroupMyReading, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing, OpWithinDays}, DefaultOp: OpHas,
|
||||
Clause: clauseNotesPresence, Suggest: suggestNone},
|
||||
{Key: "highlights", Label: "Highlights", Group: GroupMyReading, Kind: KindPresence,
|
||||
Ops: []string{OpHas, OpMissing, OpWithinDays}, DefaultOp: OpHas,
|
||||
Clause: clauseHighlightsPresence, Suggest: suggestNone},
|
||||
{Key: "note_text", Label: "Note text", Group: GroupMyReading, Kind: KindNoteText,
|
||||
// "Contains" here is substring ILIKE, NOT the system-wide fuzzy:
|
||||
// your own words deserve exact-ish. A book matches when ANY of
|
||||
// the viewer's notes on it match (EXISTS + bound values).
|
||||
Ops: []string{OpContains, OpNotContains, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseNoteText, Suggest: suggestNoteText},
|
||||
{Key: "highlight_text", Label: "Highlight text", Group: GroupMyReading, Kind: KindNoteText,
|
||||
Ops: []string{OpContains, OpNotContains, OpEmpty}, DefaultOp: OpContains,
|
||||
Clause: clauseHighlightText, Suggest: suggestHighlight},
|
||||
{Key: "time_spent", Label: "Time spent (minutes)", Group: GroupMyReading, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost}, DefaultOp: OpAtLeast,
|
||||
Clause: clauseTimeSpent, Suggest: suggestNone},
|
||||
{Key: "pages_read", Label: "Pages read", Group: GroupMyReading, Kind: KindNumber,
|
||||
Ops: []string{OpBetween, OpAtLeast, OpAtMost}, DefaultOp: OpAtLeast,
|
||||
Clause: clausePagesRead, Suggest: suggestNone},
|
||||
{Key: "collection", Label: "In collection", Group: GroupMyReading, Kind: KindCollection,
|
||||
// Membership mirrors the collection page: excluded items and
|
||||
// deleted collections never count; playlists are collections.
|
||||
Ops: []string{OpIn, OpNotIn, OpInAny, OpInNone}, DefaultOp: OpIn,
|
||||
Clause: clauseCollection, Suggest: suggestCollection},
|
||||
|
||||
// ── Library (holding structure + family lens) ─────────────────
|
||||
{Key: "library", Label: "Library", Group: GroupLibrary, Kind: KindLibrary,
|
||||
// Values are library IDs; the menu only ever offers the viewer's
|
||||
// VISIBLE libraries (family lens — a hidden library can not even
|
||||
// be offered).
|
||||
Ops: []string{OpIs, OpIsNot}, DefaultOp: OpIs,
|
||||
Clause: clauseLibrary, Suggest: suggestLibrary},
|
||||
}
|
||||
|
||||
// FilterFieldByKey returns the registry entry for a wire key.
|
||||
func FilterFieldByKey(key string) (FilterField, bool) {
|
||||
for _, f := range filterFields {
|
||||
if f.Key == key {
|
||||
return f, true
|
||||
}
|
||||
}
|
||||
return FilterField{}, false
|
||||
}
|
||||
|
||||
// FilterFieldsByGroup returns the registry grouped for the field menu,
|
||||
// groups in display order.
|
||||
func FilterFieldsByGroup() []struct {
|
||||
Group string
|
||||
Fields []FilterField
|
||||
} {
|
||||
order := []string{GroupPeople, GroupPublish, GroupTopics, GroupBookFacts, GroupComics, GroupMyReading, GroupLibrary}
|
||||
out := make([]struct {
|
||||
Group string
|
||||
Fields []FilterField
|
||||
}, 0, len(order))
|
||||
for _, g := range order {
|
||||
group := struct {
|
||||
Group string
|
||||
Fields []FilterField
|
||||
}{Group: g}
|
||||
for _, f := range filterFields {
|
||||
if f.Group == g {
|
||||
group.Fields = append(group.Fields, f)
|
||||
}
|
||||
}
|
||||
out = append(out, group)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// FilterFieldKeys returns every registered key (the completeness check
|
||||
// for tests).
|
||||
func FilterFieldKeys() []string {
|
||||
keys := make([]string, 0, len(filterFields))
|
||||
for _, f := range filterFields {
|
||||
keys = append(keys, f.Key)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
|
||||
// fieldOpsAllowed reports whether op is allowed on the field.
|
||||
func fieldOpsAllowed(f FilterField, op string) bool {
|
||||
for _, o := range f.Ops {
|
||||
if o == op {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// maxFilterDepth bounds group nesting (BookOrbit allows 5; the UI
|
||||
// discourages depth but the model allows a little headroom).
|
||||
const maxFilterDepth = 8
|
||||
|
||||
// ParseFilterTree parses + validates the qf wire JSON. Validation is
|
||||
// the allow-list enforcement: unknown fields/operators and malformed
|
||||
// values are a parse error (→ 400), never a silently-ignored token.
|
||||
func ParseFilterTree(raw string) (*FilterTree, error) {
|
||||
var t FilterTree
|
||||
if err := json.Unmarshal([]byte(raw), &t); err != nil {
|
||||
return nil, fmt.Errorf("invalid qf JSON: %w", err)
|
||||
}
|
||||
if err := validateGroup(&FilterChild{Mode: t.Mode, Children: t.Children}, 1); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &t, nil
|
||||
}
|
||||
|
||||
func validateGroup(g *FilterChild, depth int) error {
|
||||
if depth > maxFilterDepth {
|
||||
return fmt.Errorf("filter group nesting deeper than %d", maxFilterDepth)
|
||||
}
|
||||
if g.Mode != "all" && g.Mode != "any" {
|
||||
return fmt.Errorf("filter group mode must be all|any, got %q", g.Mode)
|
||||
}
|
||||
if len(g.Children) == 0 {
|
||||
return fmt.Errorf("filter group has no children")
|
||||
}
|
||||
for _, c := range g.Children {
|
||||
if c.Field == "" && c.Mode != "" {
|
||||
if err := validateGroup(&c, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err := validateRule(c); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// numericParsers map each numeric field's wire values to the Go value
|
||||
// bound into SQL — validation and clause-building share one parser per
|
||||
// field, so a value that passes validation always binds.
|
||||
var numericParsers = map[string]func(string) (any, error){
|
||||
"published_year": parseNumberInt,
|
||||
"page_count": parseNumberInt,
|
||||
"chapter_count": parseNumberInt,
|
||||
"total_characters": parseNumberInt,
|
||||
"file_size": parseNumberInt,
|
||||
"series_count": parseNumberInt,
|
||||
"volume": parseNumberInt,
|
||||
"my_rating": parseNumberInt,
|
||||
"series_number": parseNumberFloat,
|
||||
"community_rating": parseNumberFloat,
|
||||
"progress_percent": parseNumberFloat,
|
||||
"time_spent": parseNumberFloat,
|
||||
"pages_read": parseNumberFloat,
|
||||
}
|
||||
|
||||
func validateRule(c FilterChild) error {
|
||||
f, ok := FilterFieldByKey(c.Field)
|
||||
if !ok {
|
||||
return fmt.Errorf("unknown filter field %q", c.Field)
|
||||
}
|
||||
if !fieldOpsAllowed(f, c.Op) {
|
||||
return fmt.Errorf("operator %q not allowed on field %q", c.Op, c.Field)
|
||||
}
|
||||
if opsNoValue[c.Op] {
|
||||
if len(c.Values) != 0 {
|
||||
return fmt.Errorf("operator %q takes no values", c.Op)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if len(c.Values) == 0 {
|
||||
return fmt.Errorf("operator %q on field %q needs a value", c.Op, c.Field)
|
||||
}
|
||||
if c.Op == OpBetween && len(c.Values) != 2 {
|
||||
return fmt.Errorf("between takes exactly two values")
|
||||
}
|
||||
// Numeric fields: values must parse (they bind as numbers).
|
||||
if parse, ok := numericParsers[f.Key]; ok {
|
||||
for _, v := range c.Values {
|
||||
if _, err := parse(v); err != nil {
|
||||
return fmt.Errorf("field %q needs numeric values, got %q", f.Key, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Date within_days: a positive integer day count.
|
||||
if c.Op == OpWithinDays {
|
||||
n, err := strconv.Atoi(c.Values[0])
|
||||
if err != nil || n <= 0 {
|
||||
return fmt.Errorf("within_days needs a positive day count, got %q", c.Values[0])
|
||||
}
|
||||
}
|
||||
// Static domains (status, file_availability).
|
||||
if domain, ok := fixedDomains[f.Key]; ok {
|
||||
for _, v := range c.Values {
|
||||
found := false
|
||||
for _, d := range domain {
|
||||
if d == v {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return fmt.Errorf("field %q value must be one of %v, got %q", f.Key, domain, v)
|
||||
}
|
||||
}
|
||||
}
|
||||
// UUID-valued fields (collection, library, added_by "is").
|
||||
switch f.Key {
|
||||
case "library", "collection":
|
||||
for _, v := range c.Values {
|
||||
if _, err := uuid.Parse(v); err != nil {
|
||||
return fmt.Errorf("field %q values must be UUIDs, got %q", f.Key, v)
|
||||
}
|
||||
}
|
||||
case "added_by":
|
||||
if c.Op == OpIs {
|
||||
if _, err := uuid.Parse(c.Values[0]); err != nil {
|
||||
return fmt.Errorf("added_by \"is\" needs a user UUID, got %q", c.Values[0])
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func parseNumberInt(v string) (any, error) {
|
||||
n, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func parseNumberFloat(v string) (any, error) {
|
||||
f, err := strconv.ParseFloat(v, 64)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
package services
|
||||
|
||||
// Filter registry unit tests (faceted plan Phase 9): every entry is
|
||||
// complete (kind, operators, default operator, clause, suggestion
|
||||
// source), the mandated surfaces are present (including the three
|
||||
// holding structures), and the qf parse is a strict allow-list.
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestFilterRegistryCompleteness(t *testing.T) {
|
||||
keys := FilterFieldKeys()
|
||||
require.NotEmpty(t, keys)
|
||||
|
||||
seen := map[string]bool{}
|
||||
for _, key := range keys {
|
||||
require.False(t, seen[key], "duplicate registry key %q", key)
|
||||
seen[key] = true
|
||||
|
||||
f, ok := FilterFieldByKey(key)
|
||||
require.True(t, ok)
|
||||
require.NotEmpty(t, f.Label, "%s: label", key)
|
||||
require.NotEmpty(t, f.Group, "%s: menu group", key)
|
||||
require.NotEmpty(t, f.Kind, "%s: kind", key)
|
||||
require.NotEmpty(t, f.Ops, "%s: operators", key)
|
||||
require.NotNil(t, f.Clause, "%s: match clause", key)
|
||||
require.Contains(t, f.Ops, f.DefaultOp, "%s: default operator must be allowed", key)
|
||||
}
|
||||
|
||||
// The mandated surfaces — every book attribute class, the user-
|
||||
// relative class, and the three HOLDING STRUCTURES (library,
|
||||
// collections, series — the registry is the completeness check).
|
||||
mandated := []string{
|
||||
// people / publishing / topics
|
||||
"author", "narrator", "contributor", "publisher", "imprint",
|
||||
"edition", "published_year", "genre", "tags", "story_arc", "age_rating",
|
||||
// reading surface (comics & manga)
|
||||
"format_group", "mime_type", "is_reflowable", "has_fixed_layout",
|
||||
"is_black_and_white", "manga_type", "reading_direction", "explicit", "abridged",
|
||||
// series facts + numbers + dates + text + presence + identifiers
|
||||
"series", "series_number", "series_count", "volume",
|
||||
"page_count", "chapter_count", "total_characters", "file_size",
|
||||
"community_rating", "added_at", "imported_at",
|
||||
"title", "subtitle", "description", "summary", "language",
|
||||
"web_url", "metadata_notes", "isbn", "asin", "cover",
|
||||
"file_availability", "added_by",
|
||||
// user-relative (everything that pertains to a book)
|
||||
"status", "progress_percent", "last_read", "my_rating", "review",
|
||||
"notes", "highlights", "note_text", "highlight_text", "time_spent",
|
||||
"pages_read", "collection",
|
||||
// holding structure
|
||||
"library",
|
||||
}
|
||||
for _, key := range mandated {
|
||||
assert.True(t, seen[key], "mandated filter field %q missing from the registry", key)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilterParseValidation(t *testing.T) {
|
||||
valid := func(raw string) {
|
||||
_, err := ParseFilterTree(raw)
|
||||
require.NoError(t, err, "expected %s to parse", raw)
|
||||
}
|
||||
invalid := func(raw, wantContain string) {
|
||||
_, err := ParseFilterTree(raw)
|
||||
require.Error(t, err, "expected %s to be rejected", raw)
|
||||
if wantContain != "" {
|
||||
require.Contains(t, err.Error(), wantContain)
|
||||
}
|
||||
}
|
||||
|
||||
valid(`{"mode":"all","children":[{"field":"author","op":"contains","values":["Voss"]}]}`)
|
||||
valid(`{"mode":"any","children":[
|
||||
{"field":"author","op":"contains","values":["Voss"]},
|
||||
{"mode":"all","children":[{"field":"genre","op":"is","values":["fantasy"]}]}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"page_count","op":"between","values":["10","20"]}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"cover","op":"has"}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"status","op":"is","values":["reading","finished"]}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"added_at","op":"within_days","values":["14"]}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"my_rating","op":"unrated"}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"collection","op":"in_any"}]}`)
|
||||
valid(`{"mode":"all","children":[{"field":"library","op":"is","values":["` +
|
||||
"12345678-1234-1234-1234-123456789012" + `"]}]}`)
|
||||
|
||||
invalid(`{not json`, "invalid qf JSON")
|
||||
invalid(`{"mode":"all","children":[]}`, "no children")
|
||||
invalid(`{"mode":"sometimes","children":[{"field":"author","op":"contains","values":["x"]}]}`, "mode must be all|any")
|
||||
invalid(`{"mode":"all","children":[{"field":"not_a_field","op":"is","values":["x"]}]}`, "unknown filter field")
|
||||
invalid(`{"mode":"all","children":[{"field":"author","op":"explodes","values":["x"]}]}`, "not allowed on field")
|
||||
invalid(`{"mode":"all","children":[{"field":"page_count","op":"contains","values":["1"]}]}`, "not allowed on field")
|
||||
invalid(`{"mode":"all","children":[{"field":"page_count","op":"at_least","values":["abc"]}]}`, "numeric")
|
||||
invalid(`{"mode":"all","children":[{"field":"page_count","op":"between","values":["1"]}]}`, "exactly two")
|
||||
invalid(`{"mode":"all","children":[{"field":"cover","op":"has","values":["x"]}]}`, "takes no values")
|
||||
invalid(`{"mode":"all","children":[{"field":"added_at","op":"within_days","values":["0"]}]}`, "positive day count")
|
||||
invalid(`{"mode":"all","children":[{"field":"status","op":"is","values":["platypus"]}]}`, "must be one of")
|
||||
invalid(`{"mode":"all","children":[{"field":"library","op":"is","values":["not-a-uuid"]}]}`, "must be UUIDs")
|
||||
invalid(`{"mode":"all","children":[{"field":"author","op":"contains"}]}`, "needs a value")
|
||||
}
|
||||
|
||||
func TestFilterParseDepthCap(t *testing.T) {
|
||||
// Nesting beyond maxFilterDepth is rejected.
|
||||
raw := `{"mode":"all","children":[`
|
||||
for i := 0; i < maxFilterDepth+2; i++ {
|
||||
raw += `{"mode":"all","children":[`
|
||||
}
|
||||
raw += `{"field":"author","op":"contains","values":["x"]}`
|
||||
for i := 0; i < maxFilterDepth+2; i++ {
|
||||
raw += `]}`
|
||||
}
|
||||
raw += `]}`
|
||||
_, err := ParseFilterTree(raw)
|
||||
require.Error(t, err)
|
||||
require.Contains(t, err.Error(), "nesting deeper")
|
||||
}
|
||||
|
||||
func TestFilterFieldsByGroupOrdering(t *testing.T) {
|
||||
groups := FilterFieldsByGroup()
|
||||
require.NotEmpty(t, groups)
|
||||
first := groups[0].Group
|
||||
if !strings.EqualFold(first, GroupPeople) {
|
||||
t.Fatalf("first menu group should be %q, got %q", GroupPeople, first)
|
||||
}
|
||||
for _, g := range groups {
|
||||
require.NotEmpty(t, g.Fields, "group %s must not be empty", g.Group)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package services
|
||||
|
||||
import "strings"
|
||||
|
||||
// Genre model v2 (faceted-navigation plan, Batch B): the `genre` column
|
||||
// holds ONLY top-level shelves; every more specific candidate is a tag
|
||||
// mapped to its parent shelf. User rule: *a subgenre is not a genre, it's
|
||||
// a tag*.
|
||||
//
|
||||
// Sources arrive as comma-blobs ("Epic, Action & Adventure, Military,
|
||||
// Sword & Sorcery" — one Audible genre string), media-format noise
|
||||
// ("Audiobook"), maximally generic tags ("Fiction"), or library-type
|
||||
// words ("Manga", "Comics" — ComicInfo files that describe the TYPE,
|
||||
// not a shelf). Deterministic ranked rule set — no network, no guessing:
|
||||
//
|
||||
// 1. split candidate strings on , ; /
|
||||
// 2. alias-map variants to canonical forms (shelf or subgenre)
|
||||
// 3. discard demoted tokens (formats/editions/type-words) — never win
|
||||
// 4. pick: first DIRECT shelf (source order) > first MAPPED subgenre
|
||||
// shelf > Fiction/Nonfiction floor > "" (unknown-only: tags carry it)
|
||||
// 5. every candidate (split parts included) still lands in tags
|
||||
//
|
||||
// "adventure" and "action & adventure" are BOTH shelves — provider data
|
||||
// (Audible/Calibre) uses both, and the user-verified traces pin them:
|
||||
// {Pirates} → adventure (mapped), Malazan blob → action & adventure
|
||||
// (direct beats mapped). Merging them is deferred aliasing work.
|
||||
var genreShelves = map[string]bool{
|
||||
// Fiction side
|
||||
"fiction": true, "action & adventure": true, "adventure": true,
|
||||
"fantasy": true, "science fiction": true, "horror": true,
|
||||
"mystery": true, "thriller": true, "romance": true,
|
||||
"historical fiction": true, "western": true, "war & military": true,
|
||||
"young adult": true, "middle grade": true, "children's": true,
|
||||
"classics": true, "humor": true, "poetry": true, "short stories": true,
|
||||
// Nonfiction side
|
||||
"nonfiction": true, "biography & memoir": true, "history": true,
|
||||
"science": true, "true crime": true, "self-help": true,
|
||||
"philosophy": true, "business": true,
|
||||
}
|
||||
|
||||
// genreFloors are shelves that only win when NO other shelf candidate
|
||||
// exists (direct or mapped) — maximally generic positions. {Fiction,
|
||||
// Epic Fantasy} → fantasy, {Fiction} alone → fiction.
|
||||
func isGenreFloor(lower string) bool {
|
||||
return lower == "fiction" || lower == "nonfiction"
|
||||
}
|
||||
|
||||
// genreSubgenreShelf maps subgenre candidates (lowercase) to their parent
|
||||
// shelf. Anything here is a TAG by definition; its shelf wins the genre
|
||||
// column only when no direct shelf hit exists.
|
||||
var genreSubgenreShelf = map[string]string{
|
||||
// Fantasy subgenres
|
||||
"epic fantasy": "fantasy", "sword & sorcery": "fantasy",
|
||||
"dark fantasy": "fantasy", "urban fantasy": "fantasy",
|
||||
"paranormal": "fantasy", "high fantasy": "fantasy",
|
||||
"fairy tales": "fantasy", "mythology": "fantasy",
|
||||
"folklore": "fantasy", "litrpg": "fantasy",
|
||||
"progression fantasy": "fantasy",
|
||||
// Science fiction subgenres
|
||||
"space opera": "science fiction", "cyberpunk": "science fiction",
|
||||
"military science fiction": "science fiction", "dystopian": "science fiction",
|
||||
"post-apocalyptic": "science fiction", "apocalyptic": "science fiction",
|
||||
"time travel": "science fiction", "alternate history": "science fiction",
|
||||
"first contact": "science fiction", "steampunk": "science fiction",
|
||||
// Romance subgenres
|
||||
"contemporary romance": "romance", "historical romance": "romance",
|
||||
"paranormal romance": "romance",
|
||||
// Mystery subgenres
|
||||
"cozy mystery": "mystery", "crime": "mystery",
|
||||
"police procedural": "mystery", "noir": "mystery", "hardboiled": "mystery",
|
||||
// Thriller subgenres
|
||||
"suspense": "thriller", "espionage": "thriller",
|
||||
// Horror subgenres
|
||||
"gothic": "horror", "supernatural": "horror",
|
||||
// Fiction-side themes
|
||||
"literary fiction": "fiction", "coming of age": "fiction",
|
||||
"satire": "fiction", "anthology": "fiction", "alternate world": "fiction",
|
||||
// War & Military subgenres
|
||||
"military": "war & military", "war": "war & military",
|
||||
// Adventure subgenres
|
||||
"pirates": "adventure",
|
||||
// Biography & Memoir subgenres
|
||||
"biography": "biography & memoir", "autobiography": "biography & memoir",
|
||||
"memoir": "biography & memoir",
|
||||
// Remaining nonfiction topics → the nonfiction umbrella shelf
|
||||
"psychology": "nonfiction", "economics": "nonfiction",
|
||||
"politics": "nonfiction", "religion": "nonfiction",
|
||||
"spirituality": "nonfiction", "travel": "nonfiction",
|
||||
"cooking": "nonfiction", "food": "nonfiction",
|
||||
"health": "nonfiction", "fitness": "nonfiction",
|
||||
"sports": "nonfiction", "music": "nonfiction",
|
||||
"art": "nonfiction", "photography": "nonfiction",
|
||||
"education": "nonfiction", "reference": "nonfiction",
|
||||
"legal": "nonfiction", "medical": "nonfiction",
|
||||
}
|
||||
|
||||
// genreAliases maps provider variants to a canonical shelf OR subgenre
|
||||
// form (which the maps above then resolve).
|
||||
var genreAliases = map[string]string{
|
||||
"sci-fi": "science fiction", "scifi": "science fiction",
|
||||
"sf": "science fiction", "sff": "science fiction",
|
||||
"sci fi": "science fiction", "sci_fi": "science fiction",
|
||||
"speculative fiction": "science fiction",
|
||||
"detectives": "mystery", "detective": "mystery",
|
||||
"whodunnit": "mystery", "whodunit": "mystery",
|
||||
"adventure fiction": "action & adventure",
|
||||
"action and adventure": "action & adventure",
|
||||
"ya": "young adult", "mg": "middle grade",
|
||||
"coming-of-age": "coming of age",
|
||||
"histfiction": "historical fiction",
|
||||
"epic": "epic fantasy",
|
||||
"military scifi": "military science fiction",
|
||||
"military sf": "military science fiction",
|
||||
"post apocalyptic": "post-apocalyptic",
|
||||
"self help": "self-help",
|
||||
}
|
||||
|
||||
// genreDemoted never wins the genre column — media formats, editions,
|
||||
// and library-TYPE words (ComicInfo genre="Manga" describes the type,
|
||||
// not a shelf; shelves apply uniformly across all library types). They
|
||||
// still pass through to tags.
|
||||
var genreDemoted = map[string]bool{
|
||||
"audiobook": true, "unabridged": true, "abridged": true,
|
||||
"dramatization": true, "dramatized": true, "audio drama": true,
|
||||
"manga": true, "comics": true, "graphic novel": true,
|
||||
"graphic novels": true,
|
||||
}
|
||||
|
||||
// splitGenreCandidates splits compound genre strings ("Epic, Action &
|
||||
// Adventure" — one Audible blob) into individual candidates.
|
||||
func splitGenreCandidates(genres []string) []string {
|
||||
var out []string
|
||||
for _, g := range genres {
|
||||
for _, part := range strings.Split(g, ",") {
|
||||
for _, p := range strings.Split(part, ";") {
|
||||
for _, piece := range strings.Split(p, "/") {
|
||||
t := strings.TrimSpace(piece)
|
||||
if t != "" {
|
||||
out = append(out, t)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// normalizeGenreCandidate lowercases, trims, and alias-resolves one
|
||||
// candidate; "" means noise (empty input).
|
||||
func normalizeGenreCandidate(c string) string {
|
||||
lower := strings.ToLower(strings.TrimSpace(c))
|
||||
if lower == "" {
|
||||
return ""
|
||||
}
|
||||
if a, ok := genreAliases[lower]; ok {
|
||||
return a
|
||||
}
|
||||
return lower
|
||||
}
|
||||
|
||||
// pickSmartGenre chooses the SHELF for the genre column from candidate
|
||||
// strings (genre model v2). Returns the winner ("" when no shelf is
|
||||
// known — unknown-only input stays tag-only) and the full candidate
|
||||
// list for the tags array.
|
||||
func pickSmartGenre(genres []string) (genre string, all []string) {
|
||||
all = splitGenreCandidates(genres)
|
||||
direct := "" // first non-floor shelf, source order
|
||||
mapped := "" // first subgenre-resolved shelf, source order
|
||||
floor := "" // fiction / nonfiction, source order
|
||||
for _, c := range all {
|
||||
lower := normalizeGenreCandidate(c)
|
||||
if lower == "" || genreDemoted[lower] {
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case genreShelves[lower]:
|
||||
if isGenreFloor(lower) {
|
||||
if floor == "" {
|
||||
floor = lower
|
||||
}
|
||||
} else if direct == "" {
|
||||
direct = lower
|
||||
}
|
||||
case genreSubgenreShelf[lower] != "":
|
||||
if mapped == "" {
|
||||
mapped = genreSubgenreShelf[lower]
|
||||
}
|
||||
}
|
||||
// Unknown candidates are tags-only — never guess a shelf.
|
||||
}
|
||||
if direct != "" {
|
||||
return direct, all
|
||||
}
|
||||
if mapped != "" {
|
||||
return mapped, all
|
||||
}
|
||||
return floor, all
|
||||
}
|
||||
@@ -38,7 +38,11 @@ var AllowedExtensions = map[string][]string{
|
||||
// All four decided breadth formats registered from day one (the
|
||||
// DOC/LIT doctrine: extensions are a deliberate list); only m4b
|
||||
// scans richly in Batch A — the chapter abstraction carries the rest.
|
||||
LibraryTypeAudiobooks: {".m4b", ".m4a", ".mp3", ".flac"},
|
||||
// .zip joins as the zip-of-audio book container (multi-file ingest);
|
||||
// comics libraries keep using .cbz so the two never collide. The
|
||||
// browser-playable lossless/next-gen containers join for Batch B
|
||||
// metadata parity (.wma deliberately absent: browsers can't play it).
|
||||
LibraryTypeAudiobooks: {".m4b", ".m4a", ".mp3", ".flac", ".zip", ".opus", ".ogg", ".oga", ".aac", ".wav"},
|
||||
}
|
||||
|
||||
var MimeTypes = map[string]string{
|
||||
@@ -74,6 +78,15 @@ var MimeTypes = map[string]string{
|
||||
".m4a": "audio/mp4",
|
||||
".mp3": "audio/mpeg",
|
||||
".flac": "audio/flac",
|
||||
// Opus is registered as audio/ogg per RFC 7845 (Ogg encapsulation);
|
||||
// every browser <audio> element accepts it that way.
|
||||
".opus": "audio/ogg",
|
||||
".ogg": "audio/ogg",
|
||||
".oga": "audio/ogg",
|
||||
".aac": "audio/aac",
|
||||
".wav": "audio/wav",
|
||||
// Zip-of-audio book container (audiobooks libraries only).
|
||||
".zip": "application/zip",
|
||||
}
|
||||
|
||||
// GetLibraryTypes retrieves all available library types
|
||||
|
||||
+1128
-186
File diff suppressed because it is too large.
Load diff
@@ -9,10 +9,17 @@ package services
|
||||
// until a rescan runs with ffprobe present).
|
||||
//
|
||||
// The chapters array written into media_items.audio_metadata IS the
|
||||
// format abstraction: embedded m4b chapters and (later) folder-as-book
|
||||
// files both produce {index, title, start, end} entries, so the locator
|
||||
// {chapter, offset_seconds} and the /chapters endpoint never learn which
|
||||
// file model produced them.
|
||||
// format abstraction: embedded m4b chapters, folder-as-book files and
|
||||
// zip-of-audio books all produce {index, title, start, end} entries, so
|
||||
// the locator {chapter, offset_seconds} and the /chapters endpoint never
|
||||
// learn which file model produced them.
|
||||
//
|
||||
// Multi-file books (layout "folder" | "zip", see
|
||||
// media_scanner_audio_multifile.go) additionally carry a tracks array:
|
||||
// one entry per playable file, with its global-seconds start/end and the
|
||||
// path the /audio/:trackIndex streaming endpoint resolves. Single-file
|
||||
// books leave layout and tracks empty — the <audio> element streams the
|
||||
// whole file exactly as before.
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
@@ -33,6 +40,21 @@ type AudioChapter struct {
|
||||
End float64 `json:"end"`
|
||||
}
|
||||
|
||||
// AudioTrack is one playable file of a multi-file audiobook. Path is the
|
||||
// library-relative file path (layout "folder") or the in-zip entry name
|
||||
// (layout "zip"); the /audio/:trackIndex endpoint resolves it to bytes.
|
||||
// Start/End are cumulative global seconds across the book — the same
|
||||
// coordinate system chapters, bookmarks and audio_offset_seconds use.
|
||||
type AudioTrack struct {
|
||||
Index int `json:"index"`
|
||||
Path string `json:"path"`
|
||||
Title string `json:"title,omitempty"`
|
||||
Start float64 `json:"start"`
|
||||
End float64 `json:"end"`
|
||||
DurationSeconds float64 `json:"duration_seconds,omitempty"`
|
||||
SizeBytes int64 `json:"size_bytes,omitempty"`
|
||||
}
|
||||
|
||||
// AudioMetadataPayload is the JSONB stored in media_items.audio_metadata.
|
||||
// Only the audio paths read it — ebook paths never do (the kobo_metadata
|
||||
// precedent).
|
||||
@@ -44,16 +66,33 @@ type AudioMetadataPayload struct {
|
||||
ASIN string `json:"asin,omitempty"`
|
||||
Abridged *bool `json:"abridged,omitempty"`
|
||||
Chapters []AudioChapter `json:"chapters,omitempty"`
|
||||
// Layout: "" = single file (the Batch A model), "folder" = a
|
||||
// directory of track files, "zip" = a zip archive of tracks.
|
||||
Layout string `json:"layout,omitempty"`
|
||||
// Tracks: one entry per playable file for multi-file layouts. Empty
|
||||
// for single-file books (the whole file IS the track).
|
||||
Tracks []AudioTrack `json:"tracks,omitempty"`
|
||||
}
|
||||
|
||||
func isAudioFile(path string) bool {
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".m4b", ".m4a", ".mp3", ".flac":
|
||||
case ".m4b", ".m4a", ".mp3", ".flac", ".zip", ".opus", ".ogg", ".oga", ".aac", ".wav":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// isAudioTrackFile reports whether a path is a PLAYABLE track of a
|
||||
// multi-file book. Zips are book CONTAINERS (layout "zip"), never tracks
|
||||
// of a folder book — a nested zip inside a book root must not join the
|
||||
// track list, and pre-grouping one-track strays were never zips.
|
||||
func isAudioTrackFile(path string) bool {
|
||||
if strings.EqualFold(filepath.Ext(path), ".zip") {
|
||||
return false
|
||||
}
|
||||
return isAudioFile(path)
|
||||
}
|
||||
|
||||
// ffprobeAvailable is resolved once per scan process; a missing binary is
|
||||
// a degradation, not an error.
|
||||
var ffprobeAvailable = func() bool {
|
||||
@@ -61,10 +100,55 @@ var ffprobeAvailable = func() bool {
|
||||
return err == nil
|
||||
}()
|
||||
|
||||
// ffmpegAvailable mirrors ffprobeAvailable for the embedded-cover
|
||||
// extractor (ffprobe cannot write image streams; ffmpeg can).
|
||||
var ffmpegAvailable = func() bool {
|
||||
_, err := exec.LookPath("ffmpeg")
|
||||
return err == nil
|
||||
}()
|
||||
|
||||
// extractEmbeddedAudioCover pulls the embedded cover art (mp3 APIC /
|
||||
// m4b covr / flac picture) out of an audio file with one ffmpeg call —
|
||||
// the attached picture presents itself as the file's only video stream.
|
||||
// Writes outputBase+".cover.jpg" following the EPUB sidecar convention
|
||||
// ("<book>.cover.jpg"). Returns "" when ffmpeg is absent, the file
|
||||
// carries no picture, or anything fails — purely optional enrichment.
|
||||
func extractEmbeddedAudioCover(inputPath, outputBase string) string {
|
||||
if !ffmpegAvailable {
|
||||
return ""
|
||||
}
|
||||
outPath := outputBase + ".cover.jpg"
|
||||
cmd := exec.Command("ffmpeg",
|
||||
"-y", "-v", "error",
|
||||
"-i", inputPath,
|
||||
"-map", "0:v:0",
|
||||
"-frames:v", "1",
|
||||
outPath,
|
||||
)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
// "matches no streams" is the no-embedded-art case; stay quiet
|
||||
// about it, log anything unexpected.
|
||||
if len(out) > 0 && !strings.Contains(string(out), "matches no streams") {
|
||||
fmt.Printf("ffmpeg cover extraction note for %s: %s\n", inputPath, strings.TrimSpace(string(out)))
|
||||
}
|
||||
return ""
|
||||
}
|
||||
if info, err := os.Stat(outPath); err != nil || info.Size() == 0 {
|
||||
_ = os.Remove(outPath)
|
||||
return ""
|
||||
}
|
||||
return outPath
|
||||
}
|
||||
|
||||
type ffprobeReport struct {
|
||||
Format struct {
|
||||
Duration flexFloat `json:"duration"`
|
||||
BitRate flexFloat `json:"bit_rate"`
|
||||
// Tags: ffprobe normalizes well-known atoms to lowercase keys but
|
||||
// passes freeform keys (Audible's `----:com.apple.iTunes:*`)
|
||||
// through verbatim — a map with case-insensitive lookup is the
|
||||
// only honest shape. See tag().
|
||||
Tags map[string]string `json:"tags"`
|
||||
} `json:"format"`
|
||||
Streams []struct {
|
||||
CodecName string `json:"codec_name"`
|
||||
@@ -134,6 +218,24 @@ func runFFProbe(path string) *ffprobeReport {
|
||||
return &report
|
||||
}
|
||||
|
||||
// tag returns the first non-empty ffprobe tag among the candidate keys,
|
||||
// matched case-insensitively and trimmed. Empty string when none match.
|
||||
func (r *ffprobeReport) tag(keys ...string) string {
|
||||
if r == nil {
|
||||
return ""
|
||||
}
|
||||
for _, key := range keys {
|
||||
for k, v := range r.Format.Tags {
|
||||
if strings.EqualFold(k, key) {
|
||||
if trimmed := strings.TrimSpace(v); trimmed != "" {
|
||||
return trimmed
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// enrichAudioMetadata merges ffprobe facts into the (possibly sidecar-
|
||||
// seeded) audio payload, creating it when no sidecar existed. Sidecar
|
||||
// narrators/chapters win (curated); ffprobe fills duration, codec,
|
||||
@@ -192,6 +294,24 @@ func (s *MediaScanner) enrichAudioMetadata(path string, metadata *MediaMetadata)
|
||||
if payload.Abridged != nil {
|
||||
metadata.Abridged = *payload.Abridged
|
||||
}
|
||||
|
||||
// Tag fallbacks run LAST in the single-file chain (sidecars → folder
|
||||
// structure handled by the caller → tags → filename): they only fill
|
||||
// fields the earlier sources left empty.
|
||||
applyAudioTagFallbacks(metadata, report)
|
||||
|
||||
// CUE sheets are the last chapter source: sidecar chapters and
|
||||
// embedded container chapters (m4b) outrank them.
|
||||
if len(payload.Chapters) == 0 {
|
||||
if cueChapters := readCueChapters(path); len(cueChapters) > 0 {
|
||||
payload.Chapters = cueChapters
|
||||
if payload.DurationSeconds == 0 {
|
||||
if last := cueChapters[len(cueChapters)-1]; last.End > 0 {
|
||||
payload.DurationSeconds = last.End
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parseSidecarAudioFields extracts the audio-only fields from a
|
||||
@@ -261,6 +381,101 @@ func marshalAudioMetadata(payload *AudioMetadataPayload) []byte {
|
||||
return out
|
||||
}
|
||||
|
||||
// applyAudioTagFallbacks fills EMPTY media metadata from the file's own
|
||||
// tags — the last rung of the ingestion precedence chain (curated
|
||||
// sidecars → folder structure → embedded tags → filename). Field map:
|
||||
//
|
||||
// album / album_artist / artist → title / author
|
||||
// composer → narrators (Audible's own
|
||||
// convention: the narrator lives in the composer tag)
|
||||
// grouping → series
|
||||
// ----:com.apple.iTunes:SERIES[-PART] → series + index (Libation
|
||||
// preserves these Audible freeform atoms)
|
||||
// ----:com.apple.iTunes:SUBTITLE → joined into the title
|
||||
// ("Title: Subtitle"), mirroring the ABS sidecar path
|
||||
// ----:com.apple.iTunes:ASIN / ABRIDGED → asin / abridged flag
|
||||
// date → publish date; genre → genre
|
||||
//
|
||||
// Abridged is the one field where an explicit sidecar false must not be
|
||||
// overridden by a tag — the payload pointer marks sidecar presence.
|
||||
func applyAudioTagFallbacks(metadata *MediaMetadata, report *ffprobeReport) {
|
||||
if metadata == nil || report == nil {
|
||||
return
|
||||
}
|
||||
|
||||
if metadata.Title == "" {
|
||||
if album := report.tag("album"); album != "" {
|
||||
metadata.Title = album
|
||||
}
|
||||
}
|
||||
// Subtitle is a first-class column (A2b) — no more Calibre-style title
|
||||
// joining; the tool-provided subtitle lands in its own column.
|
||||
if subtitle := report.tag("----:com.apple.itunes:subtitle", "subtitle"); subtitle != "" {
|
||||
metadata.Subtitle = subtitle
|
||||
}
|
||||
if metadata.Author == "" {
|
||||
if artist := report.tag("album_artist"); artist != "" {
|
||||
metadata.Author = artist
|
||||
} else if artist := report.tag("artist"); artist != "" {
|
||||
metadata.Author = artist
|
||||
}
|
||||
}
|
||||
if len(metadata.Narrators) == 0 {
|
||||
// Kept as ONE narrator string: composer values have no reliable
|
||||
// multi-narrator delimiter, and splitting "Vance, Simon" would
|
||||
// manufacture two fake people. Admins can edit the column.
|
||||
if composer := report.tag("composer"); composer != "" {
|
||||
metadata.Narrators = []string{composer}
|
||||
if metadata.AudioMeta != nil && len(metadata.AudioMeta.Narrators) == 0 {
|
||||
metadata.AudioMeta.Narrators = []string{composer}
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.Series == "" {
|
||||
if series := report.tag("grouping", "----:com.apple.itunes:series"); series != "" {
|
||||
metadata.Series = series
|
||||
if part := report.tag("----:com.apple.itunes:series-part"); part != "" {
|
||||
if idx, err := strconv.ParseFloat(strings.TrimSpace(part), 64); err == nil {
|
||||
metadata.SeriesNumber = idx
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.ASIN == "" {
|
||||
if asin := report.tag("----:com.apple.itunes:asin", "asin"); asin != "" {
|
||||
metadata.ASIN = asin
|
||||
if metadata.AudioMeta != nil && metadata.AudioMeta.ASIN == "" {
|
||||
metadata.AudioMeta.ASIN = asin
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.AudioMeta == nil || metadata.AudioMeta.Abridged == nil {
|
||||
if !metadata.Abridged {
|
||||
switch strings.ToLower(report.tag("----:com.apple.itunes:abridged")) {
|
||||
case "1", "true", "yes":
|
||||
metadata.Abridged = true
|
||||
if metadata.AudioMeta != nil {
|
||||
abridged := true
|
||||
metadata.AudioMeta.Abridged = &abridged
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.PublishDate.IsZero() {
|
||||
if date := report.tag("date"); date != "" {
|
||||
for _, layout := range []string{"2006-01-02", "2006-01", "2006"} {
|
||||
if t, err := time.Parse(layout, strings.TrimSpace(date)); err == nil {
|
||||
metadata.PublishDate = t
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if metadata.Genre == "" {
|
||||
metadata.Genre = report.tag("genre")
|
||||
}
|
||||
}
|
||||
|
||||
// audioDurationLabel renders the duration for logs/detail surfaces.
|
||||
func audioDurationLabel(seconds float64) string {
|
||||
if seconds <= 0 {
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
package services
|
||||
|
||||
// CUE sheet support: a `.cue` sidecar describing the chapter marks of a
|
||||
// single big audio file (the classic CD-rip shape: one flac/mp3 + one
|
||||
// cue). CUE is the LOWEST-priority chapter source — curated sidecar
|
||||
// chapters and embedded m4b chapters outrank it — and a purely optional
|
||||
// one: no cue, no chapters, no error.
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// cueFramesPerSecond is the CD framing CUE INDEX timestamps use
|
||||
// (MM:SS:FF with 75 frames per second).
|
||||
const cueFramesPerSecond = 75.0
|
||||
|
||||
// cueTimeToSeconds converts a CUE INDEX timestamp ("03:25:12") to
|
||||
// seconds. Minutes may exceed 59 and hours are folded into the minutes
|
||||
// field per the spec (MM:SS:FF, MM unbounded).
|
||||
func cueTimeToSeconds(v string) float64 {
|
||||
parts := strings.Split(strings.TrimSpace(v), ":")
|
||||
if len(parts) != 3 {
|
||||
return 0
|
||||
}
|
||||
minutes, err1 := strconv.ParseFloat(strings.TrimSpace(parts[0]), 64)
|
||||
seconds, err2 := strconv.ParseFloat(strings.TrimSpace(parts[1]), 64)
|
||||
frames, err3 := strconv.ParseFloat(strings.TrimSpace(parts[2]), 64)
|
||||
if err1 != nil || err2 != nil || err3 != nil {
|
||||
return 0
|
||||
}
|
||||
return minutes*60 + seconds + frames/cueFramesPerSecond
|
||||
}
|
||||
|
||||
// readCueChapters finds a CUE sheet describing the given audio file and
|
||||
// returns its tracks as chapters (global seconds, ends derived from the
|
||||
// next start; the last end stays 0 = unknown). A cue matches when its
|
||||
// FILE directive names the audio file (basename, case-insensitive), or
|
||||
// — for single-cue directories — when the directory holds exactly one.
|
||||
func readCueChapters(path string) []AudioChapter {
|
||||
dir := filepath.Dir(path)
|
||||
base := strings.ToLower(filepath.Base(path))
|
||||
|
||||
cuePaths, err := filepath.Glob(filepath.Join(dir, "*.cue"))
|
||||
if err != nil || len(cuePaths) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
var matched string
|
||||
for _, cuePath := range cuePaths {
|
||||
if referencedByCue(cuePath, base) {
|
||||
matched = cuePath
|
||||
break
|
||||
}
|
||||
}
|
||||
if matched == "" && len(cuePaths) == 1 {
|
||||
// Single-cue directory: the cue almost certainly describes the
|
||||
// only audio file next to it.
|
||||
matched = cuePaths[0]
|
||||
}
|
||||
if matched == "" {
|
||||
return nil
|
||||
}
|
||||
return parseCueSheet(matched, base)
|
||||
}
|
||||
|
||||
// referencedByCue reports whether the cue's FILE directive names the
|
||||
// given audio basename.
|
||||
func referencedByCue(cuePath, audioBase string) bool {
|
||||
f, err := os.Open(cuePath)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer func() { _ = f.Close() }()
|
||||
|
||||
scanner := bufio.NewScanner(f)
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if len(line) >= 4 && strings.EqualFold(line[:4], "FILE") {
|
||||
rest := strings.TrimSpace(line[4:])
|
||||
rest = strings.Trim(rest, "\"")
|
||||
// FILE "name" WAVE / FILE name MP3 — drop the type word.
|
||||
if idx := strings.LastIndexAny(rest, " \t"); idx != -1 {
|
||||
rest = strings.TrimSpace(rest[:idx])
|
||||
}
|
||||
rest = strings.Trim(rest, "\"")
|
||||
return strings.EqualFold(filepath.Base(rest), audioBase)
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// parseCueSheet extracts TRACK/TITLE/INDEX 01 entries as chapters.
|
||||
// INDEX timestamps are absolute from the file start per the spec; the
|
||||
// pregap (INDEX 00) is ignored. Multi-file cues contribute only the
|
||||
// TRACK blocks under the FILE directive naming our audio file (or all
|
||||
// blocks when the cue declares no FILE at all).
|
||||
func parseCueSheet(cuePath, audioBase string) []AudioChapter {
|
||||
f, err := os.Open(cuePath)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
defer func() { _ = f.Close() }()
|
||||
|
||||
var chapters []AudioChapter
|
||||
var current *AudioChapter
|
||||
title := ""
|
||||
inOurFile := true
|
||||
|
||||
flush := func() {
|
||||
if current != nil && current.Start >= 0 {
|
||||
current.Title = strings.TrimSpace(title)
|
||||
if current.Title == "" {
|
||||
current.Title = "Chapter " + strconv.Itoa(current.Index+1)
|
||||
}
|
||||
if n := len(chapters); n > 0 && current.Start >= chapters[n-1].Start {
|
||||
chapters[n-1].End = current.Start
|
||||
}
|
||||
chapters = append(chapters, *current)
|
||||
}
|
||||
current = nil
|
||||
title = ""
|
||||
}
|
||||
|
||||
scanner := bufio.NewScanner(f)
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
upper := strings.ToUpper(line)
|
||||
switch {
|
||||
case strings.HasPrefix(upper, "FILE"):
|
||||
flush()
|
||||
rest := strings.Trim(strings.TrimSpace(line[4:]), "\"")
|
||||
if idx := strings.LastIndexAny(rest, " \t"); idx != -1 {
|
||||
rest = strings.TrimSpace(rest[:idx])
|
||||
}
|
||||
rest = strings.Trim(rest, "\"")
|
||||
inOurFile = strings.EqualFold(filepath.Base(rest), audioBase)
|
||||
case strings.HasPrefix(upper, "TRACK"):
|
||||
flush()
|
||||
if !inOurFile {
|
||||
continue
|
||||
}
|
||||
current = &AudioChapter{Index: len(chapters)}
|
||||
if fields := strings.Fields(line); len(fields) >= 2 {
|
||||
if n, err := strconv.Atoi(fields[1]); err == nil && n > 0 {
|
||||
current.Index = n - 1
|
||||
}
|
||||
}
|
||||
case strings.HasPrefix(upper, "TITLE") && current != nil:
|
||||
if q1 := strings.Index(line, "\""); q1 != -1 {
|
||||
if q2 := strings.LastIndex(line, "\""); q2 > q1 {
|
||||
title = line[q1+1 : q2]
|
||||
}
|
||||
}
|
||||
case strings.HasPrefix(upper, "INDEX"):
|
||||
if current == nil || !inOurFile {
|
||||
continue
|
||||
}
|
||||
if fields := strings.Fields(line); len(fields) >= 3 && fields[1] == "01" {
|
||||
current.Start = cueTimeToSeconds(fields[2])
|
||||
}
|
||||
}
|
||||
}
|
||||
flush()
|
||||
|
||||
if len(chapters) == 0 {
|
||||
return nil
|
||||
}
|
||||
return chapters
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestAudioPayloadIsEmpty pins the write-after-success guard on
|
||||
// updateAudioBookItem (2026-10-21 incident: multi-file audiobooks
|
||||
// surfaced as 1 second long because a failed re-extraction overwrote a
|
||||
// populated audio_metadata). An empty payload — nil, or no tracks, no
|
||||
// chapters, no duration — is the signature of a failed extraction and
|
||||
// must never replace existing audio data.
|
||||
func TestAudioPayloadIsEmpty(t *testing.T) {
|
||||
require := func(cond bool, msg string) {
|
||||
if !cond {
|
||||
t.Fatal(msg)
|
||||
}
|
||||
}
|
||||
|
||||
require(audioPayloadIsEmpty(nil), "nil payload must read as empty")
|
||||
require(audioPayloadIsEmpty(&AudioMetadataPayload{}), "zero-value payload must read as empty")
|
||||
require(audioPayloadIsEmpty(&AudioMetadataPayload{DurationSeconds: 0}), "zero-duration payload must read as empty")
|
||||
|
||||
// A real book always carries at least one of the audio facts.
|
||||
require(!audioPayloadIsEmpty(&AudioMetadataPayload{DurationSeconds: 302.6}),
|
||||
"duration-only payload (single-track book) is not empty")
|
||||
require(!audioPayloadIsEmpty(&AudioMetadataPayload{
|
||||
Tracks: []AudioTrack{{Index: 0, Path: "01.mp3"}},
|
||||
}), "track manifest is not empty")
|
||||
require(!audioPayloadIsEmpty(&AudioMetadataPayload{
|
||||
Chapters: []AudioChapter{{Index: 0, Title: "Chapter 1"}},
|
||||
}), "chapters are not empty")
|
||||
}
|
||||
@@ -0,0 +1,281 @@
|
||||
package services
|
||||
|
||||
// Unit tests for the exhaustive audiobook metadata ingestion: CUE
|
||||
// sheets, m3u playlists, tag fallbacks (incl. Audible freeform atoms)
|
||||
// and tag-based track ordering.
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCueTimeToSeconds(t *testing.T) {
|
||||
cases := map[string]float64{
|
||||
"00:00:00": 0,
|
||||
"01:00:00": 60,
|
||||
"03:25:12": 3*60 + 25 + 12.0/75.0,
|
||||
"75:00:37": 75*60 + 37.0/75.0, // minutes exceed 59 per spec
|
||||
"not-a-time": 0,
|
||||
"1:2": 0,
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := cueTimeToSeconds(in); (got-want) > 0.001 || (want-got) > 0.001 {
|
||||
t.Errorf("cueTimeToSeconds(%q) = %v, want %v", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseCueSheet(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cuePath := filepath.Join(dir, "book.cue")
|
||||
cue := `TITLE "Cue Book"
|
||||
FILE "book.flac" WAVE
|
||||
TRACK 01 AUDIO
|
||||
TITLE "Opening"
|
||||
INDEX 01 00:00:00
|
||||
TRACK 02 AUDIO
|
||||
TITLE "The Middle Bit"
|
||||
INDEX 01 00:01:30
|
||||
TRACK 03 AUDIO
|
||||
TITLE "Closing"
|
||||
INDEX 01 00:03:00
|
||||
FILE "other.flac" WAVE
|
||||
TRACK 01 AUDIO
|
||||
TITLE "Foreign Track"
|
||||
INDEX 01 00:00:00
|
||||
`
|
||||
if err := os.WriteFile(cuePath, []byte(cue), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
chapters := parseCueSheet(cuePath, "book.flac")
|
||||
if len(chapters) != 3 {
|
||||
t.Fatalf("chapters = %d (%+v), want 3 (foreign FILE block excluded)", len(chapters), chapters)
|
||||
}
|
||||
if chapters[0].Title != "Opening" || chapters[0].Start != 0 {
|
||||
t.Fatalf("first = %+v", chapters[0])
|
||||
}
|
||||
// MM:SS:FF: "00:01:30" = 1 second + 30/75 frames = 1.4s
|
||||
if chapters[1].Start != 1.4 || chapters[1].End != 3.0 {
|
||||
t.Fatalf("second = %+v", chapters[1])
|
||||
}
|
||||
if chapters[2].End != 0 {
|
||||
t.Fatalf("last end must stay 0 (unknown): %+v", chapters[2])
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadCueChaptersMatched(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
cue := "FILE \"audio.mp3\" WAVE\n TRACK 01 AUDIO\n TITLE \"One\"\n INDEX 01 00:00:00\n TRACK 02 AUDIO\n TITLE \"Two\"\n INDEX 01 00:00:10\n"
|
||||
if err := os.WriteFile(filepath.Join(dir, "audio.cue"), []byte(cue), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
chapters := readCueChapters(filepath.Join(dir, "audio.mp3"))
|
||||
if len(chapters) != 2 {
|
||||
t.Fatalf("chapters = %d", len(chapters))
|
||||
}
|
||||
|
||||
// No cue at all → nil, never an error.
|
||||
if chapters := readCueChapters(filepath.Join(dir, "other.mp3")); chapters != nil {
|
||||
t.Fatalf("expected nil for missing cue, got %+v", chapters)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFindAndApplyPlaylistHint(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
playlist := "#EXTM3U\n#EXTINF:120,Chapter Zero\n02_Second.mp3\n#EXTINF:60,Chapter One\n./01_First.mp3\nmissing.mp3\n"
|
||||
if err := os.WriteFile(filepath.Join(dir, "book.m3u"), []byte(playlist), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Exactly one playlist → hint; two → none.
|
||||
hint := findPlaylistHint(dir)
|
||||
if hint == nil {
|
||||
t.Fatal("expected hint")
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, "other.m3u8"), []byte("#EXTM3U\n"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if findPlaylistHint(dir) != nil {
|
||||
t.Fatal("multiple playlists must produce no hint")
|
||||
}
|
||||
|
||||
abs := []string{
|
||||
filepath.Join(dir, "01_First.mp3"),
|
||||
filepath.Join(dir, "02_Second.mp3"),
|
||||
filepath.Join(dir, "10_Later.mp3"), // not in playlist → appended last
|
||||
}
|
||||
ordered := hint.reorder(abs)
|
||||
if len(ordered) != 3 || ordered[0] != abs[1] || ordered[1] != abs[0] || ordered[2] != abs[2] {
|
||||
t.Fatalf("playlist order not applied: %v", ordered)
|
||||
}
|
||||
if got := hint.titleFor(abs[0]); got != "Chapter One" {
|
||||
t.Fatalf("titleFor(./01_First.mp3) = %q", got)
|
||||
}
|
||||
if got := hint.titleFor(abs[1]); got != "Chapter Zero" {
|
||||
t.Fatalf("titleFor(02_Second.mp3) = %q", got)
|
||||
}
|
||||
if got := hint.titleFor(abs[2]); got != "" {
|
||||
t.Fatalf("unlisted track must have no title, got %q", got)
|
||||
}
|
||||
if h := (*playlistHint)(nil); h.titleFor(abs[0]) != "" {
|
||||
t.Fatal("nil hint must be safe")
|
||||
}
|
||||
}
|
||||
|
||||
func syntheticReport(tags map[string]string) *ffprobeReport {
|
||||
r := &ffprobeReport{}
|
||||
r.Format.Tags = tags
|
||||
return r
|
||||
}
|
||||
|
||||
func TestApplyAudioTagFallbacks(t *testing.T) {
|
||||
t.Run("full fill", func(t *testing.T) {
|
||||
meta := &MediaMetadata{}
|
||||
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
|
||||
"album": "The Tagged Album",
|
||||
"album_artist": "Tag Author",
|
||||
"artist": "Wrong Author",
|
||||
"composer": "Tag Narrator",
|
||||
"grouping": "Tag Series",
|
||||
"genre": "Audiobook",
|
||||
"date": "2020",
|
||||
"----:com.apple.iTunes:SUBTITLE": "Or, A Life",
|
||||
"----:com.apple.iTunes:SERIES": "Freeform Series",
|
||||
"----:com.apple.iTunes:SERIES-PART": "4",
|
||||
"----:com.apple.iTunes:ASIN": "B0ASIN1234",
|
||||
"----:com.apple.iTunes:ABRIDGED": "1",
|
||||
}))
|
||||
// A2b: subtitle is a first-class column — no title joining.
|
||||
if meta.Title != "The Tagged Album" {
|
||||
t.Fatalf("title = %q", meta.Title)
|
||||
}
|
||||
if meta.Subtitle != "Or, A Life" {
|
||||
t.Fatalf("subtitle = %q", meta.Subtitle)
|
||||
}
|
||||
if meta.Author != "Tag Author" {
|
||||
t.Fatalf("author = %q", meta.Author)
|
||||
}
|
||||
if len(meta.Narrators) != 1 || meta.Narrators[0] != "Tag Narrator" {
|
||||
t.Fatalf("narrators = %v", meta.Narrators)
|
||||
}
|
||||
// grouping outranks freeform SERIES for the series name only
|
||||
// because it is checked first; both are tag-level fallbacks.
|
||||
if meta.Series == "" {
|
||||
t.Fatal("series not filled from tags")
|
||||
}
|
||||
if meta.SeriesNumber != 4 {
|
||||
t.Fatalf("series number = %v", meta.SeriesNumber)
|
||||
}
|
||||
if meta.ASIN != "B0ASIN1234" {
|
||||
t.Fatalf("asin = %q", meta.ASIN)
|
||||
}
|
||||
if !meta.Abridged {
|
||||
t.Fatal("abridged not filled")
|
||||
}
|
||||
if meta.PublishDate.Year() != 2020 {
|
||||
t.Fatalf("publish date = %v", meta.PublishDate)
|
||||
}
|
||||
if meta.Genre != "Audiobook" {
|
||||
t.Fatalf("genre = %q", meta.Genre)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("fills only empty fields", func(t *testing.T) {
|
||||
meta := &MediaMetadata{Title: "Sidecar Title", Author: "Sidecar Author", Narrators: []string{"Sidecar Narrator"}}
|
||||
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
|
||||
"album": "Tag Album", "artist": "Tag Artist", "composer": "Tag Narrator",
|
||||
}))
|
||||
if meta.Title != "Sidecar Title" || meta.Author != "Sidecar Author" {
|
||||
t.Fatalf("tags must not override earlier sources: %+v", meta)
|
||||
}
|
||||
if len(meta.Narrators) != 1 || meta.Narrators[0] != "Sidecar Narrator" {
|
||||
t.Fatalf("narrators = %v", meta.Narrators)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("explicit sidecar false survives abridged tag", func(t *testing.T) {
|
||||
abridgedFalse := false
|
||||
meta := &MediaMetadata{AudioMeta: &AudioMetadataPayload{Abridged: &abridgedFalse}}
|
||||
applyAudioTagFallbacks(meta, syntheticReport(map[string]string{
|
||||
"----:com.apple.iTunes:ABRIDGED": "1",
|
||||
}))
|
||||
if meta.Abridged {
|
||||
t.Fatal("explicit sidecar false must not be overridden")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("nil report is a no-op", func(t *testing.T) {
|
||||
meta := &MediaMetadata{}
|
||||
applyAudioTagFallbacks(meta, nil)
|
||||
if meta.Title != "" {
|
||||
t.Fatal("nil report must change nothing")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestParseTagOrdinal(t *testing.T) {
|
||||
cases := map[string]int{
|
||||
"2": 2,
|
||||
"3/9": 3,
|
||||
" 12 ": 12,
|
||||
"": 0,
|
||||
"x": 0,
|
||||
"0": 0,
|
||||
"2/": 2,
|
||||
"-1": 0,
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := parseTagOrdinal(in); got != want {
|
||||
t.Errorf("parseTagOrdinal(%q) = %d, want %d", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOrderTracksByTags(t *testing.T) {
|
||||
mk := func(disc, track int, rel string) audioTrackInfo {
|
||||
return audioTrackInfo{disc: disc, trackNum: track, relPath: rel}
|
||||
}
|
||||
|
||||
t.Run("tag order wins when complete", func(t *testing.T) {
|
||||
probed := []audioTrackInfo{
|
||||
mk(2, 1, "b/01.mp3"), mk(1, 2, "a/02.mp3"), mk(1, 10, "a/10.mp3"),
|
||||
}
|
||||
orderTracksByTags(probed)
|
||||
want := []string{"a/02.mp3", "a/10.mp3", "b/01.mp3"}
|
||||
for i, tr := range probed {
|
||||
if tr.relPath != want[i] {
|
||||
t.Fatalf("order = %v", probed)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("incomplete coverage keeps natural order", func(t *testing.T) {
|
||||
probed := []audioTrackInfo{mk(1, 1, "a/01.mp3"), mk(0, 0, "a/02.mp3")}
|
||||
orderTracksByTags(probed)
|
||||
if probed[0].relPath != "a/01.mp3" || probed[1].relPath != "a/02.mp3" {
|
||||
t.Fatalf("order = %v", probed)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestTagLookupCaseInsensitive(t *testing.T) {
|
||||
r := syntheticReport(map[string]string{
|
||||
"ALBUM": "Cased Album",
|
||||
"----:com.apple.iTunes:ASIN": "B0CASE",
|
||||
})
|
||||
if got := r.tag("album"); got != "Cased Album" {
|
||||
t.Fatalf("tag(album) = %q", got)
|
||||
}
|
||||
if got := r.tag("album", "title"); got != "Cased Album" {
|
||||
t.Fatalf("multi-key lookup = %q", got)
|
||||
}
|
||||
if got := r.tag("----:COM.APPLE.ITUNES:asin"); !strings.EqualFold(got, "B0CASE") {
|
||||
t.Fatalf("freeform case-insensitive lookup = %q", got)
|
||||
}
|
||||
if got := r.tag("missing"); got != "" {
|
||||
t.Fatalf("missing key = %q", got)
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,203 @@
|
||||
package services
|
||||
|
||||
// Unit tests for the multi-file audiobook grouping/manifest helpers
|
||||
// (media_scanner_audio_multifile.go). Pure filesystem logic — no DB.
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestNaturalLess(t *testing.T) {
|
||||
cases := []struct {
|
||||
a, b string
|
||||
less bool
|
||||
}{
|
||||
{"01 Track.mp3", "02 Track.mp3", true},
|
||||
{"02 Track.mp3", "10 Track.mp3", true}, // numeric, not lexicographic
|
||||
{"10 Track.mp3", "9 Track.mp3", false},
|
||||
{"Track 2", "Track 10", true},
|
||||
{"abc", "abd", true},
|
||||
{"01", "1", false}, // leading zeros compare equal → strict-less false
|
||||
{"a1b", "a1c", true},
|
||||
{"Disc 1", "Disc 2", true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if got := naturalLess(tc.a, tc.b); got != tc.less {
|
||||
t.Errorf("naturalLess(%q, %q) = %v, want %v", tc.a, tc.b, got, tc.less)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSortTracksNatural(t *testing.T) {
|
||||
in := []string{"/x/10.mp3", "/x/2.mp3", "/x/1.mp3"}
|
||||
sortTracksNatural(in)
|
||||
want := []string{"/x/1.mp3", "/x/2.mp3", "/x/10.mp3"}
|
||||
for i := range want {
|
||||
if in[i] != want[i] {
|
||||
t.Fatalf("sorted = %v, want %v", in, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// writeDummyTrack creates a file with plausible audio bytes (never probed
|
||||
// in these tests — only structure matters).
|
||||
func writeDummyTrack(t *testing.T, path string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte("ID3 fake audio bytes"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBookRootForTrackFile(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
|
||||
// Layout:
|
||||
// root/Author/Title/01.mp3, 02.mp3 → one book at Title
|
||||
// root/Author/Parts/CD1/01.mp3, CD2/01.mp3 → one book at Parts
|
||||
// root/Author/Lone/book.mp3 → no folder book
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "Title", "01.mp3"))
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "Title", "02.mp3"))
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "Parts", "CD1", "01.mp3"))
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "Parts", "CD2", "01.mp3"))
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "Lone", "book.mp3"))
|
||||
|
||||
t.Run("simple track folder", func(t *testing.T) {
|
||||
got, tracks, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Title", "02.mp3"), root)
|
||||
if !ok {
|
||||
t.Fatalf("expected folder book, got none")
|
||||
}
|
||||
if got != filepath.Join(root, "Author", "Title") {
|
||||
t.Fatalf("root = %q", got)
|
||||
}
|
||||
if len(tracks) != 2 {
|
||||
t.Fatalf("tracks = %v", tracks)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("disc folders merge into parent", func(t *testing.T) {
|
||||
got, tracks, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Parts", "CD1", "01.mp3"), root)
|
||||
if !ok {
|
||||
t.Fatalf("expected folder book")
|
||||
}
|
||||
if got != filepath.Join(root, "Author", "Parts") {
|
||||
t.Fatalf("root = %q (disc folders must merge up)", got)
|
||||
}
|
||||
if len(tracks) != 2 {
|
||||
t.Fatalf("tracks = %v", tracks)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("lone file is not a folder book", func(t *testing.T) {
|
||||
_, _, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Lone", "book.mp3"), root)
|
||||
if ok {
|
||||
t.Fatalf("single track must not form a folder book")
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("sibling books never merge at author level", func(t *testing.T) {
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "BookB", "01.mp3"))
|
||||
writeDummyTrack(t, filepath.Join(root, "Author", "BookB", "02.mp3"))
|
||||
got, _, ok := bookRootForTrackFile(filepath.Join(root, "Author", "Title", "01.mp3"), root)
|
||||
if !ok || got != filepath.Join(root, "Author", "Title") {
|
||||
t.Fatalf("root = %q, ok = %v; Title/BookB must stay separate", got, ok)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestBuildMultiFileAudioPayload(t *testing.T) {
|
||||
tracks := []audioTrackInfo{
|
||||
{relPath: "Author/Title/01.mp3", title: "Chapter One", duration: 100, size: 10},
|
||||
{relPath: "Author/Title/02.mp3", title: "Chapter Two", duration: 50, size: 5},
|
||||
}
|
||||
payload := buildMultiFileAudioPayload("folder", nil, tracks)
|
||||
if payload.Layout != "folder" {
|
||||
t.Fatalf("layout = %q", payload.Layout)
|
||||
}
|
||||
if payload.DurationSeconds != 150 {
|
||||
t.Fatalf("duration = %v", payload.DurationSeconds)
|
||||
}
|
||||
if len(payload.Tracks) != 2 {
|
||||
t.Fatalf("tracks = %d", len(payload.Tracks))
|
||||
}
|
||||
if payload.Tracks[1].Start != 100 || payload.Tracks[1].End != 150 {
|
||||
t.Fatalf("cumulative bounds wrong: %+v", payload.Tracks[1])
|
||||
}
|
||||
// No sidecar chapters: tracks ARE the chapters.
|
||||
if len(payload.Chapters) != 2 || payload.Chapters[0].Start != 0 || payload.Chapters[1].Start != 100 {
|
||||
t.Fatalf("chapters = %+v", payload.Chapters)
|
||||
}
|
||||
|
||||
// Curated sidecar chapters keep priority; tracks still carry bounds.
|
||||
sidecar := &AudioMetadataPayload{
|
||||
Chapters: []AudioChapter{{Index: 0, Title: "Part I", Start: 0, End: 150}},
|
||||
}
|
||||
payload2 := buildMultiFileAudioPayload("zip", sidecar, tracks)
|
||||
if len(payload2.Chapters) != 1 || payload2.Chapters[0].Title != "Part I" {
|
||||
t.Fatalf("sidecar chapters must win: %+v", payload2.Chapters)
|
||||
}
|
||||
if len(payload2.Tracks) != 2 {
|
||||
t.Fatalf("tracks lost: %+v", payload2.Tracks)
|
||||
}
|
||||
}
|
||||
|
||||
func TestZipAudioEntries(t *testing.T) {
|
||||
zipPath := filepath.Join(t.TempDir(), "book.zip")
|
||||
f, err := os.Create(zipPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
w := zip.NewWriter(f)
|
||||
for _, name := range []string{
|
||||
"__MACOSX/._junk.mp3",
|
||||
".hidden.mp3",
|
||||
"02 second.mp3",
|
||||
"10 tenth.mp3",
|
||||
"01 first.mp3",
|
||||
"cover.jpg",
|
||||
"notes.txt",
|
||||
} {
|
||||
if _, err := w.Create(name); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err := w.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = f.Close()
|
||||
|
||||
entries, err := zipAudioEntries(zipPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := []string{"01 first.mp3", "02 second.mp3", "10 tenth.mp3"}
|
||||
if len(entries) != len(want) {
|
||||
t.Fatalf("entries = %v", entries)
|
||||
}
|
||||
for i, e := range entries {
|
||||
if e.name != want[i] {
|
||||
t.Fatalf("entry[%d] = %q, want %q", i, e.name, want[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanTrackTitle(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"/x/01_Genesis001.mp3": "Genesis001",
|
||||
"/x/01 - Track.mp3": "Track",
|
||||
"/x/12_.Intro.mp3": "Intro",
|
||||
"/x/Chapter 3.mp3": "Chapter 3",
|
||||
"/x/JustTitle.m4b": "JustTitle",
|
||||
"/x/01.mp3": "01", // number-only name: number stays the title
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := cleanTrackTitle(in); got != want {
|
||||
t.Errorf("cleanTrackTitle(%q) = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package services
|
||||
|
||||
// Unit tests for the extractor-version (ev) mechanism (faceted plan
|
||||
// Phase 10): the wire normalization (absent = v1-or-unknown = 1) and the
|
||||
// per-scan catch-up budget. The full scan-state machine (ev mismatch →
|
||||
// re-extract once → stable) lives in cmd/server/tests
|
||||
// (TestExtractorVersionReextract) because the store paths need a real DB.
|
||||
|
||||
import (
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestManifestEvNormalization(t *testing.T) {
|
||||
if got := manifestEv(nil); got != 1 {
|
||||
t.Errorf("nil manifest rides v1: got %d, want 1", got)
|
||||
}
|
||||
cases := []struct {
|
||||
raw string
|
||||
want int
|
||||
}{
|
||||
{`{"v":1}`, 1}, // old shape: ev absent → v1-or-unknown
|
||||
{`{"v":1,"ev":0}`, 1}, // explicitly unset → same
|
||||
{`{"v":1,"ev":1}`, 1}, // baseline parser stamp
|
||||
{`{"v":1,"ev":2}`, 2}, // post-bump stamp
|
||||
{`{"v":1,"entries":[],"ev":3}`, 3}, // sidecar-less ev home
|
||||
}
|
||||
for _, tc := range cases {
|
||||
m := decodeSidecarManifest([]byte(tc.raw))
|
||||
if m == nil {
|
||||
t.Fatalf("%s must decode", tc.raw)
|
||||
}
|
||||
if got := manifestEv(m); got != tc.want {
|
||||
t.Errorf("manifestEv(%s) = %d, want %d", tc.raw, got, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEvBudgetTake(t *testing.T) {
|
||||
scanner := &MediaScanner{}
|
||||
scanner.evBudgetRemaining.Store(2)
|
||||
if !scanner.evBudgetTake() {
|
||||
t.Error("budget 2: first take must pass")
|
||||
}
|
||||
if !scanner.evBudgetTake() {
|
||||
t.Error("budget 1: second take must pass")
|
||||
}
|
||||
if scanner.evBudgetTake() {
|
||||
t.Error("budget 0: third take must fail")
|
||||
}
|
||||
if got := scanner.evBudgetRemaining.Load(); got != 0 {
|
||||
t.Errorf("budget must land at exactly 0, got %d", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEvBudgetTake_FailsFastAtZero(t *testing.T) {
|
||||
scanner := &MediaScanner{}
|
||||
scanner.evBudgetRemaining.Store(0)
|
||||
// Repeated takes at 0 must fail WITHOUT driving the counter negative
|
||||
// (the CAS loop only fires when cur > 0).
|
||||
for i := 0; i < 5; i++ {
|
||||
if scanner.evBudgetTake() {
|
||||
t.Fatalf("take %d must fail at budget 0", i+1)
|
||||
}
|
||||
}
|
||||
if got := scanner.evBudgetRemaining.Load(); got != 0 {
|
||||
t.Errorf("budget must stay 0, got %d", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,348 @@
|
||||
package services
|
||||
|
||||
// Sidecar manifest fingerprints (faceted plan Part 2 §3): size-stable
|
||||
// books are changed by EDITING their metadata sidecars, which the old
|
||||
// size-only detection never noticed. One manifest per book (JSONB in
|
||||
// media_scan_state) lists every sidecar the extractors consume, so the
|
||||
// manifest and the extractors can never disagree — the manifest stats
|
||||
// exactly the names extractCalibreSidecar, extractAudiobookshelfSidecar,
|
||||
// and findSidecarCover look for.
|
||||
//
|
||||
// Change evaluation ladder, in cost order:
|
||||
// 1. stat pairs all equal → unchanged, ZERO bytes read (steady state);
|
||||
// 2. a stat pair differs → read + hash that one sidecar:
|
||||
// hash equal → refresh the stored stat pair (host repoint
|
||||
// reconciliation: bounded reads, zero re-extraction);
|
||||
// hash differs → the item changed → re-extract (caller);
|
||||
// 3. stored manifest NULL → backfill only: hash + store, NO
|
||||
// re-extraction (deploy-day rule — the first scan after release
|
||||
// must not look like an mtime flood).
|
||||
//
|
||||
// mtime is a CACHE field only — it is never a change signal. Docker
|
||||
// bind-mount repoints reset every mtime, which is exactly why the hash
|
||||
// reconciliation ladder exists.
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
const sidecarManifestVersion = 1
|
||||
|
||||
type manifestSidecar struct {
|
||||
Name string `json:"n"`
|
||||
Size int64 `json:"s"`
|
||||
MTime int64 `json:"m"`
|
||||
// Sha is the content hash. Present for ALL entries — covers too:
|
||||
// the repoint reconciliation needs a baseline hash for any stat-
|
||||
// drifted sidecar, and a hashless cover would turn harmless mtime
|
||||
// drift into a re-extraction (the exact flood this design kills).
|
||||
Sha string `json:"h,omitempty"`
|
||||
}
|
||||
|
||||
type sidecarManifest struct {
|
||||
Version int `json:"v"`
|
||||
Entries []manifestSidecar `json:"entries"`
|
||||
// Ev is the extractor (parser) version that produced the item's
|
||||
// CURRENT row. Absent (0) means "v1-or-unknown": backfill and
|
||||
// extract-failure stores leave it unset so a row never claims a
|
||||
// version its data can't prove; every store that FOLLOWS a
|
||||
// successful extraction stamps the setting's current value
|
||||
// (scan_extractor_version). A stored ev below the setting re-runs
|
||||
// extraction on a later scan — parser fixes reach old rows without
|
||||
// a force rescan (faceted plan Phase 10).
|
||||
Ev int `json:"ev,omitempty"`
|
||||
}
|
||||
|
||||
// manifestEv normalizes a stored manifest's extractor version: absent
|
||||
// manifest or unset ev means version 1 (the shipped baseline parser).
|
||||
// Current-parser versions only ever grow, so "unknown" safely rides the
|
||||
// oldest live version.
|
||||
func manifestEv(m *sidecarManifest) int {
|
||||
if m == nil || m.Ev < 1 {
|
||||
return 1
|
||||
}
|
||||
return m.Ev
|
||||
}
|
||||
|
||||
// coverSidecarNames mirrors findSidecarCover's priority list — the
|
||||
// manifest must stat the same names the extractor would consume.
|
||||
var coverSidecarNames = []string{
|
||||
"cover.jpg", "cover.jpeg", "cover.png", "cover.webp",
|
||||
"folder.jpg", "folder.jpeg", "folder.png", "folder.webp",
|
||||
".folder.jpg", ".folder.jpeg", ".folder.png",
|
||||
}
|
||||
|
||||
// bookSidecarDir is the directory whose sidecars feed a book: the book
|
||||
// file's directory (multifile book roots resolve to themselves because
|
||||
// file_path IS the directory).
|
||||
func bookSidecarDir(path string) string {
|
||||
info, err := os.Stat(path)
|
||||
if err == nil && info.IsDir() {
|
||||
return path
|
||||
}
|
||||
return filepath.Dir(path)
|
||||
}
|
||||
|
||||
// sidecarNamesForBook lists every sidecar filename the extractors
|
||||
// consume for a book at path. The book basename covers the
|
||||
// findSidecarCover "<basename>.<ext>" pattern.
|
||||
func sidecarNamesForBook(path string) []string {
|
||||
names := make([]string, 0, len(coverSidecarNames)+5)
|
||||
names = append(names, "metadata.opf", "metadata.json")
|
||||
names = append(names, coverSidecarNames...)
|
||||
base := strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
|
||||
if base != "" && base != "." {
|
||||
for _, ext := range []string{".jpg", ".jpeg", ".png", ".webp"} {
|
||||
names = append(names, base+ext)
|
||||
}
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
// statSidecarManifest stats every candidate sidecar (ReadDir-free: one
|
||||
// lstat per candidate name — the walk already visited the directory).
|
||||
// Returns nil when the directory itself is gone (deleted book mid-scan).
|
||||
func statSidecarManifest(path, dir string) *sidecarManifest {
|
||||
manifest := &sidecarManifest{Version: sidecarManifestVersion}
|
||||
for _, name := range sidecarNamesForBook(path) {
|
||||
info, err := os.Stat(filepath.Join(dir, name))
|
||||
if err != nil || info.IsDir() {
|
||||
continue
|
||||
}
|
||||
manifest.Entries = append(manifest.Entries, manifestSidecar{
|
||||
Name: name,
|
||||
Size: info.Size(),
|
||||
MTime: info.ModTime().UnixNano(),
|
||||
})
|
||||
}
|
||||
return manifest
|
||||
}
|
||||
|
||||
// hashSidecarManifest fills the content hashes of every entry — text
|
||||
// sidecars (KBs) and covers (a one-time bounded read at backfill; the
|
||||
// repoint ladder needs the baseline). Returns the same manifest for
|
||||
// chaining.
|
||||
func hashSidecarManifest(path, dir string, manifest *sidecarManifest) *sidecarManifest {
|
||||
for i := range manifest.Entries {
|
||||
entry := &manifest.Entries[i]
|
||||
sha, err := sha256File(filepath.Join(dir, entry.Name))
|
||||
if err != nil {
|
||||
continue // unreadable: stays size-only this pass
|
||||
}
|
||||
entry.Sha = sha
|
||||
}
|
||||
return manifest
|
||||
}
|
||||
|
||||
func sha256File(path string) (string, error) {
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
sum := sha256.Sum256(data)
|
||||
return hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
|
||||
// decodeSidecarManifest parses a stored manifest; nil when absent or
|
||||
// corrupt (the caller then treats it as a backfill, self-healing any
|
||||
// shape breakage).
|
||||
func decodeSidecarManifest(raw []byte) *sidecarManifest {
|
||||
if len(raw) == 0 {
|
||||
return nil
|
||||
}
|
||||
var m sidecarManifest
|
||||
if err := json.Unmarshal(raw, &m); err != nil {
|
||||
return nil
|
||||
}
|
||||
return &m
|
||||
}
|
||||
|
||||
// reconcileSidecarManifest runs the evaluation ladder against the
|
||||
// freshly-statted manifest. Returns the manifest to store and whether
|
||||
// the item's metadata must be re-extracted:
|
||||
// - stat pairs all equal → fresh=nil, changed=false (zero reads);
|
||||
// - stat drift + equal hash → fresh with the refreshed pair
|
||||
// (repoint reconciliation);
|
||||
// - hash differs, a sidecar appeared/vanished, or a differing text
|
||||
// sidecar has no stored hash → changed.
|
||||
func reconcileSidecarManifest(dir string, stored, computed *sidecarManifest) (fresh *sidecarManifest, changed bool) {
|
||||
if stored == nil {
|
||||
return nil, true
|
||||
}
|
||||
if len(stored.Entries) != len(computed.Entries) {
|
||||
return nil, true // sidecar appeared or vanished
|
||||
}
|
||||
storedByName := make(map[string]manifestSidecar, len(stored.Entries))
|
||||
for _, e := range stored.Entries {
|
||||
storedByName[e.Name] = e
|
||||
}
|
||||
// Fast path: stat pairs all equal → unchanged, zero bytes read,
|
||||
// nothing to store.
|
||||
drifted := false
|
||||
for _, cur := range computed.Entries {
|
||||
old, ok := storedByName[cur.Name]
|
||||
if !ok || cur.Size != old.Size || cur.MTime != old.MTime {
|
||||
drifted = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !drifted {
|
||||
return nil, false
|
||||
}
|
||||
fresh = &sidecarManifest{Version: sidecarManifestVersion, Ev: stored.Ev} // provenance preserved: stat-only reconciliation never re-extracted anything
|
||||
for _, cur := range computed.Entries {
|
||||
old, ok := storedByName[cur.Name]
|
||||
if !ok {
|
||||
return nil, true
|
||||
}
|
||||
if cur.Size == old.Size && cur.MTime == old.MTime {
|
||||
fresh.Entries = append(fresh.Entries, old) // untouched: keep the stored hash
|
||||
continue
|
||||
}
|
||||
if old.Sha == "" {
|
||||
return nil, true // unhashable drift: fall back to re-extract
|
||||
}
|
||||
sha, err := sha256File(filepath.Join(dir, cur.Name))
|
||||
if err != nil || sha != old.Sha {
|
||||
return nil, true
|
||||
}
|
||||
cur.Sha = old.Sha // repoint: content identical, stat pair refreshed
|
||||
fresh.Entries = append(fresh.Entries, cur)
|
||||
}
|
||||
return fresh, false
|
||||
}
|
||||
|
||||
// evReextractBudgetPerScan bounds the parser-version catch-up: at most
|
||||
// this many ev-triggered re-extractions per scan pass. Each one stamps
|
||||
// the fresh manifest with the new ev, so progress is resumable — later
|
||||
// polls pick up where this one stopped, and no single scan floods the
|
||||
// disk with a full-library re-extraction (faceted plan Phase 10).
|
||||
const evReextractBudgetPerScan = 100
|
||||
|
||||
// storeManifest persists the manifest and write-throughs the map entry.
|
||||
// The stored manifest is STAMPED with the setting's current extractor
|
||||
// version: every caller here follows (or is) a full extraction under
|
||||
// the current parser, so the stamp is truthful. Backfill and
|
||||
// extract-failure stores must use storeManifestBaseline instead.
|
||||
func (s *MediaScanner) storeManifest(ctx context.Context, libraryID pgtype.UUID, entry *scanEntry, manifest *sidecarManifest) {
|
||||
manifest.Ev = s.GetExtractorVersion()
|
||||
s.storeManifestBaseline(ctx, libraryID, entry, manifest)
|
||||
}
|
||||
|
||||
// storeManifestBaseline persists a manifest WITHOUT stamping the
|
||||
// extractor version — for stores that follow NO successful extraction:
|
||||
// the deploy-day backfill (the row's provenance is unknown; leaving ev
|
||||
// unset keeps it eligible for a later version bump) and the
|
||||
// extract-failure guard (stamping would mark parser work as done that
|
||||
// never ran; leaving it stale retries on the next poll).
|
||||
func (s *MediaScanner) storeManifestBaseline(ctx context.Context, libraryID pgtype.UUID, entry *scanEntry, manifest *sidecarManifest) {
|
||||
b, err := json.Marshal(manifest)
|
||||
if err != nil {
|
||||
fmt.Printf("Warning: failed to encode sidecar manifest for %s: %v\n", entry.FilePath, err)
|
||||
return
|
||||
}
|
||||
if err := s.db.UpsertMediaScanState(ctx, database.UpsertMediaScanStateParams{
|
||||
MediaItemID: entry.ID,
|
||||
Manifest: b,
|
||||
}); err != nil {
|
||||
fmt.Printf("Warning: failed to store sidecar manifest for %s: %v\n", entry.FilePath, err)
|
||||
return
|
||||
}
|
||||
entry.Manifest = b
|
||||
s.scanStateUpsert(libraryID, entry)
|
||||
}
|
||||
|
||||
// evBudgetTake consumes one slot of the per-scan ev catch-up budget.
|
||||
// Atomic: the poll scan walks sequentially, but the watcher's
|
||||
// scanDirectory pass can overlap a poll scan (only scanDirectory holds
|
||||
// scanMutex), so the budget is shared state.
|
||||
func (s *MediaScanner) evBudgetTake() bool {
|
||||
for {
|
||||
cur := s.evBudgetRemaining.Load()
|
||||
if cur <= 0 {
|
||||
return false
|
||||
}
|
||||
if s.evBudgetRemaining.CompareAndSwap(cur, cur-1) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// detectSidecarChange fingerprints the book's sidecar directory and
|
||||
// reconciles against the stored manifest. Returns true when the item's
|
||||
// metadata must be re-extracted (a sidecar's CONTENT changed, or
|
||||
// sidecars appeared/vanished). Backfill and repoint reconciliation are
|
||||
// handled here without re-extraction; the caller stores a fresh full
|
||||
// manifest after a change-path update.
|
||||
func (s *MediaScanner) detectSidecarChange(ctx context.Context, entry *scanEntry, libraryID pgtype.UUID, path string) bool {
|
||||
dir := bookSidecarDir(path)
|
||||
return s.detectSidecarChangeWithRepresentative(ctx, entry, libraryID, path, dir)
|
||||
}
|
||||
|
||||
// detectSidecarChangeWithRepresentative is detectSidecarChange with the
|
||||
// sidecar-name derivation decoupled from the map key. Multi-file books
|
||||
// (faceted plan Phase 6) are keyed by the item's file_path (the folder
|
||||
// root / zip), but the extractor reads a folder book's sidecars through
|
||||
// the sentinel INSIDE the root — whose basename yields no "<base>.<ext>"
|
||||
// cover candidates — so the manifest must stat the representative's name
|
||||
// list to never disagree with what extraction consumes.
|
||||
func (s *MediaScanner) detectSidecarChangeWithRepresentative(ctx context.Context, entry *scanEntry, libraryID pgtype.UUID, representative, dir string) bool {
|
||||
computed := statSidecarManifest(representative, dir)
|
||||
stored := decodeSidecarManifest(entry.Manifest)
|
||||
|
||||
// Extractor-version gate (faceted plan Phase 10): a manifest stamped
|
||||
// by an older parser re-extracts even when every sidecar byte is
|
||||
// unchanged — the fingerprint says "disk same", ev says "parser
|
||||
// newer". Budget-bounded: over budget → fall through, the stale ev
|
||||
// stays stored, a later poll resumes. Checked FIRST so books with
|
||||
// no sidecars (nothing to reconcile) are still caught up.
|
||||
if stored != nil && manifestEv(stored) != s.GetExtractorVersion() && s.evBudgetTake() {
|
||||
return true
|
||||
}
|
||||
|
||||
if len(computed.Entries) == 0 {
|
||||
if stored != nil && len(stored.Entries) > 0 {
|
||||
return true // every sidecar was removed — fall back to embedded
|
||||
}
|
||||
if stored == nil {
|
||||
// Never had sidecars: nothing to fingerprint, but the book
|
||||
// still needs an ev home so future parser bumps see it. A
|
||||
// first-sighting store follows no extraction → baseline
|
||||
// (ev unset → eligible for the next bump).
|
||||
s.storeManifestBaseline(ctx, libraryID, entry, &sidecarManifest{Version: sidecarManifestVersion})
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
if stored == nil {
|
||||
// Deploy-day backfill: hash + store WITHOUT re-extraction. The
|
||||
// row was extracted by an unknown parser version — the stamp is
|
||||
// deliberately OMITTED (ev unset → "v1-or-unknown") so a later
|
||||
// scan_extractor_version bump still catches it.
|
||||
s.storeManifestBaseline(ctx, libraryID, entry, hashSidecarManifest(representative, dir, computed))
|
||||
return false
|
||||
}
|
||||
|
||||
fresh, changed := reconcileSidecarManifest(dir, stored, computed)
|
||||
if changed {
|
||||
return true
|
||||
}
|
||||
if fresh != nil {
|
||||
// Repoint reconciliation refreshed stat pairs only — the row was
|
||||
// NOT re-extracted, so the manifest must keep the row's stored
|
||||
// ev (a stamp here would claim parser work that never ran;
|
||||
// possible when the ev budget was exhausted this pass).
|
||||
s.storeManifestBaseline(ctx, libraryID, entry, fresh)
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
package services
|
||||
|
||||
// Unit tests for the sidecar manifest ladder (faceted plan Part 2 §3):
|
||||
// stat-pair fast compare, hash-based repoint reconciliation, and the
|
||||
// change verdicts. Real files only where the hash compare needs them.
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestDecodeSidecarManifest(t *testing.T) {
|
||||
if decodeSidecarManifest(nil) != nil {
|
||||
t.Error("NULL manifest decodes to nil (backfill territory)")
|
||||
}
|
||||
if decodeSidecarManifest([]byte("{not json")) != nil {
|
||||
t.Error("corrupt manifest decodes to nil (self-healing backfill)")
|
||||
}
|
||||
m := decodeSidecarManifest([]byte(`{"v":1,"entries":[{"n":"metadata.opf","s":10,"m":100,"h":"abc"}]}`))
|
||||
if m == nil || len(m.Entries) != 1 || m.Entries[0].Sha != "abc" {
|
||||
t.Errorf("valid manifest round-trip failed: %+v", m)
|
||||
}
|
||||
}
|
||||
|
||||
// TestStatSidecarManifestSentinelNames pins the Phase 6 name derivation
|
||||
// for folder books: the manifest must stat exactly the names the
|
||||
// extractor consumes through the sentinel (metadata sidecars + the
|
||||
// standard cover set) — never the directory's own basename (the
|
||||
// extractor's basename-pattern covers resolve against the sentinel's
|
||||
// basename, which generates no candidates).
|
||||
func TestStatSidecarManifestSentinelNames(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
write := func(name string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte("x"), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("metadata.json")
|
||||
write("cover.jpg")
|
||||
write(filepath.Base(dir) + ".jpg") // extractor never reads this
|
||||
sentinel := filepath.Join(dir, ".audiobook-root")
|
||||
|
||||
m := statSidecarManifest(sentinel, dir)
|
||||
if m == nil {
|
||||
t.Fatal("manifest must stat, not nil")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, e := range m.Entries {
|
||||
seen[e.Name] = true
|
||||
}
|
||||
if !seen["metadata.json"] || !seen["cover.jpg"] {
|
||||
t.Errorf("extractor-consumed sidecars missing from manifest: %v", seen)
|
||||
}
|
||||
if seen[filepath.Base(dir)+".jpg"] {
|
||||
t.Errorf("directory-basename cover must not be tracked (extractor ignores it): %v", seen)
|
||||
}
|
||||
for name := range seen {
|
||||
if name == ".audiobook-root.jpg" {
|
||||
t.Errorf("sentinel-basename candidates must not appear (they cannot exist): %v", seen)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestReconcileSidecarManifest(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
write := func(name, content string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(filepath.Join(dir, name), []byte(content), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// The book: one hashed sidecar + one size-only cover.
|
||||
write("metadata.opf", `<opf>original content</opf>`)
|
||||
originalSha, err := sha256File(filepath.Join(dir, "metadata.opf"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
write("cover.jpg", "coverbytes")
|
||||
// stored is seeded exactly the way production backfill does: a real
|
||||
// stat pass + hash fill. Hand-built fake mtimes would drift on the
|
||||
// first compare (real files' mtimes never equal a constant).
|
||||
stored := hashSidecarManifest("book.epub", dir, statSidecarManifest(filepath.Join(dir, "book.epub"), dir))
|
||||
|
||||
stat := func() *sidecarManifest {
|
||||
m := statSidecarManifest(filepath.Join(dir, "book.epub"), dir)
|
||||
if m == nil || len(m.Entries) < 2 {
|
||||
t.Fatalf("stat manifest should find both sidecars, got %+v", m)
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
// Stat pairs identical → unchanged, zero reads, nothing to store.
|
||||
fresh, changed := reconcileSidecarManifest(dir, stored, stat())
|
||||
if changed || fresh != nil {
|
||||
t.Errorf("identical stats must be unchanged, got changed=%v fresh=%v", changed, fresh)
|
||||
}
|
||||
|
||||
// Repoint: mtime drifts, content identical → reconciled fresh
|
||||
// manifest with the refreshed stat pair and the KEPT hash, no change.
|
||||
if err := os.Chtimes(filepath.Join(dir, "metadata.opf"), time.Now(), time.Now().Add(time.Minute)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fresh, changed = reconcileSidecarManifest(dir, stored, stat())
|
||||
if changed {
|
||||
t.Error("mtime drift with identical content is repoint reconciliation, not a change")
|
||||
}
|
||||
if fresh == nil || len(fresh.Entries) != 2 {
|
||||
t.Fatalf("repoint must produce a refreshed manifest, got %+v", fresh)
|
||||
}
|
||||
if fresh.Entries[0].Sha != originalSha {
|
||||
t.Error("repointed entry keeps its stored hash")
|
||||
}
|
||||
if fresh.Entries[0].MTime == stored.Entries[0].MTime {
|
||||
t.Error("repointed entry refreshes its stat pair")
|
||||
}
|
||||
|
||||
// Same-size edit: drift + hash differs → changed.
|
||||
write("metadata.opf", `<opf>mutated XYY content</opf>`) // +1 byte, and hash differs
|
||||
_, changed = reconcileSidecarManifest(dir, stored, stat())
|
||||
if !changed {
|
||||
t.Error("content drift must be a change")
|
||||
}
|
||||
|
||||
// Same-LENGTH edit (the size-only detection blind spot) → changed.
|
||||
write("metadata.opf", `<opf>original Xontent</opf>`)
|
||||
_, changed = reconcileSidecarManifest(dir, stored, stat())
|
||||
if !changed {
|
||||
t.Error("same-size content drift must be a change (hash compare)")
|
||||
}
|
||||
|
||||
// New sidecar appears → changed.
|
||||
write("metadata.json", `{"title":"X"}`)
|
||||
_, changed = reconcileSidecarManifest(dir, stored, stat())
|
||||
if !changed {
|
||||
t.Error("an appearing sidecar must be a change")
|
||||
}
|
||||
|
||||
// Sidecar vanishes AND everything else returns to the stored state
|
||||
// (opf restored; its mtime drifts but the hash proves equality) →
|
||||
// the manifest reconciles, not a change.
|
||||
if err := os.Remove(filepath.Join(dir, "metadata.json")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
write("metadata.opf", `<opf>original content</opf>`)
|
||||
fresh, changed = reconcileSidecarManifest(dir, stored, stat())
|
||||
if changed {
|
||||
t.Errorf("returning to the stored state is not a change: %+v", fresh)
|
||||
}
|
||||
if fresh == nil || len(fresh.Entries) != 2 {
|
||||
t.Errorf("reconciled manifest covers both sidecars: %+v", fresh)
|
||||
}
|
||||
}
|
||||
|
||||
// An unhashable drift (a stored entry with no baseline hash — legacy
|
||||
// or partially-written manifest) is a change: the manifest cannot prove
|
||||
// content equality for it.
|
||||
func TestReconcileUnhashableDrift(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
stored := &sidecarManifest{Version: 1, Entries: []manifestSidecar{
|
||||
{Name: "cover.jpg", Size: 10, MTime: 1000},
|
||||
}}
|
||||
computed := &sidecarManifest{Version: 1, Entries: []manifestSidecar{
|
||||
{Name: "cover.jpg", Size: 11, MTime: 1001},
|
||||
}}
|
||||
fresh, changed := reconcileSidecarManifest(dir, stored, computed)
|
||||
if !changed {
|
||||
t.Error("stat drift without a stored baseline hash must be a change")
|
||||
}
|
||||
if fresh != nil {
|
||||
t.Error("a change never returns a fresh manifest (caller builds one)")
|
||||
}
|
||||
}
|
||||
@@ -139,8 +139,8 @@ func TestParseOPFContentCalibreSeries(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("parseOPFContent() error: %v", err)
|
||||
}
|
||||
if metadata.Series != "Great Series" || metadata.SeriesNumber != 2 {
|
||||
t.Errorf("Series = %q/%d, want Great Series/2", metadata.Series, metadata.SeriesNumber)
|
||||
if metadata.Series != "Great Series" || metadata.SeriesNumber != 2.5 {
|
||||
t.Errorf("Series = %q/%v, want Great Series/2.5", metadata.Series, metadata.SeriesNumber)
|
||||
}
|
||||
if metadata.ISBN == "" {
|
||||
t.Error("schemed ISBN not extracted")
|
||||
@@ -179,8 +179,8 @@ func TestExtractAudiobookshelfSidecar(t *testing.T) {
|
||||
if m.Title != "An Book" || m.Author != "Author One" {
|
||||
t.Errorf("Title/Author = %q/%q", m.Title, m.Author)
|
||||
}
|
||||
if m.Series != "The Series" || m.SeriesNumber != 4 {
|
||||
t.Errorf("Series = %q/%d, want The Series/4", m.Series, m.SeriesNumber)
|
||||
if m.Series != "The Series" || m.SeriesNumber != 4.5 {
|
||||
t.Errorf("Series = %q/%v, want The Series/4.5", m.Series, m.SeriesNumber)
|
||||
}
|
||||
if len(m.Tags) != 2 {
|
||||
t.Errorf("Tags = %v, want genres+tags merged", m.Tags)
|
||||
|
||||
@@ -171,8 +171,8 @@ func TestParseOPFContentTitleTypeAndSeries(t *testing.T) {
|
||||
if want := "The Main Title: The Subtitle"; metadata.Title != want {
|
||||
t.Errorf("Title = %q, want %q", metadata.Title, want)
|
||||
}
|
||||
if metadata.Series != "Great Series" || metadata.SeriesNumber != 4 {
|
||||
t.Errorf("Series = %q/%d, want Great Series/4", metadata.Series, metadata.SeriesNumber)
|
||||
if metadata.Series != "Great Series" || metadata.SeriesNumber != 4.5 {
|
||||
t.Errorf("Series = %q/%v, want Great Series/4.5", metadata.Series, metadata.SeriesNumber)
|
||||
}
|
||||
if metadata.ISBN == "" {
|
||||
t.Error("urn:isbn: identifier not extracted")
|
||||
|
||||
@@ -30,3 +30,37 @@ func TestMediaScanner_GetAutoScanEnabled(t *testing.T) {
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestMediaScanner_GetExtractorVersion(t *testing.T) {
|
||||
t.Run("nil db returns default 1", func(t *testing.T) {
|
||||
scanner := &MediaScanner{
|
||||
db: nil,
|
||||
settingsCache: NewSettingsCache(30 * time.Second),
|
||||
}
|
||||
if v := scanner.GetExtractorVersion(); v != 1 {
|
||||
t.Errorf("expected 1, got %d", v)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("cached value wins (scanner read-through path)", func(t *testing.T) {
|
||||
scanner := &MediaScanner{
|
||||
db: nil,
|
||||
settingsCache: NewSettingsCache(30 * time.Second),
|
||||
}
|
||||
scanner.settingsCache.Set("scan_extractor_version", "3")
|
||||
if v := scanner.GetExtractorVersion(); v != 3 {
|
||||
t.Errorf("expected 3, got %d", v)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("cached garbage rides the default", func(t *testing.T) {
|
||||
scanner := &MediaScanner{
|
||||
db: nil,
|
||||
settingsCache: NewSettingsCache(30 * time.Second),
|
||||
}
|
||||
scanner.settingsCache.Set("scan_extractor_version", "zero")
|
||||
if v := scanner.GetExtractorVersion(); v != 1 {
|
||||
t.Errorf("expected 1, got %d", v)
|
||||
}
|
||||
})
|
||||
}
|
||||
+211
-15
@@ -2,8 +2,10 @@ package services
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/utils"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
@@ -12,12 +14,16 @@ import (
|
||||
|
||||
// SearchService handles all search and filter operations
|
||||
type SearchService struct {
|
||||
db *database.Queries
|
||||
db *database.Queries
|
||||
raw database.DBTX // the raw pgx handle — the composed filter engines' path
|
||||
}
|
||||
|
||||
// NewSearchService creates a new search service instance
|
||||
func NewSearchService(db *database.Queries) *SearchService {
|
||||
return &SearchService{db: db}
|
||||
// NewSearchService creates a new search service instance. raw is the
|
||||
// same pool db wraps (database.DBTX) — the composed filter engines run
|
||||
// hand-built, allow-listed SQL with bound parameters, which static sqlc
|
||||
// queries cannot express.
|
||||
func NewSearchService(db *database.Queries, raw database.DBTX) *SearchService {
|
||||
return &SearchService{db: db, raw: raw}
|
||||
}
|
||||
|
||||
// SearchParams contains parameters for unified search
|
||||
@@ -38,6 +44,26 @@ type SearchParams struct {
|
||||
Sort string
|
||||
Limit int
|
||||
Offset int
|
||||
|
||||
// Facet registry Batch A additions (services/facet_registry.go).
|
||||
// Empty strings / invalid Bools mean "facet not active".
|
||||
StoryArcFilter string
|
||||
ImprintFilter string
|
||||
EditionFilter string
|
||||
AgeRatingFilter string
|
||||
FormatGroupFilter string
|
||||
MangaTypeFilter string
|
||||
ReadingDirectionFilter string
|
||||
ExplicitFilter pgtype.Bool
|
||||
AbridgedFilter pgtype.Bool
|
||||
|
||||
// Filter rule builder (faceted plan Phase 9). When FilterTree is
|
||||
// set the composed engine runs INSTEAD of the flat facet params
|
||||
// (qf wins over flat on a hand-crafted URL — precedence pinned by
|
||||
// test); FilterParseErr carries a qf parse/validation failure for
|
||||
// the endpoints to answer 400.
|
||||
FilterTree *FilterTree
|
||||
FilterParseErr string
|
||||
}
|
||||
|
||||
// parseSearchQuery detects quoted strings for exact match search
|
||||
@@ -63,6 +89,14 @@ func (s *SearchService) parseSearchQuery(query string) (bool, string) {
|
||||
func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params SearchParams) ([]database.SearchMediaItemsUnifiedRow, error) {
|
||||
// Parse search query for exact match detection
|
||||
isExact, searchQuery := s.parseSearchQuery(params.SearchQuery)
|
||||
|
||||
// Language filter UX: the UI sends display names ("English"); the
|
||||
// column stores ISO codes ("en"). Normalize each comma element the
|
||||
// SAME way ingest does (normalizeLanguageCode), so a user-typed or
|
||||
// deep-linked display name matches the stored code — and unknown
|
||||
// values that fit still match verbatim rows. (Facet matching and
|
||||
// ingest can never disagree because both run the same map.)
|
||||
params.LanguageFilter = normalizeLanguageFilter(params.LanguageFilter)
|
||||
// For now implementing real exact search. If you want to reimplement wildcard patterns uncomment this and line 81 and check queries.sql line 475 and frontend.go line 211
|
||||
// searchPattern := ""
|
||||
// if isExact {
|
||||
@@ -79,11 +113,24 @@ func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params Sear
|
||||
TagsFilter: pgtype.Text{String: params.TagsFilter, Valid: true},
|
||||
NarratorFilter: pgtype.Text{String: params.NarratorFilter, Valid: true},
|
||||
LanguageFilter: pgtype.Text{String: params.LanguageFilter, Valid: true},
|
||||
YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true},
|
||||
YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true},
|
||||
HasCover: params.HasCover,
|
||||
SearchQuery: pgtype.Text{String: searchQuery, Valid: true},
|
||||
IsExactSearch: pgtype.Bool{Bool: isExact, Valid: true},
|
||||
StoryArcFilter: pgtype.Text{String: params.StoryArcFilter, Valid: true},
|
||||
ImprintFilter: pgtype.Text{String: params.ImprintFilter, Valid: true},
|
||||
EditionFilter: pgtype.Text{String: params.EditionFilter, Valid: true},
|
||||
AgeRatingFilter: pgtype.Text{String: params.AgeRatingFilter, Valid: true},
|
||||
FormatGroupFilter: pgtype.Text{
|
||||
String: params.FormatGroupFilter, Valid: true,
|
||||
},
|
||||
MangaTypeFilter: pgtype.Text{String: params.MangaTypeFilter, Valid: true},
|
||||
ReadingDirectionFilter: pgtype.Text{
|
||||
String: params.ReadingDirectionFilter, Valid: true,
|
||||
},
|
||||
ExplicitFilter: params.ExplicitFilter,
|
||||
AbridgedFilter: params.AbridgedFilter,
|
||||
YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true},
|
||||
YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true},
|
||||
HasCover: params.HasCover,
|
||||
SearchQuery: pgtype.Text{String: searchQuery, Valid: true},
|
||||
IsExactSearch: pgtype.Bool{Bool: isExact, Valid: true},
|
||||
// SearchPattern: pgtype.Text{String: searchPattern, Valid: isExact},
|
||||
Sort: pgtype.Text{String: params.Sort, Valid: true},
|
||||
Limit: pgtype.Int4{Int32: int32(params.Limit), Valid: true},
|
||||
@@ -103,6 +150,130 @@ func (s *SearchService) SearchMediaItemsUnified(ctx context.Context, params Sear
|
||||
return results, nil
|
||||
}
|
||||
|
||||
// normalizeLanguageFilter normalizes every comma element of a language
|
||||
// filter value through the ingest-side language map (display name →
|
||||
// ISO code), so filtering and ingest share one source of truth.
|
||||
func normalizeLanguageFilter(wireValue string) string {
|
||||
if wireValue == "" {
|
||||
return ""
|
||||
}
|
||||
parts := strings.Split(wireValue, ",")
|
||||
out := make([]string, 0, len(parts))
|
||||
for _, p := range parts {
|
||||
if t := strings.TrimSpace(p); t != "" {
|
||||
out = append(out, normalizeLanguageCode(t))
|
||||
}
|
||||
}
|
||||
return strings.Join(out, ",")
|
||||
}
|
||||
|
||||
// NormalizeLanguageFilter is the exported form of the search-path
|
||||
// normalization for callers that must compare RAW filter values
|
||||
// against stored codes without issuing a query (the facet panel marks
|
||||
// a value Active by matching the bound filter set).
|
||||
func NormalizeLanguageFilter(wireValue string) string {
|
||||
return normalizeLanguageFilter(wireValue)
|
||||
}
|
||||
|
||||
// FacetValue is one facet value + count for the filter panel.
|
||||
type FacetValue struct {
|
||||
Value string `json:"value"` // the raw value (rides the filter param)
|
||||
// Display is the prettified form when it differs from Value
|
||||
// (language codes → names); empty → the client shows Value.
|
||||
Display string `json:"display,omitempty"`
|
||||
Count int64 `json:"count"`
|
||||
}
|
||||
|
||||
// facetCountsLimit is the per-facet top-N bound the panel renders.
|
||||
const facetCountsLimit = 50
|
||||
|
||||
// FacetCounts computes every listable facet's top values + counts under
|
||||
// the active filters (faceted plan Phase 7): ONE UNION-ALL round trip
|
||||
// (FacetCountsUnderFilters) in which each facet is counted under the
|
||||
// OTHER facets' filters — the SQL omits each arm's own block, classic
|
||||
// faceted-search semantics. With no filters bound every block is
|
||||
// neutral and the counts are the v1 global-per-library ones, so one
|
||||
// query serves both cases. libraryID may be NULL (the All-Libraries
|
||||
// view). Display enrichment matches the wire contract: language codes
|
||||
// → display names, genre values prettified — both only when they
|
||||
// differ from the raw value (Display is "the prettified form when it
|
||||
// differs", per FacetValue).
|
||||
func (s *SearchService) FacetCounts(ctx context.Context, userID, libraryID pgtype.UUID, params SearchParams, limit int) (map[string][]FacetValue, error) {
|
||||
if limit <= 0 {
|
||||
limit = facetCountsLimit
|
||||
}
|
||||
// Language filter UX (same normalization the search path runs —
|
||||
// see SearchMediaItemsUnified): display names in, ISO codes bound.
|
||||
langFilter := normalizeLanguageFilter(params.LanguageFilter)
|
||||
|
||||
rows, err := s.db.FacetCountsUnderFilters(ctx, database.FacetCountsUnderFiltersParams{
|
||||
UserID: userID,
|
||||
LibraryID: libraryID,
|
||||
Limit: pgtype.Int4{Int32: int32(limit), Valid: true},
|
||||
AuthorFilter: pgtype.Text{String: params.AuthorFilter, Valid: true},
|
||||
SeriesFilter: pgtype.Text{String: params.SeriesFilter, Valid: true},
|
||||
GenreFilter: pgtype.Text{String: params.GenreFilter, Valid: true},
|
||||
PublisherFilter: pgtype.Text{String: params.PublisherFilter, Valid: true},
|
||||
TagsFilter: pgtype.Text{String: params.TagsFilter, Valid: true},
|
||||
NarratorFilter: pgtype.Text{String: params.NarratorFilter, Valid: true},
|
||||
LanguageFilter: pgtype.Text{String: langFilter, Valid: true},
|
||||
StoryArcFilter: pgtype.Text{String: params.StoryArcFilter, Valid: true},
|
||||
ImprintFilter: pgtype.Text{String: params.ImprintFilter, Valid: true},
|
||||
EditionFilter: pgtype.Text{String: params.EditionFilter, Valid: true},
|
||||
AgeRatingFilter: pgtype.Text{String: params.AgeRatingFilter, Valid: true},
|
||||
FormatGroupFilter: pgtype.Text{String: params.FormatGroupFilter, Valid: true},
|
||||
MangaTypeFilter: pgtype.Text{String: params.MangaTypeFilter, Valid: true},
|
||||
ReadingDirectionFilter: pgtype.Text{String: params.ReadingDirectionFilter, Valid: true},
|
||||
ExplicitFilter: params.ExplicitFilter,
|
||||
AbridgedFilter: params.AbridgedFilter,
|
||||
YearMin: pgtype.Int4{Int32: int32(params.YearMin), Valid: true},
|
||||
YearMax: pgtype.Int4{Int32: int32(params.YearMax), Valid: true},
|
||||
HasCover: params.HasCover,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
out := make(map[string][]FacetValue, 14)
|
||||
for _, r := range rows {
|
||||
v := FacetValue{Value: r.Value, Count: r.Count}
|
||||
switch r.Facet {
|
||||
case "language":
|
||||
if name := LanguageDisplayName(r.Value); name != "" && name != r.Value {
|
||||
v.Display = name
|
||||
}
|
||||
case "genre":
|
||||
if d := utils.PrettifyGenre(r.Value); d != r.Value {
|
||||
v.Display = d
|
||||
}
|
||||
}
|
||||
out[r.Facet] = append(out[r.Facet], v)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// FacetValues lists ONE facet's top values + counts — the pinned
|
||||
// single-facet shape behind GET /api/media-items/facets?facet=X. It is
|
||||
// the all-facets query with every filter neutral (v1 counts:
|
||||
// global-per-library, not recomputed under other filters).
|
||||
// Registry-driven: only FacetArray and FacetScalar facets list values —
|
||||
// bool/range facets render as toggles/ranges in the panel, not value
|
||||
// lists.
|
||||
func (s *SearchService) FacetValues(ctx context.Context, userID pgtype.UUID, facet Facet, libraryID pgtype.UUID, limit int) ([]FacetValue, error) {
|
||||
if facet.Kind != FacetArray && facet.Kind != FacetScalar {
|
||||
return nil, fmt.Errorf("facet %q does not list values", facet.Param)
|
||||
}
|
||||
counts, err := s.FacetCounts(ctx, userID, libraryID, SearchParams{}, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
values := counts[facet.Param]
|
||||
if values == nil {
|
||||
return []FacetValue{}, nil
|
||||
}
|
||||
return values, nil
|
||||
}
|
||||
|
||||
// FieldSearchParams contains parameters for field-specific search (autocomplete)
|
||||
type FieldSearchParams struct {
|
||||
UserID pgtype.UUID
|
||||
@@ -113,11 +284,15 @@ type FieldSearchParams struct {
|
||||
Offset int
|
||||
}
|
||||
|
||||
// FieldValue represents a single field value with metadata
|
||||
// FieldValue is a single field value with metadata. Display carries a
|
||||
// human label when the wire value is an ID (library/collection/added_by
|
||||
// suggestions) — omitted everywhere else, so the classic static shape
|
||||
// {value, count, score} is unchanged.
|
||||
type FieldValue struct {
|
||||
Value string `json:"value"`
|
||||
Count int64 `json:"count"`
|
||||
Score float64 `json:"score"`
|
||||
Value string `json:"value"`
|
||||
Count int64 `json:"count"`
|
||||
Score float64 `json:"score"`
|
||||
Display string `json:"display,omitempty"`
|
||||
}
|
||||
|
||||
// SearchFieldValues handles field-specific search for autocomplete dropdowns
|
||||
@@ -223,7 +398,14 @@ func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearc
|
||||
}
|
||||
fieldValues := make([]FieldValue, len(results))
|
||||
for i, r := range results {
|
||||
fieldValues[i] = FieldValue{Value: r.Value.String, Count: r.Count, Score: r.Score}
|
||||
// Serve display names ("English") — the user clicks a name,
|
||||
// the search service normalizes it back to the stored ISO
|
||||
// code ("en"). Unknown codes pass through verbatim.
|
||||
shown := r.Value.String
|
||||
if name := LanguageDisplayName(shown); name != "" {
|
||||
shown = name
|
||||
}
|
||||
fieldValues[i] = FieldValue{Value: shown, Count: r.Count, Score: r.Score}
|
||||
}
|
||||
return fieldValues, nil
|
||||
|
||||
@@ -233,8 +415,22 @@ func (s *SearchService) SearchFieldValues(ctx context.Context, params FieldSearc
|
||||
}
|
||||
|
||||
// ExecuteSearch performs search and returns results with count
|
||||
// Shared between JSON endpoint and HTML rendering
|
||||
// Shared between JSON endpoint and HTML rendering.
|
||||
// Routing: a qf filter tree → the composed engine (flat params ignored
|
||||
// — qf wins); else the static unified query. A parse/validation failure
|
||||
// surfaces as ErrInvalidFilterTree (→ 400 at the endpoints).
|
||||
func (s *SearchService) ExecuteSearch(ctx context.Context, params SearchParams) ([]database.SearchMediaItemsUnifiedRow, int, error) {
|
||||
if params.FilterParseErr != "" {
|
||||
return nil, 0, fmt.Errorf("%w: %s", ErrInvalidFilterTree, params.FilterParseErr)
|
||||
}
|
||||
if params.FilterTree != nil {
|
||||
rows, err := s.ComposedSearch(ctx, params)
|
||||
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, 0, err
|
||||
}
|
||||
count := len(rows)
|
||||
return rows, count, nil
|
||||
}
|
||||
results, err := s.SearchMediaItemsUnified(ctx, params)
|
||||
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, 0, err
|
||||
|
||||
@@ -0,0 +1,265 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Series-metadata parsing helpers for the scanner (A2c overhaul).
|
||||
//
|
||||
// Sources disagree wildly on how a series and its ordinal are spelled:
|
||||
// ABS sidecars write ["Name #3"] string arrays or [{series, sequence}]
|
||||
// objects, Libation writes a bare string, Audible prefixes ordinals with
|
||||
// "Book ", Calibre uses decimal indices. These helpers normalize all of
|
||||
// them into candidates the scanner can rank and assign.
|
||||
|
||||
// SeriesCandidate is one parsed series membership from any source.
|
||||
type SeriesCandidate struct {
|
||||
Name string
|
||||
Sequence string // raw sequence text ("3", "1.5", "1a", "Book 2")
|
||||
Number float64 // leading number of Sequence (0 = none parsed)
|
||||
HasNum bool
|
||||
FromSidecar bool
|
||||
}
|
||||
|
||||
// Playlist markers that identify ordering constructs rather than
|
||||
// bibliographic series. Deliberately explicit: this classification routes
|
||||
// entries away from the series column into curated collections, so a false
|
||||
// positive creates user-visible objects. Prefer misses over false hits.
|
||||
var playlistMarkers = []string{
|
||||
"reading order",
|
||||
"chronological order",
|
||||
"publication order",
|
||||
"in order",
|
||||
"timeline",
|
||||
}
|
||||
|
||||
var (
|
||||
seriesSeqSuffixRe = regexp.MustCompile(`\s+#([^#\s]+)$`)
|
||||
// Leading ordinal inside a sequence string: "3", "1.5", "Book 2",
|
||||
// "2, Dramatized Adaptation" → 2. Mirrors ABS cleanSeriesSequence.
|
||||
seriesLeadingNumRe = regexp.MustCompile(`\d+(?:\.\d+)?`)
|
||||
// "<Series Name>, Book N" (also ; – : separators), Audible/Amazon style.
|
||||
deriveSeriesRe = regexp.MustCompile(`(?i)^\s*(.+?)\s*[,;:–-]\s*book\s+(\d+(?:\.\d+)?)\s*$`)
|
||||
// Conservative "Title: <name>, Book N" split — only when the post-colon
|
||||
// text is itself a series-and-number tail.
|
||||
titleSubtitleSplitRe = regexp.MustCompile(`(?i)^(.{1,200}?)\s*:\s*(.+,\s*book\s+\d+(?:\.\d+)?)$`)
|
||||
whitespaceRe = regexp.MustCompile(`\s+`)
|
||||
)
|
||||
|
||||
// splitSeriesString splits "Name #1a" into name + sequence, ABS
|
||||
// parseSeriesString-compatible: the sequence is anything after the last
|
||||
// " #" up to end-of-string, containing no whitespace or further '#'.
|
||||
func splitSeriesString(s string) (name, sequence string) {
|
||||
s = strings.TrimSpace(s)
|
||||
loc := seriesSeqSuffixRe.FindStringSubmatchIndex(s)
|
||||
if loc == nil {
|
||||
return s, ""
|
||||
}
|
||||
return strings.TrimSpace(s[:loc[0]]), s[loc[2]:loc[3]]
|
||||
}
|
||||
|
||||
// seriesSequenceNumber extracts the ordinal from a raw sequence string.
|
||||
// Returns the number, whether one was found, and whether information was
|
||||
// lost in the extraction (caller should preserve the raw text, e.g. "1a").
|
||||
func seriesSequenceNumber(sequence string) (num float64, found, lossy bool) {
|
||||
trimmed := strings.TrimSpace(sequence)
|
||||
if trimmed == "" {
|
||||
return 0, false, false
|
||||
}
|
||||
token := seriesLeadingNumRe.FindString(trimmed)
|
||||
if token == "" {
|
||||
return 0, false, false
|
||||
}
|
||||
num, err := strconv.ParseFloat(token, 64)
|
||||
if err != nil {
|
||||
return 0, false, false
|
||||
}
|
||||
// "1.5" tokenizes exactly (not lossy); "1a"/"Book 2" keep a raw tail.
|
||||
return num, true, trimmed != token
|
||||
}
|
||||
|
||||
// isPlaylistSeriesName reports whether the series name is an ordering
|
||||
// construct ("The Witcher (Reading Order)") rather than a bibliography.
|
||||
// Match discipline: the marker must be a parenthetical qualifier or a
|
||||
// trailing phrase — a series merely CONTAINING the word "timeline" mid-name
|
||||
// ("The Timeline of...") is a real series, not a playlist.
|
||||
func isPlaylistSeriesName(name string) bool {
|
||||
lower := strings.ToLower(strings.TrimSpace(name))
|
||||
for _, m := range playlistMarkers {
|
||||
if strings.HasSuffix(lower, m) {
|
||||
return true
|
||||
}
|
||||
if open := strings.Index(lower, "("); open >= 0 {
|
||||
if strings.Contains(lower[open:], m) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// parseSeriesCandidates normalizes the sidecar `series` field, which is
|
||||
// written as an array of STRINGS ("Name #3" — ABS abmetadata), an array of
|
||||
// OBJECTS ({series, sequence} — some exports), or a plain JSON string
|
||||
// (Libation), into candidates.
|
||||
func parseSeriesCandidates(raw json.RawMessage) []SeriesCandidate {
|
||||
trimmed := strings.TrimSpace(string(raw))
|
||||
if trimmed == "" || trimmed == "null" {
|
||||
return nil
|
||||
}
|
||||
var out []SeriesCandidate
|
||||
|
||||
add := func(name, sequence string, fromSidecar bool) {
|
||||
name = strings.TrimSpace(name)
|
||||
if name == "" {
|
||||
return
|
||||
}
|
||||
num, found, _ := seriesSequenceNumber(sequence)
|
||||
out = append(out, SeriesCandidate{
|
||||
Name: name, Sequence: sequence, Number: num, HasNum: found,
|
||||
FromSidecar: fromSidecar,
|
||||
})
|
||||
}
|
||||
|
||||
if strings.HasPrefix(trimmed, "[") {
|
||||
// Array of objects first ({"series": ..., "sequence": ...}).
|
||||
var objs []struct {
|
||||
Series string `json:"series"`
|
||||
Name string `json:"name"`
|
||||
Sequence string `json:"sequence"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &objs); err == nil && len(objs) > 0 {
|
||||
for _, o := range objs {
|
||||
name := o.Series
|
||||
if name == "" {
|
||||
name = o.Name
|
||||
}
|
||||
add(name, o.Sequence, true)
|
||||
}
|
||||
return out
|
||||
}
|
||||
// Array of strings: ABS abmetadata form "Name #3".
|
||||
var strs []string
|
||||
if err := json.Unmarshal(raw, &strs); err == nil {
|
||||
for _, s := range strs {
|
||||
name, seq := splitSeriesString(s)
|
||||
add(name, seq, true)
|
||||
}
|
||||
return out
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Plain JSON string (Libation): "Name" or "Name #3".
|
||||
var single string
|
||||
if err := json.Unmarshal(raw, &single); err == nil {
|
||||
name, seq := splitSeriesString(single)
|
||||
add(name, seq, true)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// deriveSeriesFromText extracts "<series>, Book N" from a title/subtitle
|
||||
// string (Audible/Amazon convention). Returns ok=false when the text is not
|
||||
// obviously a series tail — this fallback must not guess.
|
||||
func deriveSeriesFromText(text string) (name string, num float64, hasNum bool, ok bool) {
|
||||
if strings.TrimSpace(text) == "" {
|
||||
return "", 0, false, false
|
||||
}
|
||||
m := deriveSeriesRe.FindStringSubmatch(text)
|
||||
if m == nil {
|
||||
return "", 0, false, false
|
||||
}
|
||||
name = whitespaceRe.ReplaceAllString(strings.TrimSpace(m[1]), " ")
|
||||
if name == "" || len(name) > 200 || strings.HasPrefix(strings.ToLower(name), "book ") {
|
||||
return "", 0, false, false
|
||||
}
|
||||
num, err := strconv.ParseFloat(m[2], 64)
|
||||
if err != nil {
|
||||
return "", 0, false, false
|
||||
}
|
||||
return name, num, true, true
|
||||
}
|
||||
|
||||
// splitTitleOnSubtitle splits "Title: <name>, Book N" into title +
|
||||
// subtitle, but ONLY when the post-colon text is itself a series-and-number
|
||||
// tail — arbitrary colons in titles are never touched.
|
||||
func splitTitleOnSubtitle(folderTitle string) (title, subtitle string, split bool) {
|
||||
m := titleSubtitleSplitRe.FindStringSubmatch(strings.TrimSpace(folderTitle))
|
||||
if m == nil {
|
||||
return folderTitle, "", false
|
||||
}
|
||||
return strings.TrimSpace(m[1]), strings.TrimSpace(m[2]), true
|
||||
}
|
||||
|
||||
// normalizeLanguageCode maps sidecar/display language names to ISO 639-1
|
||||
// codes (the media_items.language convention). Values already in code form
|
||||
// pass through; unknown display names are kept verbatim when they fit the
|
||||
// VARCHAR(10) column, else dropped (empty) — never truncate mid-word.
|
||||
func normalizeLanguageCode(v string) string {
|
||||
v = strings.TrimSpace(v)
|
||||
if v == "" {
|
||||
return ""
|
||||
}
|
||||
if code, ok := languageNameToCode[strings.ToLower(v)]; ok {
|
||||
return code
|
||||
}
|
||||
lower := strings.ToLower(v)
|
||||
if len(lower) == 2 {
|
||||
return lower // already a code (or close enough to be harmless)
|
||||
}
|
||||
if len(v) <= 10 {
|
||||
return v
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// languageCodeToName — canonical English display name per ISO code,
|
||||
// mirroring languageNameToCode (server owns display; the column stores
|
||||
// ISO 639-1).
|
||||
var languageCodeToName = map[string]string{
|
||||
"en": "English", "es": "Spanish", "fr": "French", "de": "German",
|
||||
"it": "Italian", "pt": "Portuguese", "nl": "Dutch", "ru": "Russian",
|
||||
"uk": "Ukrainian", "pl": "Polish", "cs": "Czech", "sk": "Slovak",
|
||||
"sv": "Swedish", "no": "Norwegian", "da": "Danish", "fi": "Finnish",
|
||||
"is": "Icelandic", "hu": "Hungarian", "ro": "Romanian", "bg": "Bulgarian",
|
||||
"el": "Greek", "tr": "Turkish", "ar": "Arabic", "he": "Hebrew",
|
||||
"fa": "Persian", "hi": "Hindi", "bn": "Bengali", "ur": "Urdu",
|
||||
"th": "Thai", "vi": "Vietnamese", "id": "Indonesian", "ms": "Malay",
|
||||
"ja": "Japanese", "ko": "Korean", "zh": "Chinese", "sw": "Swahili",
|
||||
"af": "Afrikaans", "ca": "Catalan", "hr": "Croatian", "sr": "Serbian",
|
||||
"sl": "Slovenian", "lt": "Lithuanian", "lv": "Latvian", "et": "Estonian",
|
||||
"ka": "Georgian", "hy": "Armenian", "cy": "Welsh", "ga": "Irish",
|
||||
"eu": "Basque", "gl": "Galician",
|
||||
}
|
||||
|
||||
// LanguageDisplayName renders a stored language value for DISPLAY.
|
||||
// Returns "" when the value is not a known code — callers keep the raw
|
||||
// value then (legacy verbatim rows like "Pig Latin" display as
|
||||
// themselves).
|
||||
func LanguageDisplayName(code string) string {
|
||||
if name, ok := languageCodeToName[strings.ToLower(strings.TrimSpace(code))]; ok {
|
||||
return name
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
var languageNameToCode = map[string]string{
|
||||
"english": "en", "spanish": "es", "español": "es", "french": "fr",
|
||||
"français": "fr", "german": "de", "deutsch": "de", "italian": "it",
|
||||
"italiano": "it", "portuguese": "pt", "dutch": "nl", "russian": "ru",
|
||||
"ukrainian": "uk", "polish": "pl", "czech": "cs", "slovak": "sk",
|
||||
"swedish": "sv", "norwegian": "no", "danish": "da", "finnish": "fi",
|
||||
"icelandic": "is", "hungarian": "hu", "romanian": "ro", "bulgarian": "bg",
|
||||
"greek": "el", "turkish": "tr", "arabic": "ar", "hebrew": "he",
|
||||
"persian": "fa", "hindi": "hi", "bengali": "bn", "urdu": "ur",
|
||||
"thai": "th", "vietnamese": "vi", "indonesian": "id", "malay": "ms",
|
||||
"japanese": "ja", "korean": "ko", "chinese": "zh", "mandarin": "zh",
|
||||
"cantonese": "zh", "swahili": "sw", "afrikaans": "af", "catalan": "ca",
|
||||
"croatian": "hr", "serbian": "sr", "slovenian": "sl", "lithuanian": "lt",
|
||||
"latvian": "lv", "estonian": "et", "georgian": "ka", "armenian": "hy",
|
||||
"welsh": "cy", "irish": "ga", "basque": "eu", "galician": "gl",
|
||||
}
|
||||
@@ -0,0 +1,322 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
func TestSplitSeriesString(t *testing.T) {
|
||||
cases := []struct{ in, name, seq string }{
|
||||
{"The Witcher #3", "The Witcher", "3"},
|
||||
{"Name #1a", "Name", "1a"},
|
||||
{"No Sequence", "No Sequence", ""},
|
||||
{"Name #1#a", "Name #1#a", ""}, // embedded # is not a sequence
|
||||
}
|
||||
for _, c := range cases {
|
||||
name, seq := splitSeriesString(c.in)
|
||||
if name != c.name || seq != c.seq {
|
||||
t.Errorf("splitSeriesString(%q) = (%q, %q), want (%q, %q)", c.in, name, seq, c.name, c.seq)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSeriesSequenceNumber(t *testing.T) {
|
||||
cases := []struct {
|
||||
in string
|
||||
num float64
|
||||
found bool
|
||||
lossy bool
|
||||
}{
|
||||
{"3", 3, true, false},
|
||||
{"1.5", 1.5, true, false},
|
||||
{"1a", 1, true, true},
|
||||
{"Book 2", 2, true, true},
|
||||
{"2, Dramatized Adaptation", 2, true, true},
|
||||
{"", 0, false, false},
|
||||
{"novella", 0, false, false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
num, found, lossy := seriesSequenceNumber(c.in)
|
||||
if num != c.num || found != c.found || lossy != c.lossy {
|
||||
t.Errorf("seriesSequenceNumber(%q) = (%v,%v,%v), want (%v,%v,%v)",
|
||||
c.in, num, found, lossy, c.num, c.found, c.lossy)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The A2c headline bug: ABS abmetadata writes series as an array of
|
||||
// "Name #seq" STRINGS. The old parser only accepted object arrays and
|
||||
// silently dropped every ABS-matched series.
|
||||
func TestParseSeriesCandidatesStringArray(t *testing.T) {
|
||||
raw := []byte(`["The Witcher #3", "The Witcher (Reading Order) #5"]`)
|
||||
candidates := parseSeriesCandidates(raw)
|
||||
if len(candidates) != 2 {
|
||||
t.Fatalf("got %d candidates, want 2", len(candidates))
|
||||
}
|
||||
if candidates[0].Name != "The Witcher" || candidates[0].Number != 3 || !candidates[0].HasNum {
|
||||
t.Errorf("candidate[0] = %+v", candidates[0])
|
||||
}
|
||||
if candidates[1].Name != "The Witcher (Reading Order)" || candidates[1].Number != 5 {
|
||||
t.Errorf("candidate[1] = %+v", candidates[1])
|
||||
}
|
||||
if !isPlaylistSeriesName(candidates[1].Name) {
|
||||
t.Errorf("reading-order entry must classify as playlist-ish")
|
||||
}
|
||||
if isPlaylistSeriesName(candidates[0].Name) {
|
||||
t.Errorf("publication series must not classify as playlist-ish")
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseSeriesCandidatesOtherShapes(t *testing.T) {
|
||||
objs := parseSeriesCandidates([]byte(`[{"series":"Great Series","sequence":"2.5"}]`))
|
||||
if len(objs) != 1 || objs[0].Name != "Great Series" || objs[0].Number != 2.5 {
|
||||
t.Errorf("object form: %+v", objs)
|
||||
}
|
||||
libation := parseSeriesCandidates([]byte(`"Libation Series #4"`))
|
||||
if len(libation) != 1 || libation[0].Name != "Libation Series" || libation[0].Number != 4 {
|
||||
t.Errorf("plain string form: %+v", libation)
|
||||
}
|
||||
if parseSeriesCandidates([]byte(`null`)) != nil {
|
||||
t.Errorf("null must yield no candidates")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsPlaylistSeriesName(t *testing.T) {
|
||||
positives := []string{
|
||||
"The Witcher (Reading Order)",
|
||||
"Discworld Reading Order",
|
||||
"Narnia Chronological Order",
|
||||
"Robin Hobb in order",
|
||||
"Series Publication Order",
|
||||
"Middle Earth Timeline",
|
||||
}
|
||||
for _, p := range positives {
|
||||
if !isPlaylistSeriesName(p) {
|
||||
t.Errorf("%q should be playlist-ish", p)
|
||||
}
|
||||
}
|
||||
negatives := []string{"The Witcher", "Witcher Saga", "The Timeline of..."}
|
||||
for _, n := range negatives {
|
||||
if isPlaylistSeriesName(n) {
|
||||
t.Errorf("%q must not be playlist-ish", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeriveSeriesFromText(t *testing.T) {
|
||||
name, num, hasNum, ok := deriveSeriesFromText("The Witcher, Book 3")
|
||||
if !ok || name != "The Witcher" || num != 3 || !hasNum {
|
||||
t.Errorf("got (%q,%v,%v,%v)", name, num, hasNum, ok)
|
||||
}
|
||||
name, num, hasNum, ok = deriveSeriesFromText("The Witcher, Book 1.5")
|
||||
if !ok || name != "The Witcher" || num != 1.5 || !hasNum {
|
||||
t.Errorf("decimal: got (%q,%v,%v,%v)", name, num, hasNum, ok)
|
||||
}
|
||||
// Not obviously a series tail → must refuse rather than guess.
|
||||
for _, negative := range []string{"1984", "The Fellowship", "Book Ends"} {
|
||||
if _, _, _, ok := deriveSeriesFromText(negative); ok {
|
||||
t.Errorf("%q must not derive a series", negative)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestSplitTitleOnSubtitle(t *testing.T) {
|
||||
title, subtitle, split := splitTitleOnSubtitle("Baptism of Fire: The Witcher, Book 3")
|
||||
if !split || title != "Baptism of Fire" || subtitle != "The Witcher, Book 3" {
|
||||
t.Errorf("got (%q,%q,%v)", title, subtitle, split)
|
||||
}
|
||||
// Arbitrary colons never split.
|
||||
_, _, split = splitTitleOnSubtitle("Star Wars: A New Hope")
|
||||
if split {
|
||||
t.Errorf("non-series colon must not split")
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyTitleDerivedSeries(t *testing.T) {
|
||||
// The user's actual naming, two-level folder, no sidecar anywhere.
|
||||
meta := &MediaMetadata{Title: "Baptism of Fire: The Witcher, Book 3"}
|
||||
applyTitleDerivedSeries(meta)
|
||||
if meta.Series != "The Witcher" || meta.SeriesNumber != 3 {
|
||||
t.Errorf("series = %q #%v, want The Witcher #3", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
if meta.Title != "Baptism of Fire: The Witcher, Book 3" {
|
||||
t.Errorf("fallback must never mutate the display title, got %q", meta.Title)
|
||||
}
|
||||
|
||||
// Subtitle already split by the folder parser: number + name both land.
|
||||
meta = &MediaMetadata{Title: "Blood of Elves", Subtitle: "The Witcher, Book 1"}
|
||||
applyTitleDerivedSeries(meta)
|
||||
if meta.Series != "The Witcher" || meta.SeriesNumber != 1 {
|
||||
t.Errorf("series = %q #%v, want The Witcher #1", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
|
||||
// Existing series data is never overwritten (blanks-only fallback)...
|
||||
meta = &MediaMetadata{Title: "X", Subtitle: "Derived, Book 9", Series: "Real Series"}
|
||||
applyTitleDerivedSeries(meta)
|
||||
if meta.Series != "Real Series" || meta.SeriesNumber != 0 {
|
||||
t.Errorf("a number from a different series name must not apply, got %q #%v", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
// ...but a number whose derived name MATCHES the existing series fills it.
|
||||
meta = &MediaMetadata{Title: "X", Subtitle: "Real Series, Book 9", Series: "Real Series"}
|
||||
applyTitleDerivedSeries(meta)
|
||||
if meta.Series != "Real Series" || meta.SeriesNumber != 9 {
|
||||
t.Errorf("matching-name number must fill, got %q #%v", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPickSmartGenre(t *testing.T) {
|
||||
// The real Malazan Audible blob: one string, four genres.
|
||||
genre, all := pickSmartGenre([]string{"Epic, Action & Adventure, Military, Sword & Sorcery"})
|
||||
if genre != "action & adventure" {
|
||||
t.Errorf("genre = %q, want action & adventure", genre)
|
||||
}
|
||||
if len(all) != 4 {
|
||||
t.Errorf("candidates = %v, want the 4 split parts", all)
|
||||
}
|
||||
// Format noise never wins.
|
||||
if genre, _ := pickSmartGenre([]string{"Audiobook"}); genre != "" {
|
||||
t.Errorf("audiobook is not a genre, got %q", genre)
|
||||
}
|
||||
// Fiction is the floor: alone it resolves, against a real shelf it loses.
|
||||
if genre, _ := pickSmartGenre([]string{"Fiction"}); genre != "fiction" {
|
||||
t.Errorf("plain Fiction should still resolve, got %q", genre)
|
||||
}
|
||||
// Genre model v2: Epic Fantasy is a SUBGENRE — the genre column gets
|
||||
// its parent shelf and the specific label lives in tags. (Intended
|
||||
// change from the old longest-match behavior, which returned
|
||||
// "epic fantasy" here.)
|
||||
if genre, _ := pickSmartGenre([]string{"Fiction", "Epic Fantasy"}); genre != "fantasy" {
|
||||
t.Errorf("subgenre must resolve to its shelf, got %q", genre)
|
||||
}
|
||||
// User-verified traces (Batch B decision):
|
||||
if genre, _ := pickSmartGenre([]string{"Pirates", "Adventure"}); genre != "adventure" {
|
||||
t.Errorf("direct shelf beats mapped subgenre, got %q", genre)
|
||||
}
|
||||
if genre, _ := pickSmartGenre([]string{"Pirates"}); genre != "adventure" {
|
||||
t.Errorf("mapped subgenre alone resolves to its shelf, got %q", genre)
|
||||
}
|
||||
if genre, _ := pickSmartGenre([]string{"Sci-Fi", "Science Fiction", "Fiction"}); genre != "science fiction" {
|
||||
t.Errorf("direct shelf beats floor, got %q", genre)
|
||||
}
|
||||
// Type-words describe the library type, not a shelf.
|
||||
if genre, _ := pickSmartGenre([]string{"Manga"}); genre != "" {
|
||||
t.Errorf("manga is a type-word, got %q", genre)
|
||||
}
|
||||
// Unknown-only input: no shelf is known — tags carry it, never guess.
|
||||
if genre, all := pickSmartGenre([]string{"Completely Made-Up Genre"}); genre != "" {
|
||||
t.Errorf("unknown-only input must leave genre empty, got %q (tags=%v)", genre, all)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNormalizeLanguageCode(t *testing.T) {
|
||||
cases := []struct{ in, want string }{
|
||||
{"English", "en"},
|
||||
{"en", "en"},
|
||||
{"German", "de"},
|
||||
{"", ""},
|
||||
{"Norwegian (Bokmål)", ""}, // too long for VARCHAR(10); dropped, never truncated
|
||||
{"Pig Latin", "Pig Latin"}, // unknown display name that fits: kept
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := normalizeLanguageCode(c.in); got != c.want {
|
||||
t.Errorf("normalizeLanguageCode(%q) = %q, want %q", c.in, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// End-to-end: a REAL ABS-shaped sidecar (string-array series, the shape
|
||||
// every ABS-matched book writes) must land series, language, explicit and
|
||||
// the full author array.
|
||||
func TestExtractAudiobookshelfSidecarStringArraySeries(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
sidecar := `{
|
||||
"title": "Blood of Elves",
|
||||
"authors": ["Andrzej Sapkowski"],
|
||||
"narrators": ["Peter Kenny"],
|
||||
"series": ["The Witcher #1", "The Witcher (Reading Order) #3"],
|
||||
"genres": ["Fiction"],
|
||||
"language": "English",
|
||||
"explicit": true,
|
||||
"publisher": "Orbit"
|
||||
}`
|
||||
if err := os.WriteFile(filepath.Join(dir, "metadata.json"), []byte(sidecar), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
meta := extractAudiobookshelfSidecar(filepath.Join(dir, "book.m4b"))
|
||||
if meta == nil {
|
||||
t.Fatal("sidecar not extracted")
|
||||
}
|
||||
if meta.Series != "The Witcher" || meta.SeriesNumber != 1 {
|
||||
t.Errorf("series = %q #%v", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
if len(meta.Authors) != 1 || meta.Authors[0] != "Andrzej Sapkowski" {
|
||||
t.Errorf("authors array = %v", meta.Authors)
|
||||
}
|
||||
if meta.Language != "en" {
|
||||
t.Errorf("language = %q, want en", meta.Language)
|
||||
}
|
||||
if !meta.Explicit {
|
||||
t.Errorf("explicit flag lost")
|
||||
}
|
||||
if len(meta.PlaylistSeries) != 1 || meta.PlaylistSeries[0].Name != "The Witcher (Reading Order)" {
|
||||
t.Errorf("playlist series = %+v", meta.PlaylistSeries)
|
||||
}
|
||||
if meta.AlternateInfo != "" {
|
||||
t.Errorf("playlist entry must not leak into alternate_info, got %q", meta.AlternateInfo)
|
||||
}
|
||||
}
|
||||
|
||||
// Additional real series (the Drizzt case: Icewind Dale #1 = Legend of
|
||||
// Drizzt #4) land in alternate_info with the comic-established shape.
|
||||
func TestExtractAudiobookshelfSidecarExtraSeriesToAlternateInfo(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
sidecar := `{
|
||||
"title": "The Crystal Shard",
|
||||
"authors": ["R.A. Salvatore"],
|
||||
"series": [
|
||||
{"series": "The Icewind Dale Trilogy", "sequence": "1"},
|
||||
{"series": "The Legend of Drizzt", "sequence": "4"}
|
||||
]
|
||||
}`
|
||||
if err := os.WriteFile(filepath.Join(dir, "metadata.json"), []byte(sidecar), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
meta := extractAudiobookshelfSidecar(filepath.Join(dir, "book.m4b"))
|
||||
if meta.Series != "The Icewind Dale Trilogy" || meta.SeriesNumber != 1 {
|
||||
t.Errorf("series = %q #%v", meta.Series, meta.SeriesNumber)
|
||||
}
|
||||
if meta.AlternateInfo == "" {
|
||||
t.Fatal("second series must land in alternate_info")
|
||||
}
|
||||
// Lossy sequence text is preserved, never silently dropped.
|
||||
meta2 := extractAudiobookshelfSidecarPath(t, dir, `{"title":"X","series":["Saga #1a"]}`)
|
||||
if meta2 == nil || meta2.Series != "Saga" || meta2.SeriesNumber != 1 {
|
||||
t.Fatalf("lossy series = %+v", meta2)
|
||||
}
|
||||
if meta2.AlternateInfo == "" || !strings.Contains(meta2.AlternateInfo, "series_sequence_raw") || !strings.Contains(meta2.AlternateInfo, "1a") {
|
||||
t.Errorf("raw sequence not preserved: %q", meta2.AlternateInfo)
|
||||
}
|
||||
}
|
||||
|
||||
func extractAudiobookshelfSidecarPath(t *testing.T, dir, json string) *MediaMetadata {
|
||||
t.Helper()
|
||||
// extractAudiobookshelfSidecar looks for exactly "metadata.json"; use a
|
||||
// fresh dir per call instead.
|
||||
fresh := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(fresh, "metadata.json"), []byte(json), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return extractAudiobookshelfSidecar(filepath.Join(fresh, "book.m4b"))
|
||||
}
|
||||
|
||||
// The series page tiebreaker rides on the same ordering contract the
|
||||
// scanner feeds: numbers sort numerically, NULLs last (§3.6).
|
||||
func TestSeriesNumberFloatOrderingValues(t *testing.T) {
|
||||
if !(pgtype.Float8{Float64: 1}.Float64 < pgtype.Float8{Float64: 1.5}.Float64) {
|
||||
t.Fatal("1 must sort before 1.5")
|
||||
}
|
||||
}
|
||||
@@ -26,7 +26,7 @@ func TestContinueSeriesRowToMediaItems_FieldsMappedCorrectly(t *testing.T) {
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
CoverImagePath: pgtype.Text{String: "/covers/test.jpg", Valid: true},
|
||||
Series: pgtype.Text{String: "Test Series", Valid: true},
|
||||
SeriesNumber: pgtype.Int4{Int32: 3, Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 3, Valid: true},
|
||||
Tags: []string{"fantasy", "adventure"},
|
||||
FormatGroup: "epub",
|
||||
AddedByAdminID: pgtype.UUID{Bytes: adminUUID, Valid: true},
|
||||
@@ -43,7 +43,7 @@ func TestContinueSeriesRowToMediaItems_FieldsMappedCorrectly(t *testing.T) {
|
||||
assert.Equal(t, pgtype.Text{String: "application/epub+zip", Valid: true}, result.MimeType, "MimeType should match")
|
||||
assert.Equal(t, pgtype.Text{String: "/covers/test.jpg", Valid: true}, result.CoverImagePath, "CoverImagePath should match")
|
||||
assert.Equal(t, pgtype.Text{String: "Test Series", Valid: true}, result.Series, "Series should match")
|
||||
assert.Equal(t, pgtype.Int4{Int32: 3, Valid: true}, result.SeriesNumber, "SeriesNumber should match")
|
||||
assert.Equal(t, pgtype.Float8{Float64: 3, Valid: true}, result.SeriesNumber, "SeriesNumber should match")
|
||||
assert.Equal(t, []string{"fantasy", "adventure"}, result.Tags, "Tags should match")
|
||||
assert.Equal(t, "epub", result.FormatGroup, "FormatGroup should match")
|
||||
}
|
||||
@@ -64,7 +64,7 @@ func TestContinueSeriesRowToMediaItems_NullFieldsHandled(t *testing.T) {
|
||||
MimeType: pgtype.Text{Valid: false},
|
||||
CoverImagePath: pgtype.Text{Valid: false},
|
||||
Series: pgtype.Text{Valid: false},
|
||||
SeriesNumber: pgtype.Int4{Valid: false},
|
||||
SeriesNumber: pgtype.Float8{Valid: false},
|
||||
Tags: nil,
|
||||
FormatGroup: "epub",
|
||||
}
|
||||
@@ -94,7 +94,7 @@ func TestContinueSeriesRowToMediaItems_AllFieldsMapped(t *testing.T) {
|
||||
MimeType: pgtype.Text{String: "epub", Valid: true},
|
||||
CoverImagePath: pgtype.Text{String: "/cover.jpg", Valid: true},
|
||||
Series: pgtype.Text{String: "Series", Valid: true},
|
||||
SeriesNumber: pgtype.Int4{Int32: 1, Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 1, Valid: true},
|
||||
Tags: []string{"tag1"},
|
||||
Asin: pgtype.Text{String: "ASIN", Valid: true},
|
||||
Publisher: pgtype.Text{String: "Pub", Valid: true},
|
||||
|
||||
+142
-26
@@ -4,6 +4,7 @@ import (
|
||||
"bookhoard/internal/database"
|
||||
wsync "bookhoard/internal/sync"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
@@ -27,6 +28,13 @@ const (
|
||||
JobStatusCancelled JobStatus = "cancelled"
|
||||
)
|
||||
|
||||
// Sentinel errors surfaced by CancelJob so HTTP handlers can map them
|
||||
// to 404/409 without string matching.
|
||||
var (
|
||||
ErrJobNotFound = errors.New("job not found")
|
||||
ErrJobNotCancellable = errors.New("job cannot be cancelled")
|
||||
)
|
||||
|
||||
type JobType string
|
||||
|
||||
const (
|
||||
@@ -45,11 +53,30 @@ var WorkerInstance *Worker
|
||||
func (w *Worker) Enqueue(job *Job) {
|
||||
select {
|
||||
case w.jobQueue <- job:
|
||||
w.recordPending(job)
|
||||
default:
|
||||
fmt.Printf("Worker queue full, rejecting job: %s\n", job.ID)
|
||||
}
|
||||
}
|
||||
|
||||
// recordPending registers a queued job's pending result so the status
|
||||
// endpoint and GetActiveScanJobs can report jobs still waiting for a
|
||||
// worker, and so CancelJob can cancel them before execution starts.
|
||||
// The entry is replaced when processJob dequeues the job.
|
||||
func (w *Worker) recordPending(job *Job) {
|
||||
libID, _ := job.Params["library_id"].(string)
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
if _, exists := w.results[job.ID]; !exists {
|
||||
w.results[job.ID] = &JobResult{
|
||||
JobID: job.ID,
|
||||
Status: JobStatusPending,
|
||||
LibraryID: libID,
|
||||
JobType: job.Type,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type Job struct {
|
||||
ID string
|
||||
Type JobType
|
||||
@@ -78,7 +105,11 @@ type JobResult struct {
|
||||
// LibraryID carries the scan job's library (from Params) so clients
|
||||
// can label progress rows after a page refresh, when the original
|
||||
// start-request context is gone.
|
||||
LibraryID string `json:"library_id,omitempty"`
|
||||
LibraryID string `json:"library_id,omitempty"`
|
||||
// JobType distinguishes scan jobs in the active set — the enqueue
|
||||
// coalescing check (HasActiveScanForLibrary) must only coalesce
|
||||
// scans, never imports/converts sharing a library id.
|
||||
JobType JobType `json:"job_type,omitempty"`
|
||||
Error string `json:"error"`
|
||||
Result interface{} `json:"result"`
|
||||
Progress float64 `json:"progress"`
|
||||
@@ -146,8 +177,14 @@ type AnalyticsJobResult struct {
|
||||
ReportPath string `json:"report_path"`
|
||||
}
|
||||
type Worker struct {
|
||||
jobQueue chan *Job
|
||||
results map[string]*JobResult
|
||||
jobQueue chan *Job
|
||||
results map[string]*JobResult
|
||||
// cancels maps a running job's id to its per-job context cancel
|
||||
// func, registered by processJob and removed at completion. Lets
|
||||
// CancelJob stop a running job without disturbing siblings (all
|
||||
// manual scans used to share the handler's process-lifetime
|
||||
// context, so no targeted cancellation was possible).
|
||||
cancels map[string]context.CancelFunc
|
||||
connManager *wsync.ConnectionManager
|
||||
mu sync.RWMutex
|
||||
wg sync.WaitGroup
|
||||
@@ -192,6 +229,7 @@ func NewWorkerWithConfig(numWorkers, queueCap int, connManager *wsync.Connection
|
||||
w := &Worker{
|
||||
jobQueue: make(chan *Job, queueCap),
|
||||
results: make(map[string]*JobResult),
|
||||
cancels: make(map[string]context.CancelFunc),
|
||||
ctx: ctx,
|
||||
cancel: cancel,
|
||||
}
|
||||
@@ -243,22 +281,41 @@ func (w *Worker) worker() {
|
||||
func (w *Worker) processJob(job *Job) {
|
||||
libID, _ := job.Params["library_id"].(string)
|
||||
w.mu.Lock()
|
||||
// A job cancelled while queued must never execute: honour the
|
||||
// pending-cancel marker and leave its result untouched.
|
||||
if result, exists := w.results[job.ID]; exists && result.Status == JobStatusCancelled {
|
||||
w.mu.Unlock()
|
||||
return
|
||||
}
|
||||
w.results[job.ID] = &JobResult{
|
||||
JobID: job.ID,
|
||||
Status: JobStatusRunning,
|
||||
LibraryID: libID,
|
||||
JobType: job.Type,
|
||||
}
|
||||
|
||||
// Derive a per-job context so cancelling this job (CancelJob) stops
|
||||
// only this job. The parent stays whatever the caller passed (the
|
||||
// handler's context for manual scans, Background for auto-scans).
|
||||
parent := job.Context
|
||||
if parent == nil {
|
||||
parent = context.Background()
|
||||
}
|
||||
jobCtx, cancel := context.WithCancel(parent)
|
||||
job.Context = jobCtx
|
||||
w.cancels[job.ID] = cancel
|
||||
w.mu.Unlock()
|
||||
|
||||
defer func() {
|
||||
w.mu.Lock()
|
||||
delete(w.cancels, job.ID)
|
||||
w.mu.Unlock()
|
||||
cancel()
|
||||
}()
|
||||
|
||||
started := time.Now()
|
||||
job.StartedAt = &started
|
||||
|
||||
w.mu.Lock()
|
||||
if result, exists := w.results[job.ID]; exists {
|
||||
result.Status = JobStatusRunning
|
||||
}
|
||||
w.mu.Unlock()
|
||||
|
||||
var err error
|
||||
var result interface{}
|
||||
|
||||
@@ -318,27 +375,43 @@ func (w *Worker) processJob(job *Job) {
|
||||
}
|
||||
}
|
||||
|
||||
// A cancelled job stopped mid-flight: keep its last real progress
|
||||
// instead of claiming 100%, and drop the meaningless "context
|
||||
// canceled" processing error.
|
||||
finalProgress := 1.0
|
||||
finalError := ""
|
||||
if status == JobStatusCancelled {
|
||||
finalError = ""
|
||||
if old, exists := w.results[job.ID]; exists {
|
||||
finalProgress = old.Progress
|
||||
}
|
||||
} else if err != nil {
|
||||
finalError = err.Error()
|
||||
}
|
||||
|
||||
w.results[job.ID] = &JobResult{
|
||||
JobID: job.ID,
|
||||
Status: status,
|
||||
LibraryID: libID,
|
||||
Error: func() string {
|
||||
if err != nil {
|
||||
return err.Error()
|
||||
} else {
|
||||
return ""
|
||||
}
|
||||
}(),
|
||||
Result: result,
|
||||
Progress: 1.0,
|
||||
Error: finalError,
|
||||
Result: result,
|
||||
Progress: finalProgress,
|
||||
|
||||
FilesScanned: filesScanned,
|
||||
NewItems: newItems,
|
||||
Errors: errors,
|
||||
}
|
||||
|
||||
if job.Type == JobTypeScan && w.connManager != nil && job.UserID != "" {
|
||||
// Cancelled scans announce scan_cancelled so clients reset
|
||||
// their scanning indicators; a "scan_complete" would read as
|
||||
// a finished scan.
|
||||
msgType := wsync.MessageTypeScanComplete
|
||||
if status == JobStatusCancelled {
|
||||
msgType = wsync.MessageTypeScanCancelled
|
||||
}
|
||||
w.connManager.BroadcastToUser(job.UserID, wsync.BroadcastMessage{
|
||||
Type: wsync.MessageTypeScanComplete,
|
||||
Type: msgType,
|
||||
Data: map[string]interface{}{
|
||||
"job_id": job.ID,
|
||||
"files_scanned": filesScanned,
|
||||
@@ -419,6 +492,12 @@ func (w *Worker) processScanJob(job *Job) (interface{}, error) {
|
||||
scanner.SetAdminID(adminUUID)
|
||||
scanner.SetForce(force)
|
||||
|
||||
// Trigger label (plan Part 2 §4): poll / watcher / admin / initial —
|
||||
// surfaces in the per-scan summary line.
|
||||
if trigger, ok := job.Params["trigger"].(string); ok {
|
||||
scanner.SetTrigger(trigger)
|
||||
}
|
||||
|
||||
var libraryUUID pgtype.UUID
|
||||
if err := libraryUUID.Scan(libraryID); err != nil {
|
||||
return nil, err
|
||||
@@ -972,6 +1051,7 @@ func (w *Worker) EnqueueJob(job *Job) error {
|
||||
|
||||
select {
|
||||
case w.jobQueue <- job:
|
||||
w.recordPending(job)
|
||||
return nil
|
||||
case <-w.ctx.Done():
|
||||
return fmt.Errorf("worker is shutting down")
|
||||
@@ -1005,19 +1085,55 @@ func (w *Worker) GetActiveScanJobs() []JobResult {
|
||||
return active
|
||||
}
|
||||
|
||||
// HasActiveScanForLibrary reports whether a pending or running SCAN job
|
||||
// already targets the library (faceted plan Part 2 §5): coalescing
|
||||
// stops poll ticks from stacking full scans when a scan outlives its
|
||||
// interval, and enforces the one-scan-per-library invariant the bulk
|
||||
// scan-state map leans on.
|
||||
func (w *Worker) HasActiveScanForLibrary(libraryID string) bool {
|
||||
if w == nil || libraryID == "" {
|
||||
return false
|
||||
}
|
||||
w.mu.RLock()
|
||||
defer w.mu.RUnlock()
|
||||
for _, result := range w.results {
|
||||
if result.JobType != JobTypeScan || result.LibraryID != libraryID {
|
||||
continue
|
||||
}
|
||||
if result.Status == JobStatusRunning || result.Status == JobStatusPending {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// CancelJob cancels a pending or running job by id. Pending jobs are
|
||||
// marked cancelled so processJob skips them at dequeue (no context
|
||||
// exists yet); running jobs have their per-job context cancelled — the
|
||||
// job stops at its next context check and processJob finalises the
|
||||
// status as cancelled.
|
||||
func (w *Worker) CancelJob(jobID string) error {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
|
||||
if result, exists := w.results[jobID]; exists {
|
||||
if result.Status == JobStatusRunning || result.Status == JobStatusPending {
|
||||
result.Status = JobStatusCancelled
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("job cannot be cancelled")
|
||||
result, exists := w.results[jobID]
|
||||
if !exists {
|
||||
return ErrJobNotFound
|
||||
}
|
||||
|
||||
return fmt.Errorf("job not found")
|
||||
switch result.Status {
|
||||
case JobStatusPending:
|
||||
result.Status = JobStatusCancelled
|
||||
return nil
|
||||
case JobStatusRunning:
|
||||
result.Status = JobStatusCancelled
|
||||
if cancel, ok := w.cancels[jobID]; ok {
|
||||
cancel()
|
||||
}
|
||||
return nil
|
||||
default:
|
||||
return ErrJobNotCancellable
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) Shutdown() {
|
||||
|
||||
@@ -58,3 +58,46 @@ func TestGetActiveScanJobsEmpty(t *testing.T) {
|
||||
t.Fatalf("expected no active jobs, got %d", len(jobs))
|
||||
}
|
||||
}
|
||||
|
||||
// HasActiveScanForLibrary is the enqueue-coalescing predicate (plan
|
||||
// Part 2 §5): a pending/running SCAN for the same library makes the
|
||||
// next poll tick / watcher event a no-op. Only scans coalesce — an
|
||||
// import or convert sharing the library id must not block one.
|
||||
func TestHasActiveScanForLibrary(t *testing.T) {
|
||||
w := NewWorker(1, nil)
|
||||
|
||||
seed := map[string]*JobResult{
|
||||
"running-scan": {JobID: "running-scan", Status: JobStatusRunning, LibraryID: "lib-1", JobType: JobTypeScan},
|
||||
"pending-scan": {JobID: "pending-scan", Status: JobStatusPending, LibraryID: "lib-2", JobType: JobTypeScan},
|
||||
"done-scan": {JobID: "done-scan", Status: JobStatusCompleted, LibraryID: "lib-3", JobType: JobTypeScan},
|
||||
"failed-scan": {JobID: "failed-scan", Status: JobStatusFailed, LibraryID: "lib-4", JobType: JobTypeScan},
|
||||
"import-lib1": {JobID: "import-lib1", Status: JobStatusRunning, LibraryID: "lib-1", JobType: JobTypeImport},
|
||||
"no-library": {JobID: "no-library", Status: JobStatusRunning, JobType: JobTypeScan},
|
||||
}
|
||||
w.mu.Lock()
|
||||
w.results = seed
|
||||
w.mu.Unlock()
|
||||
|
||||
cases := []struct {
|
||||
library string
|
||||
want bool
|
||||
}{
|
||||
{"lib-1", true}, // running scan
|
||||
{"lib-2", true}, // pending scan
|
||||
{"lib-3", false}, // completed = terminal
|
||||
{"lib-4", false}, // failed = terminal
|
||||
{"lib-9", false}, // unknown library
|
||||
{"", false}, // jobs without a library never coalesce
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := w.HasActiveScanForLibrary(c.library); got != c.want {
|
||||
t.Errorf("HasActiveScanForLibrary(%q) = %v, want %v", c.library, got, c.want)
|
||||
}
|
||||
}
|
||||
|
||||
// A nil worker (tests, early startup) must not panic.
|
||||
var nilWorker *Worker
|
||||
if nilWorker.HasActiveScanForLibrary("lib-1") {
|
||||
t.Error("nil worker must report no active scans")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,157 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// A job cancelled while still queued must never execute. processJob
|
||||
// sees the pending-cancel marker at dequeue and leaves the result
|
||||
// untouched; a scan without params would otherwise fail with
|
||||
// "library_id required" and flip the status to failed — so an empty
|
||||
// error field proves the job body never ran.
|
||||
func TestCancelQueuedJobNeverExecutes(t *testing.T) {
|
||||
// Zero workers: the job stays queued until processJob is driven by
|
||||
// hand below.
|
||||
w := NewWorkerWithConfig(0, 10, nil)
|
||||
|
||||
job := &Job{
|
||||
ID: "queued-cancel",
|
||||
Type: JobTypeScan,
|
||||
Context: context.Background(),
|
||||
Params: map[string]interface{}{},
|
||||
}
|
||||
if err := w.EnqueueJob(job); err != nil {
|
||||
t.Fatalf("enqueue failed: %v", err)
|
||||
}
|
||||
|
||||
// The pending result must exist before any worker picks the job up,
|
||||
// so status polling can see (and cancel) queued jobs.
|
||||
result, ok := w.GetJobStatus("queued-cancel")
|
||||
if !ok {
|
||||
t.Fatal("queued job has no result entry")
|
||||
}
|
||||
if result.Status != JobStatusPending {
|
||||
t.Fatalf("expected pending status, got %q", result.Status)
|
||||
}
|
||||
|
||||
if err := w.CancelJob("queued-cancel"); err != nil {
|
||||
t.Fatalf("CancelJob failed: %v", err)
|
||||
}
|
||||
|
||||
// Simulate the dequeue that races the cancellation.
|
||||
w.processJob(job)
|
||||
|
||||
result, _ = w.GetJobStatus("queued-cancel")
|
||||
if result.Status != JobStatusCancelled {
|
||||
t.Fatalf("expected cancelled status after dequeue, got %q", result.Status)
|
||||
}
|
||||
if result.Error != "" {
|
||||
t.Errorf("skipped job must carry no processing error, got %q", result.Error)
|
||||
}
|
||||
}
|
||||
|
||||
// Cancelling a running job must invoke its per-job context cancel func
|
||||
// — the old CancelJob only flipped a map status and the job kept
|
||||
// running to completion.
|
||||
func TestCancelRunningJobCancelsContext(t *testing.T) {
|
||||
w := NewWorker(1, nil)
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
job := &Job{ID: "running-cancel", Type: JobTypeScan, Context: ctx}
|
||||
|
||||
// Seed the state processJob establishes while a job runs.
|
||||
w.mu.Lock()
|
||||
w.results[job.ID] = &JobResult{JobID: job.ID, Status: JobStatusRunning}
|
||||
w.cancels[job.ID] = cancel
|
||||
w.mu.Unlock()
|
||||
|
||||
if err := w.CancelJob(job.ID); err != nil {
|
||||
t.Fatalf("CancelJob failed: %v", err)
|
||||
}
|
||||
|
||||
if ctx.Err() == nil {
|
||||
t.Fatal("CancelJob must cancel the job's context")
|
||||
}
|
||||
result, _ := w.GetJobStatus(job.ID)
|
||||
if result.Status != JobStatusCancelled {
|
||||
t.Fatalf("expected cancelled status, got %q", result.Status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCancelJobErrors(t *testing.T) {
|
||||
w := NewWorker(1, nil)
|
||||
|
||||
if err := w.CancelJob("missing-job"); !errors.Is(err, ErrJobNotFound) {
|
||||
t.Errorf("unknown job: expected ErrJobNotFound, got %v", err)
|
||||
}
|
||||
|
||||
w.mu.Lock()
|
||||
w.results["done-job"] = &JobResult{JobID: "done-job", Status: JobStatusCompleted}
|
||||
w.results["failed-job"] = &JobResult{JobID: "failed-job", Status: JobStatusFailed}
|
||||
w.results["cancelled-job"] = &JobResult{JobID: "cancelled-job", Status: JobStatusCancelled}
|
||||
w.mu.Unlock()
|
||||
|
||||
for _, id := range []string{"done-job", "failed-job", "cancelled-job"} {
|
||||
if err := w.CancelJob(id); !errors.Is(err, ErrJobNotCancellable) {
|
||||
t.Errorf("terminal job %q: expected ErrJobNotCancellable, got %v", id, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Both enqueue paths must register a pending result: auto-scans arrive
|
||||
// via Enqueue (fire-and-forget) and manual scans via EnqueueJob, and a
|
||||
// queued job must be visible to status polling and cancellable before
|
||||
// a worker dequeues it.
|
||||
func TestEnqueueRecordsPendingResult(t *testing.T) {
|
||||
w := NewWorkerWithConfig(0, 10, nil)
|
||||
|
||||
fireAndForget := &Job{
|
||||
ID: "via-enqueue",
|
||||
Type: JobTypeScan,
|
||||
Context: context.Background(),
|
||||
Params: map[string]interface{}{"library_id": "lib-auto"},
|
||||
}
|
||||
w.Enqueue(fireAndForget)
|
||||
|
||||
tracked := &Job{
|
||||
ID: "via-enqueue-job",
|
||||
Type: JobTypeScan,
|
||||
Context: context.Background(),
|
||||
Params: map[string]interface{}{"library_id": "lib-manual"},
|
||||
}
|
||||
if err := w.EnqueueJob(tracked); err != nil {
|
||||
t.Fatalf("EnqueueJob failed: %v", err)
|
||||
}
|
||||
|
||||
for id, libID := range map[string]string{
|
||||
"via-enqueue": "lib-auto",
|
||||
"via-enqueue-job": "lib-manual",
|
||||
} {
|
||||
result, ok := w.GetJobStatus(id)
|
||||
if !ok {
|
||||
t.Fatalf("%s: no pending result recorded", id)
|
||||
}
|
||||
if result.Status != JobStatusPending {
|
||||
t.Errorf("%s: expected pending, got %q", id, result.Status)
|
||||
}
|
||||
if result.LibraryID != libID {
|
||||
t.Errorf("%s: expected library_id %q, got %q", id, libID, result.LibraryID)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A scan whose context is already cancelled must abort immediately
|
||||
// with context.Canceled — before counting, walking, or running the
|
||||
// archive pass. processJob classifies that error as a cancelled job.
|
||||
func TestScanFoldersCancelledContext(t *testing.T) {
|
||||
s := NewMediaScanner(nil)
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
|
||||
if err := s.ScanFolders(ctx); !errors.Is(err, context.Canceled) {
|
||||
t.Fatalf("expected context.Canceled, got %v", err)
|
||||
}
|
||||
}
|
||||
@@ -12,6 +12,7 @@ const (
|
||||
FormatGroupReflowable FormatGroup = "reflowable"
|
||||
FormatGroupFixedLayout FormatGroup = "fixed_layout"
|
||||
FormatGroupComicArchive FormatGroup = "comic_archive"
|
||||
FormatGroupAudiobook FormatGroup = "audiobook"
|
||||
FormatGroupUnknown FormatGroup = "unknown"
|
||||
)
|
||||
|
||||
|
||||
@@ -87,6 +87,15 @@ func ConvertProgress(sourceFormat, targetFormat FormatGroup, sourceData map[stri
|
||||
result["panel"] = panel
|
||||
}
|
||||
|
||||
case FormatGroupAudiobook:
|
||||
// Audio progress is percentage + global seconds; there is no page
|
||||
// grid to re-derive, so the percentage carries across as-is and
|
||||
// the second-based locator follows when present.
|
||||
result["percentage"] = percentage
|
||||
if offset, ok := sourceData["audio_offset_seconds"].(int); ok {
|
||||
result["audio_offset_seconds"] = offset
|
||||
}
|
||||
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported target format: %s", targetFormat)
|
||||
}
|
||||
@@ -108,7 +117,7 @@ func extractPercentage(sourceFormat FormatGroup, sourceData map[string]interface
|
||||
}
|
||||
}
|
||||
|
||||
case FormatGroupFixedLayout, FormatGroupComicArchive:
|
||||
case FormatGroupFixedLayout, FormatGroupComicArchive, FormatGroupAudiobook:
|
||||
if page, ok := sourceData["page"].(int); ok {
|
||||
if total, ok := sourceData["total_pages"].(int); ok && total > 0 {
|
||||
return float64(page) / float64(total)
|
||||
|
||||
@@ -20,6 +20,7 @@ const (
|
||||
MessageTypeScanProgress = "scan_progress"
|
||||
MessageTypeScanComplete = "scan_complete"
|
||||
MessageTypeScanError = "scan_error"
|
||||
MessageTypeScanCancelled = "scan_cancelled"
|
||||
)
|
||||
|
||||
// BroadcastMessage represents a message to broadcast to connected clients
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
package utils
|
||||
|
||||
// Genre display rendering (faceted plan Batch B / Phase 7): the
|
||||
// canonically-lowercase genre value is title-cased at DISPLAY time
|
||||
// only — never in the DB. Canonical home: the filter panel (services),
|
||||
// templ (templates wraps this), and the TS port in bookshelf.ts all
|
||||
// share these exact rules.
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// genreSmallWords stay lowercase in title-case rendering (except the
|
||||
// first word of the value).
|
||||
var genreSmallWords = map[string]bool{
|
||||
"a": true, "an": true, "and": true, "of": true, "the": true,
|
||||
"in": true, "on": true, "for": true, "to": true, "or": true,
|
||||
}
|
||||
|
||||
// PrettifyGenre renders the canonically-lowercase genre value in title
|
||||
// case at display time (Batch B transform — never in the DB). Only
|
||||
// values that are ENTIRELY lowercase get transformed; anything else is
|
||||
// returned verbatim, protecting user-entered mixed case. Small words
|
||||
// stay down except in the first position; "&" has no case and passes
|
||||
// through. "science fiction" → "Science Fiction", "sword & sorcery" →
|
||||
// "Sword & Sorcery".
|
||||
func PrettifyGenre(value string) string {
|
||||
trimmed := strings.TrimSpace(value)
|
||||
if trimmed == "" {
|
||||
return ""
|
||||
}
|
||||
if trimmed != strings.ToLower(trimmed) {
|
||||
return trimmed // mixed case = user-authored; display as written
|
||||
}
|
||||
words := strings.Fields(trimmed)
|
||||
out := make([]string, len(words))
|
||||
for i, w := range words {
|
||||
if i > 0 && genreSmallWords[w] {
|
||||
out[i] = w
|
||||
continue
|
||||
}
|
||||
runes := []rune(w)
|
||||
out[i] = string(append([]rune{unicode.ToUpper(runes[0])}, runes[1:]...))
|
||||
}
|
||||
return strings.Join(out, " ")
|
||||
}
|
||||
@@ -18,7 +18,7 @@ import (
|
||||
|
||||
const (
|
||||
OverrideTitle = "title"
|
||||
OverrideAuthor = "author"
|
||||
OverrideAuthor = "author" // protects the authors[] array (legacy key preserved)
|
||||
OverrideISBN = "isbn"
|
||||
OverrideDescription = "description"
|
||||
OverrideCoverImagePath = "cover_image_path"
|
||||
@@ -29,7 +29,9 @@ const (
|
||||
OverrideTags = "tags"
|
||||
OverrideAsin = "asin"
|
||||
OverrideDatePublished = "date_published"
|
||||
OverridePublisher = "publisher"
|
||||
OverrideSubtitle = "subtitle"
|
||||
OverrideExplicit = "explicit"
|
||||
OverridePublisher = "publisher" // protects the publishers[] array (legacy key preserved)
|
||||
OverrideContributors = "contributors"
|
||||
OverrideLanguage = "language"
|
||||
OverrideEdition = "edition"
|
||||
@@ -150,17 +152,19 @@ func detectMetadataOverridesDiff(params database.UpdateMediaItemParams, existing
|
||||
}
|
||||
|
||||
add(OverrideTitle, params.Title != existing.Title)
|
||||
add(OverrideAuthor, normText(params.Author) != normText(existing.Author))
|
||||
add(OverrideSubtitle, normText(params.Subtitle) != normText(existing.Subtitle))
|
||||
add(OverrideAuthor, normStringSlice(params.Authors) != normStringSlice(existing.Authors))
|
||||
add(OverrideISBN, normText(params.Isbn) != normText(existing.Isbn))
|
||||
add(OverrideDescription, normText(params.Description) != normText(existing.Description))
|
||||
add(OverrideSeries, normText(params.Series) != normText(existing.Series))
|
||||
add(OverrideSeriesNumber, normInt(params.SeriesNumber) != normInt(existing.SeriesNumber))
|
||||
add(OverrideSeriesNumber, normFloat(params.SeriesNumber) != normFloat(existing.SeriesNumber))
|
||||
add(OverrideTags, normStringSlice(params.Tags) != normStringSlice(existing.Tags))
|
||||
add(OverrideNarrators, normStringSlice(params.Narrators) != normStringSlice(existing.Narrators))
|
||||
add(OverrideAbridged, params.Abridged != existing.Abridged)
|
||||
add(OverrideExplicit, params.Explicit != existing.Explicit)
|
||||
add(OverrideAsin, normText(params.Asin) != normText(existing.Asin))
|
||||
add(OverrideDatePublished, normDate(params.DatePublished) != normDate(existing.DatePublished))
|
||||
add(OverridePublisher, normText(params.Publisher) != normText(existing.Publisher))
|
||||
add(OverridePublisher, normStringSlice(params.Publishers) != normStringSlice(existing.Publishers))
|
||||
add(OverrideContributors, normStringSlice(params.Contributors) != normStringSlice(existing.Contributors))
|
||||
add(OverrideLanguage, normText(params.Language) != normText(existing.Language))
|
||||
add(OverrideEdition, normText(params.Edition) != normText(existing.Edition))
|
||||
@@ -205,8 +209,12 @@ func ApplyMetadataOverrides(params *database.UpdateMediaItemParams, existing dat
|
||||
if hasOverride(o, OverrideTitle) {
|
||||
params.Title = existing.Title
|
||||
}
|
||||
if hasOverride(o, OverrideSubtitle) {
|
||||
params.Subtitle = existing.Subtitle
|
||||
}
|
||||
if hasOverride(o, OverrideAuthor) {
|
||||
params.Author = existing.Author
|
||||
params.Authors = existing.Authors
|
||||
params.AuthorsSearch = existing.AuthorsSearch
|
||||
}
|
||||
if hasOverride(o, OverrideISBN) {
|
||||
params.Isbn = existing.Isbn
|
||||
@@ -241,7 +249,8 @@ func ApplyMetadataOverrides(params *database.UpdateMediaItemParams, existing dat
|
||||
params.DatePublished = existing.DatePublished
|
||||
}
|
||||
if hasOverride(o, OverridePublisher) {
|
||||
params.Publisher = existing.Publisher
|
||||
params.Publishers = existing.Publishers
|
||||
params.PublishersSearch = existing.PublishersSearch
|
||||
}
|
||||
if hasOverride(o, OverrideContributors) {
|
||||
params.Contributors = existing.Contributors
|
||||
|
||||
@@ -63,12 +63,12 @@ func TestDetectMetadataOverridesUnsetFormsEqual(t *testing.T) {
|
||||
// Zero-value params must not look "changed" against NULL-ish columns.
|
||||
existing := database.MediaItems{
|
||||
PageCount: pgtype.Int4{Int32: 0, Valid: false},
|
||||
SeriesNumber: pgtype.Int4{Int32: 5, Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 5, Valid: true},
|
||||
CommunityRating: pgtype.Float8{Float64: 0, Valid: false},
|
||||
}
|
||||
params := database.UpdateMediaItemParams{
|
||||
PageCount: pgtype.Int4{Int32: 0, Valid: false},
|
||||
SeriesNumber: pgtype.Int4{Int32: 0, Valid: false}, // cleared by user -> changed
|
||||
SeriesNumber: pgtype.Float8{Float64: 0, Valid: false}, // cleared by user -> changed
|
||||
CommunityRating: pgtype.Float8{Float64: 0, Valid: false},
|
||||
}
|
||||
got := DetectMetadataOverrides(params, existing)
|
||||
|
||||
@@ -183,6 +183,21 @@ func NormalizeNarratorsSearch(narrators []string) []string {
|
||||
return NormalizeTagsSearch(narrators)
|
||||
}
|
||||
|
||||
// SingleOrSlice returns the write-truth array for a field that has both a
|
||||
// legacy single-value form and an array form: the explicit array when
|
||||
// populated, else the single string as a one-element slice. Used by the
|
||||
// authors/publishers arrays (`author`/`publisher` varchars regenerate from
|
||||
// them via the sync_author_publisher_display trigger).
|
||||
func SingleOrSlice(single string, arr []string) []string {
|
||||
if len(arr) > 0 {
|
||||
return arr
|
||||
}
|
||||
if single == "" {
|
||||
return []string{}
|
||||
}
|
||||
return []string{single}
|
||||
}
|
||||
|
||||
// NormalizeContributors normalizes an array of contributors for display:
|
||||
// 1. Trim whitespace
|
||||
// 2. Preserve original casing (including CAPSLOCK companies)
|
||||
|
||||
+12
-5
@@ -5,11 +5,12 @@
|
||||
"scripts": {
|
||||
"build:css": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --watch",
|
||||
"build:css:prod": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --minify",
|
||||
"build:ts": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vite build",
|
||||
"build:ts:dev": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vite build --mode development",
|
||||
"build:ts:watch": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vite build --watch",
|
||||
"build:ts": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vp build",
|
||||
"build:ts:dev": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vp build --mode development",
|
||||
"build:ts:watch": "cp node_modules/htmx.org/dist/htmx.min.js web/static/htmx.min.js && vp build --watch",
|
||||
"build": "npm run build:ts && npm run build:css:prod",
|
||||
"dev": "npm run build:ts:dev && npm run build:css"
|
||||
"dev": "npm run build:ts:dev && npm run build:css",
|
||||
"test": "vp test web/src"
|
||||
},
|
||||
"dependencies": {
|
||||
"@bookhoard/foliate-js": "git+https://git.linuxhg.com/Bookhoard/foliate-js.git#f622841",
|
||||
@@ -31,8 +32,14 @@
|
||||
"@types/markdown-it": "^14.2.0",
|
||||
"@types/turndown": "^5.0.6",
|
||||
"autoprefixer": "^10.4.27",
|
||||
"jsdom": "^30.1.3",
|
||||
"playwright": "^1.64.1",
|
||||
"tailwindcss": "^3.4.19",
|
||||
"typescript": "^5.9.3",
|
||||
"vite": "^7.3.1"
|
||||
"vite": "npm:@voidzero-dev/vite-plus-core@1.3.0",
|
||||
"vite-plus": "1.3.0"
|
||||
},
|
||||
"overrides": {
|
||||
"vite": "npm:@voidzero-dev/vite-plus-core@1.3.0"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
// Filter-builder UI regression harness (faceted plan Phase 9 follow-up).
|
||||
//
|
||||
// Drives the real bookshelf in headless Chromium: opens the filter
|
||||
// drawer, types an author value, and sweeps every author operator,
|
||||
// asserting the live count fetch behaves (200s, correct numbers, no
|
||||
// console errors). Born from the mis-keyed operator table that made
|
||||
// every non-self-named operator (doesn't contain, is empty, at least…)
|
||||
// a silent 400 — the class of bug only a real browser catches.
|
||||
//
|
||||
// Prerequisites: dev stack up (make dev / docker compose), playwright
|
||||
// installed (cd web && npm i -D playwright && npx playwright install
|
||||
// chromium — the repo root's devDependency already carries it), and a
|
||||
// reachable server.
|
||||
//
|
||||
// Usage: NODE_PATH=<repo>/node_modules node scripts/filter_builder_harness.js
|
||||
// Env: HARNESS_BASE (default http://localhost:8765), HARNESS_LOGIN,
|
||||
// HARNESS_PASSWORD.
|
||||
const { chromium } = require("playwright");
|
||||
|
||||
const BASE = process.env.HARNESS_BASE || "http://localhost:8765";
|
||||
const LOGIN = process.env.HARNESS_LOGIN || "testuser@tests.bookhoard.internal";
|
||||
const PASSWORD = process.env.HARNESS_PASSWORD || "Test@Pass123!";
|
||||
const AUTHOR = "Andrzej Sapkowski";
|
||||
const OPS_SWEEP = ["doesn't contain", "is", "isn't", "is empty", "contains"];
|
||||
|
||||
|
||||
(async () => {
|
||||
const browser = await chromium.launch();
|
||||
const context = await browser.newContext();
|
||||
const page = await context.newPage();
|
||||
|
||||
const events = [];
|
||||
page.on("console", (m) => {
|
||||
if (m.type() === "error" || m.type() === "warning") {
|
||||
events.push(`[console.${m.type()}] ${m.text()}`);
|
||||
}
|
||||
});
|
||||
page.on("pageerror", (e) => events.push(`[pageerror] ${e.message}`));
|
||||
page.on("response", async (r) => {
|
||||
if (r.url().includes("filter-count")) {
|
||||
let body = "";
|
||||
try { body = (await r.text()).slice(0, 120); } catch {}
|
||||
events.push(`[net] filter-count ${r.status()} ${body}`);
|
||||
}
|
||||
});
|
||||
|
||||
const login = await context.request.post(`${BASE}/api/auth/login`, {
|
||||
data: { login: LOGIN, password: PASSWORD },
|
||||
});
|
||||
const loginBody = await login.json();
|
||||
await context.addInitScript((t) => localStorage.setItem("token", t), loginBody.access_token);
|
||||
console.log("login:", login.status());
|
||||
|
||||
await page.goto(`${BASE}/bookshelf`);
|
||||
await page.waitForLoadState("networkidle");
|
||||
await page.evaluate(() => {
|
||||
const el = document.getElementById("library-select");
|
||||
const empty = [...el.options].find((o) => o.value === "");
|
||||
if (empty) {
|
||||
el.value = "";
|
||||
el.dispatchEvent(new Event("change", { bubbles: true }));
|
||||
}
|
||||
});
|
||||
await page.waitForTimeout(800);
|
||||
|
||||
await page.click('button:has-text("Filters")');
|
||||
await page.waitForTimeout(300);
|
||||
const valueInput = page.locator(".fb-value").first();
|
||||
await valueInput.click();
|
||||
await valueInput.fill(AUTHOR);
|
||||
await page.waitForTimeout(1200);
|
||||
const countEl = page.locator('span[x-show*="matchCount"]').first();
|
||||
console.log("contains:", JSON.stringify(await countEl.textContent()));
|
||||
|
||||
for (const label of OPS_SWEEP) {
|
||||
await page.locator(".fb-picker-op").first().click();
|
||||
await page.waitForTimeout(150);
|
||||
await page
|
||||
.locator(".fb-menu-item:visible")
|
||||
.filter({ hasText: new RegExp(`^${label.replace(/[\W_]+/g, "\\$&")}$`) })
|
||||
.first()
|
||||
.click();
|
||||
await page.waitForTimeout(1300);
|
||||
console.log(`after "${label}":`, JSON.stringify(await countEl.textContent()));
|
||||
}
|
||||
|
||||
// contains-with-empty-value is incomplete (counter hidden) — refill
|
||||
// and confirm the count returns.
|
||||
await valueInput.fill(AUTHOR);
|
||||
await page.waitForTimeout(1300);
|
||||
console.log("refilled contains:", JSON.stringify(await countEl.textContent()));
|
||||
|
||||
console.log("--- warnings/errors + count calls ---");
|
||||
for (const l of events) console.log(l);
|
||||
await browser.close();
|
||||
})().catch((e) => { console.error("HARNESS ERROR:", e.message); process.exit(1); });
|
||||
@@ -97,9 +97,14 @@ templ Admin(user User, stats AdminStats) {
|
||||
@Icon("refresh", "h-5 w-5")
|
||||
Scanning Libraries
|
||||
</h3>
|
||||
<button @click="hideScanProgress()" class="icon-btn" aria-label="Close">
|
||||
@Icon("close", "h-5 w-5")
|
||||
</button>
|
||||
<div class="flex items-center gap-2">
|
||||
<button id="scan-cancel-all-btn" @click="cancelAllScans()" class="hidden icon-btn" aria-label="Cancel all scans" title="Cancel all scans">
|
||||
@Icon("x-circle", "h-5 w-5")
|
||||
</button>
|
||||
<button @click="hideScanProgress()" class="icon-btn" aria-label="Close">
|
||||
@Icon("close", "h-5 w-5")
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mb-4">
|
||||
<div class="flex justify-between text-sm mb-2">
|
||||
|
||||
Loaded 100 of 221 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user