Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
253d8d0e8e | ||
|
|
acbcc61ae4 | ||
|
|
30766020c8 | ||
|
|
ffc80f7ba7 | ||
|
|
c753780817 | ||
|
|
8d9fb0046c | ||
|
|
5c10bbb90e | ||
|
|
49b0b6eeb9 | ||
|
|
28420d56ed | ||
|
|
2235b00c59 | ||
|
|
54d7d6e7a1 | ||
|
|
45b1152a15 | ||
|
|
a6723e6ac8 | ||
|
|
4288a86b2d | ||
|
|
85a9873a75 | ||
|
|
9a148c2344 | ||
|
|
c0e6993273 | ||
|
|
0b7ad0f376 | ||
|
|
4ab4f9fa90 | ||
|
|
304ea9599f | ||
|
|
cf73eec2e4 | ||
|
|
ab4bddc784 | ||
|
|
de26544d9d | ||
|
|
efbe312f16 | ||
|
|
d49a9e6ca2 | ||
|
|
313268a7ae | ||
|
|
1457dc7289 | ||
|
|
f579696633 | ||
|
|
4d697f5fcf | ||
|
|
bb2cc26e9d | ||
|
|
385a5d9dba | ||
|
|
922b92a8b6 | ||
|
|
8c3d919aa0 | ||
|
|
004d4197a4 | ||
|
|
f4b581872f | ||
|
|
a19d7a4cbf | ||
|
|
cb258b6d20 | ||
|
|
3dcb794032 | ||
|
|
2b122f2481 | ||
|
|
f1e4b2a603 | ||
|
|
795225f097 | ||
|
|
9f39e2efd8 | ||
|
|
7b9e8e4fc2 | ||
|
|
4c43caa639 | ||
|
|
fd9292e6f2 | ||
|
|
ffbcc53083 | ||
|
|
fb1645c90c | ||
|
|
96c1228805 | ||
|
|
14285ff314 | ||
|
|
e5d4744977 | ||
|
|
b462c7fb44 | ||
|
|
59eddc1386 | ||
|
|
eaf1b2c8dd | ||
|
|
61e5f47ec0 | ||
|
|
d1f9ad3b25 | ||
|
|
79f4d668f1 | ||
|
|
c71ef4af85 | ||
|
|
7c9e561e68 |
No files matched your search
+9
-1
@@ -6,10 +6,18 @@
|
||||
# Generate with: openssl rand -hex 32
|
||||
JWT_SECRET=your-secure-jwt-secret-key-here
|
||||
|
||||
# PostgreSQL database password
|
||||
# PostgreSQL database password (the bootstrap superuser account; kept
|
||||
# for container initialization — the app itself connects as bookhoard_srv)
|
||||
# Generate with: openssl rand -hex 16
|
||||
DBPASS=your-secure-database-password-here
|
||||
|
||||
# Runtime database role password (bookhoard_srv — the non-superuser
|
||||
# account the app connects as; subject to RLS). Required for the app to
|
||||
# start. Consumed by the one-shot db-init service on every startup
|
||||
# (idempotent — creates/refreshes the role and its privileges).
|
||||
# Generate with: openssl rand -hex 16
|
||||
DBSRVPASS=your-secure-runtime-role-password-here
|
||||
|
||||
# Networking: change a port if it conflicts on your host
|
||||
# Postgres port, host + container (e.g. another local DB already uses 5432)
|
||||
# DB_PORT=15432
|
||||
|
||||
@@ -80,7 +80,11 @@ refuses, **silently at build/test time**, two authoring patterns:
|
||||
guard means recomputing the hash (the failing test prints it); adding
|
||||
a *second* inline script means hashing it too and extending
|
||||
`cspSanctionedInlineScripts` in `templates/csp_source_test.go`.
|
||||
Everything else goes in a bundled `web/src` module.
|
||||
Everything else goes in a bundled `web/src` module. This includes
|
||||
partials emitted from Go handler strings — those use data attributes
|
||||
+ `web/src/auth_forms.ts` (post-swap redirect/stash) or htmx's
|
||||
`HX-Redirect`/`HX-Refresh` headers; `internal/middleware`'s
|
||||
Go-source scan is the tripwire for that vector.
|
||||
- **No native `on*=` handler attributes** (`onclick=`, `onerror=`,
|
||||
`onsubmit=`, `onchange=`, … — quote AND templ brace forms). Inline
|
||||
handlers never go through the hash path; hash-mode CSP refuses them
|
||||
@@ -102,30 +106,50 @@ inside quoted attribute values (gotcha #1 below), so dynamic
|
||||
## Library visibility (family-lens rule) — ALWAYS apply
|
||||
|
||||
Per-user library visibility (`library_visibility` table, default-ALLOW:
|
||||
every user sees every library unless an admin hides one) must be
|
||||
enforced on **every endpoint that returns media metadata or content**.
|
||||
It is NOT automatic — enforcement is per-handler.
|
||||
every user sees every library unless an admin hides one) is enforced by
|
||||
**Postgres RLS** — one source of truth: the labeled policy block in
|
||||
`database/schema/schema.sql` + the runtime role provisioned by the
|
||||
`db-init` compose service.
|
||||
|
||||
- **New endpoints that list/filter media or series** MUST filter by the
|
||||
requesting user's visible-library set, and MUST reject an explicit
|
||||
`library_id` outside that set with 403 (never silently pass).
|
||||
- Helpers (do not hand-roll):
|
||||
- `handlers`: `(*MediaHandler).visibleLibrarySet` /
|
||||
`(*SeriesHandler).visibleLibrarySet` — membership map + id slice.
|
||||
- `router`: `visibleLibraryIDsFor` — same for SSR routes.
|
||||
- `resolveLibrary` (helpers.go) only honors requested library ids
|
||||
inside the visible set.
|
||||
- SQL pattern: `AND (sqlc.narg('visible_library_ids')::uuid[] IS NULL OR
|
||||
library_id = ANY(sqlc.narg('visible_library_ids')::uuid[]))` — see
|
||||
`ListMediaItems`, `GetDistinctSeries`, `SearchMediaItems`.
|
||||
- Content endpoints (file bytes, page images) additionally gate per item
|
||||
like `ReaderHandler.ensureItemVisible` / the download endpoint.
|
||||
- Tests must pin visibility: hidden library → 403 on the endpoint, and
|
||||
absent from unscoped aggregates.
|
||||
- **The engine is the enforcement.** Every `media_items` read is
|
||||
filtered by the policy, which joins `library_visibility` on the
|
||||
per-transaction GUC `app.user_id`. New media-listing queries need
|
||||
NO visibility clause; new endpoints need NO visibility plumbing.
|
||||
- **The app declares identity; it never computes visibility.** All
|
||||
sqlc statements run through `ScopedDB`
|
||||
(`internal/database/scoped.go`), which declares `app.user_id` from
|
||||
the stdlib request context. Three bridge sites put it there:
|
||||
`router.go` JWT, `router.go` cookie/device, and
|
||||
`middleware/device_auth.go` (KOReader/Kobo sync devices). A media
|
||||
read on a detached `context.Background()` RAISES — thread the
|
||||
request context or the acting identity (see conflicts.go and
|
||||
analytics.go for the pattern).
|
||||
- **Background jobs declare the acting identity** before detached
|
||||
work: scan jobs carry the library owner / first-admin fallback
|
||||
(the job Context carries it — `ScanFolders` self-declares the
|
||||
configured admin), the hash backfill runs as first admin. The sync
|
||||
queue processor needs nothing (`sync_queue` is not RLS-governed).
|
||||
- **NEVER**: session-scoped `app.user_id` (pooled-connection identity
|
||||
bleed); a bypass flag; connecting the app as the bootstrap
|
||||
superuser (superusers bypass RLS unconditionally — `DATABASE_USER`
|
||||
stays `bookhoard_srv`; the `db-init` compose service provisions the
|
||||
role on every startup).
|
||||
- **schema.sql re-runs on EVERY boot** (executeSchema) — everything
|
||||
added to it must be idempotent (`DROP .. IF EXISTS` + `CREATE`).
|
||||
The migration tx declares a synthetic identity
|
||||
(`schemaMigrationIdentity`) because the schema's own dedup block
|
||||
reads `media_items`.
|
||||
- **Auditors**: `TestRlsInvariant` (API black-box — add new
|
||||
media-returning surfaces to it; loosening requires a
|
||||
docs/contract_pins.md entry) and the Playwright sweep (`tests/ui`,
|
||||
new pages/controls go into `tests/ui/COVERAGE.md`). Item-scoped
|
||||
gates may answer 404 as well as 403 (row invisible vs Go-level
|
||||
check — contract_pins 2026-10-29).
|
||||
|
||||
History: 2026-10-13 (`431fec0`) the series/media listing surfaces
|
||||
leaked hidden libraries to any authenticated account — series endpoints
|
||||
were name-only across ALL libraries. Fixed; do not regress.
|
||||
leaked hidden libraries; **2026-10-29 enforcement moved to RLS** — the
|
||||
dashboard section queries had silently never carried the clauses,
|
||||
and the invariant test went red → green and stays as the auditor.
|
||||
|
||||
## Tests
|
||||
|
||||
|
||||
+3
-1
@@ -41,12 +41,14 @@ RUN npm run build:ts
|
||||
# Declared after `COPY . .` so a version bump only invalidates this layer.
|
||||
# The release workflow passes the git tag, e.g. APP_VERSION=v1.5.
|
||||
ARG APP_VERSION=dev
|
||||
# Short commit hash of the built source (stale-container drift guard).
|
||||
ARG GIT_COMMIT=none
|
||||
|
||||
# Build Go binary (cached unless Go files or generated code changes)
|
||||
RUN --mount=type=cache,target=/root/go/pkg/mod \
|
||||
--mount=type=cache,target=/root/.cache/go-build \
|
||||
CGO_ENABLED=0 GOOS=linux go build -installsuffix cgo \
|
||||
-ldflags "-X bookhoard/internal/version.Version=${APP_VERSION}" \
|
||||
-ldflags "-X bookhoard/internal/version.Version=${APP_VERSION} -X bookhoard/internal/version.Commit=${GIT_COMMIT}" \
|
||||
-o main ./cmd/server
|
||||
|
||||
# Materialize the Go module cache into an image layer so the test-runner
|
||||
|
||||
@@ -11,6 +11,10 @@ endif
|
||||
# Override with: CONTAINER_RUNTIME=podman make rebuild-app
|
||||
CONTAINER_RUNTIME ?= $(shell command -v docker 2>/dev/null || command -v podman 2>/dev/null)
|
||||
|
||||
# Build stamp (stale-container drift guard): the rebuilt image carries
|
||||
# the source commit; the app logs it at boot and /api/health reports it.
|
||||
GIT_COMMIT ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo none)
|
||||
|
||||
# Dev compose stack: base prod file merged with the dev override (local build + tests).
|
||||
# Prod deploy does NOT use this — it runs plain `docker compose` against the base file only.
|
||||
COMPOSE := $(CONTAINER_RUNTIME) compose -f docker-compose.yml -f docker-compose.dev.yml
|
||||
|
||||
@@ -21,160 +21,165 @@ A modern self-hosted media library system built with Go, PostgreSQL, HTMX, and T
|
||||
- **Podman** (recommended) or Docker
|
||||
- **5 minutes** of your time
|
||||
|
||||
### Installation
|
||||
### Installation (no source checkout needed)
|
||||
|
||||
Create a directory on the Docker host. You only need the compose file,
|
||||
an `.env`, and an `uploads` folder — everything else rides the published
|
||||
image.
|
||||
|
||||
```bash
|
||||
# 1. Clone the repository
|
||||
git clone https://git.linuxhg.com/Bookhoard/bookhoard.git
|
||||
cd bookhoard
|
||||
|
||||
# 2. Set up environment
|
||||
cp .env.example .env
|
||||
# Generate secure passwords (no special characters):
|
||||
# JWT_SECRET: openssl rand -hex 32
|
||||
# DBPASS: openssl rand -hex 16
|
||||
# Edit .env with your generated values
|
||||
|
||||
# 3. Pull images and start the server
|
||||
docker compose pull
|
||||
docker compose up -d
|
||||
# Optionally pin a specific version: set IMAGE_TAG in .env (defaults to "latest")
|
||||
|
||||
# 4. Open your browser
|
||||
open http://localhost:8765
|
||||
mkdir bookhoard && cd bookhoard
|
||||
mkdir uploads
|
||||
```
|
||||
|
||||
The first user to register automatically becomes an admin.
|
||||
**`docker-compose.yml`** — copy this wholesale:
|
||||
|
||||
---
|
||||
```yaml
|
||||
services:
|
||||
# PostgreSQL Database
|
||||
db:
|
||||
image: postgres:15-alpine
|
||||
container_name: bookhoard_db
|
||||
environment:
|
||||
POSTGRES_DB: bookhoard
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: ${DBPASS}
|
||||
# PGPORT makes Postgres listen on DB_PORT (kept in sync with the host mapping + app's DATABASE_PORT)
|
||||
PGPORT: ${DB_PORT:-5432}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
ports:
|
||||
- "127.0.0.1:${DB_PORT:-5432}:${DB_PORT:-5432}"
|
||||
- "[::1]:${DB_PORT:-5432}:${DB_PORT:-5432}"
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U postgres"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
env_file:
|
||||
- .env
|
||||
|
||||
## 📖 Key Features
|
||||
# One-shot bootstrap: creates the bookhoard_srv runtime role and
|
||||
# transfers object ownership to it (RLS prerequisite). Idempotent —
|
||||
# safe on every startup; the app waits for its success. Passwords
|
||||
# accept inline values or .env, like the rest of this file.
|
||||
# NOTE: plpgsql $$ tags are written $$$$ — compose eats one dollar
|
||||
# per pair before the container sees the SQL.
|
||||
db-init:
|
||||
image: postgres:15-alpine
|
||||
container_name: bookhoard_db_init
|
||||
environment:
|
||||
PGHOST: db
|
||||
PGPORT: ${DB_PORT:-5432}
|
||||
PGUSER: postgres
|
||||
PGDATABASE: bookhoard
|
||||
PGPASSWORD: ${DBPASS}
|
||||
DBSRVPASS: ${DBSRVPASS}
|
||||
entrypoint: ["bash", "-c"]
|
||||
command:
|
||||
- |
|
||||
set -e
|
||||
: "${DBSRVPASS:?DBSRVPASS required in .env}"
|
||||
psql -v ON_ERROR_STOP=1 -v role_pass="$DBSRVPASS" <<'EOSQL'
|
||||
DO $$$$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'bookhoard_srv') THEN
|
||||
CREATE ROLE bookhoard_srv LOGIN;
|
||||
END IF;
|
||||
END $$$$;
|
||||
ALTER ROLE bookhoard_srv LOGIN PASSWORD :'role_pass';
|
||||
DO $$$$
|
||||
DECLARE r RECORD;
|
||||
BEGIN
|
||||
FOR r IN SELECT tablename FROM pg_tables WHERE schemaname = 'public' LOOP
|
||||
EXECUTE format('ALTER TABLE public.%I OWNER TO bookhoard_srv', r.tablename);
|
||||
END LOOP;
|
||||
FOR r IN SELECT sequencename FROM pg_sequences WHERE schemaname = 'public' LOOP
|
||||
EXECUTE format('ALTER SEQUENCE public.%I OWNER TO bookhoard_srv', r.sequencename);
|
||||
END LOOP;
|
||||
FOR r IN SELECT p.oid::regprocedure AS fn FROM pg_proc p
|
||||
JOIN pg_namespace n ON p.pronamespace = n.oid
|
||||
WHERE n.nspname = 'public' AND p.prokind = 'f' LOOP
|
||||
EXECUTE format('ALTER FUNCTION %s OWNER TO bookhoard_srv', r.fn);
|
||||
END LOOP;
|
||||
END $$$$;
|
||||
ALTER SCHEMA public OWNER TO bookhoard_srv;
|
||||
ALTER DATABASE bookhoard OWNER TO bookhoard_srv;
|
||||
EOSQL
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
|
||||
### Universal Cross-Platform Sync
|
||||
# Bookhoard Application
|
||||
# In production this image is pulled from the Gitea container registry.
|
||||
# Override IMAGE_TAG in .env to pin or rollback a specific version (defaults to "latest").
|
||||
app:
|
||||
image: git.linuxhg.com/bookhoard/bookhoard:${IMAGE_TAG:-latest}
|
||||
container_name: bookhoard
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# Database Configuration
|
||||
# Runtime role = bookhoard_srv (non-superuser OWNER, subject to RLS
|
||||
# via FORCE). POSTGRES_USER above is the bootstrap account ONLY —
|
||||
# the app must NEVER connect as it (superusers bypass RLS).
|
||||
DATABASE_HOST: db
|
||||
DATABASE_PORT: ${DB_PORT:-5432}
|
||||
DATABASE_USER: bookhoard_srv
|
||||
DATABASE_PASSWORD: ${DBSRVPASS}
|
||||
DATABASE_NAME: bookhoard
|
||||
|
||||
- **Real-Time Progress**: Turn a page on your e-reader, see it in your browser
|
||||
- **Format-Aware**: EPUB CFI, page numbers, percentages - all handled correctly
|
||||
- **Offline Queue**: Changes sync when you reconnect, priority-processed
|
||||
- **Conflict Resolution**: Smart handling when same book read on multiple devices
|
||||
- **Book Matching**: Automatic matching using SHA-256, ISBN, UUID
|
||||
- **OPDS Catalog**: Wireless book delivery to e-readers over Wi-Fi
|
||||
- **Format Conversion**: On-the-fly EPUB→KEPUB conversion (for upcoming native Kobo support)
|
||||
# Application Configuration
|
||||
JWT_SECRET: ${JWT_SECRET}
|
||||
SERVER_PORT: ${SERVER_PORT:-8765}
|
||||
# IMPORTANT: Device sync requires full URL with protocol
|
||||
# Local: http://localhost:8765
|
||||
# Local network: http://192.168.1.X:8765
|
||||
# Domain: https://bookhoard.example.com
|
||||
BASE_URL: ${BASE_URL:-http://localhost:8765}
|
||||
# Mark session cookies Secure; set true behind a TLS-terminating reverse proxy (Caddy/nginx/traefik)
|
||||
COOKIE_SECURE: ${COOKIE_SECURE:-false}
|
||||
|
||||
### Media Management
|
||||
# Rate Limiting Configuration
|
||||
TEST_MODE: ${TEST_MODE:-false}
|
||||
RATE_LIMIT_ENABLED: ${RATE_LIMIT_ENABLED:-true}
|
||||
REQUESTS_PER_MINUTE: ${REQUESTS_PER_MINUTE:-10}
|
||||
|
||||
- **Calibre Integration**: Automatic metadata import from Calibre `metadata.opf` sidecar files
|
||||
- **Smart Search**: Partial matching with fuzzy search fallback for typos
|
||||
- **Advanced Filtering**: Filter by author, series, genre, language, year, cover images
|
||||
- **Dynamic Sorting**: By title, author, date added, published date, page count, series
|
||||
- **Rich Metadata**: Title, author, series, publisher, ISBN, language, edition, tags
|
||||
- **5-Star Ratings**: Half-star precision (1-10 scale)
|
||||
- **Notes & Highlights**: Color-coded annotations with linked notes
|
||||
- **Usage Analytics**: Reading statistics, device usage, popular books
|
||||
# Conversion Service Configuration
|
||||
BOOKHOARD_CONVERSION_CACHE_DIR: ${BOOKHOARD_CONVERSION_CACHE_DIR:-/app/cache/kepub}
|
||||
BOOKHOARD_CONVERSION_TOOL: ${BOOKHOARD_CONVERSION_TOOL:-/usr/bin/kepubify}
|
||||
BOOKHOARD_CONVERSION_CACHE_TTL: ${BOOKHOARD_CONVERSION_CACHE_TTL:-24h}
|
||||
|
||||
### Smart Collections
|
||||
# Library Maintenance
|
||||
# Days an archived item (file missing from disk for 2+ scans) is kept,
|
||||
# with its reading history, before library scans purge it for good.
|
||||
# Set 0 to keep archived items until purged manually on the library admin page.
|
||||
ARCHIVE_RETENTION_DAYS: ${ARCHIVE_RETENTION_DAYS:-90}
|
||||
|
||||
- **Auto-Assign Rules**: Automatically add books based on genre, author, series, tags, language, publisher, year
|
||||
- **Device Shelf Mappings**: Map collections to device shelves (used by native Kobo sync, coming soon)
|
||||
- **Test Before Creating**: Preview which books match your rules
|
||||
# System timezone (fallback for server-side time operations)
|
||||
TZ: ${TZ:-UTC}
|
||||
ports:
|
||||
- "${SERVER_PORT:-8765}:${SERVER_PORT:-8765}"
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
db-init:
|
||||
condition: service_completed_successfully
|
||||
volumes:
|
||||
- ./uploads:/app/uploads
|
||||
- bookhoard_conversion_cache:/app/cache/kepub
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "curl -f http://localhost:${SERVER_PORT:-8765}/health || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 10s
|
||||
|
||||
### Library Organization
|
||||
|
||||
- **Multi-Library Support**: Ebooks, Comics, and Manga with type-specific file formats
|
||||
- **Multiple Folders**: Add multiple scanning folders per library
|
||||
- **Visibility Control**: Admins control which libraries each user can see
|
||||
- **Background Scanning**: Auto-scan with per-user frequency settings
|
||||
- **Watch Mode**: Real-time file system monitoring for instant updates
|
||||
|
||||
### Security
|
||||
|
||||
- **JWT Authentication**: Short-lived access tokens (1 hour) with refresh tokens (7 days)
|
||||
- **Strong Passwords**: Complexity requirements enforced (8+ chars, uppercase, lowercase, number, special)
|
||||
- **Account Lockout**: 5 failed attempts = 15-minute lockout
|
||||
- **Rate Limiting**: 10 requests/minute on auth endpoints
|
||||
- **Input Validation**: Comprehensive validation on all inputs
|
||||
- **No Passwords on Devices**: Web-based device approval with QR codes
|
||||
|
||||
---
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
### For Users & Self-Hosters
|
||||
|
||||
- **[docs/user/calibre-integration.md](docs/user/calibre-integration.md)** - Calibre library integration
|
||||
- **[docs/user/sync-guide.md](docs/user/sync-guide.md)** - Understanding and using universal sync
|
||||
- **[docs/user/devices/koreader-setup.md](docs/user/devices/koreader-setup.md)** - KOReader configuration
|
||||
- **[docs/user/devices/kobo-setup.md](docs/user/devices/kobo-setup.md)** - Kobo e-reader configuration (coming soon)
|
||||
- **[docs/user/user-guide.md](docs/user/user-guide.md)** - General user guide
|
||||
- **[docs/user/admin-guide.md](docs/user/admin-guide.md)** - Admin features and configuration
|
||||
- **[docs/user/settings-guide.md](docs/user/settings-guide.md)** - Settings and preferences
|
||||
|
||||
### For Developers
|
||||
|
||||
- **[docs/developer/api/api-reference.md](docs/developer/api/api-reference.md)** - Complete API documentation
|
||||
- **[docs/developer/android-app.md](docs/developer/android-app.md)** - Android app design & roadmap
|
||||
- **[docs/contributing/development.md](docs/contributing/development.md)** - Development workflow
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Supported Devices
|
||||
|
||||
| Platform | Sync | OPDS | Status |
|
||||
| ---------------- | ---- | ---- | ------------------------------------------------------------- |
|
||||
| **Web Browser** | ✅ | ✅ | Full support |
|
||||
| **KOReader** | ✅ | ✅ | Runs on Kindle, Kobo, PocketBook hardware |
|
||||
| **Kobo Devices** | 🚧 | 🚧 | Native Kobo sync coming soon (use KOReader on Kobo today) |
|
||||
| **Mobile Apps** | 🚧 | 🚧 | Native Android app in design ([docs](docs/developer/android-app.md)); iOS later |
|
||||
|
||||
---
|
||||
|
||||
## 💾 Network Storage (NAS / NFS / SMB)
|
||||
|
||||
Libraries on network mounts are supported for everyday use — serving,
|
||||
reading, progress, and annotation sync all work over the mount — with a
|
||||
few behaviors worth knowing:
|
||||
|
||||
- **Outage hysteresis.** When the mount drops, files miss one scan
|
||||
(marked missing, hidden) then a second (archived, still recoverable).
|
||||
When the mount returns — by path or by identical content elsewhere —
|
||||
everything is restored automatically: annotations, progress, and
|
||||
covers included.
|
||||
- **Retention purge.** Archived items older than `ARCHIVE_RETENTION_DAYS`
|
||||
(default **90**, `0` = never auto-purge) are deleted for good by
|
||||
library scans, annotations included. A NAS offline longer than the
|
||||
retention window loses those books' metadata on return.
|
||||
- **Deleting while the storage is down is refused.** Deleting a book
|
||||
cascades its annotations, so deletes are rejected with a clear error
|
||||
while the item's library storage is unreachable. Archive the item
|
||||
instead, or retry when the mount is back.
|
||||
- **Watch mode does not fire on network mounts** (inotify doesn't work
|
||||
over NFS/SMB). The periodic scan is the change detector.
|
||||
- **Latency** — folder reachability checks are cached ~30s; large
|
||||
libraries on slow mounts will still feel slower than local disk.
|
||||
|
||||
---
|
||||
|
||||
## 🛠 Tech Stack
|
||||
|
||||
- **Backend**: Go 1.25+ with Echo framework
|
||||
- **Database**: PostgreSQL 15+ with pgx v5
|
||||
- **Frontend**: HTMX + Tailwind CSS + Templ
|
||||
- **Auth**: JWT tokens with bcrypt password hashing
|
||||
- **Container**: Podman (Docker compatible)
|
||||
|
||||
---
|
||||
|
||||
## 🧪 Testing
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
make test-all
|
||||
|
||||
# Run integration tests (with test mode)
|
||||
make test-integration
|
||||
|
||||
# Run Bruno OpenCollection YAML API tests
|
||||
npm install -g @usebruno/cli
|
||||
bruno run
|
||||
# Named Volumes
|
||||
volumes:
|
||||
postgres_data:
|
||||
bookhoard_conversion_cache:
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -60,3 +60,13 @@ variables:
|
||||
name: book_sha256
|
||||
- secret: true
|
||||
name: book_uuid
|
||||
- secret: true
|
||||
name: podcasts_library_id
|
||||
- name: podcast_feed_url
|
||||
value: https://example.com/feed.xml
|
||||
- secret: true
|
||||
name: podcast_feed_id
|
||||
- secret: true
|
||||
name: podcast_episode_id
|
||||
- name: podcast_search_query
|
||||
value: episode
|
||||
@@ -0,0 +1,30 @@
|
||||
info:
|
||||
name: Add Podcast Feed
|
||||
type: http
|
||||
seq: 1
|
||||
|
||||
http:
|
||||
method: POST
|
||||
url: '{{base_url}}/api/podcasts/feeds'
|
||||
auth: inherit
|
||||
body:
|
||||
type: json
|
||||
data: |-
|
||||
{
|
||||
"library_id": "{{podcasts_library_id}}",
|
||||
"url": "{{podcast_feed_url}}",
|
||||
"download_all": false
|
||||
}
|
||||
settings:
|
||||
encodeUrl: true
|
||||
timeout: 30000
|
||||
followRedirects: true
|
||||
maxRedirects: 5
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 201", function() {
|
||||
expect(res.status).to.eql(201);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Delete Feed (admin; ?files=false keeps files)
|
||||
type: http
|
||||
seq: 12
|
||||
|
||||
http:
|
||||
method: DELETE
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Download All (every non-downloaded episode)
|
||||
type: http
|
||||
seq: 8
|
||||
|
||||
http:
|
||||
method: POST
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/episodes/all/download'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 202", function() {
|
||||
expect(res.status).to.eql(202);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Download / Retry Episode (idempotent state machine)
|
||||
type: http
|
||||
seq: 7
|
||||
|
||||
http:
|
||||
method: POST
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/episodes/{{podcast_episode_id}}/download'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 202", function() {
|
||||
expect(res.status).to.eql(202);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Feed Detail (episodes + download states)
|
||||
type: http
|
||||
seq: 3
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200 (404 would be scoping/RLS)", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: List My Podcast Feeds
|
||||
type: http
|
||||
seq: 2
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: '{{base_url}}/api/podcasts/feeds'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Manual Refresh (poll one feed)
|
||||
type: http
|
||||
seq: 6
|
||||
|
||||
http:
|
||||
method: POST
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/refresh'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Effective Sequence (override applied, downloaded-only, gaps)
|
||||
type: http
|
||||
seq: 9
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/sequence'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,23 @@
|
||||
info:
|
||||
name: Set Order Override (native | oldest | newest)
|
||||
type: http
|
||||
seq: 10
|
||||
|
||||
http:
|
||||
method: PUT
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/order'
|
||||
auth: inherit
|
||||
body:
|
||||
type: json
|
||||
data: |-
|
||||
{
|
||||
"order": "oldest"
|
||||
}
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Subscription-scoped Search (covers un-downloaded + historical)
|
||||
type: http
|
||||
seq: 11
|
||||
|
||||
http:
|
||||
method: GET
|
||||
url: '{{base_url}}/api/podcasts/search?q={{podcast_search_query}}'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
info:
|
||||
name: Unsubscribe (files retained)
|
||||
type: http
|
||||
seq: 5
|
||||
|
||||
http:
|
||||
method: DELETE
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/subscription'
|
||||
auth: inherit
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
info:
|
||||
name: Update Subscription (display name + auto-download)
|
||||
type: http
|
||||
seq: 4
|
||||
|
||||
http:
|
||||
method: PUT
|
||||
url: '{{base_url}}/api/podcasts/feeds/{{podcast_feed_id}}/subscription'
|
||||
auth: inherit
|
||||
body:
|
||||
type: json
|
||||
data: |-
|
||||
{
|
||||
"display_name": "My Serialized Feed",
|
||||
"auto_download": "newest",
|
||||
"auto_download_count": 3
|
||||
}
|
||||
|
||||
runtime:
|
||||
scripts:
|
||||
- type: tests
|
||||
code: |-
|
||||
test("status is 200", function() {
|
||||
expect(res.status).to.eql(200);
|
||||
});
|
||||
+21
-2
@@ -10,6 +10,7 @@ import (
|
||||
"bookhoard/internal/router"
|
||||
"bookhoard/internal/services"
|
||||
"bookhoard/internal/sync"
|
||||
"bookhoard/internal/version"
|
||||
"bookhoard/templates"
|
||||
"context"
|
||||
"log"
|
||||
@@ -33,6 +34,7 @@ func (cv *CustomValidator) Validate(i interface{}) error {
|
||||
}
|
||||
|
||||
func main() {
|
||||
log.Printf("Bookhoard %s (commit %s) starting", version.Version, version.Commit)
|
||||
cfg := config.LoadConfig()
|
||||
|
||||
dbPool, err := pgxpool.New(context.Background(), cfg.DatabaseURL())
|
||||
@@ -41,7 +43,12 @@ func main() {
|
||||
}
|
||||
defer dbPool.Close()
|
||||
|
||||
queries := database.New(dbPool)
|
||||
// The search service's raw DBTX path must ALSO carry the RLS
|
||||
// identity — hand it the ScopedDB itself (a DBTX), never the bare
|
||||
// pool, or search queries bypass the per-user GUC and raise under
|
||||
// the policies.
|
||||
scopedDB := database.NewScopedDB(dbPool)
|
||||
queries := database.New(scopedDB)
|
||||
|
||||
// Initialize database schema
|
||||
log.Println("🔧 Ensuring database schema is initialized...")
|
||||
@@ -53,6 +60,9 @@ func main() {
|
||||
|
||||
// Load tunable settings from the DB into the registry. All values fall back
|
||||
// to compiled defaults if a row is missing, so this never blocks startup.
|
||||
// (Startup has no end-user identity; system_settings is not RLS-
|
||||
// governed, and ScopedDB declares nothing when no identity exists —
|
||||
// the engine only raises on governed tables.)
|
||||
registry := database.NewSettingsRegistry(queries)
|
||||
if err := registry.Load(ctx); err != nil {
|
||||
log.Printf("⚠️ Could not load system settings (using defaults): %v", err)
|
||||
@@ -165,10 +175,16 @@ func main() {
|
||||
dashboardHandler := handlers.NewDashboardHandler(queries)
|
||||
seriesHandler := handlers.NewSeriesHandler(queries)
|
||||
filtersHandler := handlers.NewFiltersHandler(queries)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, dbPool, libraryService, progressService, annotationService)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, scopedDB, libraryService, progressService, annotationService)
|
||||
matchingHandler := handlers.NewMatchingHandler(queries, connManager)
|
||||
jobsHandler := handlers.NewJobsHandler(queries, worker)
|
||||
|
||||
// Podcast subsystem: fetcher service + sequential download worker +
|
||||
// poller. One instance trio per server (per-feed locks dedupe).
|
||||
podcastService := services.NewPodcastService(queries)
|
||||
podcastWorker := services.NewPodcastDownloadWorker(queries, podcastService)
|
||||
podcastHandler := handlers.NewPodcastHandler(queries, podcastService, podcastWorker)
|
||||
|
||||
e := echo.New()
|
||||
|
||||
// Set up validator
|
||||
@@ -229,6 +245,9 @@ func main() {
|
||||
CollectionHandler: collectionHandler,
|
||||
FiltersHandler: filtersHandler,
|
||||
DashboardHandler: dashboardHandler,
|
||||
PodcastHandler: podcastHandler,
|
||||
PodcastService: podcastService,
|
||||
PodcastWorker: podcastWorker,
|
||||
DashboardService: dashboardService,
|
||||
SeriesHandler: seriesHandler,
|
||||
OPDSHandler: opdsHandler,
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
package main
|
||||
|
||||
// Regression tests for the archive reconciliation pass: a multi-folder
|
||||
// library must be reconciled against the UNION of its folders' walks, not
|
||||
// once per folder. The old per-folder loop listed every library item
|
||||
// against each single folder's walk, so with two registered folders every
|
||||
// item living under the other folder was flagged "missing (1/2)" on every
|
||||
// scan (prod: 743/743 audiobooks stuck at count 1 — unarchivable only
|
||||
// because each scan's walk kept clearing them), and with three or more
|
||||
// folders the crossed marks would have reached the archive threshold
|
||||
// mid-pass and archived real content. Folder-book roots (directory
|
||||
// file_path) must also count as present across folder boundaries — the
|
||||
// file walk never emits a directory.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/services"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// scanTestFolders runs a synchronous scan over several folders — the
|
||||
// multi-folder shape the archive pass must get right (scanTestFolder
|
||||
// covers the single-folder case).
|
||||
func scanTestFolders(t *testing.T, setup *TestServerSetup, adminID pgtype.UUID, folders []string) {
|
||||
t.Helper()
|
||||
scanner := services.NewMediaScanner(setup.DB)
|
||||
defer func() { _ = scanner.Close() }()
|
||||
scanner.SetAdminID(adminID)
|
||||
require.NoError(t, scanner.SetFolders(folders, false))
|
||||
require.NoError(t, scanner.ScanFolders(context.Background()))
|
||||
}
|
||||
|
||||
func TestArchivePassMultiFolderLibraryKeepsEverythingPresent(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
t.Cleanup(func() { _ = setup.Close() })
|
||||
|
||||
libID, folderA := createTypedTestLibrary(t, setup, fmt.Sprintf("Test Archive MultiFolder %d", time.Now().UnixNano()), "audiobooks")
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) }) // after setup.Close defer (LIFO): pool still open
|
||||
|
||||
// Second registered folder in the SAME library — the shape the
|
||||
// per-folder reconciliation got wrong.
|
||||
folderB := t.TempDir()
|
||||
_, err := setup.DB.AddLibraryFolder(context.Background(), database.AddLibraryFolderParams{
|
||||
LibraryID: libID,
|
||||
FolderPath: folderB,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
admin, _ := moveTestAdminID(t, setup)
|
||||
|
||||
// Folder A holds a lone single-file book; folder B holds a
|
||||
// folder-of-tracks book whose file_path is a DIRECTORY.
|
||||
writeTestTrack(t, filepath.Join(folderA, "Author A", "Lone Book", "lonely.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folderB, "Author B", "Folder Book", "01 First.mp3"))
|
||||
writeTestTrack(t, filepath.Join(folderB, "Author B", "Folder Book", "02 Second.mp3"))
|
||||
|
||||
// Two consecutive scans: the first proves no flags are raised, the
|
||||
// second proves the flag/walk bounce (clear → re-flag) is gone too.
|
||||
for scanNo := 1; scanNo <= 2; scanNo++ {
|
||||
scanTestFolders(t, setup, admin, []string{folderA, folderB})
|
||||
|
||||
items := listLibraryAudioItems(t, setup, libID)
|
||||
var singleFile, folderBook bool
|
||||
for _, item := range items {
|
||||
assert.False(t, item.ArchivedAt.Valid, "scan %d: item %q must not be archived", scanNo, item.FilePath)
|
||||
assert.Equal(t, int32(0), item.MissingScanCount, "scan %d: item %q must not be flagged missing", scanNo, item.FilePath)
|
||||
switch item.FilePath {
|
||||
case "Author A/Lone Book/lonely.mp3":
|
||||
singleFile = true
|
||||
case "Author B/Folder Book":
|
||||
folderBook = true
|
||||
}
|
||||
}
|
||||
require.True(t, singleFile, "scan %d: single-file book must exist", scanNo)
|
||||
require.True(t, folderBook, "scan %d: folder book must exist at its directory path", scanNo)
|
||||
require.Len(t, items, 2, "scan %d: exactly the two books, no per-track strays", scanNo)
|
||||
}
|
||||
}
|
||||
@@ -25,6 +25,7 @@ import (
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
@@ -288,7 +289,12 @@ func TestAudiobookTrackEndpoints(t *testing.T) {
|
||||
resp, err := client.Do(req)
|
||||
require.NoError(t, err)
|
||||
_ = resp.Body.Close()
|
||||
require.Equal(t, http.StatusForbidden, resp.StatusCode, "%s must be gated", path)
|
||||
// PIN WIDENED for RLS (docs/contract_pins.md 2026-10-29): the
|
||||
// handler's own Go-level gate still answers 403, but under
|
||||
// engine-level visibility the hidden row is simply ABSENT —
|
||||
// 404 is equally gated (no existence leak). Both accepted.
|
||||
require.True(t, resp.StatusCode == http.StatusForbidden || resp.StatusCode == http.StatusNotFound,
|
||||
"%s must be gated (403 or 404), got %d", path, resp.StatusCode)
|
||||
}
|
||||
|
||||
// Restore visibility so harness cleanup ordering never sees a
|
||||
@@ -411,7 +417,12 @@ func TestAudiobookTagFallbacks(t *testing.T) {
|
||||
require.Len(t, item.Narrators, 1, "composer tag fills narrators (Audible convention)")
|
||||
require.Equal(t, "Tag Narrator", item.Narrators[0])
|
||||
require.Equal(t, "Tag Series", item.Series.String, "grouping tag fills series")
|
||||
require.Equal(t, "Audiobook", item.Genre.String, "genre tag fills genre")
|
||||
// 885d6140 demoted format words ("Audiobook") to tags-only: the
|
||||
// genre column stays empty and the raw candidate lands in tags.
|
||||
// (The commit message declares this test's update; the edit itself
|
||||
// never landed — pinned to the demoted contract now.)
|
||||
require.Empty(t, item.Genre.String, "demoted format words never win the genre column")
|
||||
assert.Contains(t, item.Tags, "Audiobook", "the demoted candidate lands as a tag")
|
||||
require.False(t, item.DatePublished.Time.IsZero(), "date tag fills publish date")
|
||||
require.Equal(t, 2020, item.DatePublished.Time.Year())
|
||||
// ASIN/abridged/subtitle arrive only through Audible freeform atoms,
|
||||
|
||||
@@ -2,7 +2,6 @@ package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
@@ -367,8 +366,6 @@ func TestPreviewCollection(t *testing.T) {
|
||||
}
|
||||
|
||||
func createPreviewTestMediaItems(t *testing.T, db *database.Queries, libraryID string, count int) []string {
|
||||
ctx := context.Background()
|
||||
|
||||
libUUID, err := uuid.Parse(libraryID)
|
||||
require.NoError(t, err, "Should parse library ID")
|
||||
|
||||
@@ -379,7 +376,7 @@ func createPreviewTestMediaItems(t *testing.T, db *database.Queries, libraryID s
|
||||
for i := 0; i < count; i++ {
|
||||
genre := genres[i%len(genres)]
|
||||
|
||||
media, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := db.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: "Preview Test Book " + string(rune('A'+i)),
|
||||
Authors: []string{"Test Author " + string(rune('A'+i))},
|
||||
|
||||
@@ -108,7 +108,7 @@ func smartCreateLibraryAndBooks(t *testing.T, setup *TestServerSetup, userID uui
|
||||
})
|
||||
|
||||
for title, genre := range titlesByGenre {
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: title,
|
||||
Authors: []string{"Smart Rule Author"},
|
||||
|
||||
@@ -113,7 +113,7 @@ func TestComicMetadataDisplay_ComicDetailPage(t *testing.T) {
|
||||
params.AlternateInfo = []byte(v)
|
||||
}
|
||||
|
||||
mediaItem, err := setup.DB.CreateMediaItem(ctx, params)
|
||||
mediaItem, err := setup.DB.CreateMediaItem(seedCtx(), params)
|
||||
require.NoError(t, err, "Should create media item")
|
||||
|
||||
mediaUUID, err := uuid.FromBytes(mediaItem.ID.Bytes[0:16])
|
||||
@@ -596,7 +596,7 @@ func TestComicMetadataDisplay_ComicDetailPage(t *testing.T) {
|
||||
Isbn: pgtype.Text{String: "9781234567890", Valid: true},
|
||||
}
|
||||
|
||||
mediaItem, err := setup.DB.CreateMediaItem(ctx, params)
|
||||
mediaItem, err := setup.DB.CreateMediaItem(seedCtx(), params)
|
||||
require.NoError(t, err, "Should create ebook")
|
||||
|
||||
mediaUUID, err := uuid.FromBytes(mediaItem.ID.Bytes[0:16])
|
||||
@@ -740,7 +740,7 @@ func TestComicMetadataDisplay_AllFieldsTogether(t *testing.T) {
|
||||
// Create a media item with ALL metadata fields populated
|
||||
alternateInfoStr := `{"alternate_series":"Ultimate X-Men","alternate_number":1,"alternate_count":12}`
|
||||
var mediaItem database.MediaItems
|
||||
mediaItem, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
mediaItem, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: "Complete Metadata Comic",
|
||||
FilePath: "/test/complete.cbz",
|
||||
|
||||
@@ -25,7 +25,7 @@ func TestComicMetadataExtraction(t *testing.T) {
|
||||
require.NoError(t, err, "Should parse library UUID")
|
||||
|
||||
// Insert test media item with full comic metadata
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: "Test Manga",
|
||||
FilePath: "/test/manga.cbz",
|
||||
@@ -64,7 +64,7 @@ func TestComicMetadataExtraction(t *testing.T) {
|
||||
parsedLibraryID, err := uuid.Parse(libraryID)
|
||||
require.NoError(t, err, "Should parse library UUID")
|
||||
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: "Test Comic",
|
||||
FilePath: "/test/comic.cbz",
|
||||
@@ -94,7 +94,7 @@ func TestComicMetadataExtraction(t *testing.T) {
|
||||
parsedLibraryID, err := uuid.Parse(libraryID)
|
||||
require.NoError(t, err, "Should parse library UUID")
|
||||
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: "Minimal Comic",
|
||||
FilePath: "/test/minimal.cbz",
|
||||
@@ -136,7 +136,7 @@ func TestReadingDirectionAPI(t *testing.T) {
|
||||
}
|
||||
|
||||
for _, tc := range testCases {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: tc.title,
|
||||
FilePath: "/test/" + tc.title + ".cbz",
|
||||
@@ -205,7 +205,7 @@ func TestUniversalMetadataFields(t *testing.T) {
|
||||
require.NoError(t, err, "Should parse ebook library UUID")
|
||||
|
||||
t.Run("Comic with universal fields", func(t *testing.T) {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedComicLibraryID, Valid: true},
|
||||
Title: "Comic with Universal Metadata",
|
||||
FilePath: "/test/comic.cbz",
|
||||
@@ -231,7 +231,7 @@ func TestUniversalMetadataFields(t *testing.T) {
|
||||
})
|
||||
|
||||
t.Run("Ebook with universal fields", func(t *testing.T) {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedEbookLibraryID, Valid: true},
|
||||
Title: "Ebook with Universal Metadata",
|
||||
FilePath: "/test/book.epub",
|
||||
@@ -270,7 +270,7 @@ func TestComicSpecificFields(t *testing.T) {
|
||||
t.Run("Alternate series info as JSONB", func(t *testing.T) {
|
||||
alternateInfo := `{"alternate_series":"Ultimate X-Men","alternate_number":1,"alternate_count":12}`
|
||||
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: parsedLibraryID, Valid: true},
|
||||
Title: "X-Men with Alternate Series",
|
||||
FilePath: "/test/xmen.cbz",
|
||||
|
||||
@@ -120,7 +120,7 @@ func createContractFixture(t *testing.T, setup *TestServerSetup) contractFixture
|
||||
|
||||
seriesName := fmt.Sprintf("Test Contract Series %d", time.Now().UnixNano())
|
||||
itemTitle := fmt.Sprintf("Test Contract Item %d", time.Now().UnixNano())
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: itemTitle,
|
||||
Authors: []string{"Test Contract Author"},
|
||||
@@ -452,9 +452,13 @@ func TestContractPin_KoboSyncFromServer(t *testing.T) {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// bookInfoKeys is the BookInfo JSON every card surface emits.
|
||||
// 2026-10-27 (search-progress batch): + viewer_percentage (nil = no
|
||||
// viewer data; only the search-unified adapter populates it today)
|
||||
// and + library_name (same deal). docs/contract_pins.md changelog.
|
||||
var bookInfoKeys = []string{
|
||||
"media_item_id", "title", "author", "cover_image_path",
|
||||
"has_conflict", "format_group", "badge",
|
||||
"viewer_percentage", "library_name",
|
||||
}
|
||||
|
||||
func collectionURL(setup *TestServerSetup, id string) string {
|
||||
@@ -667,6 +671,12 @@ func TestContractPin_SearchJSON(t *testing.T) {
|
||||
// (docs/contract_pins.md changelog).
|
||||
"subtitle", "explicit", "authors", "authors_search",
|
||||
"publishers", "publishers_search", "narrators_search",
|
||||
// 2026-10-27 (search-progress batch): + viewer_percentage —
|
||||
// the requester's reading_progress.percentage (LEFT JOIN on
|
||||
// the same user_id param the visibility join binds), null
|
||||
// when they never opened the book (docs/contract_pins.md
|
||||
// changelog).
|
||||
"viewer_percentage",
|
||||
}
|
||||
|
||||
url := fmt.Sprintf("%s/api/media-items/search?q=%s&library_id=%s", setup.Server.URL, url.QueryEscape(fx.itemTitle), fx.libraryID)
|
||||
@@ -742,6 +752,15 @@ func TestContractPin_MediaListBothBranches(t *testing.T) {
|
||||
"tags_search", "contributors_search", "narrators_search", "file_sha256",
|
||||
"opf_identifier", "opf_uuid", "hash_confidence", "library_name",
|
||||
"library_type_name_2",
|
||||
// Podcast episode columns (2026-10-29 additive widening — plan
|
||||
// podcast-integration.md tasks 1.1/1.10): columns on
|
||||
// media_items, NULL for scan rows. See docs/contract_pins.md
|
||||
// changelog.
|
||||
"source", "source_feed_id", "episode_guid", "feed_position",
|
||||
"season_number", "episode_number", "enclosure_url",
|
||||
"declared_duration_seconds", "transcript_url", "chapters_url",
|
||||
"hosts", "guests", "download_state", "download_error",
|
||||
"download_attempts",
|
||||
}
|
||||
resolvedKeys := []string{
|
||||
"id", "library_id", "title", "subtitle", "author", "authors", "isbn", "description",
|
||||
|
||||
@@ -89,7 +89,7 @@ func TestFacetRegistryParamsFilter(t *testing.T) {
|
||||
if mutate != nil {
|
||||
mutate(&p)
|
||||
}
|
||||
_, err := setup.DB.CreateMediaItem(ctx, p)
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), p)
|
||||
require.NoError(t, err, "create %s", title)
|
||||
}
|
||||
|
||||
@@ -194,7 +194,7 @@ func TestFacetsEndpoint(t *testing.T) {
|
||||
|
||||
// Two items share the genre; one carries the language + narrator.
|
||||
for _, title := range []string{"Facet List One", "Facet List Two"} {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: title,
|
||||
Authors: []string{"Facet Listing Author"},
|
||||
@@ -206,7 +206,7 @@ func TestFacetsEndpoint(t *testing.T) {
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Facet List Three",
|
||||
Authors: []string{"Facet Listing Author"},
|
||||
@@ -318,7 +318,6 @@ func mergeParams(base, extra url.Values) url.Values {
|
||||
// "Baen Books") cross-match at 0.3+ (session gotcha).
|
||||
func facetFixtureBooks(t *testing.T, setup *TestServerSetup, libraryID pgtype.UUID, prefix string) {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
books := []struct {
|
||||
title string
|
||||
genre string
|
||||
@@ -329,7 +328,7 @@ func facetFixtureBooks(t *testing.T, setup *TestServerSetup, libraryID pgtype.UU
|
||||
{prefix + " Gamma", "history", "Pinecrest"},
|
||||
}
|
||||
for _, b := range books {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: libraryID,
|
||||
Title: b.title,
|
||||
Authors: []string{prefix + " Author"},
|
||||
@@ -426,7 +425,7 @@ func TestFacetCountsUnderFilters(t *testing.T) {
|
||||
facetFixtureBooks(t, setup, library.ID, "Counts")
|
||||
// A book in the HIDDEN library: distinctive values that must never
|
||||
// surface for the regular user.
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: hiddenLib.ID,
|
||||
Title: "Counts Hidden Book",
|
||||
Authors: []string{"Counts Secret Author"},
|
||||
@@ -559,7 +558,7 @@ func TestFieldValuesAllLibraries(t *testing.T) {
|
||||
id pgtype.UUID
|
||||
name string
|
||||
}{{visibleLib.ID, "Visible"}, {hiddenLib.ID, "Hidden"}} {
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: lib.id,
|
||||
Title: fmt.Sprintf("Field Values %s Book", lib.name),
|
||||
Authors: []string{"Verdana Scribe"},
|
||||
|
||||
@@ -156,7 +156,7 @@ func builderFixtureLib(t *testing.T, setup *TestServerSetup, userID uuid.UUID, n
|
||||
}
|
||||
books[i].TagsSearch = lowered
|
||||
}
|
||||
_, err := setup.DB.CreateMediaItem(ctx, books[i])
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), books[i])
|
||||
require.NoError(t, err, "create %s", books[i].Title)
|
||||
}
|
||||
return libUUID, library.ID
|
||||
@@ -267,11 +267,9 @@ func TestFilterComposedSemantics(t *testing.T) {
|
||||
assert.Len(t, got, 4, "mime_type is")
|
||||
|
||||
// Series full membership: the alternate_info series matches too.
|
||||
ctx := context.Background()
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`UPDATE media_items SET alternate_info = '{"alternate_series":"Quill Chronicles"}'::jsonb
|
||||
WHERE title = $1`, "Sem Fen")
|
||||
require.NoError(t, err)
|
||||
got = titles(qfAll(qfRule("series", "contains", "Quill Chron")))
|
||||
assert.Equal(t, []string{"Sem Fen"}, got, "series matches alternate_info membership")
|
||||
got = titles(qfAll(qfRule("series", "is", "Quill Chronicles")))
|
||||
@@ -327,43 +325,34 @@ func TestFilterUserRelativeFields(t *testing.T) {
|
||||
// Reader A: reading Tide (50%), finished Stone, rated Tide 8,
|
||||
// reviewed Fen, noted on Tide ("the catharsis scene"), highlighted
|
||||
// Stone, time on Wick (2h), collection holding Tide.
|
||||
_, err := setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at) VALUES ($1,$2,0.5, now()), ($3,$4,1.0, now())`,
|
||||
tideID, aID, stoneID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at) VALUES ($1,$2,1.0, now())`,
|
||||
tideID, bID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO media_ratings (media_item_id, user_id, rating) VALUES ($1,$2,8)`, tideID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO media_ratings (media_item_id, user_id, rating) VALUES ($1,$2,3)`, tideID, bID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO media_reviews (media_item_id, user_id, review) VALUES ($1,$2,'splendid')`, fenID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO media_notes (media_item_id, user_id, content) VALUES ($1,$2,'remember the catharsis scene')`, tideID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO media_highlights (media_item_id, user_id, selection_text) VALUES ($1,$2,'the dark forest road')`, stoneID, aID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO reading_history (user_id, media_item_id, time_spent_seconds, pages_read) VALUES ($1,$2,7200,120), ($3,$4,600,30)`,
|
||||
aID, wickID, aID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
var collID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx,
|
||||
err := setup.DBPool.QueryRow(context.Background(),
|
||||
`INSERT INTO collections (user_id, name) VALUES ($1, $2) RETURNING id`,
|
||||
aID, fmt.Sprintf("Test Builder Coll %d", time.Now().UnixNano())).Scan(&collID)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteCollection(ctx, collID) })
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO collection_items (collection_id, media_item_id) VALUES ($1,$2)`, collID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
titlesFor := func(token string, qf string) []string {
|
||||
p := url.Values{"library_id": {libUUID.String()}, "qf": {qf}}
|
||||
@@ -445,7 +434,7 @@ func TestFilterSuggestionsUnderRules(t *testing.T) {
|
||||
|
||||
// Add a second distinct author+genre pair (trigram-disjoint).
|
||||
ctx := context.Background()
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: libUUID, Valid: true},
|
||||
Title: "Sug Dune",
|
||||
Authors: []string{"Zephyrine Hale"},
|
||||
@@ -572,18 +561,17 @@ func TestFilterSuggestionsUnderRules(t *testing.T) {
|
||||
// offered with the surviving count; zero-match containers never are.
|
||||
userA := getRegularUserID(t, setup.DB)
|
||||
var collID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx,
|
||||
err = setup.DBPool.QueryRow(context.Background(),
|
||||
`INSERT INTO collections (user_id, name) VALUES ($1, $2) RETURNING id`,
|
||||
pgtype.UUID{Bytes: [16]byte(userA), Valid: true},
|
||||
fmt.Sprintf("Test Builder Sug Coll %d", time.Now().UnixNano())).Scan(&collID)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteCollection(ctx, collID) })
|
||||
var tideID pgtype.UUID
|
||||
err = setup.DBPool.QueryRow(ctx, `SELECT id FROM media_items WHERE title = $1`, "Sug Tide").Scan(&tideID)
|
||||
err = setup.DBPool.QueryRow(context.Background(), `SELECT id FROM media_items WHERE title = $1`, "Sug Tide").Scan(&tideID)
|
||||
require.NoError(t, err)
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO collection_items (collection_id, media_item_id) VALUES ($1, $2)`, collID, tideID)
|
||||
require.NoError(t, err)
|
||||
|
||||
status, results = suggest("collection", " ", "")
|
||||
require.Equal(t, http.StatusOK, status)
|
||||
@@ -632,7 +620,7 @@ func TestFilterFamilyLens(t *testing.T) {
|
||||
id pgtype.UUID
|
||||
title string
|
||||
}{{visibleID, "Lens Vis Book"}, {hiddenID, "Lens Hid Book"}} {
|
||||
_, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: lib.id, Title: lib.title,
|
||||
Authors: []string{"Lens Secret Author"},
|
||||
FilePath: fmt.Sprintf("/tmp/builder/%s-%d.epub", lib.title, time.Now().UnixNano()),
|
||||
@@ -641,10 +629,9 @@ func TestFilterFamilyLens(t *testing.T) {
|
||||
})
|
||||
require.NoError(t, err)
|
||||
}
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
seedExec(t, setup.DBPool,
|
||||
`INSERT INTO library_visibility (library_id, user_id, is_visible) VALUES ($1, $2, false)`,
|
||||
hiddenID, pgtype.UUID{Bytes: [16]byte(userID), Valid: true})
|
||||
require.NoError(t, err)
|
||||
|
||||
search := func(params url.Values) (int, []map[string]interface{}) {
|
||||
return qfSearch(t, setup, setup.RegularToken, params)
|
||||
|
||||
@@ -0,0 +1,444 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestMetadataWriteToFile pins the write-to-file endpoint (2026-10-27):
|
||||
// POST /api/media-items/:id/write-metadata (admin only) persists the
|
||||
// item's CURRENT database metadata into the artifact — embedded OPF for
|
||||
// EPUBs, a Calibre-style sidecar for formats that can't carry metadata
|
||||
// — then re-extracts. Sidecar naming: metadata.opf when the item is the
|
||||
// only book in its directory, <basename>.opf otherwise (one book's
|
||||
// sidecar must never poison dir-mates).
|
||||
//
|
||||
// Fixture files live under /app/uploads (the container's library
|
||||
// folder) and are removed on cleanup; the test library delete
|
||||
// cascades the items.
|
||||
func TestMetadataWriteToFile(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
setup := setupTestServer(t)
|
||||
ctx := context.Background()
|
||||
|
||||
libID := createTestLibraryWithFolder(t, setup.Server, setup.Token, "Test Metadata Write Library", true)
|
||||
uploadsDir := "/app/uploads"
|
||||
|
||||
insertItem := func(relPath, title string) string {
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, $2, $3, 'ebook', 'application/epub+zip', 100, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, title, relPath,
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
return itemID
|
||||
}
|
||||
|
||||
writeEPUB := func(absPath, title string) {
|
||||
t.Helper()
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(absPath), 0o755))
|
||||
var buf bytes.Buffer
|
||||
w := zip.NewWriter(&buf)
|
||||
mimetype, _ := w.CreateHeader(&zip.FileHeader{Name: "mimetype", Method: zip.Store})
|
||||
_, _ = mimetype.Write([]byte("application/epub+zip"))
|
||||
container, _ := w.Create("META-INF/container.xml")
|
||||
_, _ = container.Write([]byte(`<?xml version="1.0"?><container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles><rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/></rootfiles></container>`))
|
||||
opf, _ := w.Create("OEBPS/content.opf")
|
||||
_, _ = opf.Write([]byte(fmt.Sprintf(`<?xml version="1.0"?><package version="3.0" xmlns="http://www.idpf.org/2007/opf" unique-identifier="pub-id"><metadata xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title id="t1">%s</dc:title><dc:creator>Original Author</dc:creator><dc:identifier id="pub-id">urn:uuid:abcd</dc:identifier></metadata><manifest><item id="c1" href="c1.xhtml" media-type="application/xhtml+xml"/></manifest><spine><itemref idref="c1"/></spine></package>`, title)))
|
||||
ch, _ := w.Create("OEBPS/c1.xhtml")
|
||||
_, _ = ch.Write([]byte("<html><body>x</body></html>"))
|
||||
require.NoError(t, w.Close())
|
||||
require.NoError(t, os.WriteFile(absPath, buf.Bytes(), 0o644))
|
||||
}
|
||||
|
||||
callWrite := func(token, itemID string) (int, map[string]interface{}) {
|
||||
req, _ := http.NewRequest("POST", fmt.Sprintf("%s/api/media-items/%s/write-metadata", setup.Server.URL, itemID), nil)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := (&http.Client{}).Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
var body map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&body))
|
||||
return resp.StatusCode, body
|
||||
}
|
||||
|
||||
// Edit the item's metadata the way the editor does (PUT), then ask
|
||||
// the endpoint to persist it.
|
||||
editItem := func(itemID string, fields map[string]interface{}) {
|
||||
body, _ := json.Marshal(fields)
|
||||
req, _ := http.NewRequest("PUT", fmt.Sprintf("%s/api/media-items/%s", setup.Server.URL, itemID), bytes.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := (&http.Client{}).Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode, "PUT should succeed")
|
||||
}
|
||||
|
||||
t.Run("non-admin is forbidden", func(t *testing.T) {
|
||||
rel := fmt.Sprintf("test-metadata-write-%d/forbidden/book.epub", os.Getpid())
|
||||
abs := filepath.Join(uploadsDir, rel)
|
||||
writeEPUB(abs, "Forbidden Book")
|
||||
itemID := insertItem(rel, "Forbidden Book")
|
||||
code, _ := callWrite(setup.RegularToken, itemID)
|
||||
assert.Equal(t, http.StatusForbidden, code)
|
||||
})
|
||||
|
||||
t.Run("missing file errors without sidecar litter", func(t *testing.T) {
|
||||
itemID := insertItem(fmt.Sprintf("test-metadata-write-%d/ghost/book.epub", os.Getpid()), "Ghost Book")
|
||||
code, body := callWrite(setup.Token, itemID)
|
||||
assert.Equal(t, http.StatusInternalServerError, code)
|
||||
assert.Contains(t, body["error"], "missing on disk")
|
||||
})
|
||||
|
||||
t.Run("EPUB metadata is embedded and rescan keeps it", func(t *testing.T) {
|
||||
rel := fmt.Sprintf("test-metadata-write-%d/solo/book.epub", os.Getpid())
|
||||
abs := filepath.Join(uploadsDir, rel)
|
||||
writeEPUB(abs, "Original EPUB Title")
|
||||
itemID := insertItem(rel, "Original EPUB Title")
|
||||
|
||||
editItem(itemID, map[string]interface{}{
|
||||
"title": "Embedded New Title",
|
||||
"authors": []string{"Embedded Author"},
|
||||
"series": "Embedded Series",
|
||||
"series_number": 2,
|
||||
})
|
||||
|
||||
code, body := callWrite(setup.Token, itemID)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Equal(t, "embedded-opf", body["metadata_target"])
|
||||
|
||||
// The zip on disk now carries the new values — verified by
|
||||
// PARSING the rewritten OPF (not string matching), with the
|
||||
// ORIGINAL unique-identifier surviving the splice.
|
||||
reader, err := zip.OpenReader(abs)
|
||||
require.NoError(t, err)
|
||||
var opfContent []byte
|
||||
for _, f := range reader.File {
|
||||
if f.Name == "OEBPS/content.opf" {
|
||||
rc, err := f.Open()
|
||||
require.NoError(t, err)
|
||||
opfContent, err = io.ReadAll(rc)
|
||||
require.NoError(t, rc.Close())
|
||||
require.NoError(t, err)
|
||||
}
|
||||
}
|
||||
reader.Close()
|
||||
|
||||
var opf struct {
|
||||
Metadata struct {
|
||||
Titles []string `xml:"http://purl.org/dc/elements/1.1/ title"`
|
||||
Creators []string `xml:"http://purl.org/dc/elements/1.1/ creator"`
|
||||
Metas []struct {
|
||||
Name string `xml:"name,attr"`
|
||||
Content string `xml:"content,attr"`
|
||||
} `xml:"meta"`
|
||||
} `xml:"metadata"`
|
||||
}
|
||||
require.NoError(t, xml.Unmarshal(opfContent, &opf))
|
||||
require.NotEmpty(t, opf.Metadata.Titles, "dc:title present in written OPF")
|
||||
assert.Equal(t, "Embedded New Title", opf.Metadata.Titles[0])
|
||||
assert.NotContains(t, opf.Metadata.Titles, "Original EPUB Title", "old title replaced")
|
||||
require.NotEmpty(t, opf.Metadata.Creators)
|
||||
var seriesMeta string
|
||||
for _, m := range opf.Metadata.Metas {
|
||||
if m.Name == "calibre:series" {
|
||||
seriesMeta = m.Content
|
||||
}
|
||||
}
|
||||
assert.Equal(t, "Embedded Series", seriesMeta)
|
||||
assert.Contains(t, string(opfContent), "urn:uuid:abcd", "original identifier preserved for unique-identifier")
|
||||
|
||||
// The inline rescan re-extracted from the file: the DB title
|
||||
// still reads the written value.
|
||||
var title string
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT title FROM media_items WHERE id = $1`, itemID).Scan(&title))
|
||||
assert.Equal(t, "Embedded New Title", title)
|
||||
})
|
||||
|
||||
t.Run("single-book dir gets a Calibre-style metadata.opf sidecar", func(t *testing.T) {
|
||||
rel := fmt.Sprintf("test-metadata-write-%d/alone/book.pdf", os.Getpid())
|
||||
abs := filepath.Join(uploadsDir, rel)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(abs), 0o755))
|
||||
require.NoError(t, os.WriteFile(abs, []byte("%PDF-1.4 fake"), 0o644))
|
||||
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, 'Sidecar Book', $2, 'ebook', 'application/pdf', 10, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, rel,
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
editItem(itemID, map[string]interface{}{"title": "Sidecar New Title"})
|
||||
|
||||
code, body := callWrite(setup.Token, itemID)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Equal(t, "sidecar-opf", body["metadata_target"])
|
||||
|
||||
sidecar := filepath.Join(filepath.Dir(abs), "metadata.opf")
|
||||
data, err := os.ReadFile(sidecar)
|
||||
require.NoError(t, err, "metadata.opf must exist next to a lone book")
|
||||
|
||||
// Parse the sidecar like the scanner does: title must be present
|
||||
// as a dc:title ELEMENT, not just a substring.
|
||||
var opf struct {
|
||||
Metadata struct {
|
||||
Titles []string `xml:"http://purl.org/dc/elements/1.1/ title"`
|
||||
} `xml:"metadata"`
|
||||
}
|
||||
require.NoError(t, xml.Unmarshal(data, &opf))
|
||||
require.NotEmpty(t, opf.Metadata.Titles)
|
||||
assert.Equal(t, "Sidecar New Title", opf.Metadata.Titles[0])
|
||||
})
|
||||
|
||||
t.Run("shared dir falls back to <basename>.opf", func(t *testing.T) {
|
||||
// PDFs can't carry metadata, so they take the sidecar path —
|
||||
// and with two books in one directory, the sidecar must be
|
||||
// per-file or A's write would poison B's next scan.
|
||||
dir := fmt.Sprintf("test-metadata-write-%d/shared", os.Getpid())
|
||||
relA, relB := dir+"/a.pdf", dir+"/b.pdf"
|
||||
absA, absB := filepath.Join(uploadsDir, relA), filepath.Join(uploadsDir, relB)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(absA), 0o755))
|
||||
require.NoError(t, os.WriteFile(absA, []byte("%PDF-1.4 fake a"), 0o644))
|
||||
require.NoError(t, os.WriteFile(absB, []byte("%PDF-1.4 fake b"), 0o644))
|
||||
|
||||
var itemA string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, 'Book A', $2, 'ebook', 'application/pdf', 10, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, relA,
|
||||
).Scan(&itemA)
|
||||
require.NoError(t, err)
|
||||
var itemB string
|
||||
err = setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, 'Book B', $2, 'ebook', 'application/pdf', 10, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, relB,
|
||||
).Scan(&itemB)
|
||||
require.NoError(t, err)
|
||||
|
||||
editItem(itemA, map[string]interface{}{"title": "Book A Rewritten"})
|
||||
|
||||
code, body := callWrite(setup.Token, itemA)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Equal(t, "sidecar-opf", body["metadata_target"])
|
||||
|
||||
// a.opf carries A's metadata; NO dir-scoped metadata.opf
|
||||
// may appear (it would poison B's next scan), and B gets no
|
||||
// sidecar from A's write. The name strips the extension,
|
||||
// matching the scanner's {basename}.jpg cover convention.
|
||||
data, err := os.ReadFile(filepath.Join(uploadsDir, dir, "a.opf"))
|
||||
require.NoError(t, err, "shared dir must use <basename>.opf")
|
||||
assert.Contains(t, string(data), "Book A Rewritten")
|
||||
_, err = os.Stat(filepath.Join(uploadsDir, dir, "metadata.opf"))
|
||||
assert.True(t, os.IsNotExist(err), "no dir-scoped sidecar over a multi-book folder")
|
||||
_, err = os.Stat(filepath.Join(uploadsDir, dir, "b.opf"))
|
||||
assert.True(t, os.IsNotExist(err), "B gets no sidecar from A's write")
|
||||
_ = itemB
|
||||
})
|
||||
|
||||
t.Run("audio tags rewrite round-trips for every container", func(t *testing.T) {
|
||||
// Each single-file container gets its own generated fixture
|
||||
// (ffmpeg at test runtime — no binary media in git), its own
|
||||
// write, and an INDEPENDENT verification pass: ffprobe the
|
||||
// rewritten file and assert every mapped key landed. This is the
|
||||
// "check the file's metadata yourself" contract per format:
|
||||
//
|
||||
// album/artist/composer/grouping/date/genre → all four
|
||||
// subtitle → mp3 + flac only
|
||||
// (the mp4 muxer
|
||||
// drops it)
|
||||
cases := []struct {
|
||||
ext string
|
||||
audioCodec string
|
||||
wantSubtitle bool
|
||||
}{
|
||||
{ext: ".mp3", audioCodec: "libmp3lame", wantSubtitle: true},
|
||||
{ext: ".flac", audioCodec: "flac", wantSubtitle: true},
|
||||
{ext: ".m4a", audioCodec: "aac", wantSubtitle: false},
|
||||
{ext: ".m4b", audioCodec: "aac", wantSubtitle: false},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.ext, func(t *testing.T) {
|
||||
rel := fmt.Sprintf("test-metadata-write-%d/audio-%s/book%s", os.Getpid(), strings.TrimPrefix(tc.ext, "."), tc.ext)
|
||||
abs := filepath.Join(uploadsDir, rel)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(abs), 0o755))
|
||||
gen := exec.Command("ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
|
||||
"-f", "lavfi", "-i", "anullsrc=r=44100:cl=mono",
|
||||
"-t", "2", "-c:a", tc.audioCodec,
|
||||
"-metadata", "title=Generated Fixture",
|
||||
"-metadata", "album=Generated Fixture",
|
||||
abs)
|
||||
if out, err := gen.CombinedOutput(); err != nil {
|
||||
t.Skipf("fixture generation failed for %s (%v): %s", tc.ext, err, string(out))
|
||||
}
|
||||
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, 'Generated Fixture', $2, 'audiobook', $3, 100, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, rel, "audio/"+strings.TrimPrefix(tc.ext, "."),
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Follow the real editor flow: PUT first (records
|
||||
// metadata_overrides, so the inline rescan keeps the
|
||||
// edited values instead of folder-structure derivations).
|
||||
editItem(itemID, map[string]interface{}{
|
||||
"title": "Audio New Title",
|
||||
"subtitle": "Audio Sub",
|
||||
"authors": []string{"Audio Author"},
|
||||
"narrators": []string{"Narrator Person"},
|
||||
"series": "Audio Series",
|
||||
"series_number": 1.5,
|
||||
"date_published": "1953-04-16",
|
||||
"tags": []string{"Science"},
|
||||
})
|
||||
|
||||
code, body := callWrite(setup.Token, itemID)
|
||||
require.Equal(t, http.StatusOK, code, "%s write", tc.ext)
|
||||
assert.Equal(t, "audio-tags", body["metadata_target"], tc.ext)
|
||||
|
||||
// ---- verify the FILE's metadata independently ----
|
||||
probe := func() map[string]string {
|
||||
out, err := exec.Command("ffprobe", "-v", "error", "-show_entries",
|
||||
"format_tags", "-of", "json", abs).Output()
|
||||
require.NoError(t, err)
|
||||
var parsed struct {
|
||||
Format struct {
|
||||
Tags map[string]string `json:"tags"`
|
||||
} `json:"format"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal(out, &parsed))
|
||||
return parsed.Format.Tags
|
||||
}
|
||||
tags := probe()
|
||||
assert.Equal(t, "Audio New Title", tags["album"], "%s: the scanner's album rung", tc.ext)
|
||||
assert.Equal(t, "Audio New Title", tags["title"], tc.ext)
|
||||
assert.Equal(t, "Audio Author", tags["artist"], tc.ext)
|
||||
assert.Equal(t, "Narrator Person", tags["composer"], "%s: narrators ride composer", tc.ext)
|
||||
assert.Equal(t, "Audio Series", tags["grouping"], tc.ext)
|
||||
assert.Equal(t, "Science", tags["genre"], tc.ext)
|
||||
assert.Equal(t, "1953-04-16", tags["date"], tc.ext)
|
||||
assert.NotContains(t, tags["album"], "Generated Fixture", "%s: old tag replaced", tc.ext)
|
||||
if tc.wantSubtitle {
|
||||
assert.Equal(t, "Audio Sub", tags["subtitle"], "%s: subtitle round-trips", tc.ext)
|
||||
} else {
|
||||
assert.Empty(t, tags["subtitle"], "%s: subtitle must not be written where it can't round-trip", tc.ext)
|
||||
writeMap := body["write"].(map[string]interface{})
|
||||
assert.NotContains(t, writeMap["fields_written"], "subtitle", "%s: honest fields_written", tc.ext)
|
||||
}
|
||||
|
||||
// Still valid, same-length audio after the rewrite.
|
||||
durOut, err := exec.Command("ffprobe", "-v", "error", "-show_entries",
|
||||
"format=duration", "-of", "default=noprint_wrappers=1:nokey=1", abs).Output()
|
||||
require.NoError(t, err)
|
||||
dur, err := strconv.ParseFloat(strings.TrimSpace(string(durOut)), 64)
|
||||
require.NoError(t, err)
|
||||
assert.Greater(t, dur, 1.5, "%s: audio survived the rewrite", tc.ext)
|
||||
|
||||
// The inline rescan re-read the tags: the DB title stays
|
||||
// the written one.
|
||||
var title string
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT title FROM media_items WHERE id = $1`, itemID).Scan(&title))
|
||||
assert.Equal(t, "Audio New Title", title, tc.ext)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("CBZ gets ComicInfo.xml injected and rescan keeps it", func(t *testing.T) {
|
||||
rel := fmt.Sprintf("test-metadata-write-%d/comic/book.cbz", os.Getpid())
|
||||
abs := filepath.Join(uploadsDir, rel)
|
||||
require.NoError(t, os.MkdirAll(filepath.Dir(abs), 0o755))
|
||||
// Minimal archive: one "page" file (a text stub — cover
|
||||
// extraction failing is tolerated by the scanner) and no
|
||||
// ComicInfo.xml yet.
|
||||
var buf bytes.Buffer
|
||||
w := zip.NewWriter(&buf)
|
||||
page, _ := w.Create("page_0001.jpg")
|
||||
_, _ = page.Write([]byte("not-a-real-jpeg"))
|
||||
require.NoError(t, w.Close())
|
||||
require.NoError(t, os.WriteFile(abs, buf.Bytes(), 0o644))
|
||||
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ($1, 'Generated Comic', $2, 'comic_archive', 'application/zip', 100, NOW(), NOW(), NOW()) RETURNING id`,
|
||||
libID, rel,
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
|
||||
editItem(itemID, map[string]interface{}{
|
||||
"title": "Comic New Title",
|
||||
"series": "Comic Series",
|
||||
"series_number": 4,
|
||||
"authors": []string{"Comic Writer"},
|
||||
"tags": []string{"Superhero"},
|
||||
})
|
||||
|
||||
code, body := callWrite(setup.Token, itemID)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Equal(t, "comicinfo", body["metadata_target"])
|
||||
|
||||
// ---- verify the FILE's ComicInfo.xml by parsing it ----
|
||||
reader, err := zip.OpenReader(abs)
|
||||
require.NoError(t, err)
|
||||
var infoData []byte
|
||||
for _, f := range reader.File {
|
||||
if f.Name == "ComicInfo.xml" {
|
||||
rc, err := f.Open()
|
||||
require.NoError(t, err)
|
||||
infoData, err = io.ReadAll(rc)
|
||||
require.NoError(t, rc.Close())
|
||||
require.NoError(t, err)
|
||||
}
|
||||
}
|
||||
reader.Close()
|
||||
require.NotEmpty(t, infoData, "ComicInfo.xml must exist inside the CBZ")
|
||||
|
||||
var info struct {
|
||||
Title string `xml:"Title"`
|
||||
Series string `xml:"Series"`
|
||||
Number int `xml:"Number"`
|
||||
Writer string `xml:"Writer"`
|
||||
}
|
||||
require.NoError(t, xml.Unmarshal(infoData, &info))
|
||||
assert.Equal(t, "Comic New Title", info.Title)
|
||||
assert.Equal(t, "Comic Series", info.Series)
|
||||
assert.Equal(t, 4, info.Number)
|
||||
assert.Equal(t, "Comic Writer", info.Writer)
|
||||
|
||||
// The inline rescan reads the ComicInfo back: the DB title and
|
||||
// series stay the written values.
|
||||
var title, series string
|
||||
require.NoError(t, setup.DBPool.QueryRow(ctx,
|
||||
`SELECT title, series FROM media_items WHERE id = $1`, itemID).Scan(&title, &series))
|
||||
assert.Equal(t, "Comic New Title", title)
|
||||
assert.Equal(t, "Comic Series", series)
|
||||
})
|
||||
|
||||
t.Cleanup(func() {
|
||||
_ = os.RemoveAll(filepath.Join(uploadsDir, fmt.Sprintf("test-metadata-write-%d", os.Getpid())))
|
||||
})
|
||||
}
|
||||
@@ -85,7 +85,7 @@ func (s *OwnershipAndAdminTestSuite) createItemViaDB(withFolder bool) (itemID, l
|
||||
}
|
||||
}
|
||||
|
||||
item, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
item, err := db.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Ownership Suite Item",
|
||||
Authors: []string{"Suite Author"},
|
||||
@@ -269,7 +269,7 @@ func (s *OwnershipAndAdminTestSuite) TestDelete_StorageDownRefused() {
|
||||
})
|
||||
require.NoError(s.T(), err)
|
||||
|
||||
item, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
item, err := db.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Offline Storage Item",
|
||||
FilePath: "offline.epub",
|
||||
|
||||
@@ -134,7 +134,7 @@ func TestManuallyEditedFilter(t *testing.T) {
|
||||
t.Cleanup(func() { deleteLibraries(t, setup, libID) })
|
||||
|
||||
createBook := func(path string) pgtype.UUID {
|
||||
created, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
created, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: libID,
|
||||
Title: path,
|
||||
Authors: []string{},
|
||||
|
||||
@@ -0,0 +1,808 @@
|
||||
// Podcast integration tests (plan task 1.10): RLS surfaces (the
|
||||
// auditor — hidden-library + cross-user invisibility through every
|
||||
// podcast surface), add-feed flow with allow_member_feeds gating,
|
||||
// upsert-without-delete (shrunken feeds), edit segregation, search
|
||||
// scoping, and the download worker state machine (ffmpeg-generated
|
||||
// audio at runtime — no binary fixtures in the repo).
|
||||
//
|
||||
// Doctrine: shared dev DB — test libraries/feeds named "test*"; the
|
||||
// library delete at cleanup cascades feeds/episodes/subscriptions.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/rls"
|
||||
"bookhoard/internal/services"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// podcastFixture builds a minimal RSS 2.0 doc with the given episodes
|
||||
// (guid/title pairs) — committable-text fixtures only, generated inline.
|
||||
func podcastFixture(channelTitle string, episodes map[string]string) string {
|
||||
items := ""
|
||||
for guid, title := range episodes {
|
||||
items += fmt.Sprintf(`
|
||||
<item>
|
||||
<guid>%s</guid>
|
||||
<title>%s</title>
|
||||
<itunes:subtitle>sub of %s</itunes:subtitle>
|
||||
<description>Description of %s</description>
|
||||
<itunes:duration>600</itunes:duration>
|
||||
<pubDate>Mon, 26 Oct 2026 08:00:00 +0000</pubDate>
|
||||
<enclosure url="https://cdn.invalid/%s.mp3" length="65536" type="audio/mpeg"/>
|
||||
</item>`, guid, title, guid, title, guid)
|
||||
}
|
||||
return `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<rss version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd">
|
||||
<channel>
|
||||
<title>` + channelTitle + `</title>
|
||||
<language>en-us</language>
|
||||
<description>Integration test feed</description>
|
||||
<itunes:author>Test Host</itunes:author>
|
||||
<itunes:type>serial</itunes:type>` + items + `
|
||||
</channel>
|
||||
</rss>`
|
||||
}
|
||||
|
||||
// swapFeedServer hosts feed XML over httptest TLS and wires the
|
||||
// fetcher's test client (the production client refuses loopback by
|
||||
// design — the SSRF boundary; restored on cleanup).
|
||||
func swapFeedServer(t *testing.T, handler http.HandlerFunc) *httptest.Server {
|
||||
t.Helper()
|
||||
srv := httptest.NewTLSServer(handler)
|
||||
t.Cleanup(func() {
|
||||
services.SetPodcastHTTPClientForTests(nil)
|
||||
srv.Close()
|
||||
})
|
||||
services.SetPodcastHTTPClientForTests(srv.Client())
|
||||
return srv
|
||||
}
|
||||
|
||||
// createPodcastsLibrary provisions a podcasts-type library with a
|
||||
// download root in a temp dir (test-prefixed name; cleanup cascades).
|
||||
// Returns the download root for file-layout assertions.
|
||||
func createPodcastsLibrary(t *testing.T, setup *TestServerSetup, name string) (database.Libraries, string) {
|
||||
t.Helper()
|
||||
ctx := seedCtx()
|
||||
libType, err := setup.DB.GetLibraryTypeByName(context.Background(), "podcasts")
|
||||
require.NoError(t, err, "podcasts library type must be seeded")
|
||||
lib, err := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: name,
|
||||
Description: pgtype.Text{String: "podcast integration test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
root := t.TempDir() // download root
|
||||
_, err = setup.DB.AddLibraryFolder(ctx, database.AddLibraryFolderParams{
|
||||
LibraryID: lib.ID,
|
||||
FolderPath: root,
|
||||
Purpose: services.FolderPurposePodcast,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, lib.ID) })
|
||||
return lib, root
|
||||
}
|
||||
|
||||
// upsertTestEpisode seeds one metadata-only episode row.
|
||||
func upsertTestEpisode(t *testing.T, setup *TestServerSetup, feed database.PodcastFeeds, guid, title string, position int) database.MediaItems {
|
||||
t.Helper()
|
||||
ep, err := setup.DB.UpsertPodcastEpisode(adminSeedCtx(t, setup), database.UpsertPodcastEpisodeParams{
|
||||
LibraryID: feed.LibraryID,
|
||||
Title: title,
|
||||
Authors: []string{},
|
||||
AuthorsSearch: []string{},
|
||||
FilePath: fmt.Sprintf("test-feed/%04d-%s.mp3", position, guid),
|
||||
Tags: []string{},
|
||||
SourceFeedID: feed.ID,
|
||||
EpisodeGuid: pgtype.Text{String: guid, Valid: true},
|
||||
FeedPosition: pgtype.Int4{Int32: int32(position), Valid: true},
|
||||
EnclosureUrl: pgtype.Text{String: "https://cdn.invalid/" + guid + ".mp3", Valid: true},
|
||||
Hosts: []string{},
|
||||
Guests: []string{},
|
||||
})
|
||||
require.NoError(t, err, "seed episode %q", title)
|
||||
return ep
|
||||
}
|
||||
|
||||
// subscribeRaw inserts a subscription row (podcast_subscriptions is
|
||||
// ungoverned by design — the read policy's lookup table).
|
||||
func subscribeRaw(t *testing.T, setup *TestServerSetup, userID uuid.UUID, feedID pgtype.UUID) {
|
||||
t.Helper()
|
||||
_, err := setup.DBPool.Exec(context.Background(),
|
||||
`INSERT INTO podcast_subscriptions (user_id, feed_id) VALUES ($1, $2) ON CONFLICT DO NOTHING`,
|
||||
userID, feedID)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
// adminSeedCtx declares the FIRST-ADMIN identity — the same one the
|
||||
// fetcher/worker use (invariant 12). The harness seed identity has
|
||||
// neither a subscription nor a users row, so podcast INSERT..RETURNING
|
||||
// (which evaluates the SELECT policy under FORCE) 42501s for it; real
|
||||
// admin writes are exactly what production pods run as.
|
||||
func adminSeedCtx(t *testing.T, setup *TestServerSetup) context.Context {
|
||||
t.Helper()
|
||||
admin, err := setup.DB.GetFirstAdmin(context.Background())
|
||||
require.NoError(t, err, "podcast tests need an admin identity")
|
||||
return rls.WithUser(context.Background(), admin)
|
||||
}
|
||||
|
||||
// adminCtxValue is the raw pgtype.UUID behind adminSeedCtx (for
|
||||
// ScopedDB.Exec calls that want the same identity).
|
||||
func adminCtxValue(t *testing.T, setup *TestServerSetup) pgtype.UUID {
|
||||
t.Helper()
|
||||
admin, err := setup.DB.GetFirstAdmin(context.Background())
|
||||
require.NoError(t, err)
|
||||
return admin
|
||||
}
|
||||
|
||||
func podcastAuthed(t *testing.T, setup *TestServerSetup, token, method, path, body string) (int, []byte) {
|
||||
t.Helper()
|
||||
var req *http.Request
|
||||
if body == "" {
|
||||
req, _ = http.NewRequest(method, setup.Server.URL+path, nil)
|
||||
} else {
|
||||
req, _ = http.NewRequest(method, setup.Server.URL+path, bytes.NewReader([]byte(body)))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
var buf bytes.Buffer
|
||||
_, _ = buf.ReadFrom(resp.Body)
|
||||
return resp.StatusCode, buf.Bytes()
|
||||
}
|
||||
|
||||
// ---- the RLS auditor for podcast surfaces (plan task 1.10 / invariant 13) ----
|
||||
|
||||
func TestPodcastRLSSurfaces(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
regularID := getRegularUserID(t, setup.DB)
|
||||
marker := "PODRLS" + uuid.New().String()[:8]
|
||||
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test RLS Podcasts "+marker)
|
||||
feed, err := setup.DB.CreatePodcastFeed(adminSeedCtx(t, setup), database.CreatePodcastFeedParams{
|
||||
LibraryID: lib.ID,
|
||||
FeedUrl: "https://feed.invalid/" + marker + ".xml",
|
||||
Title: "RLS Probe Feed " + marker,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
ep := upsertTestEpisode(t, setup, feed, "guid-1", marker+" Episode One", 1)
|
||||
|
||||
adminToken := setup.Token
|
||||
regularToken := setup.RegularToken
|
||||
|
||||
t.Run("non_subscriber_gets_404_everywhere", func(t *testing.T) {
|
||||
code, _ := podcastAuthed(t, setup, regularToken, "GET", "/api/podcasts/feeds/"+uuid.UUID(feed.ID.Bytes).String(), "")
|
||||
assert.Equal(t, http.StatusNotFound, code, "unsubscribed feed must be invisible (existence leak)")
|
||||
|
||||
code, _ = podcastAuthed(t, setup, regularToken, "GET", "/api/podcasts/search?q="+marker, "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
// empty via body check below in subscriber test's mirror
|
||||
|
||||
code, _ = podcastAuthed(t, setup, regularToken, "GET", "/podcasts/"+uuid.UUID(feed.ID.Bytes).String(), "")
|
||||
assert.Equal(t, http.StatusNotFound, code, "detail page must 404 for non-subscribers")
|
||||
})
|
||||
|
||||
t.Run("episode_hidden_from_generic_media_reads", func(t *testing.T) {
|
||||
code, body := podcastAuthed(t, setup, regularToken, "GET", "/api/media-items?limit=500", "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.NotContains(t, string(body), marker, "LEAK: podcast episode surfaced through generic media list")
|
||||
})
|
||||
|
||||
t.Run("admin_sees_unsubscribed_feed", func(t *testing.T) {
|
||||
code, body := podcastAuthed(t, setup, adminToken, "GET", "/api/podcasts/feeds/"+uuid.UUID(feed.ID.Bytes).String(), "")
|
||||
assert.Equal(t, http.StatusOK, code, "admin escape must see any feed")
|
||||
assert.Contains(t, string(body), marker)
|
||||
_ = ep
|
||||
})
|
||||
|
||||
t.Run("subscriber_sees_feed_then_hidden_library_hides_it", func(t *testing.T) {
|
||||
feedIDStr := uuid.UUID(feed.ID.Bytes).String()
|
||||
subscribeRaw(t, setup, regularID, feed.ID)
|
||||
|
||||
code, body := podcastAuthed(t, setup, regularToken, "GET", "/api/podcasts/feeds/"+feedIDStr, "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Contains(t, string(body), marker, "subscriber must see their feed")
|
||||
|
||||
code, body = podcastAuthed(t, setup, regularToken, "GET", "/api/podcasts/search?q="+marker, "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Contains(t, string(body), marker, "subscription-scoped search covers the subscriber's feed")
|
||||
|
||||
// Hide the library → the engine hides the feed row again (the
|
||||
// feed policy carries the library-visibility join).
|
||||
_, err := setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
|
||||
LibraryID: lib.ID,
|
||||
IsVisible: false,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() {
|
||||
_, _ = setup.DB.SetLibraryVisibility(context.Background(), database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: regularID, Valid: true},
|
||||
LibraryID: lib.ID,
|
||||
IsVisible: true,
|
||||
})
|
||||
})
|
||||
code, _ = podcastAuthed(t, setup, regularToken, "GET", "/api/podcasts/feeds/"+feedIDStr, "")
|
||||
assert.Equal(t, http.StatusNotFound, code, "hidden library must hide its feeds even from subscribers")
|
||||
})
|
||||
}
|
||||
|
||||
// ---- add-feed flow: fetch+parse+upsert+subscribe; allow_member_feeds gate ----
|
||||
|
||||
func TestPodcastAddFeedFlow(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
regularID := getRegularUserID(t, setup.DB)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test AddFeed Podcasts")
|
||||
|
||||
mux := http.NewServeMux()
|
||||
feedXML := podcastFixture("AddFlow Feed", map[string]string{"ep-a": "Episode A", "ep-b": "Episode B"})
|
||||
mux.HandleFunc("/feed.xml", func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(feedXML))
|
||||
})
|
||||
mux.HandleFunc("/other.xml", func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(podcastFixture("Other Feed", map[string]string{"other-1": "Other Episode"})))
|
||||
})
|
||||
srv := swapFeedServer(t, mux.ServeHTTP)
|
||||
feedURL := srv.URL + "/feed.xml"
|
||||
|
||||
libID := uuid.UUID(lib.ID.Bytes).String()
|
||||
addBody := fmt.Sprintf(`{"library_id":%q,"url":%q}`, libID, feedURL)
|
||||
regularPG := pgtype.UUID{Bytes: regularID, Valid: true}
|
||||
|
||||
t.Run("member_adds_feed_subscribes_and_episodes_upsert_metadata_only", func(t *testing.T) {
|
||||
resp, body := podcastAuthed(t, setup, setup.RegularToken, "POST", "/api/podcasts/feeds", addBody)
|
||||
require.Equal(t, http.StatusCreated, resp, "member add should succeed: %s", string(body))
|
||||
|
||||
// The subscription exists; the feed is on the member's list.
|
||||
code, body := podcastAuthed(t, setup, setup.RegularToken, "GET", "/api/podcasts/feeds", "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Contains(t, string(body), "AddFlow Feed")
|
||||
|
||||
// Episode rows are metadata-only: pending, no file facts.
|
||||
feeds, err := setup.DB.ListMyPodcastFeeds(rls.WithUser(context.Background(), regularPG), regularPG)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, feeds, 1)
|
||||
eps, err := setup.DB.ListPodcastEpisodes(rls.WithUser(context.Background(), pgtype.UUID{Bytes: regularID, Valid: true}), feeds[0].ID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, eps, 2)
|
||||
for _, ep := range eps {
|
||||
assert.Equal(t, "pending", ep.DownloadState, "metadata-only rows stay pending until a download")
|
||||
assert.NotEmpty(t, ep.EnclosureUrl.String, "enclosure URL persisted for the download worker")
|
||||
assert.NotZero(t, ep.FeedPosition.Int32)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("duplicate_url_attaches_without_new_row", func(t *testing.T) {
|
||||
resp, _ := podcastAuthed(t, setup, setup.RegularToken, "POST", "/api/podcasts/feeds", addBody)
|
||||
assert.Equal(t, http.StatusCreated, resp)
|
||||
feeds, err := setup.DB.ListMyPodcastFeeds(rls.WithUser(context.Background(), regularPG), regularPG)
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, feeds, 1, "URL dedupe: no second feed row")
|
||||
})
|
||||
|
||||
t.Run("allow_member_feeds_false_rejects_members_allows_admins", func(t *testing.T) {
|
||||
_, err := setup.DBPool.Exec(context.Background(),
|
||||
`UPDATE libraries SET allow_member_feeds = false WHERE id = $1`, lib.ID)
|
||||
require.NoError(t, err)
|
||||
t.Cleanup(func() {
|
||||
_, _ = setup.DBPool.Exec(context.Background(),
|
||||
`UPDATE libraries SET allow_member_feeds = true WHERE id = $1`, lib.ID)
|
||||
})
|
||||
|
||||
// A second feed URL from a member → rejected.
|
||||
otherURL := srv.URL + "/other.xml"
|
||||
resp, body := podcastAuthed(t, setup, setup.RegularToken, "POST", "/api/podcasts/feeds",
|
||||
fmt.Sprintf(`{"library_id":%q,"url":%q}`, libID, otherURL))
|
||||
assert.Equal(t, http.StatusForbidden, resp, "member add must be refused: %s", string(body))
|
||||
|
||||
// Admin keeps the right.
|
||||
resp, _ = podcastAuthed(t, setup, setup.Token, "POST", "/api/podcasts/feeds",
|
||||
fmt.Sprintf(`{"library_id":%q,"url":%q}`, libID, otherURL))
|
||||
assert.Equal(t, http.StatusCreated, resp, "admin add must succeed")
|
||||
|
||||
// Subscribing to an EXISTING feed stays allowed for members
|
||||
// (the gate governs ADDING feeds, not subscribing).
|
||||
code, _ := podcastAuthed(t, setup, setup.RegularToken, "GET", "/api/podcasts/feeds", "")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
})
|
||||
}
|
||||
|
||||
// ---- upsert-without-delete: shrunken feeds keep their history (D14) ----
|
||||
|
||||
func TestPodcastShrunkenFeedRetention(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test Shrink Podcasts")
|
||||
|
||||
current := podcastFixture("Shrink Feed", map[string]string{"ep-1": "Kept One", "ep-2": "Doomed Two"})
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/feed.xml", func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(current))
|
||||
})
|
||||
srv := swapFeedServer(t, mux.ServeHTTP)
|
||||
|
||||
libID := uuid.UUID(lib.ID.Bytes).String()
|
||||
resp, body := podcastAuthed(t, setup, setup.Token, "POST", "/api/podcasts/feeds",
|
||||
fmt.Sprintf(`{"library_id":%q,"url":%q}`, libID, srv.URL+"/feed.xml"))
|
||||
require.Equal(t, http.StatusCreated, resp, string(body))
|
||||
|
||||
adminCtx := adminSeedCtx(t, setup)
|
||||
allFeeds, err := setup.DB.ListPodcastFeedsForPolling(adminCtx)
|
||||
require.NoError(t, err)
|
||||
var feedID pgtype.UUID
|
||||
for _, f := range allFeeds {
|
||||
if f.Title == "Shrink Feed" {
|
||||
feedID = f.ID
|
||||
}
|
||||
}
|
||||
feedIDStr := uuid.UUID(feedID.Bytes).String()
|
||||
|
||||
eps, err := setup.DB.ListPodcastEpisodes(adminSeedCtx(t, setup), feedID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, eps, 2)
|
||||
|
||||
// The publisher purges an episode from the XML.
|
||||
current = podcastFixture("Shrink Feed", map[string]string{"ep-1": "Kept One"})
|
||||
resp, _ = podcastAuthed(t, setup, setup.Token, "POST", "/api/podcasts/feeds/"+feedIDStr+"/refresh", "")
|
||||
require.Equal(t, http.StatusOK, resp)
|
||||
|
||||
eps, err = setup.DB.ListPodcastEpisodes(adminCtx, feedID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, eps, 2, "PIN: shrunken feeds NEVER delete rows — history accumulates for search")
|
||||
}
|
||||
|
||||
// ---- edit segregation (D10) ----
|
||||
|
||||
func TestPodcastEditSegregation(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test EditSeg Podcasts")
|
||||
feed, err := setup.DB.CreatePodcastFeed(adminSeedCtx(t, setup), database.CreatePodcastFeedParams{
|
||||
LibraryID: lib.ID,
|
||||
FeedUrl: "https://feed.invalid/edit-seg.xml",
|
||||
Title: "EditSeg Feed",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
ep := upsertTestEpisode(t, setup, feed, "seg-1", "Segregated Episode", 1)
|
||||
epID := uuid.UUID(ep.ID.Bytes).String()
|
||||
|
||||
t.Run("generic_edit_rejects_podcast_episodes", func(t *testing.T) {
|
||||
resp, body := podcastAuthed(t, setup, setup.Token, "PUT", "/api/media-items/"+epID,
|
||||
`{"title":"Hacked Title","description":"clobbered"}`)
|
||||
assert.Equal(t, http.StatusBadRequest, resp, "PIN: generic edit must reject episodes: %s", string(body))
|
||||
})
|
||||
|
||||
t.Run("write_to_file_rejects_podcast_episodes", func(t *testing.T) {
|
||||
resp, body := podcastAuthed(t, setup, setup.Token, "POST", "/api/media-items/"+epID+"/write-metadata", "")
|
||||
assert.NotEqual(t, http.StatusOK, resp, "PIN: metadata write-to-file must refuse episodes: %s", string(body))
|
||||
})
|
||||
|
||||
t.Run("book_edit_still_works", func(t *testing.T) {
|
||||
// Control: the rejection is podcast-specific, not a broken route.
|
||||
libType, err := setup.DB.GetLibraryTypeByName(seedCtx(), "audiobooks")
|
||||
require.NoError(t, err)
|
||||
book, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: func() pgtype.UUID {
|
||||
lib2, cerr := setup.DB.CreateLibrary(seedCtx(), database.CreateLibraryParams{
|
||||
Name: "test EditSeg Books",
|
||||
LibraryTypeID: libType.ID,
|
||||
})
|
||||
require.NoError(t, cerr)
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(seedCtx(), lib2.ID) })
|
||||
return lib2.ID
|
||||
}(),
|
||||
Title: "Edit Control Book",
|
||||
FilePath: "control-" + uuid.New().String() + ".m4b",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
resp, body := podcastAuthed(t, setup, setup.Token, "PUT", "/api/media-items/"+uuid.UUID(book.ID.Bytes).String(),
|
||||
`{"title":"Edited Control"}`)
|
||||
assert.True(t, resp == http.StatusOK || resp == http.StatusBadRequest,
|
||||
"control book edit must not 404/500 (podcast-specific gate): %d %s", resp, string(body))
|
||||
})
|
||||
}
|
||||
|
||||
// ---- download worker: state machine, idempotency, file layout ----
|
||||
|
||||
func TestPodcastDownloadWorker(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
if _, err := exec.LookPath("ffmpeg"); err != nil {
|
||||
if _, err := os.Stat("/usr/bin/ffmpeg"); err != nil {
|
||||
t.Skip("ffmpeg unavailable: worker pipeline test needs runtime-generated audio")
|
||||
}
|
||||
}
|
||||
|
||||
lib, root := createPodcastsLibrary(t, setup, "test Download Podcasts")
|
||||
|
||||
// Generate a small valid MP3 (1s tone) at runtime — no binary fixtures.
|
||||
mp3Path := filepath.Join(t.TempDir(), "tone.mp3")
|
||||
gen := exec.Command("ffmpeg", "-y", "-v", "error", "-f", "lavfi",
|
||||
"-i", "sine=frequency=440:duration=1", "-b:a", "32k", mp3Path)
|
||||
out, err := gen.CombinedOutput()
|
||||
require.NoError(t, err, "ffmpeg generate: %s", string(out))
|
||||
audio, err := os.ReadFile(mp3Path)
|
||||
require.NoError(t, err)
|
||||
|
||||
var enclosureHits int
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/audio/ep.mp3", func(w http.ResponseWriter, _ *http.Request) {
|
||||
enclosureHits++
|
||||
w.Header().Set("Content-Type", "audio/mpeg")
|
||||
_, _ = w.Write(audio)
|
||||
})
|
||||
mux.HandleFunc("/feed.xml", func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(podcastFixture("Download Feed", map[string]string{"dl-1": "Downloaded Episode"})))
|
||||
})
|
||||
srv := swapFeedServer(t, mux.ServeHTTP)
|
||||
|
||||
libID := uuid.UUID(lib.ID.Bytes).String()
|
||||
resp, body := podcastAuthed(t, setup, setup.Token, "POST", "/api/podcasts/feeds",
|
||||
fmt.Sprintf(`{"library_id":%q,"url":%q}`, libID, srv.URL+"/feed.xml"))
|
||||
require.Equal(t, http.StatusCreated, resp, string(body))
|
||||
|
||||
feeds, err := setup.DB.ListPodcastFeedsForPolling(adminSeedCtx(t, setup))
|
||||
require.NoError(t, err)
|
||||
var feedRow database.PodcastFeeds
|
||||
for _, f := range feeds {
|
||||
if f.Title == "Download Feed" {
|
||||
feedRow = f
|
||||
}
|
||||
}
|
||||
require.NotZero(t, feedRow.ID, "feed must exist after add")
|
||||
eps, err := setup.DB.ListPodcastEpisodes(adminSeedCtx(t, setup), feedRow.ID)
|
||||
require.NoError(t, err)
|
||||
require.Len(t, eps, 1)
|
||||
ep := eps[0]
|
||||
|
||||
// Point the enclosure at the TEST server (fixtures use cdn.invalid).
|
||||
// The write goes through an ADMIN-declared ScopedDB — the episode
|
||||
// read policy hides such rows from the raw pool's seed identity, and
|
||||
// onUpdate UPDATE FOR UPDATE reads the target row first.
|
||||
adminCtx := adminSeedCtx(t, setup)
|
||||
adminScope := database.NewScopedDB(setup.DBPool)
|
||||
_, err = adminScope.Exec(rls.WithUser(context.Background(), adminCtxValue(t, setup)),
|
||||
`UPDATE media_items SET enclosure_url = $2 WHERE id = $1`,
|
||||
ep.ID, srv.URL+"/audio/ep.mp3")
|
||||
require.NoError(t, err)
|
||||
|
||||
updated, err := setup.DB.GetPodcastEpisode(adminCtx, ep.ID)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, srv.URL+"/audio/ep.mp3", updated.EnclosureUrl.String, "enclosure re-pointed at the test server")
|
||||
|
||||
worker := services.NewPodcastDownloadWorker(setup.DB, services.NewPodcastService(setup.DB))
|
||||
workerCtx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
worker.Start(workerCtx)
|
||||
|
||||
feedIDStr := uuid.UUID(feedRow.ID.Bytes).String()
|
||||
epIDStr := uuid.UUID(ep.ID.Bytes).String()
|
||||
resp, _ = podcastAuthed(t, setup, setup.Token, "POST",
|
||||
"/api/podcasts/feeds/"+feedIDStr+"/episodes/"+epIDStr+"/download", "")
|
||||
require.Equal(t, http.StatusAccepted, resp)
|
||||
|
||||
// Terminal state within the worker's drain cadence.
|
||||
var done database.MediaItems
|
||||
require.Eventually(t, func() bool {
|
||||
row, err := setup.DB.GetPodcastEpisode(adminCtx, ep.ID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
done = row
|
||||
return row.DownloadState == "downloaded" || row.DownloadState == "failed"
|
||||
}, 30*time.Second, 250*time.Millisecond, "download must reach a terminal state (got %s)", done.DownloadState)
|
||||
|
||||
require.Equal(t, "downloaded", done.DownloadState, "inline reason (if failed): %s", done.DownloadError.String)
|
||||
assert.True(t, done.FileSize.Int64 > 0)
|
||||
assert.NotEmpty(t, done.AudioMetadata, "ffprobe facts must land in audio_metadata")
|
||||
assert.FileExists(t, filepath.Join(root, done.FilePath), "episode lands under <download-root>/<feed-slug>/ per the pinned layout")
|
||||
|
||||
t.Run("redownload_of_downloaded_episode_is_a_noop", func(t *testing.T) {
|
||||
hitsBefore := enclosureHits
|
||||
resp, _ := podcastAuthed(t, setup, setup.Token, "POST",
|
||||
"/api/podcasts/feeds/"+feedIDStr+"/episodes/"+epIDStr+"/download", "")
|
||||
require.Equal(t, http.StatusAccepted, resp)
|
||||
time.Sleep(1500 * time.Millisecond)
|
||||
assert.Equal(t, hitsBefore, enclosureHits, "PIN: re-download of a downloaded episode must not re-fetch")
|
||||
})
|
||||
|
||||
t.Run("failed_download_persists_reason_inline", func(t *testing.T) {
|
||||
// A dead enclosure: fetch fails, state+reason persist, existing
|
||||
// metadata is never cleared. Writes run under the acting admin
|
||||
// (the seed identity would read-match zero rows under the
|
||||
// subscription policy).
|
||||
// The failure path only exercises with the file ABSENT — an
|
||||
// on-disk, probe-clean file re-marks downloaded without fetching
|
||||
// (the idempotency reconciliation, already pinned by the no-op
|
||||
// subtest above).
|
||||
require.NoError(t, os.Remove(filepath.Join(root, done.FilePath)))
|
||||
_, err := adminScope.Exec(rls.WithUser(context.Background(), adminCtxValue(t, setup)),
|
||||
`UPDATE media_items SET enclosure_url = 'https://127.0.0.1:1/none.mp3',
|
||||
download_state = 'pending', download_error = NULL, download_attempts = 0
|
||||
WHERE id = $1`, ep.ID)
|
||||
require.NoError(t, err)
|
||||
|
||||
resp, _ := podcastAuthed(t, setup, setup.Token, "POST",
|
||||
"/api/podcasts/feeds/"+feedIDStr+"/episodes/"+epIDStr+"/download", "")
|
||||
require.Equal(t, http.StatusAccepted, resp)
|
||||
require.Eventually(t, func() bool {
|
||||
r, err := setup.DB.GetPodcastEpisode(adminCtx, ep.ID)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
if r.DownloadState != "failed" {
|
||||
t.Logf("still not failed: state=%s enclosure=%q attempts=%d err=%q",
|
||||
r.DownloadState, r.EnclosureUrl.String, r.DownloadAttempts, r.DownloadError.String)
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}, 30*time.Second, 2*time.Second, "dead enclosure must end failed")
|
||||
|
||||
r, err := setup.DB.GetPodcastEpisode(adminCtx, ep.ID)
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, r.DownloadError.String, "PIN: the reason text persists (renders inline in the episode list)")
|
||||
assert.Equal(t, "Downloaded Episode", r.Title, "PIN: failures never clear episode metadata")
|
||||
})
|
||||
}
|
||||
|
||||
// ---- invariant 4 pin: the scanner never sees podcasts folders ----
|
||||
|
||||
func TestPodcastScannerExclusion(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
lib, root := createPodcastsLibrary(t, setup, "test ScanSkip Podcasts")
|
||||
|
||||
// A stray audio file inside the download root must never ingest.
|
||||
require.NoError(t, os.MkdirAll(filepath.Join(root, "some-feed"), 0o755))
|
||||
require.NoError(t, os.WriteFile(filepath.Join(root, "some-feed", "01-ep.mp3"), []byte("not really audio"), 0o644))
|
||||
|
||||
scanner := services.NewMediaScanner(setup.DB)
|
||||
err := scanner.SetFolders([]string{root}, false)
|
||||
if err != nil {
|
||||
// Zero scannable folders errors at the watcher (watch=true) —
|
||||
// scan jobs (watch=false) still configure; assert either way.
|
||||
assert.Contains(t, err.Error(), "feed-managed")
|
||||
}
|
||||
assert.Empty(t, scanner.FolderCount(), "PIN: podcasts download roots are never scanner folders")
|
||||
assert.NotZero(t, lib.ID, "library must exist for the pin to be meaningful")
|
||||
}
|
||||
|
||||
// ---- invariant 3 / task 2.6: order flips never move progress ----
|
||||
|
||||
func TestPodcastReorderSafety(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
adminCtx := adminSeedCtx(t, setup)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test Reorder Podcasts")
|
||||
|
||||
feed, err := setup.DB.CreatePodcastFeed(adminCtx, database.CreatePodcastFeedParams{
|
||||
LibraryID: lib.ID,
|
||||
FeedUrl: "https://feed.invalid/reorder.xml",
|
||||
Title: "Reorder Feed",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
e1 := upsertTestEpisode(t, setup, feed, "ro-1", "Reorder One", 1)
|
||||
e2 := upsertTestEpisode(t, setup, feed, "ro-2", "Reorder Two", 2)
|
||||
|
||||
// Mark both DOWNLOADED so they enter the sequence (the lifecycle
|
||||
// itself is pinned by TestPodcastDownloadWorker).
|
||||
adminScopeExec(t, setup, adminCtxValue(t, setup),
|
||||
`UPDATE media_items SET download_state = 'downloaded', file_size = 1000
|
||||
WHERE id IN ($1, $2) AND source = 'podcast_episode'`, e1.ID, e2.ID)
|
||||
|
||||
adminID := adminCtxValue(t, setup)
|
||||
|
||||
// Listener is 40% through episode TWO (direct row write — the pin
|
||||
// is about the row's LIFE under renumbering, not the write API).
|
||||
adminScopeExec(t, setup, adminID,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, chapter, character_offset)
|
||||
VALUES ($1, $2, 0.4, 0, 0)
|
||||
ON CONFLICT (media_item_id, user_id) DO UPDATE SET percentage = 0.4, last_read_at = NOW()`,
|
||||
e2.ID, adminID)
|
||||
|
||||
before, err := setup.DB.GetReadingProgress(adminCtx, database.GetReadingProgressParams{
|
||||
MediaItemID: e2.ID, UserID: adminID,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
pctBefore := before.Percentage.Float64
|
||||
|
||||
// FLIP the feed's native positions (the fetcher renumbering case:
|
||||
// the publisher reorders; positions renumber on the next fetch).
|
||||
adminScopeExec(t, setup, adminID,
|
||||
`UPDATE media_items SET feed_position = CASE id WHEN $1 THEN 2 WHEN $2 THEN 1 END
|
||||
WHERE id IN ($1, $2)`, e1.ID, e2.ID)
|
||||
|
||||
after, err := setup.DB.GetReadingProgress(adminCtx, database.GetReadingProgressParams{
|
||||
MediaItemID: e2.ID, UserID: adminID,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, pctBefore, after.Percentage.Float64,
|
||||
"PIN: progress anchors on the episode (guid/identity), never the ordinal — reordering must not move it")
|
||||
|
||||
// The sequence still lists both downloaded episodes; the override
|
||||
// only changes ORDER, never membership.
|
||||
memberID := getRegularUserID(t, setup.DB)
|
||||
subscribeRaw(t, setup, memberID, feed.ID)
|
||||
_, body := podcastAuthed(t, setup, setup.Token, "GET",
|
||||
"/api/podcasts/feeds/"+uuid.UUID(feed.ID.Bytes).String()+"/sequence", "")
|
||||
var seq struct {
|
||||
Episodes []struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"episodes"`
|
||||
Gaps []interface{} `json:"gaps"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal(body, &seq))
|
||||
assert.Len(t, seq.Episodes, 2, "sequence membership survives renumbering")
|
||||
assert.Empty(t, seq.Gaps, "no un-downloaded rows → no gaps")
|
||||
}
|
||||
|
||||
// adminScopeExec runs raw SQL under the first-admin identity (helper
|
||||
// for pins that need direct row surgery the Queries API lacks).
|
||||
func adminScopeExec(t *testing.T, setup *TestServerSetup, adminID pgtype.UUID, sql string, args ...any) {
|
||||
t.Helper()
|
||||
scope := database.NewScopedDB(setup.DBPool)
|
||||
_, err := scope.Exec(rls.WithUser(context.Background(), adminID), sql, args...)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
|
||||
|
||||
// ---- task 1.9 residue: the htmx wire format is FORM-encoded (issue #2) ----
|
||||
|
||||
func TestPodcastOrderFormEncoding(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test OrderForm Podcasts")
|
||||
feed, err := setup.DB.CreatePodcastFeed(adminSeedCtx(t, setup), database.CreatePodcastFeedParams{
|
||||
LibraryID: lib.ID,
|
||||
FeedUrl: "https://feed.invalid/order-form.xml",
|
||||
Title: "Order Form Feed",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
feedIDStr := uuid.UUID(feed.ID.Bytes).String()
|
||||
|
||||
// The UI's hx-put submits urlencoded form bodies (htmx non-GET
|
||||
// default) — the binder must map them (form tag), not just JSON.
|
||||
formPut := func(body string) int {
|
||||
req, _ := http.NewRequest(http.MethodPut,
|
||||
setup.Server.URL+"/api/podcasts/feeds/"+feedIDStr+"/order",
|
||||
strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.Header.Set("Authorization", "Bearer "+setup.Token)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
return resp.StatusCode
|
||||
}
|
||||
|
||||
assert.Equal(t, http.StatusOK, formPut("order=oldest"), "form-encoded order must bind")
|
||||
assert.Equal(t, http.StatusOK, formPut("order=native"), "native binds too")
|
||||
assert.Equal(t, http.StatusBadRequest, formPut(""), "missing order stays rejected (required)")
|
||||
assert.Equal(t, http.StatusBadRequest, formPut("order=sideways"), "invalid order stays rejected")
|
||||
}
|
||||
|
||||
// ---- issue #8: page search hits EPISODE titles/descriptions too ----
|
||||
|
||||
func TestPodcastPageSearchEpisodes(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test PageSearch Podcasts")
|
||||
// Feed title deliberately does NOT contain the needle.
|
||||
feed, err := setup.DB.CreatePodcastFeed(adminSeedCtx(t, setup), database.CreatePodcastFeedParams{
|
||||
LibraryID: lib.ID,
|
||||
FeedUrl: "https://feed.invalid/page-search.xml",
|
||||
Title: "Trader Tales Show",
|
||||
})
|
||||
require.NoError(t, err)
|
||||
upsertTestEpisode(t, setup, feed, "ps-1", "Zarkorrin Blaster", 1)
|
||||
|
||||
subscribeRaw(t, setup, getRegularUserID(t, setup.DB), feed.ID)
|
||||
|
||||
// SSR page search: the needle exists only in an EPISODE title.
|
||||
req, _ := http.NewRequest(http.MethodGet, setup.Server.URL+"/podcasts?q=Zarkorrin", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var buf bytes.Buffer
|
||||
_, _ = buf.ReadFrom(resp.Body)
|
||||
body := buf.String()
|
||||
|
||||
assert.Contains(t, body, "Trader Tales Show",
|
||||
"PIN: a feed surfaces when an EPISODE matches the query, even when the feed's own metadata does not")
|
||||
assert.Contains(t, body, "matching episode",
|
||||
"the match count renders on the card")
|
||||
}
|
||||
|
||||
// ---- issue #3: Download All flips the CALLER's auto-download to all ----
|
||||
|
||||
func TestPodcastDownloadAllPolicyFlip(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
regularID := getRegularUserID(t, setup.DB)
|
||||
lib, _ := createPodcastsLibrary(t, setup, "test Flip Podcasts")
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/feed.xml", func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(podcastFixture("Flip Feed", map[string]string{"f-1": "Flip One"})))
|
||||
})
|
||||
srv := swapFeedServer(t, mux.ServeHTTP)
|
||||
|
||||
libID := uuid.UUID(lib.ID.Bytes).String()
|
||||
regularPG := pgtype.UUID{Bytes: regularID, Valid: true}
|
||||
|
||||
t.Run("add_feed_with_download_all_sets_policy_all", func(t *testing.T) {
|
||||
resp, body := podcastAuthed(t, setup, setup.RegularToken, "POST", "/api/podcasts/feeds",
|
||||
fmt.Sprintf(`{"library_id":%q,"url":%q,"download_all":true}`, libID, srv.URL+"/feed.xml"))
|
||||
require.Equal(t, http.StatusCreated, resp, string(body))
|
||||
|
||||
sub, err := setup.DB.GetPodcastSubscription(rls.WithUser(context.Background(), regularPG), database.GetPodcastSubscriptionParams{
|
||||
UserID: regularPG, FeedID: func() pgtype.UUID {
|
||||
feeds, _ := setup.DB.ListPodcastFeedsForPolling(rls.WithUser(context.Background(), regularPG))
|
||||
for _, f := range feeds {
|
||||
if f.Title == "Flip Feed" {
|
||||
return f.ID
|
||||
}
|
||||
}
|
||||
return pgtype.UUID{}
|
||||
}(),
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "all", sub.AutoDownload, "PIN: the add-feed checkbox flips the caller's policy to all")
|
||||
})
|
||||
|
||||
t.Run("download_all_button_flips_policy_and_returns_oob_select", func(t *testing.T) {
|
||||
// Set the policy to newest first: the flip must override it.
|
||||
feeds, _ := setup.DB.ListPodcastFeedsForPolling(rls.WithUser(context.Background(), regularPG))
|
||||
var feedID pgtype.UUID
|
||||
for _, f := range feeds {
|
||||
if f.Title == "Flip Feed" {
|
||||
feedID = f.ID
|
||||
}
|
||||
}
|
||||
_, err := setup.DB.UpdatePodcastSubscription(rls.WithUser(context.Background(), regularPG), database.UpdatePodcastSubscriptionParams{
|
||||
UserID: regularPG, FeedID: feedID,
|
||||
DisplayName: pgtype.Text{}, AutoDownload: "newest",
|
||||
AutoDownloadCount: pgtype.Int4{Int32: 3, Valid: true},
|
||||
})
|
||||
require.NoError(t, err)
|
||||
|
||||
feedIDStr := uuid.UUID(feedID.Bytes).String()
|
||||
req, _ := http.NewRequest(http.MethodPost,
|
||||
setup.Server.URL+"/api/podcasts/feeds/"+feedIDStr+"/episodes/all/download", nil)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
req.Header.Set("HX-Request", "true")
|
||||
hxResp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err)
|
||||
defer func() { _ = hxResp.Body.Close() }()
|
||||
require.Equal(t, http.StatusOK, hxResp.StatusCode, "HX action answers with the refreshed partial")
|
||||
var buf bytes.Buffer
|
||||
_, _ = buf.ReadFrom(hxResp.Body)
|
||||
body := buf.String()
|
||||
assert.Contains(t, body, `id="episode-list"`, "direct swap: the episode list partial IS the response")
|
||||
assert.Contains(t, body, `hx-swap-oob="true"`, "issue #3: the policy select rides along out-of-band")
|
||||
|
||||
sub, err := setup.DB.GetPodcastSubscription(rls.WithUser(context.Background(), regularPG), database.GetPodcastSubscriptionParams{
|
||||
UserID: regularPG, FeedID: feedID,
|
||||
})
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "all", sub.AutoDownload, "PIN: Download All flips newest → all (dynamic + visible)")
|
||||
})
|
||||
}
|
||||
@@ -48,7 +48,7 @@ func createTestQueueItem(t *testing.T, db *database.Queries, userID uuid.UUID) (
|
||||
require.NoError(t, err, "Should create library")
|
||||
|
||||
// 4. Create media item
|
||||
mediaItem, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
mediaItem, err := db.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: library.ID,
|
||||
Title: "Test Book for Queue",
|
||||
Authors: []string{"Test Author"},
|
||||
|
||||
@@ -0,0 +1,283 @@
|
||||
package main
|
||||
|
||||
// rls_visibility_test.go — the PERMANENT black-box auditor for per-user
|
||||
// library visibility (the "family lens").
|
||||
//
|
||||
// The one sentence it asserts, in the user's language:
|
||||
//
|
||||
// A user with a library hidden sees NOTHING from that library, on
|
||||
// ANY media-returning surface — while an admin still sees it (the
|
||||
// hide is per-user, never global).
|
||||
//
|
||||
// Implementation-agnostic on purpose: it speaks only HTTP + response
|
||||
// bodies, so it stays valid regardless of HOW enforcement is done
|
||||
// (per-query visible_library_ids clauses today, Postgres RLS in the
|
||||
// RLS rollout, anything tomorrow). It is the auditor that makes a
|
||||
// forgotten surface a failing test instead of a silent leak.
|
||||
//
|
||||
// Red/green history: RED at introduction on the dashboard-section and
|
||||
// collection-detail (mobile View All) surfaces, whose queries never
|
||||
// carried the visible_library_ids clause; GREEN once enforcement
|
||||
// covers every surface (plan:
|
||||
// ~/.opencode/plan/server-rls-library-visibility.md).
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
)
|
||||
|
||||
// uuidInBody finds candidate collection ids in a dashboard-sections
|
||||
// body so the View-All detail endpoint can be probed without coupling
|
||||
// the test to unrelated ids in the body.
|
||||
|
||||
func TestRlsInvariant(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
ctx := context.Background()
|
||||
|
||||
adminToken := setup.Token
|
||||
regularToken := setup.RegularToken
|
||||
regularID := getRegularUserID(t, setup.DB)
|
||||
|
||||
// Unique markers: one for the hidden library's whole subtree
|
||||
// (library name + both book titles — a single string that must
|
||||
// appear NOWHERE), one for the visible control book.
|
||||
hiddenMarker := fmt.Sprintf("RLSHIDE%s", uuid.New().String()[:8])
|
||||
controlMarker := fmt.Sprintf("RLSCTRL%s", uuid.New().String()[:8])
|
||||
|
||||
libType, err := setup.DB.GetLibraryTypeByName(ctx, "ebooks")
|
||||
require.NoError(t, err, "ebooks library type should exist")
|
||||
|
||||
createLib := func(name string) database.Libraries {
|
||||
lib, cerr := setup.DB.CreateLibrary(ctx, database.CreateLibraryParams{
|
||||
Name: name,
|
||||
Description: pgtype.Text{String: "rls invariant test library", Valid: true},
|
||||
LibraryTypeID: libType.ID,
|
||||
CreatedByAdminID: pgtype.UUID{Bytes: [16]byte(regularID), Valid: true},
|
||||
})
|
||||
require.NoError(t, cerr, "create library %q", name)
|
||||
// Deleting the library cascades its media_items AND its
|
||||
// library_visibility rows (FK ON DELETE CASCADE) — one cleanup
|
||||
// covers the whole subtree.
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteLibrary(ctx, lib.ID) })
|
||||
return lib
|
||||
}
|
||||
libH := createLib("RLS Test Hidden Library " + hiddenMarker)
|
||||
libV := createLib("RLS Test Visible Library " + controlMarker)
|
||||
|
||||
createBook := func(libraryID pgtype.UUID, title string) {
|
||||
_, cerr := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: libraryID,
|
||||
Title: title,
|
||||
Authors: []string{"RLS Invariant Author"},
|
||||
FilePath: fmt.Sprintf("/tmp/rls-invariant/%s-%s.epub", title, uuid.New().String()),
|
||||
FileSize: pgtype.Int8{Int64: 1024, Valid: true},
|
||||
MimeType: pgtype.Text{String: "application/epub+zip", Valid: true},
|
||||
})
|
||||
require.NoError(t, cerr, "create book %q", title)
|
||||
}
|
||||
createBook(libH.ID, hiddenMarker+" Hidden Alpha")
|
||||
createBook(libH.ID, hiddenMarker+" Hidden Beta")
|
||||
createBook(libV.ID, controlMarker+" Visible Gamma")
|
||||
|
||||
// Hide libH from the REGULAR user — the same row the admin
|
||||
// visibility UI writes (harness idiom: SetLibraryVisibility).
|
||||
_, err = setup.DB.SetLibraryVisibility(ctx, database.SetLibraryVisibilityParams{
|
||||
UserID: pgtype.UUID{Bytes: [16]byte(regularID), Valid: true},
|
||||
LibraryID: libH.ID,
|
||||
IsVisible: false,
|
||||
})
|
||||
require.NoError(t, err, "hide libH from regular user")
|
||||
|
||||
// OPDS device owned by the REGULAR user (OPDS identity = device owner).
|
||||
opdsToken := "dev_" + uuid.New().String()
|
||||
device, err := setup.DB.CreateDevice(ctx, database.CreateDeviceParams{
|
||||
UserID: pgtype.UUID{Bytes: [16]byte(regularID), Valid: true},
|
||||
DeviceName: "RLS Invariant OPDS Device",
|
||||
DeviceType: "koreader",
|
||||
DeviceIdentifier: "rls-invariant-" + uuid.New().String()[:8],
|
||||
AuthToken: opdsToken,
|
||||
SyncEnabled: pgtype.Bool{Bool: true, Valid: true},
|
||||
AutoSync: pgtype.Bool{Bool: true, Valid: true},
|
||||
SyncFrequencyMinutes: pgtype.Int4{Int32: 60, Valid: true},
|
||||
DeviceMetadata: []byte("{}"),
|
||||
})
|
||||
require.NoError(t, err, "create OPDS device for regular user")
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteDevice(ctx, device.ID) })
|
||||
|
||||
rlsGet := func(token, path string) (int, string) {
|
||||
req, _ := http.NewRequest(http.MethodGet, setup.Server.URL+path, nil)
|
||||
if token != "" {
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
}
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
require.NoError(t, err, "GET %s", path)
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
var sb strings.Builder
|
||||
buf := make([]byte, 32*1024)
|
||||
for {
|
||||
n, err := resp.Body.Read(buf)
|
||||
if n > 0 {
|
||||
sb.Write(buf[:n])
|
||||
}
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
return resp.StatusCode, sb.String()
|
||||
}
|
||||
|
||||
// Surfaces that render media TITLES (books appear by name).
|
||||
titleSurfaces := []struct {
|
||||
name string
|
||||
path string
|
||||
auth string
|
||||
}{
|
||||
{"media-items list", "/api/media-items?limit=500", regularToken},
|
||||
{"dashboard sections", "/api/dashboard/sections?limit=500", regularToken},
|
||||
{"opds catalog", "/opds/devices/" + device.ID.String() + "/catalog", opdsToken},
|
||||
}
|
||||
// Surfaces that return media-adjacent data (counts, series names,
|
||||
// aggregates) — hidden-subtree leakage would show up as marker
|
||||
// series names / inventory rows, not book titles.
|
||||
aggregateSurfaces := []struct {
|
||||
name string
|
||||
path string
|
||||
auth string
|
||||
}{
|
||||
{"collections list", "/api/collections", regularToken},
|
||||
{"series list", "/api/series", regularToken},
|
||||
{"stats summary", "/api/stats/summary", regularToken},
|
||||
{"stats library-usage", "/api/stats/library-usage", regularToken},
|
||||
}
|
||||
|
||||
t.Run("restricted_user_hidden_library_absent_everywhere", func(t *testing.T) {
|
||||
for _, s := range titleSurfaces {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
code, body := rlsGet(s.auth, s.path)
|
||||
require.Equal(t, http.StatusOK, code, "surface %s must be authorized (a 401 here is a harness bug, not a pass)", s.path)
|
||||
// THE INVARIANT: nothing from the hidden subtree —
|
||||
// neither its books nor the library itself.
|
||||
assert.NotContains(t, body, hiddenMarker,
|
||||
"LEAK: hidden library content surfaced on %s", s.path)
|
||||
// Control: the visible library's book IS present —
|
||||
// guards against false greens from empty/anonymous
|
||||
// responses.
|
||||
if s.name != "opds catalog" { // root catalog may nest books behind library subsections
|
||||
assert.Contains(t, body, controlMarker,
|
||||
"control book missing from %s — response is not listing media (false-green guard)", s.path)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, s := range aggregateSurfaces {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
code, body := rlsGet(s.auth, s.path)
|
||||
require.Equal(t, http.StatusOK, code, "surface %s must be authorized", s.path)
|
||||
assert.NotContains(t, body, hiddenMarker,
|
||||
"LEAK: hidden library content surfaced on %s", s.path)
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("restricted_user_dashboard_scoped_to_hidden_library", func(t *testing.T) {
|
||||
code, body := rlsGet(regularToken, "/api/dashboard/sections?library_id="+uuid.UUID(libH.ID.Bytes).String())
|
||||
assert.NotContains(t, body, hiddenMarker,
|
||||
"LEAK: dashboard scoped to the hidden library leaked its content (status %d)", code)
|
||||
})
|
||||
|
||||
t.Run("restricted_user_collection_detail_view_all", func(t *testing.T) {
|
||||
// The mobile View All path: GET /api/collections/{id} for every
|
||||
// collection the dashboard references (system sections included).
|
||||
// Structured decode — probe exactly the collection ids the
|
||||
// sections payload declares (its own contract field), not
|
||||
// whatever UUIDs happen to appear in the body.
|
||||
_, sectionsBody := rlsGet(regularToken, "/api/dashboard/sections?limit=500")
|
||||
var sections struct {
|
||||
Sections []struct {
|
||||
CollectionID string `json:"collection_id"`
|
||||
} `json:"sections"`
|
||||
}
|
||||
require.NoError(t, json.Unmarshal([]byte(sectionsBody), §ions),
|
||||
"dashboard sections should decode; body: %s", sectionsBody)
|
||||
seen := map[string]bool{}
|
||||
for _, sec := range sections.Sections {
|
||||
id := sec.CollectionID
|
||||
if id == "" || seen[id] {
|
||||
continue
|
||||
}
|
||||
seen[id] = true
|
||||
code, body := rlsGet(regularToken, "/api/collections/"+id)
|
||||
if code != http.StatusOK {
|
||||
continue // not visible to this user — absence trivially holds
|
||||
}
|
||||
assert.NotContains(t, body, hiddenMarker,
|
||||
"LEAK: collection detail %s (View All surface) returned hidden-library content", id)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("restricted_user_explicit_hidden_library_scoped_requests_rejected", func(t *testing.T) {
|
||||
// Pinned contract (AGENTS.md family-lens rule): naming a hidden
|
||||
// library explicitly is a 403, never a silent pass.
|
||||
libID := uuid.UUID(libH.ID.Bytes).String()
|
||||
for _, path := range []string{
|
||||
"/api/media-items?library_id=" + libID,
|
||||
"/api/series?library_id=" + libID,
|
||||
"/api/series/books?library_id=" + libID + "&name=RLS",
|
||||
} {
|
||||
code, body := rlsGet(regularToken, path)
|
||||
assert.Equal(t, http.StatusForbidden, code,
|
||||
"scoped request %s must be 403 for a hidden library (got %d)", path, code)
|
||||
assert.NotContains(t, body, hiddenMarker)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("admin_still_sees_hidden_library_user_scoped_not_global", func(t *testing.T) {
|
||||
// The hide is PER-USER: an admin (no hidden rows) must still see
|
||||
// everything. This catches "fixing" the leak by hiding the
|
||||
// library globally.
|
||||
for _, s := range []struct{ name, path string }{
|
||||
{"media-items list", "/api/media-items?limit=500"},
|
||||
{"dashboard sections", "/api/dashboard/sections?limit=500"},
|
||||
} {
|
||||
t.Run(s.name, func(t *testing.T) {
|
||||
code, body := rlsGet(adminToken, s.path)
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Contains(t, body, hiddenMarker,
|
||||
"admin must still see the hidden library on %s (hide is user-scoped)", s.path)
|
||||
})
|
||||
}
|
||||
t.Run("opds catalog", func(t *testing.T) {
|
||||
// OPDS identity = device owner; an ADMIN-owned device sees all.
|
||||
adminDevToken := "dev_" + uuid.New().String()
|
||||
adminID := getTestUserID(t, setup.DB)
|
||||
adminDevice, derr := setup.DB.CreateDevice(ctx, database.CreateDeviceParams{
|
||||
UserID: pgtype.UUID{Bytes: [16]byte(adminID), Valid: true},
|
||||
DeviceName: "RLS Invariant Admin OPDS Device",
|
||||
DeviceType: "koreader",
|
||||
DeviceIdentifier: "rls-invariant-admin-" + uuid.New().String()[:8],
|
||||
AuthToken: adminDevToken,
|
||||
SyncEnabled: pgtype.Bool{Bool: true, Valid: true},
|
||||
AutoSync: pgtype.Bool{Bool: true, Valid: true},
|
||||
SyncFrequencyMinutes: pgtype.Int4{Int32: 60, Valid: true},
|
||||
DeviceMetadata: []byte("{}"),
|
||||
})
|
||||
require.NoError(t, derr, "create OPDS device for admin")
|
||||
t.Cleanup(func() { _ = setup.DB.DeleteDevice(ctx, adminDevice.ID) })
|
||||
|
||||
code, body := rlsGet(adminDevToken, "/opds/devices/"+adminDevice.ID.String()+"/catalog")
|
||||
require.Equal(t, http.StatusOK, code)
|
||||
assert.Contains(t, body, hiddenMarker,
|
||||
"admin's OPDS catalog must still expose the hidden library (user-scoped hide)")
|
||||
})
|
||||
})
|
||||
}
|
||||
@@ -78,7 +78,7 @@ func TestScanStateBulkLoad(t *testing.T) {
|
||||
require.NoError(t, os.WriteFile(apiFile, []byte("api made book content"), 0644))
|
||||
apiStat, err := os.Stat(apiFile)
|
||||
require.NoError(t, err)
|
||||
created, err := setup.DB.CreateMediaItem(context.Background(), database.CreateMediaItemParams{
|
||||
created, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: libID,
|
||||
Title: "API Made Book",
|
||||
Authors: []string{"API Author"},
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestSearchViewerProgress pins the search-progress batch (2026-10-27):
|
||||
// the unified search row carries the REQUESTING user's reading progress
|
||||
// (viewer_percentage) via a LEFT JOIN on the same user_id param the
|
||||
// visibility join binds, the JSON endpoint serializes it, and the
|
||||
// layout=dropdown HX branch renders templ rows with the completed check
|
||||
// / partial bar. Fixture: three books in a test library; the FRESH
|
||||
// regular user finished one (1.0), is mid-read on another (0.42) and
|
||||
// never opened the third (NULL). The shared admin's search response is
|
||||
// read but never written — their progress stays untouched.
|
||||
func TestSearchViewerProgress(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
setup := setupTestServer(t)
|
||||
ctx := context.Background()
|
||||
|
||||
libID := createTestLibraryWithFolder(t, setup.Server, setup.Token, "Test Search Progress Library", false)
|
||||
viewerID := getRegularUserID(t, setup.DB)
|
||||
|
||||
books := []struct {
|
||||
title string
|
||||
percentage string // "" = no progress row at all
|
||||
}{
|
||||
{title: "Progress Probe Alpha", percentage: "1.0"},
|
||||
{title: "Progress Probe Beta", percentage: "0.42"},
|
||||
{title: "Progress Probe Gamma", percentage: ""},
|
||||
}
|
||||
itemIDs := map[string]string{}
|
||||
t.Cleanup(func() {
|
||||
_, _ = setup.DBPool.Exec(ctx, "DELETE FROM libraries WHERE id = $1", libID)
|
||||
})
|
||||
|
||||
for _, b := range books {
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, created_at, imported_at, updated_at)
|
||||
VALUES ($1, $2, $3, '2026-02-01 00:00:00+00', '2026-02-01 00:00:00+00', '2026-02-01 00:00:00+00') RETURNING id`,
|
||||
libID, b.title, fmt.Sprintf("/tmp/test-search-progress/%s.epub", b.title),
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
itemIDs[b.title] = itemID
|
||||
if b.percentage != "" {
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at)
|
||||
VALUES ($1, $2, $3::float, '2026-02-02 00:00:00+00')`,
|
||||
itemID, viewerID, b.percentage,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
}
|
||||
|
||||
t.Run("JSON rows carry the viewer's own percentage", func(t *testing.T) {
|
||||
rows := searchJSON(t, setup, setup.RegularToken, "Progress Probe", libID)
|
||||
require.Len(t, rows, 3)
|
||||
byTitle := map[string]map[string]interface{}{}
|
||||
for _, row := range rows {
|
||||
byTitle[row["title"].(string)] = row
|
||||
}
|
||||
assert.Equal(t, float64(1), byTitle["Progress Probe Alpha"]["viewer_percentage"])
|
||||
assert.InDelta(t, 0.42, byTitle["Progress Probe Beta"]["viewer_percentage"], 0.0001)
|
||||
require.Nil(t, byTitle["Progress Probe Gamma"]["viewer_percentage"], "never-opened book must be null")
|
||||
|
||||
// Per-user join: the shared admin (no progress on any fixture
|
||||
// book) sees null across the board — the join binds the
|
||||
// REQUESTER, not a global.
|
||||
adminRows := searchJSON(t, setup, setup.Token, "Progress Probe", libID)
|
||||
require.Len(t, adminRows, 3)
|
||||
for _, row := range adminRows {
|
||||
assert.Nil(t, row["viewer_percentage"], "admin must see null for %v", row["title"])
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("dropdown partial renders progress overlays", func(t *testing.T) {
|
||||
req, err := http.NewRequest("GET",
|
||||
setup.Server.URL+"/api/media-items/search?layout=dropdown&q=Progress+Probe&library_id="+libID, nil)
|
||||
require.NoError(t, err)
|
||||
req.Header.Set("Cookie", "token="+setup.RegularToken)
|
||||
req.Header.Set("HX-Request", "true")
|
||||
resp, err := (&http.Client{}).Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
bodyBytes, err := io.ReadAll(resp.Body)
|
||||
require.NoError(t, err)
|
||||
body := string(bodyBytes)
|
||||
|
||||
assert.Contains(t, body, "3 results for", "count label")
|
||||
// Completed book: the check overlay + title (completed ≥ 0.999).
|
||||
assert.Contains(t, body, "Completed", "completed overlay")
|
||||
// Partial book: the bar at 42%.
|
||||
assert.Contains(t, body, "width: 42.0%", "partial progress bar")
|
||||
// Server-rendered rows, one per hit.
|
||||
assert.Equal(t, 3, strings.Count(body, `class="search-result-item`), "one row per result")
|
||||
})
|
||||
}
|
||||
|
||||
// searchJSON runs a search as the given token and decodes the row array
|
||||
// (library-scoped: the unified search is fuzzy, so an unscoped q would
|
||||
// sweep in live dev rows).
|
||||
func searchJSON(t *testing.T, setup *TestServerSetup, token, query, libraryID string) []map[string]interface{} {
|
||||
t.Helper()
|
||||
req, err := http.NewRequest("GET",
|
||||
fmt.Sprintf("%s/api/media-items/search?q=%s&library_id=%s", setup.Server.URL, url.QueryEscape(query), libraryID), nil)
|
||||
require.NoError(t, err)
|
||||
req.Header.Set("Authorization", "Bearer "+token)
|
||||
resp, err := (&http.Client{}).Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
require.Equal(t, http.StatusOK, resp.StatusCode)
|
||||
var rows []map[string]interface{}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&rows))
|
||||
return rows
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// TestSeriesSortAndFilter pins the series page's sort/filter contract
|
||||
// (2026-10-27 feature): the /series SSR page and the /api/series JSON
|
||||
// endpoint share one sort registry (services.ResolveSeriesSort) and one
|
||||
// SQL CASE cascade (GetDistinctSeries). The fixture gives three series
|
||||
// DISTINCT answers per sort key so ordering bugs can't hide:
|
||||
//
|
||||
// created author books viewer progress
|
||||
// Probe Zebra 01-03 Zeta Zed 2 1 of 2 read (01-09) → reading
|
||||
// Probe Mango 01-02 Alpha One 3 3 of 3 read (01-10) → finished
|
||||
// Probe Apple 01-01 Midway Two 1 none → unstarted
|
||||
//
|
||||
// All viewer progress belongs to the FRESH regular user
|
||||
// (getRegularUserID) and cascades away with the test library — the
|
||||
// shared admin's reading data is never touched.
|
||||
func TestSeriesSortAndFilter(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
setup := setupTestServer(t)
|
||||
ctx := context.Background()
|
||||
|
||||
libID := createTestLibraryWithFolder(t, setup.Server, setup.Token, "Test Series Filter Library", false)
|
||||
viewerID := getRegularUserID(t, setup.DB)
|
||||
|
||||
type seriesFixture struct {
|
||||
name string
|
||||
author string
|
||||
createdAt string
|
||||
books []struct {
|
||||
title string
|
||||
percentage string
|
||||
lastReadAt string
|
||||
}
|
||||
}
|
||||
fixtures := []seriesFixture{
|
||||
{
|
||||
name: "Probe Zebra Series", author: "Zeta Zed", createdAt: "2026-01-03 12:00:00+00",
|
||||
books: []struct {
|
||||
title string
|
||||
percentage string
|
||||
lastReadAt string
|
||||
}{
|
||||
{title: "Zebra Book 1", percentage: "1.0", lastReadAt: "2026-01-09 08:00:00+00"},
|
||||
{title: "Zebra Book 2", percentage: "0.5", lastReadAt: "2026-01-09 08:00:00+00"},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Probe Mango Series", author: "Alpha One", createdAt: "2026-01-02 12:00:00+00",
|
||||
books: []struct {
|
||||
title string
|
||||
percentage string
|
||||
lastReadAt string
|
||||
}{
|
||||
{title: "Mango Book 1", percentage: "1.0", lastReadAt: "2026-01-10 08:00:00+00"},
|
||||
{title: "Mango Book 2", percentage: "1.0", lastReadAt: "2026-01-10 08:00:00+00"},
|
||||
{title: "Mango Book 3", percentage: "1.0", lastReadAt: "2026-01-10 08:00:00+00"},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "Probe Apple Series", author: "Midway Two", createdAt: "2026-01-01 12:00:00+00",
|
||||
books: []struct {
|
||||
title string
|
||||
percentage string
|
||||
lastReadAt string
|
||||
}{
|
||||
{title: "Apple Book 1", percentage: "0", lastReadAt: ""},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
stripComments := regexp.MustCompile(`<!--.*?-->`)
|
||||
// Card headings only: the empty-state panels use different h3
|
||||
// classes ("text-xl font-semibold mb-2"), and "No Series Match"
|
||||
// must not read as a rendered series.
|
||||
nameRegex := regexp.MustCompile(`<h3 class="font-semibold text-sm line-clamp-2"[^>]*>([^<]+)</h3>`)
|
||||
|
||||
renderedNames := func(query string) []string {
|
||||
code, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID+query, setup.RegularToken)
|
||||
if code != 200 {
|
||||
t.Logf("series request %q -> %d; body head: %.500s", query, code, body)
|
||||
}
|
||||
require.Equal(t, 200, code)
|
||||
var names []string
|
||||
for _, m := range nameRegex.FindAllStringSubmatch(stripComments.ReplaceAllString(body, ""), -1) {
|
||||
names = append(names, strings.TrimSpace(m[1]))
|
||||
}
|
||||
return names
|
||||
}
|
||||
|
||||
t.Cleanup(func() {
|
||||
_, _ = setup.DBPool.Exec(ctx, "DELETE FROM libraries WHERE id = $1", libID)
|
||||
})
|
||||
|
||||
for _, fx := range fixtures {
|
||||
for _, b := range fx.books {
|
||||
var itemID string
|
||||
err := setup.DBPool.QueryRow(ctx,
|
||||
`INSERT INTO media_items (library_id, title, file_path, series, authors, created_at, imported_at, updated_at)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $6, $6) RETURNING id`,
|
||||
libID, b.title, fmt.Sprintf("/tmp/test-series-filter/%s.epub", b.title), fx.name, []string{fx.author}, fx.createdAt,
|
||||
).Scan(&itemID)
|
||||
require.NoError(t, err)
|
||||
if b.percentage != "0" && b.percentage != "" {
|
||||
_, err = setup.DBPool.Exec(ctx,
|
||||
`INSERT INTO reading_progress (media_item_id, user_id, percentage, last_read_at)
|
||||
VALUES ($1, $2, $3::float, $4::timestamptz)`,
|
||||
itemID, viewerID, b.percentage, b.lastReadAt,
|
||||
)
|
||||
require.NoError(t, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
t.Run("default sort is recently added", func(t *testing.T) {
|
||||
assert.Equal(t, []string{"Probe Zebra Series", "Probe Mango Series", "Probe Apple Series"}, renderedNames(""))
|
||||
})
|
||||
|
||||
t.Run("every sort key orders the grid", func(t *testing.T) {
|
||||
cases := []struct {
|
||||
sort string
|
||||
want []string
|
||||
}{
|
||||
{"sort=recent", []string{"Zebra", "Mango", "Apple"}},
|
||||
{"sort=recent_asc", []string{"Apple", "Mango", "Zebra"}},
|
||||
{"sort=name_asc", []string{"Apple", "Mango", "Zebra"}},
|
||||
{"sort=name_desc", []string{"Zebra", "Mango", "Apple"}},
|
||||
{"sort=author_asc", []string{"Mango", "Apple", "Zebra"}},
|
||||
{"sort=author_desc", []string{"Zebra", "Apple", "Mango"}},
|
||||
{"sort=books_desc", []string{"Mango", "Zebra", "Apple"}},
|
||||
{"sort=books_asc", []string{"Apple", "Zebra", "Mango"}},
|
||||
{"sort=progress_desc", []string{"Mango", "Zebra", "Apple"}},
|
||||
{"sort=progress_asc", []string{"Apple", "Zebra", "Mango"}},
|
||||
{"sort=last_read_desc", []string{"Mango", "Zebra", "Apple"}},
|
||||
{"sort=last_read_asc", []string{"Zebra", "Mango", "Apple"}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
got := renderedNames("&" + tc.sort)
|
||||
require.Len(t, got, 3, tc.sort)
|
||||
for i, want := range tc.want {
|
||||
assert.Contains(t, got[i], want, "%s: position %d", tc.sort, i)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("unknown sort falls back to the default", func(t *testing.T) {
|
||||
// Unknown keys resolve to "recent" in the registry; the SQL never
|
||||
// interpolates the value (bound-param CASE cascade), so a hostile
|
||||
// sort value is inert by construction.
|
||||
assert.Equal(t, []string{"Probe Zebra Series", "Probe Mango Series", "Probe Apple Series"}, renderedNames("&sort=bogus-key"))
|
||||
})
|
||||
|
||||
t.Run("base sort + direction toggle composes registry keys", func(t *testing.T) {
|
||||
// The toolbar sends base keys + dir; defaults apply when dir is
|
||||
// absent, and both directions of every base key are reachable.
|
||||
cases := []struct {
|
||||
query string
|
||||
want []string
|
||||
}{
|
||||
{"sort=name", []string{"Apple", "Mango", "Zebra"}}, // name asc default
|
||||
{"sort=name&dir=desc", []string{"Zebra", "Mango", "Apple"}}, // toggle flips
|
||||
{"sort=books&dir=asc", []string{"Apple", "Zebra", "Mango"}}, // books desc default, flipped
|
||||
{"sort=progress&dir=asc", []string{"Apple", "Zebra", "Mango"}}, // progress desc default, flipped
|
||||
{"sort=author", []string{"Mango", "Apple", "Zebra"}}, // author asc default
|
||||
// THE toggle regression (2026-10-27): "recent" is both a base
|
||||
// key and a registry key — dir=asc used to be swallowed.
|
||||
{"sort=recent&dir=asc", []string{"Apple", "Mango", "Zebra"}},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
got := renderedNames("&" + tc.query)
|
||||
require.Len(t, got, 3, tc.query)
|
||||
for i, want := range tc.want {
|
||||
assert.Contains(t, got[i], want, "%s: position %d", tc.query, i)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("name query filters", func(t *testing.T) {
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&q=Mango"))
|
||||
// ILIKE is case-insensitive and substring-shaped.
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&q=mango"))
|
||||
})
|
||||
|
||||
t.Run("multi-field query matches authors and book titles", func(t *testing.T) {
|
||||
// Author of Mango is "Alpha One" — the query no longer needs the
|
||||
// separate author box.
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&q=alpha+one"))
|
||||
// "Zebra Book 2" is a book TITLE inside the Zebra series: the
|
||||
// series must surface for a title match too.
|
||||
assert.Equal(t, []string{"Probe Zebra Series"}, renderedNames("&q=zebra+book+2"))
|
||||
// Nothing matches anywhere.
|
||||
assert.Empty(t, renderedNames("&q=nobody-nowhere"))
|
||||
})
|
||||
|
||||
t.Run("author param (API back-compat) filters on any book's author", func(t *testing.T) {
|
||||
// The toolbar no longer renders a separate author box (the
|
||||
// multi-field q covers authors), but the param stays live for
|
||||
// the JSON API and deep links.
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&author=alpha"))
|
||||
assert.Empty(t, renderedNames("&author=nobody"))
|
||||
})
|
||||
|
||||
t.Run("status filter matches the viewer's progress", func(t *testing.T) {
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&status=finished"), "all books finished")
|
||||
assert.Equal(t, []string{"Probe Zebra Series"}, renderedNames("&status=reading"), "some book mid-read")
|
||||
assert.Equal(t, []string{"Probe Apple Series"}, renderedNames("&status=unstarted"), "no progress anywhere")
|
||||
})
|
||||
|
||||
t.Run("filters compose", func(t *testing.T) {
|
||||
assert.Equal(t, []string{"Probe Zebra Series"}, renderedNames("&q=probe&status=reading"))
|
||||
assert.Equal(t, []string{"Probe Mango Series"}, renderedNames("&q=probe&status=finished&sort=progress_desc"))
|
||||
})
|
||||
|
||||
t.Run("cards carry the viewer's read count", func(t *testing.T) {
|
||||
_, body := ssrGet(t, setup.Server.URL+"/series?library_id="+libID, setup.RegularToken)
|
||||
assert.Contains(t, body, "3 of 3 read")
|
||||
assert.Contains(t, body, "1 of 2 read")
|
||||
assert.NotContains(t, body, "of 1 read", "unstarted series must not render a progress line")
|
||||
})
|
||||
|
||||
t.Run("JSON API shares the sort/filter contract", func(t *testing.T) {
|
||||
req, err := http.NewRequest("GET", setup.Server.URL+"/api/series?library_id="+libID+"&sort=author_asc&status=finished", nil)
|
||||
require.NoError(t, err)
|
||||
req.Header.Set("Authorization", "Bearer "+setup.RegularToken)
|
||||
resp, err := (&http.Client{}).Do(req)
|
||||
require.NoError(t, err)
|
||||
defer resp.Body.Close()
|
||||
require.Equal(t, 200, resp.StatusCode)
|
||||
|
||||
var payload struct {
|
||||
Series []struct {
|
||||
Name string `json:"name"`
|
||||
ReadCount int64 `json:"read_count"`
|
||||
} `json:"series"`
|
||||
Total int `json:"total"`
|
||||
}
|
||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&payload))
|
||||
require.Len(t, payload.Series, 1)
|
||||
assert.Equal(t, "Probe Mango Series", payload.Series[0].Name)
|
||||
assert.Equal(t, int64(3), payload.Series[0].ReadCount)
|
||||
assert.Equal(t, 1, payload.Total)
|
||||
})
|
||||
}
|
||||
@@ -253,7 +253,7 @@ func TestReadingStatsSummary(t *testing.T) {
|
||||
media, err = setup.DB.GetMediaItem(ctx, uuidToPGType(uuid.MustParse(reuseMediaID[0])))
|
||||
require.NoError(t, err, "reuse seeded media item")
|
||||
} else {
|
||||
media, err = setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err = setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: [16]byte(libUUID), Valid: true},
|
||||
Title: title,
|
||||
FilePath: fmt.Sprintf("/tmp/stats-%s-%d.epub", title, time.Now().UnixNano()),
|
||||
@@ -448,7 +448,7 @@ func TestReadingStatsWeekLibraryScope(t *testing.T) {
|
||||
libB := uuid.MustParse(createLibraryDirect(t, setup.DB, userID, fmt.Sprintf("Test Scope LibB %d", time.Now().UnixNano()), "ebooks"))
|
||||
|
||||
seed := func(lib uuid.UUID, title string, daysAgo int, minutes int32) {
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: [16]byte(lib), Valid: true},
|
||||
Title: title,
|
||||
FilePath: fmt.Sprintf("/tmp/%s-%d.epub", title, time.Now().UnixNano()),
|
||||
@@ -526,7 +526,6 @@ func TestReadingStatsWeekLibraryScope(t *testing.T) {
|
||||
func TestLibraryUsageStats(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
ebooksID := uuid.MustParse(createLibraryDirect(t, setup.DB, userID,
|
||||
fmt.Sprintf("Test Usage Ebooks %d", time.Now().UnixNano()), "ebooks"))
|
||||
@@ -541,7 +540,7 @@ func TestLibraryUsageStats(t *testing.T) {
|
||||
if durationSeconds > 0 {
|
||||
audioMeta = []byte(fmt.Sprintf(`{"duration_seconds":%d,"chapters":[]}`, durationSeconds))
|
||||
}
|
||||
media, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: [16]byte(lib), Valid: true},
|
||||
Title: title,
|
||||
Authors: []string{author},
|
||||
@@ -689,7 +688,6 @@ func TestLibraryUsageStats(t *testing.T) {
|
||||
func TestLibraryUsageStats_Visibility(t *testing.T) {
|
||||
setup := setupTestServer(t)
|
||||
userID := getRegularUserID(t, setup.DB)
|
||||
ctx := context.Background()
|
||||
|
||||
visibleID := uuid.MustParse(createLibraryDirect(t, setup.DB, userID,
|
||||
fmt.Sprintf("Test Usage Visible %d", time.Now().UnixNano()), "ebooks"))
|
||||
@@ -697,7 +695,7 @@ func TestLibraryUsageStats_Visibility(t *testing.T) {
|
||||
fmt.Sprintf("Test Usage Hidden %d", time.Now().UnixNano()), "ebooks"))
|
||||
|
||||
seed := func(lib uuid.UUID, title string, size int64) {
|
||||
_, err := setup.DB.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
_, err := setup.DB.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: pgtype.UUID{Bytes: [16]byte(lib), Valid: true},
|
||||
Title: title,
|
||||
FilePath: fmt.Sprintf("/tmp/usage-vis-%s-%d.bin", title, time.Now().UnixNano()),
|
||||
|
||||
@@ -3,6 +3,7 @@ package main
|
||||
import (
|
||||
"bookhoard/internal/config"
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/rls"
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
@@ -10,6 +11,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
"github.com/stretchr/testify/assert"
|
||||
@@ -26,11 +28,16 @@ func setupSyncTestDB(t *testing.T) *database.Queries {
|
||||
dbConfig, err := pgxpool.ParseConfig(dbURL)
|
||||
require.NoError(t, err, "Failed to parse database URL")
|
||||
dbConfig.MaxConns = 1
|
||||
// Harness-only session identity (plan §5): see setupTestServer.
|
||||
dbConfig.BeforeAcquire = func(ctx context.Context, conn *pgx.Conn) bool {
|
||||
_, err := conn.Exec(ctx, `SELECT set_config('app.user_id', $1, false)`, rls.SeedIdentity.String())
|
||||
return err == nil
|
||||
}
|
||||
|
||||
dbPool, err := pgxpool.NewWithConfig(ctx, dbConfig)
|
||||
require.NoError(t, err, "Failed to connect to test database")
|
||||
|
||||
db := database.New(dbPool)
|
||||
db := database.New(database.NewSeedDB(dbPool))
|
||||
t.Cleanup(func() {
|
||||
// Scoped wipes ONLY: reading_history now carries real captured
|
||||
// sessions for the shared dev accounts, so a table-wide DELETE
|
||||
@@ -90,9 +97,8 @@ func createSyncTestDevice(t *testing.T, db *database.Queries, userID pgtype.UUID
|
||||
}
|
||||
|
||||
func createSyncTestMedia(t *testing.T, db *database.Queries, libraryID pgtype.UUID) pgtype.UUID {
|
||||
ctx := context.Background()
|
||||
|
||||
media, err := db.CreateMediaItem(ctx, database.CreateMediaItemParams{
|
||||
media, err := db.CreateMediaItem(seedCtx(), database.CreateMediaItemParams{
|
||||
LibraryID: libraryID,
|
||||
Title: "Sync Test Book",
|
||||
Authors: []string{"Test Author"},
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/middleware"
|
||||
ratelimit "bookhoard/internal/middleware"
|
||||
"bookhoard/internal/rls"
|
||||
"bookhoard/internal/router"
|
||||
"bookhoard/internal/services"
|
||||
wsync "bookhoard/internal/sync"
|
||||
@@ -317,6 +318,37 @@ func createRegularUserOnce(t *testing.T, db *database.Queries) UserTestData {
|
||||
}
|
||||
}
|
||||
|
||||
// seedCtx is the context harness-level media seeding runs under. It
|
||||
// carries rls.SeedIdentity — a valid UUID with NO library_visibility
|
||||
// rows, so RLS grants it full visibility (needed because sqlc :one
|
||||
// inserts use RETURNING, which requires SELECT visibility of the new
|
||||
// row). setup.DB additionally falls back to this identity for calls
|
||||
// that pass a bare context. Product code never uses this; requests
|
||||
// carry real personas.
|
||||
func seedCtx() context.Context {
|
||||
return rls.WithUser(context.Background(), rls.SeedIdentity)
|
||||
}
|
||||
|
||||
// seedExec runs a raw statement against the pool under the seed
|
||||
// identity. The bare pool bypasses ScopedDB (nothing reads the ctx for
|
||||
// the GUC), so governed tables need the identity declared manually
|
||||
// inside a transaction.
|
||||
func seedExec(t *testing.T, pool *pgxpool.Pool, sql string, args ...any) {
|
||||
t.Helper()
|
||||
ctx := rls.WithUser(context.Background(), rls.SeedIdentity)
|
||||
tx, err := pool.Begin(ctx)
|
||||
require.NoError(t, err)
|
||||
if _, err := tx.Exec(ctx, `SELECT set_config('app.user_id', $1, true)`, rls.SeedIdentity.String()); err != nil {
|
||||
_ = tx.Rollback(ctx)
|
||||
require.NoError(t, err, "declare seed identity")
|
||||
}
|
||||
if _, err := tx.Exec(ctx, sql, args...); err != nil {
|
||||
_ = tx.Rollback(ctx)
|
||||
require.NoError(t, err, "seed exec")
|
||||
}
|
||||
require.NoError(t, tx.Commit(ctx))
|
||||
}
|
||||
|
||||
// uuidToPGType converts uuid.UUID to pgtype.UUID
|
||||
func uuidToPGType(u uuid.UUID) pgtype.UUID {
|
||||
return pgtype.UUID{Bytes: [16]byte(u), Valid: true}
|
||||
@@ -491,10 +523,29 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
dbConfig, err := pgxpool.ParseConfig(cfg.DatabaseURL())
|
||||
require.NoError(t, err, "Failed to parse database URL")
|
||||
dbConfig.MaxConns = 1
|
||||
// Harness-only session identity (plan §5): every connection in this
|
||||
// pool runs as rls.SeedIdentity at SESSION scope, so raw-pool
|
||||
// statements in tests (DBPool.Exec/QueryRow against governed tables)
|
||||
// pass the read policy. No bleed risk: this pool never serves a
|
||||
// second identity — A3 applies to the server's request pool.
|
||||
dbConfig.BeforeAcquire = func(ctx context.Context, conn *pgx.Conn) bool {
|
||||
_, err := conn.Exec(ctx, `SELECT set_config('app.user_id', $1, false)`, rls.SeedIdentity.String())
|
||||
return err == nil
|
||||
}
|
||||
dbPool, err := pgxpool.NewWithConfig(context.Background(), dbConfig)
|
||||
require.NoError(t, err, "Failed to connect to test database")
|
||||
|
||||
queries := database.New(dbPool)
|
||||
// ScopedDB: the test server must behave EXACTLY like the production
|
||||
// wiring (main.go) — every statement through Queries declares the
|
||||
// acting user for RLS, including the search service's raw DBTX path.
|
||||
// TestServerSetup.DB below is the SEED handle — also ScopedDB-wired
|
||||
// (required: sqlc :one inserts use RETURNING, which evaluates the
|
||||
// SELECT policy even on INSERT) — but its callers pass seedCtx(), a
|
||||
// synthetic identity with no visibility rows, so seeding sees and
|
||||
// writes everything. Real personas only exist at the HTTP layer.
|
||||
scopedDB := database.NewScopedDB(dbPool)
|
||||
queries := database.New(scopedDB)
|
||||
seedDB := database.New(database.NewSeedDB(dbPool))
|
||||
|
||||
// Wire the tunable settings registry exactly like main.go: Load
|
||||
// populates the cache (missing rows fall back to compiled defaults)
|
||||
@@ -552,13 +603,21 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
statisticsService := services.NewStatisticsService(queries)
|
||||
statsHandler := handlers.NewStatsHandler(statisticsService)
|
||||
seriesHandler := handlers.NewSeriesHandler(queries)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, dbPool, libraryService, progressService, annotationService, worker)
|
||||
mediaHandler := handlers.NewMediaHandler(queries, scopedDB, libraryService, progressService, annotationService, worker)
|
||||
matchingHandler := handlers.NewMatchingHandler(queries, connManager)
|
||||
|
||||
// Create conversion service for OPDS
|
||||
conversionService := services.NewConversionService(queries, getCachePath())
|
||||
opdsHandler := handlers.NewOPDSHandler(queries, libraryService, conversionService)
|
||||
|
||||
// Podcast subsystem (mirrors main.go wiring): fetcher service +
|
||||
// download worker + handler. The poller is NOT started in tests
|
||||
// (no background ticks against the shared dev DB); the worker loop
|
||||
// only drains when a test queues work.
|
||||
podcastService := services.NewPodcastService(queries)
|
||||
podcastWorker := services.NewPodcastDownloadWorker(queries, podcastService)
|
||||
podcastHandler := handlers.NewPodcastHandler(queries, podcastService, podcastWorker)
|
||||
|
||||
// Create Echo instance
|
||||
e := echo.New()
|
||||
|
||||
@@ -621,6 +680,9 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
// hands the kobo/kepub CFI converter a typed-nil (kobo markup
|
||||
// panic-recovered as 500s in TestProgressWeb_KoboIntegration).
|
||||
LibraryService: libraryService,
|
||||
PodcastHandler: podcastHandler,
|
||||
PodcastService: podcastService,
|
||||
PodcastWorker: podcastWorker,
|
||||
}
|
||||
|
||||
router.RegisterRoutes(routerConfig)
|
||||
@@ -756,7 +818,7 @@ func setupTestServer(t *testing.T) *TestServerSetup {
|
||||
// Create TestServerSetup struct with all resources
|
||||
setup := &TestServerSetup{
|
||||
Server: ts,
|
||||
DB: queries,
|
||||
DB: seedDB, // harness seeding/cleanup handle (see comment above)
|
||||
DBPool: dbPool,
|
||||
Config: cfg,
|
||||
ConnManager: connManager,
|
||||
|
||||
+266
-2
@@ -18,7 +18,8 @@ INSERT INTO library_types (name, description, allowed_extensions) VALUES
|
||||
('ebooks', 'Ebook files including EPUB, PDF, MOBI, etc.', ARRAY['.epub', '.pdf', '.mobi', '.azw', '.azw3', '.txt', '.rtf', '.doc', '.docx', '.lit', '.fb2', '.pdb']),
|
||||
('comics', 'Comic book archives and image formats', ARRAY['.cbz', '.cbr', '.cb7', '.cbt', '.epub', '.pdf']),
|
||||
('manga', 'Manga files including archives and image folders', ARRAY['.cbz', '.cbr', '.epub', '.pdf', '.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp', '.avif', '.tiff', '.tif']),
|
||||
('audiobooks', 'Audiobook files; m4b first, breadth formats registered from day one', ARRAY['.m4b', '.m4a', '.mp3', '.flac'])
|
||||
('audiobooks', 'Audiobook files; m4b first, breadth formats registered from day one', ARRAY['.m4b', '.m4a', '.mp3', '.flac']),
|
||||
('podcasts', 'RSS podcast/feed libraries: feeds subscribe-and-download; the scanner never walks these folders', ARRAY[]::TEXT[])
|
||||
ON CONFLICT (name) DO NOTHING;
|
||||
|
||||
-- Create users table
|
||||
@@ -99,7 +100,9 @@ INSERT INTO system_settings (setting_key, setting_value, description, setting_ty
|
||||
('sync_queue_interval_seconds', '5', 'How often the sync queue flushes', 'int', '1', '3600', TRUE, 'sync'),
|
||||
('sync_queue_batch_size', '50', 'Maximum items processed per sync queue flush', 'int', '1', '10000', TRUE, 'sync'),
|
||||
('worker_pool_size', '3', 'Number of background worker goroutines', 'int', '1', '100', TRUE, 'performance'),
|
||||
('worker_queue_cap', '100', 'Background worker job queue capacity', 'int', '1', '10000', TRUE, 'performance')
|
||||
('worker_queue_cap', '100', 'Background worker job queue capacity', 'int', '1', '10000', TRUE, 'performance'),
|
||||
-- podcasts
|
||||
('podcast_poll_interval_seconds', '3600', 'How often to poll podcast feeds for new episodes (seconds; complete feeds back off)', 'int', '300', '604800', FALSE, 'podcasts')
|
||||
ON CONFLICT (setting_key) DO NOTHING;
|
||||
|
||||
-- Create refresh_tokens table
|
||||
@@ -1877,3 +1880,264 @@ CREATE INDEX IF NOT EXISTS idx_hash_conflicts_status ON hash_conflicts(status);
|
||||
|
||||
-- Widen for databases created before the resolution format settled (no-op otherwise)
|
||||
ALTER TABLE hash_conflicts ALTER COLUMN resolution TYPE VARCHAR(50);
|
||||
|
||||
-- ============================================================
|
||||
-- PODCASTS / RSS FEEDS
|
||||
-- Plan: ~/.opencode/plan/podcast-integration.md (§6).
|
||||
-- Feeds are per-library entities (UNIQUE(library_id, feed_url)); users
|
||||
-- subscribe personally (podcast_subscriptions). Episodes are
|
||||
-- media_items rows with source='podcast_episode' — per-episode
|
||||
-- progress/stats for free — grouped by source_feed_id. Visibility is
|
||||
-- RLS-enforced (see the policy block at the end of this file): an
|
||||
-- episode/feed row is SELECT-visible only to subscribers of the feed
|
||||
-- and to admins. The fetcher/download worker is the SOLE writer of
|
||||
-- feed/episode metadata; users own only their subscription row and the
|
||||
-- reader_settings order override. ALL DDL IDEMPOTENT: schema.sql
|
||||
-- re-executes on EVERY boot.
|
||||
-- ============================================================
|
||||
|
||||
-- First per-library boolean knob (plain column, not KV): when false,
|
||||
-- only admins may add feeds to the library (members keep subscribe
|
||||
-- rights to existing feeds).
|
||||
ALTER TABLE libraries ADD COLUMN IF NOT EXISTS allow_member_feeds BOOLEAN NOT NULL DEFAULT TRUE;
|
||||
|
||||
-- Folder purpose: 'scan' (default; the FS walker's root) vs 'podcast'
|
||||
-- (a podcasts library's download root — scanner skips it wholesale,
|
||||
-- belt-and-braces on top of the library-type skip).
|
||||
ALTER TABLE library_folders ADD COLUMN IF NOT EXISTS purpose VARCHAR(20) NOT NULL DEFAULT 'scan';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS podcast_feeds (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
library_id UUID NOT NULL REFERENCES libraries(id) ON DELETE CASCADE,
|
||||
feed_url TEXT NOT NULL,
|
||||
-- podcast:guid — the feed's canonical identity; dedupes across URL
|
||||
-- migrations (fetcher honors itunes:new-feed-url + redirects).
|
||||
podcast_guid TEXT,
|
||||
title VARCHAR(500) NOT NULL,
|
||||
description TEXT,
|
||||
-- Channel itunes:author (semantically the host) and itunes:owner.
|
||||
host VARCHAR(500),
|
||||
publisher VARCHAR(500),
|
||||
owner_name VARCHAR(500),
|
||||
owner_email VARCHAR(500),
|
||||
language VARCHAR(50),
|
||||
tags TEXT[] DEFAULT '{}',
|
||||
explicit BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
artwork_path VARCHAR(500),
|
||||
-- itunes:type: 'serial' | 'episodic'
|
||||
feed_type VARCHAR(20),
|
||||
-- podcast:medium: 'audiobook' | 'podcast' | 'music' | ...
|
||||
medium VARCHAR(20),
|
||||
-- itunes:complete: poller backs off once the publisher ends the feed.
|
||||
is_complete BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
-- Conditional-GET validators (opaque; echoed verbatim).
|
||||
etag VARCHAR(500),
|
||||
last_modified VARCHAR(500),
|
||||
last_fetched_at TIMESTAMPTZ,
|
||||
last_build_date TIMESTAMPTZ,
|
||||
created_by_admin_id UUID REFERENCES users(id) ON DELETE SET NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
UNIQUE(library_id, feed_url)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_podcast_feeds_library ON podcast_feeds(library_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_podcast_feeds_guid ON podcast_feeds(podcast_guid);
|
||||
|
||||
-- User-owned subscription row: the ONLY podcast state a user writes.
|
||||
-- display_name NULL renders as the feed title. auto_download:
|
||||
-- 'off' | 'all' | 'newest' (+ auto_download_count for newest N).
|
||||
CREATE TABLE IF NOT EXISTS podcast_subscriptions (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
feed_id UUID NOT NULL REFERENCES podcast_feeds(id) ON DELETE CASCADE,
|
||||
display_name VARCHAR(500),
|
||||
auto_download VARCHAR(20) NOT NULL DEFAULT 'off',
|
||||
auto_download_count INT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
UNIQUE(user_id, feed_id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_podcast_subs_user ON podcast_subscriptions(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_podcast_subs_feed ON podcast_subscriptions(feed_id);
|
||||
|
||||
-- Episode columns (NULL for scan-sourced items). See the plan §5 for
|
||||
-- the full RSS/iTunes/Podcasting-2.0 field mapping — existing columns
|
||||
-- (title, subtitle, description, authors, explicit, date_published,
|
||||
-- cover_image_path, file_size, mime_type, language, genre/tags) carry
|
||||
-- the shared fields; only genuinely podcast-only storage lives here.
|
||||
ALTER TABLE media_items
|
||||
ADD COLUMN IF NOT EXISTS source VARCHAR(20) NOT NULL DEFAULT 'scan',
|
||||
ADD COLUMN IF NOT EXISTS source_feed_id UUID REFERENCES podcast_feeds(id) ON DELETE CASCADE,
|
||||
-- Stable reorder-proof addressing key (invariant 3): progress
|
||||
-- anchors on guid, never ordinal.
|
||||
ADD COLUMN IF NOT EXISTS episode_guid TEXT,
|
||||
-- Native order as of the latest fetch (renumbered per fetch).
|
||||
ADD COLUMN IF NOT EXISTS feed_position INTEGER,
|
||||
ADD COLUMN IF NOT EXISTS season_number INTEGER,
|
||||
ADD COLUMN IF NOT EXISTS episode_number INTEGER,
|
||||
ADD COLUMN IF NOT EXISTS enclosure_url TEXT,
|
||||
-- itunes:duration claim (often HH:MM:SS or prose); display until
|
||||
-- ffprobe reconciles after download.
|
||||
ADD COLUMN IF NOT EXISTS declared_duration_seconds INTEGER,
|
||||
ADD COLUMN IF NOT EXISTS transcript_url TEXT,
|
||||
ADD COLUMN IF NOT EXISTS chapters_url TEXT,
|
||||
ADD COLUMN IF NOT EXISTS hosts TEXT[] DEFAULT '{}',
|
||||
ADD COLUMN IF NOT EXISTS guests TEXT[] DEFAULT '{}',
|
||||
-- Download state machine: pending | queued | downloading |
|
||||
-- downloaded | failed (+ reason text + attempt count). The episode
|
||||
-- list renders this inline (D8/D19) — never Processing Issues.
|
||||
ADD COLUMN IF NOT EXISTS download_state VARCHAR(20) NOT NULL DEFAULT 'pending',
|
||||
ADD COLUMN IF NOT EXISTS download_error TEXT,
|
||||
ADD COLUMN IF NOT EXISTS download_attempts INT NOT NULL DEFAULT 0;
|
||||
|
||||
-- Identity + ordering + worker-worklist indexes (all partial: scan
|
||||
-- rows never touch these columns).
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_media_items_feed_guid
|
||||
ON media_items(source_feed_id, episode_guid) WHERE source_feed_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_media_items_feed_pos
|
||||
ON media_items(source_feed_id, feed_position) WHERE source_feed_id IS NOT NULL;
|
||||
CREATE INDEX IF NOT EXISTS idx_media_items_feed_undownloaded
|
||||
ON media_items(source_feed_id) WHERE source_feed_id IS NOT NULL AND download_state <> 'downloaded';
|
||||
|
||||
|
||||
-- ============================================================
|
||||
-- RLS: PER-USER LIBRARY VISIBILITY (ONE SOURCE OF TRUTH)
|
||||
-- Plan: ~/.opencode/plan/server-rls-library-visibility.md REV 2.
|
||||
-- The policy joins library_visibility DIRECTLY — the same table the
|
||||
-- visibility UI writes. The app declares the acting user per
|
||||
-- transaction via the LOCAL GUC app.user_id (ScopedDB, set_config
|
||||
-- is_local=true). Unset identity FAILS LOUD, never fails open:
|
||||
-- - GUC never set in the session -> current_setting() raises
|
||||
-- ("unrecognized configuration parameter")
|
||||
-- - GUC set-then-transaction-ended -> reverts to '' -> the ::uuid
|
||||
-- cast raises
|
||||
-- - NULL GUC (unreachable via ScopedDB) -> IS NOT NULL guard ->
|
||||
-- zero rows (fail-closed)
|
||||
-- The runtime role bookhoard_srv OWNS these tables and is subject to
|
||||
-- the policies via FORCE (z_rls_role.sh). Never connect the app as
|
||||
-- the bootstrap superuser — superusers bypass RLS unconditionally.
|
||||
-- IDEMPOTENT: schema.sql re-executes on EVERY app boot
|
||||
-- (database.Initialize -> executeSchema); DROP .. IF EXISTS + CREATE
|
||||
-- matches the repo's trigger idiom. DO NOT make this block
|
||||
-- non-idempotent — the second boot would brick the app.
|
||||
-- ============================================================
|
||||
|
||||
DROP POLICY IF EXISTS p_media_items_visible ON media_items;
|
||||
DROP POLICY IF EXISTS p_media_items_insert ON media_items;
|
||||
DROP POLICY IF EXISTS p_media_items_update ON media_items;
|
||||
DROP POLICY IF EXISTS p_media_items_delete ON media_items;
|
||||
|
||||
ALTER TABLE media_items ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE media_items FORCE ROW LEVEL SECURITY;
|
||||
|
||||
-- READ policy: EXACTLY the app's COALESCE(lv.is_visible, true)
|
||||
-- semantics — row absent -> visible; row true -> visible; row false
|
||||
-- -> hidden. No new index needed: library_visibility's
|
||||
-- UNIQUE(user_id, library_id) constraint already indexes the join.
|
||||
-- PODCAST EPISODES (plan D20): source='podcast_episode' rows carry the
|
||||
-- ADDITIONAL subscription predicate — visible only to subscribers of
|
||||
-- source_feed_id, or to admins (admins manage feeds regardless of
|
||||
-- subscription; the escape also satisfies background jobs, which
|
||||
-- declare a first-admin identity). Scan rows pass unchanged via the
|
||||
-- IS DISTINCT FROM guard — the predicate costs them one NULL compare.
|
||||
-- Unset identity FAILS LOUD, never fails open (see the block header).
|
||||
CREATE POLICY p_media_items_visible ON media_items
|
||||
FOR SELECT
|
||||
USING (
|
||||
current_setting('app.user_id') IS NOT NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM library_visibility lv
|
||||
WHERE lv.user_id = current_setting('app.user_id')::uuid
|
||||
AND lv.library_id = media_items.library_id
|
||||
AND lv.is_visible = false
|
||||
)
|
||||
AND (
|
||||
media_items.source IS DISTINCT FROM 'podcast_episode'
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM podcast_subscriptions s
|
||||
WHERE s.user_id = current_setting('app.user_id')::uuid
|
||||
AND s.feed_id = media_items.source_feed_id
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM users u
|
||||
WHERE u.id = current_setting('app.user_id')::uuid
|
||||
AND u.role = 'admin'
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
-- WRITE policies: PERMISSIVE STAND-INS for the deferred scoped write
|
||||
-- lens. MANDATORY at ENABLE time — with RLS enabled and no applicable
|
||||
-- policy, INSERT fails (42501) and UPDATE/DELETE silently affect zero
|
||||
-- rows, bricking the media scanner. Per-command split is deliberate:
|
||||
-- a single FOR ALL policy with USING (true) would OR into SELECT and
|
||||
-- disable the read policy above. Writes still need SELECT visibility
|
||||
-- of the target rows (the read policy governs that), which internal
|
||||
-- identities satisfy (they have no hidden rows).
|
||||
-- ON CONFLICT upserts (scanner: (library_id, file_path)) need the
|
||||
-- conflicting row SELECT-visible to the acting identity — holds for
|
||||
-- every internal identity by construction.
|
||||
CREATE POLICY p_media_items_insert ON media_items
|
||||
FOR INSERT WITH CHECK (true);
|
||||
CREATE POLICY p_media_items_update ON media_items
|
||||
FOR UPDATE USING (true) WITH CHECK (true);
|
||||
CREATE POLICY p_media_items_delete ON media_items
|
||||
FOR DELETE USING (true);
|
||||
|
||||
-- Other tables: collection_items reaches media only via INNER
|
||||
-- JOIN media_items (the join drops filtered rows). collections
|
||||
-- metadata (book_count) is a separate counts question — plan A13.
|
||||
|
||||
-- PODCAST FEEDS (plan D20): governed the same way. SELECT-visible to
|
||||
-- subscribers of the feed (their subscription IS the grant — this is
|
||||
-- the predicate's lookup table, podcast_subscriptions itself stays
|
||||
-- UNGOVERNED) plus the library-visibility join (a hidden library
|
||||
-- hides its feeds too), and to admins. WRITE policies are permissive
|
||||
-- stand-ins for the same reason as media_items above: mandatory at
|
||||
-- ENABLE time, and the fetcher/worker (first-admin identity) plus the
|
||||
-- add-feed upsert (ON CONFLICT DO UPDATE needs no separate SELECT)
|
||||
-- cover every legitimate writer.
|
||||
DROP POLICY IF EXISTS p_podcast_feeds_visible ON podcast_feeds;
|
||||
DROP POLICY IF EXISTS p_podcast_feeds_insert ON podcast_feeds;
|
||||
DROP POLICY IF EXISTS p_podcast_feeds_update ON podcast_feeds;
|
||||
DROP POLICY IF EXISTS p_podcast_feeds_delete ON podcast_feeds;
|
||||
|
||||
ALTER TABLE podcast_feeds ENABLE ROW LEVEL SECURITY;
|
||||
ALTER TABLE podcast_feeds FORCE ROW LEVEL SECURITY;
|
||||
|
||||
CREATE POLICY p_podcast_feeds_visible ON podcast_feeds
|
||||
FOR SELECT
|
||||
USING (
|
||||
current_setting('app.user_id') IS NOT NULL
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM library_visibility lv
|
||||
WHERE lv.user_id = current_setting('app.user_id')::uuid
|
||||
AND lv.library_id = podcast_feeds.library_id
|
||||
AND lv.is_visible = false
|
||||
)
|
||||
AND (
|
||||
EXISTS (
|
||||
SELECT 1 FROM podcast_subscriptions s
|
||||
WHERE s.user_id = current_setting('app.user_id')::uuid
|
||||
AND s.feed_id = podcast_feeds.id
|
||||
)
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM users u
|
||||
WHERE u.id = current_setting('app.user_id')::uuid
|
||||
AND u.role = 'admin'
|
||||
)
|
||||
)
|
||||
);
|
||||
|
||||
CREATE POLICY p_podcast_feeds_insert ON podcast_feeds
|
||||
FOR INSERT WITH CHECK (true);
|
||||
CREATE POLICY p_podcast_feeds_update ON podcast_feeds
|
||||
FOR UPDATE USING (true) WITH CHECK (true);
|
||||
CREATE POLICY p_podcast_feeds_delete ON podcast_feeds
|
||||
FOR DELETE USING (true);
|
||||
|
||||
-- podcast_subscriptions: UNGOVERNED, deliberately — it IS the read
|
||||
-- policy's lookup table, and a user must always see their own
|
||||
-- subscription rows (subscribe -> visibility is a single INSERT).
|
||||
-- ============================================================
|
||||
@@ -12,6 +12,8 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: ./Dockerfile
|
||||
args:
|
||||
GIT_COMMIT: ${GIT_COMMIT:-none}
|
||||
|
||||
# Integration Tests - runs against containerized app and db (dev only)
|
||||
tests:
|
||||
@@ -22,10 +24,12 @@ services:
|
||||
container_name: bookhoard_tests
|
||||
environment:
|
||||
# Database Configuration
|
||||
# Runtime role = bookhoard_srv (non-superuser OWNER, subject to RLS
|
||||
# via FORCE) — same role the app service uses.
|
||||
DATABASE_HOST: db
|
||||
DATABASE_PORT: ${DB_PORT:-5432}
|
||||
DATABASE_USER: postgres
|
||||
DATABASE_PASSWORD: ${DBPASS}
|
||||
DATABASE_USER: bookhoard_srv
|
||||
DATABASE_PASSWORD: ${DBSRVPASS}
|
||||
DATABASE_NAME: bookhoard
|
||||
COOKIE_SECURE: false
|
||||
|
||||
|
||||
+59
-5
@@ -11,9 +11,6 @@ services:
|
||||
PGPORT: ${DB_PORT:-5432}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
- ./database/schema:/docker-entrypoint-initdb.d
|
||||
# Make other volumes as needed
|
||||
- ./uploads:/app/uploads
|
||||
ports:
|
||||
- "127.0.0.1:${DB_PORT:-5432}:${DB_PORT:-5432}"
|
||||
- "[::1]:${DB_PORT:-5432}:${DB_PORT:-5432}"
|
||||
@@ -26,6 +23,58 @@ services:
|
||||
env_file:
|
||||
- .env
|
||||
|
||||
# One-shot bootstrap: creates the bookhoard_srv runtime role and
|
||||
# transfers object ownership to it (RLS prerequisite). Idempotent —
|
||||
# safe on every startup; the app waits for its success. Passwords
|
||||
# accept inline values or .env, like the rest of this file.
|
||||
# NOTE: plpgsql $$ tags are written $$$$ — compose eats one dollar
|
||||
# per pair before the container sees the SQL.
|
||||
db-init:
|
||||
image: postgres:15-alpine
|
||||
container_name: bookhoard_db_init
|
||||
environment:
|
||||
PGHOST: db
|
||||
PGPORT: ${DB_PORT:-5432}
|
||||
PGUSER: postgres
|
||||
PGDATABASE: bookhoard
|
||||
PGPASSWORD: ${DBPASS}
|
||||
DBSRVPASS: ${DBSRVPASS}
|
||||
entrypoint: ["bash", "-c"]
|
||||
command:
|
||||
- |
|
||||
set -e
|
||||
: "${DBSRVPASS:?DBSRVPASS required in .env}"
|
||||
psql -v ON_ERROR_STOP=1 -v role_pass="$DBSRVPASS" <<'EOSQL'
|
||||
DO $$$$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'bookhoard_srv') THEN
|
||||
CREATE ROLE bookhoard_srv LOGIN;
|
||||
END IF;
|
||||
END $$$$;
|
||||
ALTER ROLE bookhoard_srv LOGIN PASSWORD :'role_pass';
|
||||
DO $$$$
|
||||
DECLARE r RECORD;
|
||||
BEGIN
|
||||
FOR r IN SELECT tablename FROM pg_tables WHERE schemaname = 'public' LOOP
|
||||
EXECUTE format('ALTER TABLE public.%I OWNER TO bookhoard_srv', r.tablename);
|
||||
END LOOP;
|
||||
FOR r IN SELECT sequencename FROM pg_sequences WHERE schemaname = 'public' LOOP
|
||||
EXECUTE format('ALTER SEQUENCE public.%I OWNER TO bookhoard_srv', r.sequencename);
|
||||
END LOOP;
|
||||
FOR r IN SELECT p.oid::regprocedure AS fn FROM pg_proc p
|
||||
JOIN pg_namespace n ON p.pronamespace = n.oid
|
||||
WHERE n.nspname = 'public' AND p.prokind = 'f' LOOP
|
||||
EXECUTE format('ALTER FUNCTION %s OWNER TO bookhoard_srv', r.fn);
|
||||
END LOOP;
|
||||
END $$$$;
|
||||
ALTER SCHEMA public OWNER TO bookhoard_srv;
|
||||
ALTER DATABASE bookhoard OWNER TO bookhoard_srv;
|
||||
EOSQL
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
restart: "no"
|
||||
|
||||
# Bookhoard Application
|
||||
# In production this image is pulled from the Gitea container registry.
|
||||
# Override IMAGE_TAG in .env to pin or rollback a specific version (defaults to "latest").
|
||||
@@ -35,10 +84,13 @@ services:
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
# Database Configuration
|
||||
# Runtime role = bookhoard_srv (non-superuser OWNER, subject to RLS
|
||||
# via FORCE). POSTGRES_USER above is the bootstrap account ONLY —
|
||||
# the app must NEVER connect as it (superusers bypass RLS).
|
||||
DATABASE_HOST: db
|
||||
DATABASE_PORT: ${DB_PORT:-5432}
|
||||
DATABASE_USER: postgres
|
||||
DATABASE_PASSWORD: ${DBPASS}
|
||||
DATABASE_USER: bookhoard_srv
|
||||
DATABASE_PASSWORD: ${DBSRVPASS}
|
||||
DATABASE_NAME: bookhoard
|
||||
|
||||
# Application Configuration
|
||||
@@ -75,6 +127,8 @@ services:
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
db-init:
|
||||
condition: service_completed_successfully
|
||||
volumes:
|
||||
- ./uploads:/app/uploads
|
||||
- bookhoard_conversion_cache:/app/cache/kepub
|
||||
|
||||
+47
-2
@@ -37,14 +37,15 @@ These are the response SHAPES (JSON key sets, not values) frozen by the
|
||||
| `CollectionsJSON` | `GET /api/collections/:id` (all 4 branches) | envelope `id, user_id, name, description, color, icon, auto_assign_rules, view_settings, books[], book_count, is_smart, created_at`; books = `BookInfo` |
|
||||
| | `POST /api/collections/preview` | bare `{items: BookInfo[]}` |
|
||||
| `SeriesJSON` | `GET /api/series/books?name=&library_id=` | `{name, library_id, books: BookInfo[], total}` |
|
||||
| `SearchJSON` | `GET /api/media-items/search?q=&library_id=` | top-level array of `SearchMediaItemsUnifiedRow` (47 keys) |
|
||||
| `SearchJSON` | `GET /api/media-items/search?q=&library_id=` | top-level array of `SearchMediaItemsUnifiedRow` (47 keys + viewer_percentage) |
|
||||
| `DashboardSections` | `GET /api/dashboard/sections` | `{sections[]: {id, collection_id, is_system, title, description, icon, items: BookInfo[], view_all_url, priority}}` |
|
||||
| `MediaListBothBranches` | `GET /api/media-items` (unscoped) | `{data: [38-key resolved map]}` (intentionally divergent from the scoped branch until the row-shape collapse batch) |
|
||||
| | `GET /api/media-items?library_id=&sort=` | `{data: [raw ListMediaItemsSortedRow, 70 sqlc keys]}` |
|
||||
| `SSRPages` | `/bookshelf`, `/collections`, `/collections/:id`, `/series`, `/series/detail`, `/tags/detail` | 200 + stable title/substring markers |
|
||||
|
||||
`BookInfo` (every card surface): `media_item_id, title, author,
|
||||
cover_image_path, has_conflict, format_group, badge`.
|
||||
cover_image_path, has_conflict, format_group, badge, viewer_percentage,
|
||||
library_name`.
|
||||
|
||||
## Changelog (additive keys per batch)
|
||||
|
||||
@@ -414,3 +415,47 @@ cover_image_path, has_conflict, format_group, badge`.
|
||||
BEFORE Rescan All — force rescan stamps books with the current
|
||||
setting, so bump-first + force = one pass of work; force-first +
|
||||
bump = the trickle re-derives everything again (wasted, not harmful).
|
||||
|
||||
## 2026-10-27 — Search-progress batch: viewer progress on cards
|
||||
- **`SearchJSON`**: `SearchMediaItemsUnifiedRow` + `viewer_percentage`
|
||||
(float 0..1, NULL when the requester never opened the book). The
|
||||
query LEFT JOINs `reading_progress` on the SAME `user_id` param the
|
||||
library-visibility join already binds — one requester param, no new
|
||||
wire inputs; the join is row-neutral (UNIQUE(media_item, user)).
|
||||
- **`BookInfo`** (CollectionsJSON / SeriesJSON / preview /
|
||||
DashboardSections / search cards): + `viewer_percentage` (JSON null
|
||||
when nil) and + `library_name` (empty on feeds that don't load it —
|
||||
only the search-unified adapter populates both today). Adapters
|
||||
other than BookCardsFromUnified keep emitting null/""; templ cards
|
||||
render the progress bar/completed check only when populated.
|
||||
- The header-search dropdown now renders the templ `SearchResults`
|
||||
partial (layout=dropdown on the same endpoint, HX-Request) instead
|
||||
of the retired innerHTML renderer in web/src/search.ts — same
|
||||
endpoint, same auth, content server-rendered.
|
||||
|
||||
## 2026-10-29 — RLS visibility: item-scoped gates may answer 404 as well as 403
|
||||
|
||||
With library visibility enforced by Postgres RLS (see
|
||||
database/schema/schema.sql), a hidden library's rows are INVISIBLE to
|
||||
the engine — item-scoped endpoints (chapters, tracks, download, detail)
|
||||
can no longer run their Go-level 403 gate because the row never arrives.
|
||||
Both answers are gated: 403 (Go-level explicit-library checks remain on
|
||||
listing endpoints) and 404 (row-level invisibility — no existence leak).
|
||||
Pins asserting 403 on item-scoped content endpoints widen to accept
|
||||
{403, 404}. Listing/scoped endpoints keep the strict 403 contract.
|
||||
|
||||
## 2026-10-29 — Podcast episode columns join media_items (additive widening)
|
||||
|
||||
The podcast integration (plan podcast-integration.md tasks 1.1/1.10) adds
|
||||
fifteen nullable/no-null-default columns to `media_items` — they surface in
|
||||
raw sqlc rows ONLY (`library-scoped list emits raw sqlc rows`), never in the
|
||||
resolved unscoped map:
|
||||
|
||||
`source` ('scan' default), `source_feed_id`, `episode_guid`, `feed_position`,
|
||||
`season_number`, `episode_number`, `enclosure_url`,
|
||||
`declared_duration_seconds`, `transcript_url`, `chapters_url`, `hosts`,
|
||||
`guests`, `download_state`, `download_error`, `download_attempts`.
|
||||
|
||||
Scan-sourced rows carry the inert defaults ('scan' / NULLs / 'pending');
|
||||
raw-row consumers that filter `source='scan'` (scanner paths) are unaffected.
|
||||
Nothing removed, nothing renamed.
|
||||
@@ -298,6 +298,35 @@ A: Calibre Web uses the same `metadata.opf` files, so yes - Bookhoard can scan a
|
||||
|
||||
A: Bookhoard has its own sorting logic and doesn't import Calibre sort fields. This allows for consistent sorting across all books.
|
||||
|
||||
## Writing metadata back to files (admin)
|
||||
|
||||
The book page's metadata editor has a **Write to file** action alongside
|
||||
Save. Save updates Bookhoard's database only; Write to file persists the
|
||||
currently saved values into the artifact on disk, then re-extracts so
|
||||
everything stays in sync:
|
||||
|
||||
- **EPUB/KEPUB** — the OPF embedded in the book is rewritten in place
|
||||
(cover, chapters and reading position are untouched). The file's
|
||||
checksum changes, so device syncs (Kobo, KOReader) re-key the book on
|
||||
their next sync.
|
||||
- **Single-file audio (MP3/FLAC/M4A/M4B)** — the container's own tags
|
||||
are rewritten (audio streams copied untouched, chapters and cover art
|
||||
preserved). Only fields the format can carry are written: title,
|
||||
subtitle (MP3/FLAC), authors, narrators (composer tag — Audible's
|
||||
convention), series, date and genre.
|
||||
- **CBZ** — a ComicInfo.xml is written inside the archive (injected or
|
||||
replaced). CBR/CB7 archives can't be rewritten in place, so they take
|
||||
the sidecar path.
|
||||
- **Everything else** (PDF, folder books, CBR/CB7) — a Calibre-style
|
||||
sidecar is written next to the file:
|
||||
`metadata.opf` when the book is alone in its folder (Calibre's own
|
||||
convention, so Calibre reads it too), or `<filename>.opf` when several
|
||||
books share a folder, so one book's sidecar can never leak onto its
|
||||
neighbours.
|
||||
|
||||
The write mirrors exactly what the scanner reads back, and the original
|
||||
file is never touched if anything fails (temp file + atomic rename).
|
||||
|
||||
## Resources
|
||||
|
||||
- [Calibre Website](https://calibre-ebook.com/)
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
import { test, expect, type APIRequestContext, type Page } from "@playwright/test";
|
||||
|
||||
const BASE = "http://localhost:8765";
|
||||
const STAMP = Date.now();
|
||||
const EMAIL = `e2e-search-${STAMP}@example.com`;
|
||||
const USERNAME = `e2e-search-${STAMP}`;
|
||||
const PASSWORD = "E2e-Playwright-2026!";
|
||||
|
||||
let token = "";
|
||||
|
||||
const consoleFailures: string[] = [];
|
||||
|
||||
function watchPage(page: Page): void {
|
||||
page.on("pageerror", (err) => {
|
||||
consoleFailures.push(`pageerror: ${err.message}`);
|
||||
});
|
||||
page.on("console", (msg) => {
|
||||
if (msg.type() === "error") {
|
||||
consoleFailures.push(`console.error: ${msg.text()}`);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
function expectNoClientErrors(): void {
|
||||
const bad = consoleFailures.filter((f) =>
|
||||
/(is not defined|is not a function|headerSearch|alpine)/i.test(f),
|
||||
);
|
||||
expect(bad, `uncaught client errors:\n${bad.join("\n")}`).toEqual([]);
|
||||
}
|
||||
|
||||
async function apiLogin(request: APIRequestContext): Promise<string> {
|
||||
const res = await request.post("/api/auth/login", {
|
||||
data: { login: EMAIL, password: PASSWORD },
|
||||
});
|
||||
expect(res.status(), "login should succeed").toBe(200);
|
||||
const body = await res.json();
|
||||
return body.access_token as string;
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
|
||||
test.describe("Header search + audio scrubber (client wiring)", () => {
|
||||
let page: Page;
|
||||
|
||||
test.beforeAll(async ({ browser, request }) => {
|
||||
const reg = await request.post("/api/auth/register", {
|
||||
data: { email: EMAIL, username: USERNAME, password: PASSWORD },
|
||||
});
|
||||
expect([201, 200, 409]).toContain(reg.status());
|
||||
token = await apiLogin(request);
|
||||
expect(token).toBeTruthy();
|
||||
|
||||
page = await browser.newPage();
|
||||
watchPage(page);
|
||||
await page.context().addCookies([{ name: "token", value: token, url: BASE }]);
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
expectNoClientErrors();
|
||||
await page?.close();
|
||||
});
|
||||
|
||||
test("header search dropdown opens on typing (regression: tree-shaken module)", async () => {
|
||||
// Drive the header on TWO pages: the bundle is shared, but this
|
||||
// catches any page-scoped wiring surprise.
|
||||
for (const pagePath of ["/bookshelf", "/series"]) {
|
||||
await page.goto(pagePath);
|
||||
const input = page.locator("#header-search");
|
||||
await expect(input).toBeVisible();
|
||||
|
||||
// The regression this pins: search.ts (which registers the
|
||||
// headerSearch Alpine component) was tree-shaken from the bundle,
|
||||
// so typing did nothing — no request, no dropdown. Type with REAL
|
||||
// key events (not fill) and assert the request carries the query.
|
||||
await input.click();
|
||||
await input.pressSequentially("algo", { delay: 90 });
|
||||
|
||||
const requestPromise = page.waitForRequest(
|
||||
(req) =>
|
||||
req.url().includes("/api/media-items/search") &&
|
||||
req.url().includes("q=algo") &&
|
||||
(req.headers()["hx-request"] === "true" || req.headers()["hx-request"] === "true"),
|
||||
{ timeout: 10_000 },
|
||||
);
|
||||
const req = await requestPromise;
|
||||
expect(req.url()).toContain("layout=dropdown");
|
||||
|
||||
const results = page.locator("#search-results");
|
||||
await expect(results).toBeVisible({ timeout: 10_000 });
|
||||
await expect(results.locator(".search-result-item").first()).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
|
||||
// Escape closes the dropdown (keyboard contract).
|
||||
await page.keyboard.press("Escape");
|
||||
await expect(results).toBeHidden();
|
||||
}
|
||||
});
|
||||
|
||||
test("audio scrubber: grabbing the handle does not teleport (plain and Shift)", async ({ request }) => {
|
||||
// Find an audiobook in the shared dev library (plain media list,
|
||||
// then filter client-side — the search endpoint's facet behavior
|
||||
// with empty q isn't a listing API).
|
||||
const res = await request.get("/api/media-items", {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
expect(res.status()).toBe(200);
|
||||
const body = (await res.json()) as { data?: Array<{ id: string; format_group?: string }> };
|
||||
const rows = (body.data ?? []).filter((r) => r.format_group === "audiobook");
|
||||
test.skip(rows.length === 0, "no audiobooks in the dev library");
|
||||
const bookId = rows[0].id;
|
||||
|
||||
await page.goto(`/readers/${bookId}`);
|
||||
const scrubArea = page.locator("#scrub-area");
|
||||
await expect(scrubArea).toBeVisible();
|
||||
|
||||
// The fresh user has never played this book — park the playhead at
|
||||
// 30% via the progress API so the grab tests run mid-book.
|
||||
const put = await request.put(`/api/media-items/${bookId}/progress`, {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
data: { percentage: 0.3, device_id: "e2e-scrubber" },
|
||||
});
|
||||
expect([200, 204]).toContain(put.status());
|
||||
await page.reload();
|
||||
await expect(scrubArea).toBeVisible();
|
||||
|
||||
const box = await scrubArea.boundingBox();
|
||||
test.skip(!box, "scrub bar not laid out");
|
||||
const y = box!.y + box!.height / 2;
|
||||
|
||||
// Where is the thumb right now? The native range input mirrors the
|
||||
// playhead; read its aria-valuenow + max to compute the thumb X.
|
||||
const thumbX = await page.evaluate(() => {
|
||||
const input = document.getElementById("audio-scrub") as HTMLInputElement;
|
||||
const area = document.getElementById("scrub-area")!;
|
||||
const rect = area.getBoundingClientRect();
|
||||
const value = Number(input.value || 0);
|
||||
const max = Number(input.max || 1);
|
||||
return rect.left + (value / max) * rect.width;
|
||||
});
|
||||
|
||||
const readScrubValue = () =>
|
||||
page.evaluate(() => Number((document.getElementById("audio-scrub") as HTMLInputElement).value));
|
||||
|
||||
const before = await readScrubValue();
|
||||
expect(before).toBeGreaterThan(0);
|
||||
|
||||
// --- plain grab: pointerdown ON the thumb must not move it ---
|
||||
await page.mouse.move(thumbX, y);
|
||||
await page.mouse.down();
|
||||
await page.waitForTimeout(150);
|
||||
const atGrab = await readScrubValue();
|
||||
expect(Math.abs(atGrab - before)).toBeLessThan(
|
||||
Math.max(1, before * 0.0005),
|
||||
"grabbing the handle must not jump the playhead",
|
||||
);
|
||||
|
||||
// Drag left by 60px: the handle follows RELATIVELY.
|
||||
await page.mouse.move(thumbX - 60, y, { steps: 5 });
|
||||
await page.waitForTimeout(100);
|
||||
const afterDrag = await readScrubValue();
|
||||
expect(afterDrag).toBeLessThan(atGrab, "dragging left scrubs backward");
|
||||
|
||||
await page.mouse.up();
|
||||
await page.waitForTimeout(400); // seek commits
|
||||
|
||||
// --- Shift+grab: the precision rail appears, no teleport ---
|
||||
const railBefore = await readScrubValue();
|
||||
// Re-locate the thumb after the commit.
|
||||
const thumbX2 = await page.evaluate(() => {
|
||||
const input = document.getElementById("audio-scrub") as HTMLInputElement;
|
||||
const area = document.getElementById("scrub-area")!;
|
||||
const rect = area.getBoundingClientRect();
|
||||
const value = Number(input.value || 0);
|
||||
const max = Number(input.max || 1);
|
||||
return rect.left + (value / max) * rect.width;
|
||||
});
|
||||
|
||||
await page.keyboard.down("Shift");
|
||||
await page.mouse.move(thumbX2, y);
|
||||
await page.mouse.down();
|
||||
await page.waitForTimeout(150);
|
||||
const railAtGrab = await readScrubValue();
|
||||
expect(Math.abs(railAtGrab - railBefore)).toBeLessThan(
|
||||
Math.max(1, railBefore * 0.0005),
|
||||
"Shift+grab must not teleport either",
|
||||
);
|
||||
// The rail is the feedback for Shift+grab — it must be visible.
|
||||
await expect(page.locator('div[x-show="railVisible"]')).toBeVisible();
|
||||
|
||||
await page.keyboard.up("Shift");
|
||||
await page.mouse.up();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,160 @@
|
||||
import { test, expect, type APIRequestContext, type Page } from "@playwright/test";
|
||||
import { execSync } from "child_process";
|
||||
|
||||
const BASE = "http://localhost:8765";
|
||||
const STAMP = Date.now();
|
||||
const EMAIL = `e2e-setfb-${STAMP}@example.com`;
|
||||
const USERNAME = `e2e-setfb-${STAMP}`;
|
||||
const PASSWORD = "E2e-Playwright-2026!";
|
||||
|
||||
// Settings-page save feedback: every save surface must acknowledge the
|
||||
// click (toast) and the scan form must NOT swap the handler's JSON body
|
||||
// over its card (the pre-toast behavior — raw JSON replaced the form).
|
||||
//
|
||||
// Dev-state safety: every success save re-submits the value ALREADY in
|
||||
// the input (read from the page first), so nothing in the shared dev DB
|
||||
// changes; the error-path request is rejected server-side before any
|
||||
// write.
|
||||
|
||||
let token = "";
|
||||
let page = null as Page | null;
|
||||
// system_config.updated_by audit restore: the config save handler stamps
|
||||
// the requesting admin's id onto system_config rows. The harness cleanup
|
||||
// deletes transient example.com users, and this FK (no cascade) would
|
||||
// block the delete and fail unrelated integration tests. Capture before
|
||||
// any save, restore after.
|
||||
let e2eUserID = "";
|
||||
let origUpdatedBy = "";
|
||||
|
||||
function psql(sql: string): string {
|
||||
const flat = sql.replace(/\s+/g, " ");
|
||||
return execSync(
|
||||
`docker exec bookhoard_db psql -U postgres -d bookhoard -tAc ${JSON.stringify(flat)}`,
|
||||
)
|
||||
.toString()
|
||||
.trim();
|
||||
}
|
||||
|
||||
async function apiLogin(request: APIRequestContext): Promise<string> {
|
||||
const res = await request.post("/api/auth/login", {
|
||||
data: { login: EMAIL, password: PASSWORD },
|
||||
});
|
||||
expect(res.status(), "login should succeed").toBe(200);
|
||||
return (await res.json()).access_token as string;
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
|
||||
test.describe("Settings save feedback (toasts + disabled buttons)", () => {
|
||||
test.beforeAll(async ({ browser, request }) => {
|
||||
// Fresh user, promoted to admin in the dev DB (the harness cleanup
|
||||
// sweeps transient example.com users afterwards).
|
||||
const reg = await request.post("/api/auth/register", {
|
||||
data: { email: EMAIL, username: USERNAME, password: PASSWORD },
|
||||
});
|
||||
expect([201, 200, 409]).toContain(reg.status());
|
||||
psql(`UPDATE users SET role='admin' WHERE email='${EMAIL}'`);
|
||||
token = await apiLogin(request);
|
||||
expect(token).toBeTruthy();
|
||||
|
||||
e2eUserID = psql(`SELECT id FROM users WHERE email='${EMAIL}'`);
|
||||
origUpdatedBy = psql(`SELECT COALESCE(updated_by::text, '') FROM system_config WHERE key='base_url'`);
|
||||
|
||||
page = await browser.newPage();
|
||||
await page.context().addCookies([{ name: "token", value: token, url: BASE }]);
|
||||
await page.goto("/admin/settings");
|
||||
await expect(page.locator("h1")).toHaveText("System Settings");
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
await page?.close();
|
||||
// Hand system_config.updated_by back to whoever held it (or NULL),
|
||||
// so the transient user can be swept.
|
||||
if (e2eUserID) {
|
||||
const restore = origUpdatedBy
|
||||
? `'${origUpdatedBy}'::uuid`
|
||||
: "NULL";
|
||||
psql(`UPDATE system_config SET updated_by = ${restore} WHERE updated_by = '${e2eUserID}'::uuid`);
|
||||
}
|
||||
});
|
||||
|
||||
test("Base URL save shows a success toast — twice (twin-form drift pin)", async () => {
|
||||
const input = page.locator("#settings-form input[name='base_url']");
|
||||
await expect(input).toBeVisible();
|
||||
|
||||
// First save goes through the templ form…
|
||||
await page.locator("#settings-form button[type='submit']").first().click();
|
||||
await expect(page.getByText("Settings saved", { exact: true })).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
|
||||
// …the second through the Go-source twin the handler swaps in after
|
||||
// the first save. If the twin ever drops data-settings-save (the
|
||||
// handlers→templates import cycle is why the twin exists), THIS is
|
||||
// the assertion that fails.
|
||||
await page.waitForTimeout(300); // let the swap settle
|
||||
await expect(page.locator("#settings-form")).toBeVisible();
|
||||
await page.locator("#settings-form button[type='submit']").first().click();
|
||||
await expect(page.getByText("Settings saved", { exact: true })).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
});
|
||||
|
||||
test("Scan settings save shows a toast and never swaps raw JSON over the card", async () => {
|
||||
const section = page.locator("#scan-settings-section");
|
||||
await expect(section).toBeVisible();
|
||||
|
||||
await section.locator("button[type='submit']").click();
|
||||
await expect(page.getByText("Scan settings saved", { exact: true })).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
|
||||
// The regression: the scan handler responds JSON; with the old
|
||||
// hx-swap="outerHTML" the card was replaced by the literal JSON text.
|
||||
await expect(section).toBeVisible();
|
||||
await expect(section.locator("form")).toBeVisible();
|
||||
const body = await page.locator("body").innerText();
|
||||
expect(body).not.toContain("scan settings updated successfully");
|
||||
});
|
||||
|
||||
test("Tunable row save shows a toast and the per-row status span", async () => {
|
||||
const row = page.locator("form[data-settings-save='tunable']").first();
|
||||
await expect(row).toBeVisible();
|
||||
const key = await row.locator("input[name='key']").getAttribute("value");
|
||||
expect(key).toBeTruthy();
|
||||
|
||||
await row.locator("button[type='submit']").click();
|
||||
await expect(page.getByText("Setting saved", { exact: true })).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
// Server-driven per-row status (kept alongside the toast; it carries
|
||||
// the "restart required" nuance).
|
||||
await expect(page.locator(`#status-${key}`)).toContainText("Saved", {
|
||||
timeout: 10_000,
|
||||
});
|
||||
});
|
||||
|
||||
test("Invalid value shows an error toast and the form survives", async () => {
|
||||
// Inject a genuinely invalid option and select it: setting a select
|
||||
// to a value with no matching option silently serializes as NO field
|
||||
// at all (no selected option → nothing submitted), which would save
|
||||
// fine instead of failing. With a real option present, the server
|
||||
// rejects with 400 {"error":"invalid timezone"} before any write.
|
||||
await page.evaluate(() => {
|
||||
const select = document.getElementById("default_timezone") as HTMLSelectElement;
|
||||
const opt = document.createElement("option");
|
||||
opt.value = "Not/AZone";
|
||||
opt.textContent = "E2E invalid";
|
||||
select.appendChild(opt);
|
||||
select.value = "Not/AZone";
|
||||
});
|
||||
await page.locator("#settings-form button[type='submit']").first().click();
|
||||
|
||||
await expect(page.getByText("Save failed: invalid timezone")).toBeVisible({
|
||||
timeout: 10_000,
|
||||
});
|
||||
// htmx does not swap non-2xx bodies: the form is intact.
|
||||
await expect(page.locator("#settings-form")).toBeVisible();
|
||||
await expect(page.locator("#settings-form input[name='base_url']")).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,259 @@
|
||||
import { test, expect, type APIRequestContext, type Page } from "@playwright/test";
|
||||
import { execSync } from "child_process";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
import path from "path";
|
||||
|
||||
const REPO = process.cwd(); // uploads/ here is bind-mounted at /app/uploads
|
||||
const BASE = "http://localhost:8765";
|
||||
const STAMP = Date.now();
|
||||
const EMAIL = `e2e-meta-${STAMP}@example.com`;
|
||||
const USERNAME = `e2e-meta-${STAMP}`;
|
||||
const PASSWORD = "E2e-Playwright-2026!";
|
||||
|
||||
// Host-side upload dir (bind-mounted into the app container).
|
||||
const UPLOADS_HOST = path.join(REPO, "uploads", `e2e-meta-${STAMP}`);
|
||||
const UPLOADS_CONT = `/app/uploads/e2e-meta-${STAMP}`;
|
||||
|
||||
interface FormatSeed {
|
||||
ext: string;
|
||||
group: string;
|
||||
mime: string;
|
||||
id?: string;
|
||||
}
|
||||
|
||||
// Every write-capable file type, through the REAL admin UI, verified by
|
||||
// reading each artifact back independently afterwards (this test IS the
|
||||
// "choose each file type, then check the file's metadata yourself"
|
||||
// contract):
|
||||
//
|
||||
// epub → embedded OPF splice; verify by parsing the OPF XML
|
||||
// pdf → Calibre metadata.opf sidecar (lone book); parse XML
|
||||
// cbz → ComicInfo.xml injected; parse XML
|
||||
// mp3/flac/m4a/m4b → ffprobe the rewritten tags
|
||||
const FORMATS: FormatSeed[] = [
|
||||
{ ext: "epub", group: "ebook", mime: "application/epub+zip" },
|
||||
{ ext: "pdf", group: "ebook", mime: "application/pdf" },
|
||||
{ ext: "cbz", group: "comic_archive", mime: "application/zip" },
|
||||
{ ext: "mp3", group: "audiobook", mime: "audio/mpeg" },
|
||||
{ ext: "flac", group: "audiobook", mime: "audio/flac" },
|
||||
{ ext: "m4a", group: "audiobook", mime: "audio/mp4" },
|
||||
{ ext: "m4b", group: "audiobook", mime: "audio/mp4" },
|
||||
];
|
||||
|
||||
let token = "";
|
||||
let libraryID = "";
|
||||
|
||||
function psql(sql: string): string {
|
||||
// Collapse whitespace (JSON.stringify would bind newlines as literal
|
||||
// "\n" through the shell and break the SQL).
|
||||
const flat = sql.replace(/\s+/g, " ");
|
||||
return execSync(
|
||||
`docker exec bookhoard_db psql -U postgres -d bookhoard -tAc ${JSON.stringify(flat)}`,
|
||||
)
|
||||
.toString()
|
||||
.trim();
|
||||
}
|
||||
|
||||
function run(cmd: string): string {
|
||||
return execSync(cmd, { encoding: "utf8" }).trim();
|
||||
}
|
||||
|
||||
function writePython(script: string): void {
|
||||
run(`python3 - <<'PYEOF'\n${script}\nPYEOF`);
|
||||
}
|
||||
|
||||
function writeEPUB(dest: string, title: string): void {
|
||||
const script = `import zipfile, shutil
|
||||
arc = ${JSON.stringify(dest)}
|
||||
tmp = arc + ".tmp"
|
||||
with zipfile.ZipFile(tmp, "w") as z:
|
||||
z.writestr(zipfile.ZipInfo("mimetype"), "application/epub+zip", compress_type=zipfile.ZIP_STORED)
|
||||
z.writestr("META-INF/container.xml", '<container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles><rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/></rootfiles></container>')
|
||||
z.writestr("OEBPS/content.opf", '<?xml version="1.0"?><package version="3.0" xmlns="http://www.idpf.org/2007/opf" unique-identifier="pub-id"><metadata xmlns:dc="http://purl.org/dc/elements/1.1/"><dc:title id="t1">' + ${JSON.stringify(title)} + '</dc:title><dc:identifier id="pub-id">urn:uuid:e2e</dc:identifier></metadata><manifest><item id="c1" href="c1.xhtml" media-type="application/xhtml+xml"/></manifest><spine><itemref idref="c1"/></spine></package>')
|
||||
z.writestr("OEBPS/c1.xhtml", "<html><body>chapter</body></html>")
|
||||
shutil.move(tmp, arc)`;
|
||||
writePython(script);
|
||||
}
|
||||
|
||||
function writeCBZ(dest: string): void {
|
||||
writePython(`import zipfile, shutil
|
||||
arc = ${JSON.stringify(dest)}
|
||||
tmp = arc + ".tmp"
|
||||
with zipfile.ZipFile(tmp, "w") as z:
|
||||
z.writestr("page_0001.jpg", "not-a-real-jpeg-but-fine")
|
||||
shutil.move(tmp, arc)`);
|
||||
}
|
||||
|
||||
/** Independent artifact readers — no app code involved. The python is
|
||||
* written to a temp file (never -c through the shell: paths with
|
||||
* quotes/newlines are unreadable that way). */
|
||||
function pythonRead(script: string, absPath: string): string {
|
||||
const scriptPath = path.join(os.tmpdir(), `e2e-meta-read-${STAMP}.py`);
|
||||
fs.writeFileSync(scriptPath, script);
|
||||
return run(`python3 ${JSON.stringify(scriptPath)} ${JSON.stringify(absPath)}`);
|
||||
}
|
||||
|
||||
function artifactTitle(ext: string, absPath: string): string {
|
||||
switch (ext) {
|
||||
case "epub":
|
||||
return pythonRead(`import zipfile, sys, xml.etree.ElementTree as ET
|
||||
with zipfile.ZipFile(sys.argv[1]) as z:
|
||||
root = ET.fromstring(z.read('OEBPS/content.opf'))
|
||||
ns = {'dc': 'http://purl.org/dc/elements/1.1/'}
|
||||
titles = [t.text or '' for t in root.findall('.//dc:title', ns)]
|
||||
print(titles[0] if titles else '')`, absPath);
|
||||
case "cbz":
|
||||
return pythonRead(`import zipfile, sys, xml.etree.ElementTree as ET
|
||||
with zipfile.ZipFile(sys.argv[1]) as z:
|
||||
root = ET.fromstring(z.read('ComicInfo.xml'))
|
||||
print(root.findtext('Title') or '')`, absPath);
|
||||
case "pdf":
|
||||
// one-book dir → the Calibre-style sidecar carries the metadata
|
||||
return pythonRead(`import os, sys, xml.etree.ElementTree as ET
|
||||
sidecar = os.path.join(os.path.dirname(sys.argv[1]), 'metadata.opf')
|
||||
root = ET.parse(sidecar).getroot()
|
||||
ns = {'dc': 'http://purl.org/dc/elements/1.1/'}
|
||||
print(root.findtext('.//dc:title', default='', namespaces=ns))`, absPath);
|
||||
default:
|
||||
// audio containers: ffprobe the rewritten album tag
|
||||
return run(
|
||||
`ffprobe -v error -show_entries format_tags=album -of default=noprint_wrappers=1:nokey=1 ${JSON.stringify(absPath)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function apiLogin(request: APIRequestContext): Promise<string> {
|
||||
const res = await request.post("/api/auth/login", {
|
||||
data: { login: EMAIL, password: PASSWORD },
|
||||
});
|
||||
expect(res.status(), "login should succeed").toBe(200);
|
||||
return (await res.json()).access_token as string;
|
||||
}
|
||||
|
||||
test.describe.configure({ mode: "serial" });
|
||||
|
||||
test.describe("Write-to-file per format (real admin UI + artifact verification)", () => {
|
||||
let page: Page;
|
||||
|
||||
test.beforeAll(async ({ browser, request }) => {
|
||||
fs.mkdirSync(UPLOADS_HOST, { recursive: true });
|
||||
|
||||
// Fresh user, promoted to admin in the dev DB (the harness cleanup
|
||||
// sweeps transient example.com users afterwards).
|
||||
const reg = await request.post("/api/auth/register", {
|
||||
data: { email: EMAIL, username: USERNAME, password: PASSWORD },
|
||||
});
|
||||
expect([201, 200, 409]).toContain(reg.status());
|
||||
|
||||
psql(`UPDATE users SET role='admin' WHERE email='${EMAIL}'`);
|
||||
token = await apiLogin(request);
|
||||
expect(token).toBeTruthy();
|
||||
|
||||
// Test library with a folder at our upload dir.
|
||||
const libRes = await request.post("/api/libraries", {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
data: { name: `E2E Meta Write ${STAMP}`, type: "ebooks" },
|
||||
});
|
||||
expect(libRes.status()).toBe(201);
|
||||
libraryID = ((await libRes.json()) as { id: string }).id;
|
||||
const foldRes = await request.post(`/api/libraries/${libraryID}/folders`, {
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
data: { folder_path: UPLOADS_CONT },
|
||||
});
|
||||
expect(foldRes.status()).toBe(201);
|
||||
|
||||
// Seed one fixture per format: file on disk + media_items row.
|
||||
for (const f of FORMATS) {
|
||||
const rel = `${f.group}/${f.ext}/e2e-fixture.${f.ext}`;
|
||||
const abs = path.join(UPLOADS_HOST, rel);
|
||||
fs.mkdirSync(path.dirname(abs), { recursive: true });
|
||||
if (f.ext === "epub") writeEPUB(abs, "E2E Original Title");
|
||||
else if (f.ext === "cbz") writeCBZ(abs);
|
||||
else if (f.ext === "pdf") fs.writeFileSync(abs, "%PDF-1.4 e2e stub\n");
|
||||
else {
|
||||
const codec = f.ext === "mp3" ? "libmp3lame" : f.ext === "flac" ? "flac" : "aac";
|
||||
run(
|
||||
`ffmpeg -hide_banner -loglevel error -y -f lavfi -i anullsrc=r=44100:cl=mono -t 2 -c:a ${codec} -metadata title='E2E Original Title' -metadata album='E2E Original Title' ${JSON.stringify(abs)}`,
|
||||
);
|
||||
}
|
||||
|
||||
f.id = psql(
|
||||
`INSERT INTO media_items (library_id, title, file_path, format_group, mime_type, file_size, created_at, imported_at, updated_at)
|
||||
VALUES ('${libraryID}', 'E2E Original Title', '${rel}', '${f.group}', '${f.mime}', 100, NOW(), NOW(), NOW())
|
||||
RETURNING id`,
|
||||
);
|
||||
// -tAc prints the id row first, then psql's command tag
|
||||
// ("INSERT 0 1") last; take the first non-empty line.
|
||||
f.id = f.id.split("\n").find((l) => l.trim())!.trim();
|
||||
expect(f.id).toMatch(/^[0-9a-f-]{36}$/i);
|
||||
}
|
||||
|
||||
page = await browser.newPage();
|
||||
await page.context().addCookies([{ name: "token", value: token, url: BASE }]);
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
await page?.close();
|
||||
// Best-effort cleanup: the library delete cascades the items; the
|
||||
// psql fallback + rm sweep anything the API couldn't.
|
||||
try {
|
||||
await fetch(`${BASE}/api/libraries/${libraryID}`, {
|
||||
method: "DELETE",
|
||||
headers: { Authorization: `Bearer ${token}` },
|
||||
});
|
||||
} catch {
|
||||
/* best effort */
|
||||
}
|
||||
fs.rmSync(UPLOADS_HOST, { recursive: true, force: true });
|
||||
try {
|
||||
psql(`DELETE FROM libraries WHERE id='${libraryID}'`);
|
||||
} catch {
|
||||
/* already gone */
|
||||
}
|
||||
});
|
||||
|
||||
for (const f of FORMATS) {
|
||||
test(`writes metadata into ${f.ext} and the file content verifies`, async () => {
|
||||
const newTitle = `E2E WRITTEN ${f.ext.toUpperCase()}`;
|
||||
const abs = path.join(UPLOADS_HOST, f.group, f.ext, `e2e-fixture.${f.ext}`);
|
||||
expect(f.id).toBeTruthy();
|
||||
|
||||
// Real admin UI flow: book page → Edit → change title → the
|
||||
// in-app dialog → Save & write.
|
||||
await page.goto(`/media/${f.id}`);
|
||||
await page.click('button:has-text("Edit")');
|
||||
const modal = page.locator("#metadata-editor-modal");
|
||||
await expect(modal).toBeVisible();
|
||||
await page.fill('#metadata-editor-modal input[name="title"]', newTitle);
|
||||
|
||||
await page.click('#metadata-editor-modal button:has-text("Write to file")');
|
||||
const dialog = page.locator("#write-confirm-dialog");
|
||||
await expect(dialog).toBeVisible();
|
||||
await dialog.locator('button:has-text("Save & write")').click();
|
||||
|
||||
// Wait for the request + inline rescan + page reload to settle.
|
||||
await page.waitForTimeout(2500);
|
||||
|
||||
// Then verify the FILE's metadata — independently of the app
|
||||
// (read the artifact back with python/ffprobe, not via the app).
|
||||
let seen = "";
|
||||
let lastErr = "";
|
||||
for (let i = 0; i < 20; i++) {
|
||||
try {
|
||||
seen = artifactTitle(f.ext, abs);
|
||||
} catch (e) {
|
||||
lastErr = String(e);
|
||||
seen = "";
|
||||
}
|
||||
if (seen.includes(newTitle)) break;
|
||||
await page.waitForTimeout(500);
|
||||
}
|
||||
if (!seen.includes(newTitle)) {
|
||||
console.error(`[${f.ext}] artifact read failed: ${lastErr}`);
|
||||
}
|
||||
expect(seen, `${f.ext}: the FILE must carry the written metadata`).toContain(newTitle);
|
||||
expect(seen, `${f.ext}: the old title must be replaced`).not.toContain("E2E Original Title");
|
||||
});
|
||||
}
|
||||
});
|
||||
@@ -150,6 +150,7 @@ type Libraries struct {
|
||||
CreatedByAdminID pgtype.UUID `db:"created_by_admin_id" json:"created_by_admin_id"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
AllowMemberFeeds bool `db:"allow_member_feeds" json:"allow_member_feeds"`
|
||||
}
|
||||
|
||||
type LibraryFolders struct {
|
||||
@@ -157,6 +158,7 @@ type LibraryFolders struct {
|
||||
LibraryID pgtype.UUID `db:"library_id" json:"library_id"`
|
||||
FolderPath string `db:"folder_path" json:"folder_path"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
Purpose string `db:"purpose" json:"purpose"`
|
||||
}
|
||||
|
||||
type LibraryTypes struct {
|
||||
@@ -338,6 +340,21 @@ type MediaItems struct {
|
||||
OpfIdentifier pgtype.Text `db:"opf_identifier" json:"opf_identifier"`
|
||||
OpfUuid pgtype.Text `db:"opf_uuid" json:"opf_uuid"`
|
||||
HashConfidence pgtype.Text `db:"hash_confidence" json:"hash_confidence"`
|
||||
Source string `db:"source" json:"source"`
|
||||
SourceFeedID pgtype.UUID `db:"source_feed_id" json:"source_feed_id"`
|
||||
EpisodeGuid pgtype.Text `db:"episode_guid" json:"episode_guid"`
|
||||
FeedPosition pgtype.Int4 `db:"feed_position" json:"feed_position"`
|
||||
SeasonNumber pgtype.Int4 `db:"season_number" json:"season_number"`
|
||||
EpisodeNumber pgtype.Int4 `db:"episode_number" json:"episode_number"`
|
||||
EnclosureUrl pgtype.Text `db:"enclosure_url" json:"enclosure_url"`
|
||||
DeclaredDurationSeconds pgtype.Int4 `db:"declared_duration_seconds" json:"declared_duration_seconds"`
|
||||
TranscriptUrl pgtype.Text `db:"transcript_url" json:"transcript_url"`
|
||||
ChaptersUrl pgtype.Text `db:"chapters_url" json:"chapters_url"`
|
||||
Hosts []string `db:"hosts" json:"hosts"`
|
||||
Guests []string `db:"guests" json:"guests"`
|
||||
DownloadState string `db:"download_state" json:"download_state"`
|
||||
DownloadError pgtype.Text `db:"download_error" json:"download_error"`
|
||||
DownloadAttempts int32 `db:"download_attempts" json:"download_attempts"`
|
||||
}
|
||||
|
||||
type MediaNotes struct {
|
||||
@@ -405,6 +422,43 @@ type PanelData struct {
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
type PodcastFeeds struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
LibraryID pgtype.UUID `db:"library_id" json:"library_id"`
|
||||
FeedUrl string `db:"feed_url" json:"feed_url"`
|
||||
PodcastGuid pgtype.Text `db:"podcast_guid" json:"podcast_guid"`
|
||||
Title string `db:"title" json:"title"`
|
||||
Description pgtype.Text `db:"description" json:"description"`
|
||||
Host pgtype.Text `db:"host" json:"host"`
|
||||
Publisher pgtype.Text `db:"publisher" json:"publisher"`
|
||||
OwnerName pgtype.Text `db:"owner_name" json:"owner_name"`
|
||||
OwnerEmail pgtype.Text `db:"owner_email" json:"owner_email"`
|
||||
Language pgtype.Text `db:"language" json:"language"`
|
||||
Tags []string `db:"tags" json:"tags"`
|
||||
Explicit bool `db:"explicit" json:"explicit"`
|
||||
ArtworkPath pgtype.Text `db:"artwork_path" json:"artwork_path"`
|
||||
FeedType pgtype.Text `db:"feed_type" json:"feed_type"`
|
||||
Medium pgtype.Text `db:"medium" json:"medium"`
|
||||
IsComplete bool `db:"is_complete" json:"is_complete"`
|
||||
Etag pgtype.Text `db:"etag" json:"etag"`
|
||||
LastModified pgtype.Text `db:"last_modified" json:"last_modified"`
|
||||
LastFetchedAt pgtype.Timestamptz `db:"last_fetched_at" json:"last_fetched_at"`
|
||||
LastBuildDate pgtype.Timestamptz `db:"last_build_date" json:"last_build_date"`
|
||||
CreatedByAdminID pgtype.UUID `db:"created_by_admin_id" json:"created_by_admin_id"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
UpdatedAt pgtype.Timestamptz `db:"updated_at" json:"updated_at"`
|
||||
}
|
||||
|
||||
type PodcastSubscriptions struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
UserID pgtype.UUID `db:"user_id" json:"user_id"`
|
||||
FeedID pgtype.UUID `db:"feed_id" json:"feed_id"`
|
||||
DisplayName pgtype.Text `db:"display_name" json:"display_name"`
|
||||
AutoDownload string `db:"auto_download" json:"auto_download"`
|
||||
AutoDownloadCount pgtype.Int4 `db:"auto_download_count" json:"auto_download_count"`
|
||||
CreatedAt pgtype.Timestamptz `db:"created_at" json:"created_at"`
|
||||
}
|
||||
|
||||
// Tracks media that failed processing in a library: item-backed issues (library-type mismatches, metadata extraction failures) and row-less import failures (no media_items row exists; keyed by library + file_path). Auto-resolved by later successful scans.
|
||||
type ProcessingIssues struct {
|
||||
ID pgtype.UUID `db:"id" json:"id"`
|
||||
|
||||
@@ -22,6 +22,8 @@ type Querier interface {
|
||||
// ============================================
|
||||
AddBookToKoboShelf(ctx context.Context, arg AddBookToKoboShelfParams) (KoboShelves, error)
|
||||
// Library Folders queries
|
||||
// purpose: 'scan' (FS-walker root, default) | 'podcast' (download root;
|
||||
// scanner-skipped, written by the podcast download worker instead).
|
||||
AddLibraryFolder(ctx context.Context, arg AddLibraryFolderParams) (LibraryFolders, error)
|
||||
// Second consecutive missing scan: hide the item from all browsing while
|
||||
// preserving reading history in case the file returns.
|
||||
@@ -53,6 +55,19 @@ type Querier interface {
|
||||
// collections page); same live filter as GetCollectionItems, plus the
|
||||
// future-hybrid excluded exclusion and an optional library scope.
|
||||
CountCollectionLiveItems(ctx context.Context, arg CountCollectionLiveItemsParams) (int32, error)
|
||||
CountDownloadedEpisodes(ctx context.Context, sourceFeedID pgtype.UUID) (int64, error)
|
||||
// Sibling-book count for sidecar naming (MetadataWriter): a dir-scoped
|
||||
// metadata.opf may only be written when NO other non-archived item
|
||||
// shares the directory; otherwise the sidecar goes to
|
||||
// <basename>.opf so one book's metadata can never poison dir-mates.
|
||||
CountOtherMediaItemsInDir(ctx context.Context, arg CountOtherMediaItemsInDirParams) (int32, error)
|
||||
CountPodcastEpisodesForFeed(ctx context.Context, sourceFeedID pgtype.UUID) (int64, error)
|
||||
CountPodcastFeedSubscribers(ctx context.Context, feedID pgtype.UUID) (int64, error)
|
||||
// Sidebar/page gate (D13): >=1 podcasts-type library the user can see
|
||||
// (COALESCE(is_visible,true) semantics — mirrors GetUserVisibleLibraries).
|
||||
CountPodcastLibrariesForUser(ctx context.Context, userID pgtype.UUID) (int64, error)
|
||||
CountQueuedEpisodes(ctx context.Context) (int64, error)
|
||||
CountUndownloadedEpisodes(ctx context.Context, sourceFeedID pgtype.UUID) (int64, error)
|
||||
// Count unlinked books for a device (ignored rows stay hidden)
|
||||
CountUnlinkedBooks(ctx context.Context, deviceID pgtype.UUID) (int64, error)
|
||||
// Banner count: unresolved, non-ignored queue rows across the caller's own
|
||||
@@ -111,6 +126,22 @@ type Querier interface {
|
||||
// ============================================
|
||||
CreateOrUpdateKoboEntitlement(ctx context.Context, arg CreateOrUpdateKoboEntitlementParams) (KoboEntitlements, error)
|
||||
// ============================================================================
|
||||
// PODCASTS / RSS FEEDS (plan ~/.opencode/plan/podcast-integration.md §6/§8)
|
||||
// Visibility is RLS-enforced (schema policy block) — NO query here carries
|
||||
// visibility clauses. Feed reads run under the acting user's identity
|
||||
// (subscriber-or-admin policy); the fetcher/worker run as first-admin.
|
||||
// The fetcher/download worker is the SOLE writer of feed/episode metadata;
|
||||
// users write only podcast_subscriptions rows.
|
||||
// ============================================================================
|
||||
// Add-feed upsert keyed on the per-library UNIQUE(library_id, feed_url):
|
||||
// a second subscriber adding the same URL attaches to the existing row
|
||||
// instead of erroring. DO UPDATE (not DO NOTHING) so the RETURNING works
|
||||
// even when the conflicting row is invisible to the actor under RLS
|
||||
// (non-admin creators cannot SELECT other users' feeds). Metadata columns
|
||||
// are filled by the fetcher right after; this only establishes identity.
|
||||
CreatePodcastFeed(ctx context.Context, arg CreatePodcastFeedParams) (PodcastFeeds, error)
|
||||
CreatePodcastLibraryFolder(ctx context.Context, arg CreatePodcastLibraryFolderParams) (LibraryFolders, error)
|
||||
// ============================================================================
|
||||
// PROCESSING ISSUES QUERIES
|
||||
// ============================================================================
|
||||
// Widened upsert (faceted plan Phase 11): media_item_id and file_path
|
||||
@@ -155,6 +186,11 @@ type Querier interface {
|
||||
DeleteMediaItemFormat(ctx context.Context, id pgtype.UUID) error
|
||||
DeleteMediaRating(ctx context.Context, arg DeleteMediaRatingParams) error
|
||||
DeleteMediaReview(ctx context.Context, arg DeleteMediaReviewParams) error
|
||||
// Admin-only removal (handler checks role); episodes cascade via
|
||||
// media_items.source_feed_id ON DELETE CASCADE.
|
||||
DeletePodcastFeed(ctx context.Context, id pgtype.UUID) (int64, error)
|
||||
// Unsubscribe NEVER deletes feed/episode/file state (D6).
|
||||
DeletePodcastSubscription(ctx context.Context, arg DeletePodcastSubscriptionParams) (int64, error)
|
||||
DeleteProcessingIssue(ctx context.Context, id pgtype.UUID) (ProcessingIssues, error)
|
||||
DeleteReadingProgress(ctx context.Context, arg DeleteReadingProgressParams) error
|
||||
DeleteSavedFilter(ctx context.Context, arg DeleteSavedFilterParams) (SavedFilters, error)
|
||||
@@ -165,7 +201,14 @@ type Querier interface {
|
||||
// Delete unlinked book
|
||||
DeleteUnlinkedBook(ctx context.Context, id pgtype.UUID) error
|
||||
DeleteUser(ctx context.Context, id pgtype.UUID) error
|
||||
DeleteUserSettingByKey(ctx context.Context, arg DeleteUserSettingByKeyParams) error
|
||||
DeleteUserSystemCollection(ctx context.Context, arg DeleteUserSystemCollectionParams) error
|
||||
// Device deletion (deregister on logout): detach the device's reading
|
||||
// history instead of deleting it — these rows are real captured stats
|
||||
// (the Users section reads them); reading_history.device_id is NO ACTION
|
||||
// so a blind device delete 500s once any history anchors to the device.
|
||||
// device_id is nullable: NULL keeps the session, loses the association.
|
||||
DetachReadingHistoryFromDevice(ctx context.Context, deviceID pgtype.UUID) error
|
||||
// Device existence probe for capture: the web/app progress paths
|
||||
// historically stuff the user id into DeviceID (it only fed a broadcast
|
||||
// label before stats); a capture row must only reference a REAL device.
|
||||
@@ -275,12 +318,24 @@ type Querier interface {
|
||||
// Series are grouped PER LIBRARY: the same series name in two libraries
|
||||
// (novels vs manga adaptations, LN splits) stays two entries. Visible
|
||||
// filter enforces per-user library visibility at the source.
|
||||
// Tiebreaker REQUIRED for LIMIT/OFFSET pagination: bulk imports in one
|
||||
// transaction share a single NOW() timestamp, leaving many groups tied on
|
||||
// MAX(created_at). Postgres does not guarantee tie order between
|
||||
// executions, so page boundaries could repeat or drop series. The GROUP BY
|
||||
// keys make the sort total and deterministic.
|
||||
//
|
||||
// Series-page sort/filter contract (extensible registry in
|
||||
// services.ResolveSeriesSort): SeriesQuery is a series-NAME substring so
|
||||
// it filters in WHERE (the group key — counts stay true); AuthorQuery
|
||||
// and Status filter via HAVING so COUNT(*) keeps reporting the whole
|
||||
// series. ViewerUserId scopes the per-user progress aggregates (the rp
|
||||
// join can't multiply rows: reading_progress is UNIQUE per
|
||||
// media_item+user). Sort is a BOUND PARAM compared inside the CASE
|
||||
// cascade — never interpolated — and the GROUP BY tiebreaker stays
|
||||
// REQUIRED for LIMIT/OFFSET determinism: bulk imports in one
|
||||
// transaction share a single NOW() timestamp, leaving many groups tied
|
||||
// on MAX(created_at), and Postgres does not guarantee tie order between
|
||||
// executions, so page boundaries could repeat or drop series.
|
||||
GetDistinctSeries(ctx context.Context, arg GetDistinctSeriesParams) ([]GetDistinctSeriesRow, error)
|
||||
// Count counterpart of GetDistinctSeries: identical WHERE + JOIN +
|
||||
// HAVING so the pagination total matches what the page query returns
|
||||
// under the same filters (Sort is a page-ORDER concern, not a set
|
||||
// membership one, so it is absent here by design).
|
||||
GetDistinctSeriesCount(ctx context.Context, arg GetDistinctSeriesCountParams) (int32, error)
|
||||
GetFailedSyncQueueItems(ctx context.Context, limit int32) ([]SyncQueue, error)
|
||||
GetFirstAdmin(ctx context.Context) (pgtype.UUID, error)
|
||||
@@ -391,6 +446,8 @@ type Querier interface {
|
||||
// GetMediaScanState reads one item's manifest (per-item rescan paths
|
||||
// and the manifest integration tests).
|
||||
GetMediaScanState(ctx context.Context, mediaItemID pgtype.UUID) (MediaScanState, error)
|
||||
// Sequential download worker's claim source (single worker, day one).
|
||||
GetNextQueuedEpisode(ctx context.Context) (MediaItems, error)
|
||||
GetNextRetryTime(ctx context.Context) (interface{}, error)
|
||||
GetNotStartedItems(ctx context.Context, arg GetNotStartedItemsParams) ([]MediaItems, error)
|
||||
// Get OPDS token
|
||||
@@ -398,6 +455,25 @@ type Querier interface {
|
||||
// Get OPDS tokens by device
|
||||
GetOpdsTokensByDevice(ctx context.Context, deviceID pgtype.UUID) ([]OpdsTokens, error)
|
||||
GetPanelData(ctx context.Context, arg GetPanelDataParams) (PanelData, error)
|
||||
// Continue aggregation (plan 2.5): the caller's IN-PROGRESS episodes
|
||||
// (0 < pct < finishedThreshold) across ALL subscribed feeds, most
|
||||
// recent first. Go groups per feed — the first row per feed IS the
|
||||
// continue target.
|
||||
GetPodcastContinueForUser(ctx context.Context, userID pgtype.UUID) ([]GetPodcastContinueForUserRow, error)
|
||||
GetPodcastEpisode(ctx context.Context, id pgtype.UUID) (MediaItems, error)
|
||||
// Per-user progress for a feed's episodes (row badges + the Phase-2
|
||||
// continue fields). reading_progress is UNIQUE per (media_item, user).
|
||||
GetPodcastEpisodeProgress(ctx context.Context, arg GetPodcastEpisodeProgressParams) ([]GetPodcastEpisodeProgressRow, error)
|
||||
GetPodcastFeed(ctx context.Context, id pgtype.UUID) (PodcastFeeds, error)
|
||||
// URL-migration dedupe: same canonical guid within the library reuses
|
||||
// the existing row.
|
||||
GetPodcastFeedByGuid(ctx context.Context, arg GetPodcastFeedByGuidParams) (PodcastFeeds, error)
|
||||
// The download root (purpose='podcast') of a podcasts library.
|
||||
GetPodcastLibraryFolder(ctx context.Context, libraryID pgtype.UUID) ([]LibraryFolders, error)
|
||||
GetPodcastSubscription(ctx context.Context, arg GetPodcastSubscriptionParams) (PodcastSubscriptions, error)
|
||||
// The podcasts library/ies visible to the user (add-feed target list;
|
||||
// a self-hosted instance typically has exactly one).
|
||||
GetPodcastsLibraryForUser(ctx context.Context, userID pgtype.UUID) ([]GetPodcastsLibraryForUserRow, error)
|
||||
// Get most popular books for a user
|
||||
GetPopularBooks(ctx context.Context, arg GetPopularBooksParams) ([]GetPopularBooksRow, error)
|
||||
// Single-issue read for the resolve/delete endpoints' family-lens gate.
|
||||
@@ -498,6 +574,12 @@ type Querier interface {
|
||||
// END instant (the push time; NULL falls back to created_at) — the same
|
||||
// value the capture stamps every write with.
|
||||
GetUserReadingSessionsRange(ctx context.Context, arg GetUserReadingSessionsRangeParams) ([]GetUserReadingSessionsRangeRow, error)
|
||||
// ============================================================================
|
||||
// GENERIC reader_settings KV (podcast order override, plan D5: key
|
||||
// podcast_order:<feed_id>). The pre-existing reader_settings queries are
|
||||
// hard-keyed to 'reader_settings'; these two are the generic form.
|
||||
// ============================================================================
|
||||
GetUserSettingByKey(ctx context.Context, arg GetUserSettingByKeyParams) ([]byte, error)
|
||||
GetUserVisibleLibraries(ctx context.Context, userID pgtype.UUID) ([]GetUserVisibleLibrariesRow, error)
|
||||
GetVisibleLibraryMediaCounts(ctx context.Context, userID pgtype.UUID) ([]GetVisibleLibraryMediaCountsRow, error)
|
||||
// Smart ("dynamic") collections: rules are evaluated LIVE at render, so
|
||||
@@ -529,6 +611,9 @@ type Querier interface {
|
||||
ListDeletedAnnotationsForBook(ctx context.Context, arg ListDeletedAnnotationsForBookParams) ([]ListDeletedAnnotationsForBookRow, error)
|
||||
ListDevicesByType(ctx context.Context, deviceType string) ([]Devices, error)
|
||||
ListDevicesByUser(ctx context.Context, userID pgtype.UUID) ([]Devices, error)
|
||||
// Download-worker policy union (D6): most aggressive policy wins for
|
||||
// what arrives. Runs under an admin identity (background job).
|
||||
ListFeedSubscribers(ctx context.Context, feedID pgtype.UUID) ([]ListFeedSubscribersRow, error)
|
||||
// Items hidden from libraries: either missing from disk (waiting on the
|
||||
// second scan that archives them) or already archived and pending purge.
|
||||
// Feeds the admin archived-items page. match_* columns identify an active
|
||||
@@ -540,6 +625,9 @@ type Querier interface {
|
||||
// collapsible "Ignored" section so a misclick is recoverable.
|
||||
ListIgnoredUnlinkedBooksForUser(ctx context.Context, userID pgtype.UUID) ([]ListIgnoredUnlinkedBooksForUserRow, error)
|
||||
ListLibraries(ctx context.Context) ([]ListLibrariesRow, error)
|
||||
// Podcasts exclusion (plan §9): generic book surfaces never list
|
||||
// podcast episodes (library_type_name is the denormalized, trigger/
|
||||
// fetcher-maintained discriminator; NULL rows predate it and stay).
|
||||
ListMediaItems(ctx context.Context, arg ListMediaItemsParams) ([]ListMediaItemsRow, error)
|
||||
ListMediaItemsByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]ListMediaItemsByLibraryRow, error)
|
||||
ListMediaItemsByLibraryIncludingArchived(ctx context.Context, libraryID pgtype.UUID) ([]ListMediaItemsByLibraryIncludingArchivedRow, error)
|
||||
@@ -548,11 +636,21 @@ type Querier interface {
|
||||
// List media items that have no stored SHA-256 (imported before hashing existed)
|
||||
ListMediaItemsMissingHash(ctx context.Context) ([]MediaItems, error)
|
||||
ListMediaItemsSorted(ctx context.Context, arg ListMediaItemsSortedParams) ([]ListMediaItemsSortedRow, error)
|
||||
// The podcasts page: the acting user's subscriptions with feed metadata
|
||||
// and the user-owned subscription fields. Facet values derive in Go.
|
||||
ListMyPodcastFeeds(ctx context.Context, userID pgtype.UUID) ([]ListMyPodcastFeedsRow, error)
|
||||
ListPendingHashConflicts(ctx context.Context) ([]ListPendingHashConflictsRow, error)
|
||||
// DeviceID NULL = every device's pending items (the queue processor's
|
||||
// enumerator passes NULL; a bare `device_id = NULL` never matches, which
|
||||
// silently disabled the offline flush until 2026-10-17).
|
||||
ListPendingSyncQueueItems(ctx context.Context, arg ListPendingSyncQueueItemsParams) ([]SyncQueue, error)
|
||||
// Feed detail / sequence source. RLS scopes to subscriber-or-admin;
|
||||
// ordering: native position, then pub date, then guid (stable across
|
||||
// re-fetches for historical rows whose position went stale).
|
||||
ListPodcastEpisodes(ctx context.Context, sourceFeedID pgtype.UUID) ([]MediaItems, error)
|
||||
// Poller worklist (admin identity): every feed with its validators and
|
||||
// the back-off flag (is_complete feeds poll rarely, never disappear).
|
||||
ListPodcastFeedsForPolling(ctx context.Context) ([]PodcastFeeds, error)
|
||||
// LEFT JOIN (Phase 11): row-less import failures have NO media_items
|
||||
// row — an inner join would silently drop the ledger's newest kind.
|
||||
// Item-side projections COALESCE to keep them non-nullable strings;
|
||||
@@ -560,6 +658,8 @@ type Querier interface {
|
||||
ListProcessingIssuesByLibrary(ctx context.Context, libraryID pgtype.UUID) ([]ListProcessingIssuesByLibraryRow, error)
|
||||
ListSyncConflictsByMediaItem(ctx context.Context, arg ListSyncConflictsByMediaItemParams) ([]SyncConflicts, error)
|
||||
ListSyncConflictsByUser(ctx context.Context, userID pgtype.UUID) ([]ListSyncConflictsByUserRow, error)
|
||||
// Download All + policy union worklist.
|
||||
ListUndownloadedEpisodes(ctx context.Context, sourceFeedID pgtype.UUID) ([]MediaItems, error)
|
||||
// List unresolved unlinked books with pagination
|
||||
ListUnresolvedUnlinkedBooks(ctx context.Context, arg ListUnresolvedUnlinkedBooksParams) ([]ListUnresolvedUnlinkedBooksRow, error)
|
||||
// Page list: unresolved queue rows for the caller's own devices (ignored
|
||||
@@ -568,6 +668,14 @@ type Querier interface {
|
||||
ListUsers(ctx context.Context) ([]ListUsersRow, error)
|
||||
// First consecutive scan that cannot find the file on disk.
|
||||
MarkMediaItemMissing(ctx context.Context, id pgtype.UUID) error
|
||||
// SUCCESS-only write (invariant 7): the file is on disk and probed.
|
||||
// ffprobe facts (real duration via audio_metadata, precise size/mime)
|
||||
// reconcile the feed's declared values.
|
||||
MarkPodcastEpisodeDownloaded(ctx context.Context, arg MarkPodcastEpisodeDownloadedParams) (MediaItems, error)
|
||||
// Failure/rejection write: state + reason + whether a human already
|
||||
// intervened (manual_retry resets the auto-retry budget). Existing
|
||||
// metadata/file facts are never cleared.
|
||||
MarkPodcastEpisodeFailed(ctx context.Context, arg MarkPodcastEpisodeFailedParams) error
|
||||
// Content reappeared at a new location while the old path vanished: treat as
|
||||
// a move. Repoints the row (and refreshes file-level fields) so reading
|
||||
// history follows the book; archive/missing state is cleared separately.
|
||||
@@ -631,14 +739,30 @@ type Querier interface {
|
||||
// Search Media Items queries
|
||||
SearchMediaItems(ctx context.Context, arg SearchMediaItemsParams) ([]SearchMediaItemsRow, error)
|
||||
SearchMediaItemsUnified(ctx context.Context, arg SearchMediaItemsUnifiedParams) ([]SearchMediaItemsUnifiedRow, error)
|
||||
// Subscription-scoped episode search (D14): title + description cover
|
||||
// un-downloaded AND historical rows (fetcher persists metadata-only).
|
||||
// RLS already restricts to the caller's subscriptions — the explicit
|
||||
// source filter keeps scan items out of podcast search.
|
||||
SearchMyPodcastEpisodes(ctx context.Context, searchQuery string) ([]MediaItems, error)
|
||||
// Feed-title AND subscription-display-name search (D14) + the facet
|
||||
// fields the podcasts page filters on.
|
||||
SearchMyPodcastFeeds(ctx context.Context, arg SearchMyPodcastFeedsParams) ([]SearchMyPodcastFeedsRow, error)
|
||||
SearchNarratorValues(ctx context.Context, arg SearchNarratorValuesParams) ([]SearchNarratorValuesRow, error)
|
||||
SearchSeriesValues(ctx context.Context, arg SearchSeriesValuesParams) ([]SearchSeriesValuesRow, error)
|
||||
SearchTagsValues(ctx context.Context, arg SearchTagsValuesParams) ([]SearchTagsValuesRow, error)
|
||||
SetDeviceSyncEnabled(ctx context.Context, arg SetDeviceSyncEnabledParams) (Devices, error)
|
||||
// Library Visibility queries
|
||||
SetLibraryVisibility(ctx context.Context, arg SetLibraryVisibilityParams) (LibraryVisibility, error)
|
||||
// Worker claim: pending|failed -> queued. Attempts incremented at
|
||||
// claim time so a crashed worker still counts the try.
|
||||
SetPodcastEpisodeDownloadQueued(ctx context.Context, id pgtype.UUID) (int64, error)
|
||||
SetPodcastEpisodeDownloading(ctx context.Context, id pgtype.UUID) error
|
||||
// Manual retry (same endpoint as download): reset to pending so the
|
||||
// worker re-claims it; the reason text clears with the fresh attempt.
|
||||
SetPodcastEpisodePending(ctx context.Context, id pgtype.UUID) error
|
||||
// Set system config
|
||||
SetSystemConfig(ctx context.Context, arg SetSystemConfigParams) (SystemConfig, error)
|
||||
SubscribePodcastFeed(ctx context.Context, arg SubscribePodcastFeedParams) (PodcastSubscriptions, error)
|
||||
SyncLibraryTypeExtensions(ctx context.Context, arg SyncLibraryTypeExtensionsParams) error
|
||||
TombstoneMediaBookmarkByDedupKey(ctx context.Context, arg TombstoneMediaBookmarkByDedupKeyParams) error
|
||||
TombstoneMediaBookmarkByID(ctx context.Context, arg TombstoneMediaBookmarkByIDParams) error
|
||||
@@ -703,6 +827,20 @@ type Querier interface {
|
||||
UpdateMediaNoteForSync(ctx context.Context, arg UpdateMediaNoteForSyncParams) (MediaNotes, error)
|
||||
UpdateMediaRating(ctx context.Context, arg UpdateMediaRatingParams) (MediaRatings, error)
|
||||
UpdatePassword(ctx context.Context, arg UpdatePasswordParams) error
|
||||
UpdatePodcastFeedArtwork(ctx context.Context, arg UpdatePodcastFeedArtworkParams) error
|
||||
// Post-fetch bookkeeping: validators for conditional GETs, the fetch
|
||||
// timestamp, and the feed's own last-build marker.
|
||||
UpdatePodcastFeedFetch(ctx context.Context, arg UpdatePodcastFeedFetchParams) (PodcastFeeds, error)
|
||||
// Fetcher-only full metadata refresh (parse result). podcast_guid is
|
||||
// set once a feed declares it; new_feed_url handling is a dedicated
|
||||
// query below.
|
||||
UpdatePodcastFeedMetadata(ctx context.Context, arg UpdatePodcastFeedMetadataParams) (PodcastFeeds, error)
|
||||
// itunes:new-feed-url (or a permanent HTTP redirect chain): the feed has
|
||||
// MOVED. Rewrite feed_url so future polls hit the new location; the row
|
||||
// (and every subscription/episode) stays put.
|
||||
UpdatePodcastFeedRedirect(ctx context.Context, arg UpdatePodcastFeedRedirectParams) error
|
||||
// User-owned fields only (D9): display name + auto-download policy.
|
||||
UpdatePodcastSubscription(ctx context.Context, arg UpdatePodcastSubscriptionParams) (PodcastSubscriptions, error)
|
||||
UpdateReadingProgress(ctx context.Context, arg UpdateReadingProgressParams) (ReadingProgress, error)
|
||||
UpdateReadingSpeed(ctx context.Context, arg UpdateReadingSpeedParams) (ReadingSpeed, error)
|
||||
UpdateSavedFilter(ctx context.Context, arg UpdateSavedFilterParams) (SavedFilters, error)
|
||||
@@ -728,11 +866,23 @@ type Querier interface {
|
||||
// content identical), and after a manifest-gated re-extract.
|
||||
UpsertMediaScanState(ctx context.Context, arg UpsertMediaScanStateParams) error
|
||||
UpsertPanelData(ctx context.Context, arg UpsertPanelDataParams) (PanelData, error)
|
||||
// ============================================================================
|
||||
// Podcast episodes — media_items rows with source='podcast_episode'
|
||||
// ============================================================================
|
||||
// Fetcher-only upsert keyed on the partial unique index
|
||||
// (source_feed_id, episode_guid). DO UPDATE refreshes FETCH-OWNED
|
||||
// metadata only. Write-after-success guard (AGENTS.md gotcha #7):
|
||||
// download state/error/attempts are NEVER touched here, and the
|
||||
// downloaded file's facts (path/size/mime/cover) survive renumbering —
|
||||
// only un-downloaded rows follow the re-derived file_path.
|
||||
// authors_search mirrors authors (trigram search column).
|
||||
UpsertPodcastEpisode(ctx context.Context, arg UpsertPodcastEpisodeParams) (MediaItems, error)
|
||||
UpsertReaderSettings(ctx context.Context, arg UpsertReaderSettingsParams) (ReaderSettings, error)
|
||||
UpsertSystemSetting(ctx context.Context, arg UpsertSystemSettingParams) (SystemSettings, error)
|
||||
// Ingest: insert or refresh the queue row for a device file that could not
|
||||
// be identified. One row per (device, content_id); re-syncs bump last_seen.
|
||||
UpsertUnlinkedBook(ctx context.Context, arg UpsertUnlinkedBookParams) (UnlinkedBooks, error)
|
||||
UpsertUserSettingByKey(ctx context.Context, arg UpsertUserSettingByKeyParams) error
|
||||
}
|
||||
|
||||
var _ Querier = (*Queries)(nil)
|
||||
+2976
-65
File diff suppressed because it is too large.
Load diff
@@ -97,7 +97,9 @@ DELETE FROM libraries WHERE id = $1;
|
||||
|
||||
-- Library Folders queries
|
||||
-- name: AddLibraryFolder :one
|
||||
INSERT INTO library_folders (library_id, folder_path) VALUES ($1, $2) RETURNING *;
|
||||
-- purpose: 'scan' (FS-walker root, default) | 'podcast' (download root;
|
||||
-- scanner-skipped, written by the podcast download worker instead).
|
||||
INSERT INTO library_folders (library_id, folder_path, purpose) VALUES ($1, $2, $3) RETURNING *;
|
||||
|
||||
-- name: GetLibraryFolders :many
|
||||
SELECT * FROM library_folders WHERE library_id = $1 ORDER BY created_at;
|
||||
@@ -178,12 +180,16 @@ SELECT * FROM media_items WHERE library_id = $1 AND file_path = $2;
|
||||
SELECT * FROM media_items WHERE id = $1;
|
||||
|
||||
-- name: ListMediaItems :many
|
||||
-- Podcasts exclusion (plan §9): generic book surfaces never list
|
||||
-- podcast episodes (library_type_name is the denormalized, trigger/
|
||||
-- fetcher-maintained discriminator; NULL rows predate it and stay).
|
||||
SELECT mi.*, l.name as library_name, lt.name as library_type_name
|
||||
FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
JOIN library_types lt ON l.library_type_id = lt.id
|
||||
WHERE mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts')
|
||||
AND (sqlc.narg('visible_library_ids')::uuid[] IS NULL OR mi.library_id = ANY(sqlc.narg('visible_library_ids')::uuid[]))
|
||||
ORDER BY mi.created_at DESC LIMIT $1 OFFSET $2;
|
||||
|
||||
@@ -212,6 +218,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
JOIN library_types lt ON l.library_type_id = lt.id
|
||||
WHERE mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.library_id = sqlc.narg('library_id')
|
||||
ORDER BY
|
||||
CASE
|
||||
@@ -658,6 +665,7 @@ LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.na
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (
|
||||
mi.title ILIKE sqlc.narg('search_pattern') OR
|
||||
@@ -734,14 +742,21 @@ SELECT
|
||||
mi.opf_uuid,
|
||||
mi.hash_confidence,
|
||||
l.name as library_name,
|
||||
lt.name as library_type_name
|
||||
lt.name as library_type_name,
|
||||
-- Viewer's own progress (search-result badges): NULL when the
|
||||
-- requester never opened the book. Reuses the SAME requester param
|
||||
-- the visibility join binds; UNIQUE(media_item, user) keeps the
|
||||
-- join row-neutral.
|
||||
rp.percentage as viewer_percentage
|
||||
FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
JOIN library_types lt ON l.library_type_id = lt.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
LEFT JOIN reading_progress rp ON rp.media_item_id = mi.id AND rp.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL
|
||||
OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
-- Fuzzy author filter — per-element against the authors[] array (A2c
|
||||
@@ -926,6 +941,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -945,6 +961,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -965,6 +982,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.tags_search) AS tag
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND tag IS NOT NULL
|
||||
AND word_similarity(sqlc.narg('search_query'), tag) > 0.3
|
||||
@@ -982,6 +1000,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.narrators) AS narrator
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND narrator IS NOT NULL
|
||||
AND word_similarity(sqlc.narg('search_query'), narrator) > 0.3
|
||||
@@ -998,6 +1017,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1017,6 +1037,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1055,6 +1076,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.authors) AS a
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1123,6 +1145,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.publishers) AS p
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1190,6 +1213,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.narrators) AS n
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1259,6 +1283,7 @@ JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
CROSS JOIN LATERAL unnest(mi.tags) AS t
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1327,6 +1352,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1398,6 +1424,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1469,6 +1496,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1540,6 +1568,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1607,6 +1636,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1674,6 +1704,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1741,6 +1772,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1812,6 +1844,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1883,6 +1916,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -1954,6 +1988,7 @@ FROM media_items mi
|
||||
JOIN libraries l ON mi.library_id = l.id
|
||||
LEFT JOIN library_visibility lv ON l.id = lv.library_id AND lv.user_id = sqlc.narg('user_id')
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
@@ -2598,6 +2633,7 @@ SELECT
|
||||
FROM media_items mi
|
||||
JOIN reading_history rh ON rh.media_item_id = mi.id
|
||||
WHERE rh.user_id = $1
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
GROUP BY mi.id, mi.title, mi.author
|
||||
ORDER BY read_count DESC
|
||||
LIMIT $2;
|
||||
@@ -2660,6 +2696,7 @@ SELECT
|
||||
FROM reading_history rh
|
||||
JOIN media_items mi ON mi.id = rh.media_item_id
|
||||
WHERE rh.user_id = sqlc.arg('user_id')
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND COALESCE(rh.reading_session_end, rh.created_at) >= sqlc.arg('start_time')
|
||||
AND COALESCE(rh.reading_session_end, rh.created_at) <= sqlc.arg('end_time')
|
||||
AND (sqlc.narg('visible_library_ids')::uuid[] IS NULL OR mi.library_id = ANY(sqlc.narg('visible_library_ids')::uuid[]))
|
||||
@@ -2803,6 +2840,14 @@ SET
|
||||
updated_at = NOW()
|
||||
WHERE id = $1;
|
||||
|
||||
-- name: DetachReadingHistoryFromDevice :exec
|
||||
-- Device deletion (deregister on logout): detach the device's reading
|
||||
-- history instead of deleting it — these rows are real captured stats
|
||||
-- (the Users section reads them); reading_history.device_id is NO ACTION
|
||||
-- so a blind device delete 500s once any history anchors to the device.
|
||||
-- device_id is nullable: NULL keeps the session, loses the association.
|
||||
UPDATE reading_history SET device_id = NULL WHERE device_id = $1;
|
||||
|
||||
-- name: DeleteDevice :exec
|
||||
DELETE FROM devices WHERE id = $1;
|
||||
|
||||
@@ -3948,6 +3993,7 @@ INNER JOIN (
|
||||
ORDER BY media_item_id, last_read_at DESC
|
||||
) rp ON rp.media_item_id = mi.id
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
ORDER BY rp.last_read_at DESC
|
||||
@@ -3956,6 +4002,7 @@ LIMIT sqlc.narg('limit');
|
||||
-- name: GetRecentlyAddedItems :many
|
||||
SELECT mi.* FROM media_items mi
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
ORDER BY mi.imported_at DESC NULLS LAST, mi.created_at DESC
|
||||
@@ -3971,6 +4018,7 @@ INNER JOIN (
|
||||
ORDER BY media_item_id, last_read_at DESC
|
||||
) rp ON rp.media_item_id = mi.id
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
ORDER BY rp.last_read_at DESC
|
||||
@@ -3979,6 +4027,7 @@ LIMIT sqlc.narg('limit');
|
||||
-- name: GetNotStartedItems :many
|
||||
SELECT mi.* FROM media_items mi
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND NOT EXISTS (
|
||||
@@ -4065,38 +4114,135 @@ SELECT mi.* FROM media_items mi
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR mi.library_id = sqlc.narg('library_id')::uuid)
|
||||
AND mi.archived_at IS NULL
|
||||
AND mi.missing_scan_count = 0
|
||||
AND (mi.library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9: collections picker)
|
||||
ORDER BY mi.created_at DESC;
|
||||
|
||||
-- name: GetDistinctSeries :many
|
||||
-- Series are grouped PER LIBRARY: the same series name in two libraries
|
||||
-- (novels vs manga adaptations, LN splits) stays two entries. Visible
|
||||
-- filter enforces per-user library visibility at the source.
|
||||
--
|
||||
-- Series-page sort/filter contract (extensible registry in
|
||||
-- services.ResolveSeriesSort): SeriesQuery is a series-NAME substring so
|
||||
-- it filters in WHERE (the group key — counts stay true); AuthorQuery
|
||||
-- and Status filter via HAVING so COUNT(*) keeps reporting the whole
|
||||
-- series. ViewerUserId scopes the per-user progress aggregates (the rp
|
||||
-- join can't multiply rows: reading_progress is UNIQUE per
|
||||
-- media_item+user). Sort is a BOUND PARAM compared inside the CASE
|
||||
-- cascade — never interpolated — and the GROUP BY tiebreaker stays
|
||||
-- REQUIRED for LIMIT/OFFSET determinism: bulk imports in one
|
||||
-- transaction share a single NOW() timestamp, leaving many groups tied
|
||||
-- on MAX(created_at), and Postgres does not guarantee tie order between
|
||||
-- executions, so page boundaries could repeat or drop series.
|
||||
SELECT library_id, series, COUNT(*) as book_count,
|
||||
MAX(series_count) as total_in_series,
|
||||
MAX(created_at) as last_entry_at
|
||||
MAX(created_at) as last_entry_at,
|
||||
COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) >= 0.999) AS finished_count,
|
||||
MAX(rp.last_read_at) AS last_read_at
|
||||
FROM media_items
|
||||
LEFT JOIN reading_progress rp
|
||||
ON rp.media_item_id = media_items.id
|
||||
AND rp.user_id = sqlc.narg('viewer_user_id')::uuid
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (sqlc.narg('visible_library_ids')::uuid[] IS NULL OR library_id = ANY(sqlc.narg('visible_library_ids')::uuid[]))
|
||||
AND archived_at IS NULL
|
||||
AND missing_scan_count = 0
|
||||
AND (library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9: series pages)
|
||||
AND series IS NOT NULL AND series != ''
|
||||
-- Multi-field search (series page): the query matches the series
|
||||
-- NAME, any book's AUTHOR, or any book's TITLE in the group. It is
|
||||
-- a WHERE filter: a group qualifies when ANY of its rows matches,
|
||||
-- so COUNT(*) keeps reporting the whole series.
|
||||
AND (sqlc.narg('series_query')::text IS NULL OR
|
||||
series ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
OR author ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM media_items sib
|
||||
WHERE sib.library_id = media_items.library_id
|
||||
AND sib.series = media_items.series
|
||||
AND sib.title ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
))
|
||||
GROUP BY library_id, series
|
||||
-- Tiebreaker REQUIRED for LIMIT/OFFSET pagination: bulk imports in one
|
||||
-- transaction share a single NOW() timestamp, leaving many groups tied on
|
||||
-- MAX(created_at). Postgres does not guarantee tie order between
|
||||
-- executions, so page boundaries could repeat or drop series. The GROUP BY
|
||||
-- keys make the sort total and deterministic.
|
||||
ORDER BY MAX(created_at) DESC, library_id, series
|
||||
HAVING (sqlc.narg('author_query')::text IS NULL OR BOOL_OR(author ILIKE '%' || sqlc.narg('author_query')::text || '%'))
|
||||
AND (
|
||||
sqlc.narg('status')::text IS NULL
|
||||
OR CASE sqlc.narg('status')::text
|
||||
WHEN 'finished' THEN COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) >= 0.999) = COUNT(*)
|
||||
WHEN 'reading' THEN COUNT(*) FILTER (WHERE rp.percentage > 0 AND rp.percentage < 0.999) >= 1
|
||||
WHEN 'unstarted' THEN COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) > 0) = 0
|
||||
ELSE true
|
||||
END
|
||||
)
|
||||
ORDER BY
|
||||
CASE WHEN COALESCE(sqlc.narg('sort')::text, 'recent') = 'recent' THEN MAX(created_at) END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'recent_asc' THEN MAX(created_at) END ASC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'name_asc' THEN series END ASC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'name_desc' THEN series END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'author_asc' THEN MIN(COALESCE(author, '')) END ASC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'author_desc' THEN MIN(COALESCE(author, '')) END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'books_desc' THEN COUNT(*) END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'books_asc' THEN COUNT(*) END ASC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'progress_desc'
|
||||
THEN (COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) >= 0.999))::float / NULLIF(COUNT(*), 0)
|
||||
END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'progress_asc'
|
||||
THEN (COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) >= 0.999))::float / NULLIF(COUNT(*), 0)
|
||||
END ASC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'last_read_desc' THEN MAX(rp.last_read_at) END DESC NULLS LAST,
|
||||
CASE WHEN sqlc.narg('sort')::text = 'last_read_asc' THEN MAX(rp.last_read_at) END ASC NULLS LAST,
|
||||
library_id, series
|
||||
LIMIT sqlc.narg('limit') OFFSET sqlc.narg('offset');
|
||||
|
||||
-- name: GetDistinctSeriesCount :one
|
||||
SELECT COUNT(DISTINCT (library_id, series))::int
|
||||
FROM media_items
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR library_id = sqlc.narg('library_id')::uuid)
|
||||
-- Count counterpart of GetDistinctSeries: identical WHERE + JOIN +
|
||||
-- HAVING so the pagination total matches what the page query returns
|
||||
-- under the same filters (Sort is a page-ORDER concern, not a set
|
||||
-- membership one, so it is absent here by design).
|
||||
SELECT COUNT(*)::int FROM (
|
||||
SELECT library_id, series
|
||||
FROM media_items
|
||||
LEFT JOIN reading_progress rp
|
||||
ON rp.media_item_id = media_items.id
|
||||
AND rp.user_id = sqlc.narg('viewer_user_id')::uuid
|
||||
WHERE (sqlc.narg('library_id')::uuid IS NULL OR library_id = sqlc.narg('library_id')::uuid)
|
||||
AND (sqlc.narg('visible_library_ids')::uuid[] IS NULL OR library_id = ANY(sqlc.narg('visible_library_ids')::uuid[]))
|
||||
AND archived_at IS NULL
|
||||
AND missing_scan_count = 0
|
||||
AND series IS NOT NULL AND series != '';
|
||||
AND (library_type_name IS DISTINCT FROM 'podcasts') -- podcasts exclusion (plan §9: series pages)
|
||||
AND series IS NOT NULL AND series != ''
|
||||
AND (sqlc.narg('series_query')::text IS NULL OR
|
||||
series ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
OR author ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
OR EXISTS (
|
||||
SELECT 1 FROM media_items sib
|
||||
WHERE sib.library_id = media_items.library_id
|
||||
AND sib.series = media_items.series
|
||||
AND sib.title ILIKE '%' || sqlc.narg('series_query')::text || '%'
|
||||
))
|
||||
GROUP BY library_id, series
|
||||
HAVING (sqlc.narg('author_query')::text IS NULL OR BOOL_OR(author ILIKE '%' || sqlc.narg('author_query')::text || '%'))
|
||||
AND (
|
||||
sqlc.narg('status')::text IS NULL
|
||||
OR CASE sqlc.narg('status')::text
|
||||
WHEN 'finished' THEN COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) >= 0.999) = COUNT(*)
|
||||
WHEN 'reading' THEN COUNT(*) FILTER (WHERE rp.percentage > 0 AND rp.percentage < 0.999) >= 1
|
||||
WHEN 'unstarted' THEN COUNT(*) FILTER (WHERE COALESCE(rp.percentage, 0) > 0) = 0
|
||||
ELSE true
|
||||
END
|
||||
)
|
||||
) groups;
|
||||
|
||||
-- name: CountOtherMediaItemsInDir :one
|
||||
-- Sibling-book count for sidecar naming (MetadataWriter): a dir-scoped
|
||||
-- metadata.opf may only be written when NO other non-archived item
|
||||
-- shares the directory; otherwise the sidecar goes to
|
||||
-- <basename>.opf so one book's metadata can never poison dir-mates.
|
||||
SELECT COUNT(*)::int
|
||||
FROM media_items
|
||||
WHERE library_id = sqlc.narg('library_id')::uuid
|
||||
AND id != sqlc.narg('id')::uuid
|
||||
AND archived_at IS NULL
|
||||
AND file_path LIKE (sqlc.narg('dir_prefix')::text || '%');
|
||||
|
||||
-- name: GetSeriesCovers :many
|
||||
SELECT cover_image_path, library_id
|
||||
@@ -4482,3 +4628,403 @@ SET file_size = $2,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1
|
||||
RETURNING id, library_id, title, author, isbn, description, file_path, file_size, mime_type, cover_image_path, series, series_number, tags, narrators, abridged, asin, date_published, publisher, contributors, language, edition, page_count, genre, copyright_year, goodreads_id, openlibrary_id, google_books_id, added_by_admin_id, created_at, imported_at, updated_at, format_group, format_mimetype, is_reflowable, has_fixed_layout, total_characters, chapter_count, entitlement_id, revision_number, kobo_content_id, kobo_metadata, manga_type, reading_direction, series_count, volume, imprint, age_rating, web_url, story_arc, is_black_and_white, metadata_notes, community_rating, alternate_info, scan_information, summary, metadata_overrides, missing_scan_count, archived_at, chapter_metadata, library_type_name, audio_metadata, tags_search, contributors_search, narrators_search, file_sha256, opf_identifier, opf_uuid, hash_confidence;
|
||||
|
||||
-- ============================================================================
|
||||
-- PODCASTS / RSS FEEDS (plan ~/.opencode/plan/podcast-integration.md §6/§8)
|
||||
-- Visibility is RLS-enforced (schema policy block) — NO query here carries
|
||||
-- visibility clauses. Feed reads run under the acting user's identity
|
||||
-- (subscriber-or-admin policy); the fetcher/worker run as first-admin.
|
||||
-- The fetcher/download worker is the SOLE writer of feed/episode metadata;
|
||||
-- users write only podcast_subscriptions rows.
|
||||
-- ============================================================================
|
||||
|
||||
-- name: CreatePodcastFeed :one
|
||||
-- Add-feed upsert keyed on the per-library UNIQUE(library_id, feed_url):
|
||||
-- a second subscriber adding the same URL attaches to the existing row
|
||||
-- instead of erroring. DO UPDATE (not DO NOTHING) so the RETURNING works
|
||||
-- even when the conflicting row is invisible to the actor under RLS
|
||||
-- (non-admin creators cannot SELECT other users' feeds). Metadata columns
|
||||
-- are filled by the fetcher right after; this only establishes identity.
|
||||
INSERT INTO podcast_feeds (library_id, feed_url, title, created_by_admin_id)
|
||||
VALUES ($1, $2, $3, $4)
|
||||
ON CONFLICT (library_id, feed_url) DO UPDATE SET updated_at = NOW()
|
||||
RETURNING *;
|
||||
|
||||
-- name: GetPodcastFeed :one
|
||||
SELECT * FROM podcast_feeds WHERE id = $1;
|
||||
|
||||
-- name: ListMyPodcastFeeds :many
|
||||
-- The podcasts page: the acting user's subscriptions with feed metadata
|
||||
-- and the user-owned subscription fields. Facet values derive in Go.
|
||||
SELECT f.*,
|
||||
s.display_name,
|
||||
s.auto_download,
|
||||
s.auto_download_count,
|
||||
s.created_at AS subscribed_at
|
||||
FROM podcast_subscriptions s
|
||||
JOIN podcast_feeds f ON f.id = s.feed_id
|
||||
WHERE s.user_id = $1
|
||||
ORDER BY COALESCE(s.display_name, f.title) ASC;
|
||||
|
||||
-- name: GetPodcastSubscription :one
|
||||
SELECT * FROM podcast_subscriptions WHERE user_id = $1 AND feed_id = $2;
|
||||
|
||||
-- name: SubscribePodcastFeed :one
|
||||
INSERT INTO podcast_subscriptions (user_id, feed_id)
|
||||
VALUES ($1, $2)
|
||||
ON CONFLICT (user_id, feed_id) DO NOTHING
|
||||
RETURNING *;
|
||||
|
||||
-- name: UpdatePodcastSubscription :one
|
||||
-- User-owned fields only (D9): display name + auto-download policy.
|
||||
UPDATE podcast_subscriptions
|
||||
SET display_name = $3,
|
||||
auto_download = $4,
|
||||
auto_download_count = $5
|
||||
WHERE user_id = $1 AND feed_id = $2
|
||||
RETURNING *;
|
||||
|
||||
-- name: DeletePodcastSubscription :execrows
|
||||
-- Unsubscribe NEVER deletes feed/episode/file state (D6).
|
||||
DELETE FROM podcast_subscriptions WHERE user_id = $1 AND feed_id = $2;
|
||||
|
||||
-- name: ListFeedSubscribers :many
|
||||
-- Download-worker policy union (D6): most aggressive policy wins for
|
||||
-- what arrives. Runs under an admin identity (background job).
|
||||
SELECT user_id, auto_download, auto_download_count
|
||||
FROM podcast_subscriptions
|
||||
WHERE feed_id = $1;
|
||||
|
||||
-- name: CountPodcastFeedSubscribers :one
|
||||
SELECT COUNT(*) FROM podcast_subscriptions WHERE feed_id = $1;
|
||||
|
||||
-- name: ListPodcastFeedsForPolling :many
|
||||
-- Poller worklist (admin identity): every feed with its validators and
|
||||
-- the back-off flag (is_complete feeds poll rarely, never disappear).
|
||||
SELECT * FROM podcast_feeds ORDER BY last_fetched_at ASC NULLS FIRST;
|
||||
|
||||
-- name: UpdatePodcastFeedFetch :one
|
||||
-- Post-fetch bookkeeping: validators for conditional GETs, the fetch
|
||||
-- timestamp, and the feed's own last-build marker.
|
||||
UPDATE podcast_feeds
|
||||
SET etag = $2,
|
||||
last_modified = $3,
|
||||
last_fetched_at = NOW(),
|
||||
last_build_date = $4,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1
|
||||
RETURNING *;
|
||||
|
||||
-- name: UpdatePodcastFeedMetadata :one
|
||||
-- Fetcher-only full metadata refresh (parse result). podcast_guid is
|
||||
-- set once a feed declares it; new_feed_url handling is a dedicated
|
||||
-- query below.
|
||||
UPDATE podcast_feeds
|
||||
SET title = $2,
|
||||
description = $3,
|
||||
host = $4,
|
||||
publisher = $5,
|
||||
owner_name = $6,
|
||||
owner_email = $7,
|
||||
language = $8,
|
||||
tags = $9,
|
||||
explicit = $10,
|
||||
feed_type = $11,
|
||||
medium = $12,
|
||||
is_complete = $13,
|
||||
podcast_guid = COALESCE($14, podcast_guid),
|
||||
updated_at = NOW()
|
||||
WHERE id = $1
|
||||
RETURNING *;
|
||||
|
||||
-- name: UpdatePodcastFeedArtwork :exec
|
||||
UPDATE podcast_feeds SET artwork_path = $2, updated_at = NOW() WHERE id = $1;
|
||||
|
||||
-- name: UpdatePodcastFeedRedirect :exec
|
||||
-- itunes:new-feed-url (or a permanent HTTP redirect chain): the feed has
|
||||
-- MOVED. Rewrite feed_url so future polls hit the new location; the row
|
||||
-- (and every subscription/episode) stays put.
|
||||
UPDATE podcast_feeds
|
||||
SET feed_url = $2,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1;
|
||||
|
||||
-- name: GetPodcastFeedByGuid :one
|
||||
-- URL-migration dedupe: same canonical guid within the library reuses
|
||||
-- the existing row.
|
||||
SELECT * FROM podcast_feeds
|
||||
WHERE library_id = $1 AND podcast_guid = $2
|
||||
LIMIT 1;
|
||||
|
||||
-- name: DeletePodcastFeed :execrows
|
||||
-- Admin-only removal (handler checks role); episodes cascade via
|
||||
-- media_items.source_feed_id ON DELETE CASCADE.
|
||||
DELETE FROM podcast_feeds WHERE id = $1;
|
||||
|
||||
-- name: CountPodcastLibrariesForUser :one
|
||||
-- Sidebar/page gate (D13): >=1 podcasts-type library the user can see
|
||||
-- (COALESCE(is_visible,true) semantics — mirrors GetUserVisibleLibraries).
|
||||
SELECT COUNT(*)
|
||||
FROM libraries l
|
||||
JOIN library_types lt ON lt.id = l.library_type_id AND lt.name = 'podcasts'
|
||||
LEFT JOIN library_visibility lv ON lv.library_id = l.id AND lv.user_id = $1
|
||||
WHERE COALESCE(lv.is_visible, true) = true;
|
||||
|
||||
-- name: GetPodcastsLibraryForUser :many
|
||||
-- The podcasts library/ies visible to the user (add-feed target list;
|
||||
-- a self-hosted instance typically has exactly one).
|
||||
SELECT l.id, l.name
|
||||
FROM libraries l
|
||||
JOIN library_types lt ON lt.id = l.library_type_id AND lt.name = 'podcasts'
|
||||
LEFT JOIN library_visibility lv ON lv.library_id = l.id AND lv.user_id = $1
|
||||
WHERE COALESCE(lv.is_visible, true) = true
|
||||
ORDER BY l.created_at ASC;
|
||||
|
||||
-- name: GetPodcastLibraryFolder :many
|
||||
-- The download root (purpose='podcast') of a podcasts library.
|
||||
SELECT lf.*
|
||||
FROM library_folders lf
|
||||
WHERE lf.library_id = $1 AND lf.purpose = 'podcast';
|
||||
|
||||
-- name: CreatePodcastLibraryFolder :one
|
||||
INSERT INTO library_folders (library_id, folder_path, purpose)
|
||||
VALUES ($1, $2, 'podcast')
|
||||
RETURNING *;
|
||||
|
||||
-- ============================================================================
|
||||
-- Podcast episodes — media_items rows with source='podcast_episode'
|
||||
-- ============================================================================
|
||||
|
||||
-- name: UpsertPodcastEpisode :one
|
||||
-- Fetcher-only upsert keyed on the partial unique index
|
||||
-- (source_feed_id, episode_guid). DO UPDATE refreshes FETCH-OWNED
|
||||
-- metadata only. Write-after-success guard (AGENTS.md gotcha #7):
|
||||
-- download state/error/attempts are NEVER touched here, and the
|
||||
-- downloaded file's facts (path/size/mime/cover) survive renumbering —
|
||||
-- only un-downloaded rows follow the re-derived file_path.
|
||||
-- authors_search mirrors authors (trigram search column).
|
||||
INSERT INTO media_items (
|
||||
library_id, title, subtitle, description, authors, authors_search,
|
||||
file_path, file_size, mime_type, cover_image_path, explicit,
|
||||
date_published, language, tags, genre,
|
||||
format_group, format_mimetype, is_reflowable, library_type_name,
|
||||
source, source_feed_id, episode_guid, feed_position,
|
||||
season_number, episode_number, enclosure_url,
|
||||
declared_duration_seconds, transcript_url, chapters_url,
|
||||
hosts, guests
|
||||
) VALUES (
|
||||
$1, $2, $3, $4, $5, $6,
|
||||
$7, $8, $9, $10, $11,
|
||||
$12, $13, $14, $15,
|
||||
'audiobook', $9, FALSE, 'podcasts',
|
||||
'podcast_episode', $16, $17, $18,
|
||||
$19, $20, $21,
|
||||
$22, $23, $24,
|
||||
$25, $26
|
||||
)
|
||||
ON CONFLICT (source_feed_id, episode_guid) WHERE source_feed_id IS NOT NULL DO UPDATE SET
|
||||
title = EXCLUDED.title,
|
||||
subtitle = EXCLUDED.subtitle,
|
||||
description = EXCLUDED.description,
|
||||
authors = EXCLUDED.authors,
|
||||
authors_search = EXCLUDED.authors_search,
|
||||
explicit = EXCLUDED.explicit,
|
||||
date_published = EXCLUDED.date_published,
|
||||
language = EXCLUDED.language,
|
||||
tags = EXCLUDED.tags,
|
||||
genre = EXCLUDED.genre,
|
||||
season_number = EXCLUDED.season_number,
|
||||
episode_number = EXCLUDED.episode_number,
|
||||
enclosure_url = EXCLUDED.enclosure_url,
|
||||
declared_duration_seconds = EXCLUDED.declared_duration_seconds,
|
||||
transcript_url = EXCLUDED.transcript_url,
|
||||
chapters_url = EXCLUDED.chapters_url,
|
||||
hosts = EXCLUDED.hosts,
|
||||
guests = EXCLUDED.guests,
|
||||
feed_position = EXCLUDED.feed_position,
|
||||
cover_image_path = CASE WHEN media_items.download_state = 'downloaded' THEN media_items.cover_image_path ELSE EXCLUDED.cover_image_path END,
|
||||
file_path = CASE WHEN media_items.download_state = 'downloaded' THEN media_items.file_path ELSE EXCLUDED.file_path END,
|
||||
file_size = CASE WHEN media_items.download_state = 'downloaded' THEN media_items.file_size ELSE EXCLUDED.file_size END,
|
||||
mime_type = CASE WHEN media_items.download_state = 'downloaded' THEN media_items.mime_type ELSE EXCLUDED.mime_type END,
|
||||
updated_at = NOW()
|
||||
RETURNING *;
|
||||
|
||||
-- name: ListPodcastEpisodes :many
|
||||
-- Feed detail / sequence source. RLS scopes to subscriber-or-admin;
|
||||
-- ordering: native position, then pub date, then guid (stable across
|
||||
-- re-fetches for historical rows whose position went stale).
|
||||
SELECT * FROM media_items
|
||||
WHERE source_feed_id = $1 AND source = 'podcast_episode'
|
||||
ORDER BY feed_position ASC NULLS LAST, date_published ASC NULLS LAST, episode_guid ASC;
|
||||
|
||||
-- name: GetPodcastEpisode :one
|
||||
SELECT * FROM media_items
|
||||
WHERE id = $1 AND source = 'podcast_episode';
|
||||
|
||||
-- name: SetPodcastEpisodeDownloadQueued :execrows
|
||||
-- Worker claim: pending|failed -> queued. Attempts incremented at
|
||||
-- claim time so a crashed worker still counts the try.
|
||||
UPDATE media_items
|
||||
SET download_state = 'queued',
|
||||
download_error = NULL,
|
||||
download_attempts = download_attempts + 1,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1
|
||||
AND source = 'podcast_episode'
|
||||
AND download_state IN ('pending', 'failed');
|
||||
|
||||
-- name: SetPodcastEpisodeDownloading :exec
|
||||
UPDATE media_items
|
||||
SET download_state = 'downloading', updated_at = NOW()
|
||||
WHERE id = $1 AND source = 'podcast_episode';
|
||||
|
||||
-- name: MarkPodcastEpisodeDownloaded :one
|
||||
-- SUCCESS-only write (invariant 7): the file is on disk and probed.
|
||||
-- ffprobe facts (real duration via audio_metadata, precise size/mime)
|
||||
-- reconcile the feed's declared values.
|
||||
UPDATE media_items
|
||||
SET download_state = 'downloaded',
|
||||
download_error = NULL,
|
||||
file_path = $2,
|
||||
file_size = $3,
|
||||
mime_type = $4,
|
||||
cover_image_path = COALESCE($5, cover_image_path),
|
||||
audio_metadata = $6,
|
||||
chapter_count = $7,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1 AND source = 'podcast_episode'
|
||||
RETURNING *;
|
||||
|
||||
-- name: MarkPodcastEpisodeFailed :exec
|
||||
-- Failure/rejection write: state + reason + whether a human already
|
||||
-- intervened (manual_retry resets the auto-retry budget). Existing
|
||||
-- metadata/file facts are never cleared.
|
||||
UPDATE media_items
|
||||
SET download_state = 'failed',
|
||||
download_error = $2,
|
||||
download_attempts = CASE WHEN $3::boolean THEN 0 ELSE download_attempts END,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1 AND source = 'podcast_episode';
|
||||
|
||||
-- name: SetPodcastEpisodePending :exec
|
||||
-- Manual retry (same endpoint as download): reset to pending so the
|
||||
-- worker re-claims it; the reason text clears with the fresh attempt.
|
||||
UPDATE media_items
|
||||
SET download_state = 'pending',
|
||||
download_error = NULL,
|
||||
download_attempts = 0,
|
||||
updated_at = NOW()
|
||||
WHERE id = $1 AND source = 'podcast_episode';
|
||||
|
||||
-- name: ListUndownloadedEpisodes :many
|
||||
-- Download All + policy union worklist.
|
||||
SELECT * FROM media_items
|
||||
WHERE source_feed_id = $1
|
||||
AND source = 'podcast_episode'
|
||||
AND download_state <> 'downloaded'
|
||||
ORDER BY feed_position ASC NULLS LAST, date_published ASC NULLS LAST, episode_guid ASC;
|
||||
|
||||
-- name: CountUndownloadedEpisodes :one
|
||||
SELECT COUNT(*) FROM media_items
|
||||
WHERE source_feed_id = $1
|
||||
AND source = 'podcast_episode'
|
||||
AND download_state <> 'downloaded';
|
||||
|
||||
-- name: CountDownloadedEpisodes :one
|
||||
SELECT COUNT(*) FROM media_items
|
||||
WHERE source_feed_id = $1
|
||||
AND source = 'podcast_episode'
|
||||
AND download_state = 'downloaded';
|
||||
|
||||
-- name: SearchMyPodcastEpisodes :many
|
||||
-- Subscription-scoped episode search (D14): title + description cover
|
||||
-- un-downloaded AND historical rows (fetcher persists metadata-only).
|
||||
-- RLS already restricts to the caller's subscriptions — the explicit
|
||||
-- source filter keeps scan items out of podcast search.
|
||||
SELECT * FROM media_items
|
||||
WHERE source = 'podcast_episode'
|
||||
AND (sqlc.arg('search_query')::text = '' OR title ILIKE '%' || sqlc.arg('search_query')::text || '%' OR description ILIKE '%' || sqlc.arg('search_query')::text || '%')
|
||||
ORDER BY feed_position ASC NULLS LAST, date_published ASC NULLS LAST
|
||||
LIMIT 50;
|
||||
|
||||
-- name: SearchMyPodcastFeeds :many
|
||||
-- Feed-title AND subscription-display-name search (D14) + the facet
|
||||
-- fields the podcasts page filters on.
|
||||
SELECT f.*,
|
||||
s.display_name,
|
||||
s.auto_download,
|
||||
s.auto_download_count,
|
||||
s.created_at AS subscribed_at
|
||||
FROM podcast_subscriptions s
|
||||
JOIN podcast_feeds f ON f.id = s.feed_id
|
||||
WHERE s.user_id = sqlc.arg('user_id')
|
||||
AND (sqlc.arg('search_query')::text = '' OR f.title ILIKE '%' || sqlc.arg('search_query')::text || '%'
|
||||
OR COALESCE(s.display_name, '') ILIKE '%' || sqlc.arg('search_query')::text || '%'
|
||||
OR COALESCE(f.host, '') ILIKE '%' || sqlc.arg('search_query')::text || '%'
|
||||
OR COALESCE(f.publisher, '') ILIKE '%' || sqlc.arg('search_query')::text || '%')
|
||||
ORDER BY COALESCE(s.display_name, f.title) ASC;
|
||||
|
||||
-- name: GetNextQueuedEpisode :one
|
||||
-- Sequential download worker's claim source (single worker, day one).
|
||||
SELECT * FROM media_items
|
||||
WHERE source = 'podcast_episode' AND download_state = 'queued'
|
||||
ORDER BY updated_at ASC
|
||||
LIMIT 1;
|
||||
|
||||
-- name: CountQueuedEpisodes :one
|
||||
SELECT COUNT(*) FROM media_items
|
||||
WHERE source = 'podcast_episode' AND download_state IN ('queued', 'downloading');
|
||||
|
||||
-- name: CountPodcastEpisodesForFeed :one
|
||||
SELECT COUNT(*) FROM media_items
|
||||
WHERE source_feed_id = $1 AND source = 'podcast_episode';
|
||||
|
||||
-- ============================================================================
|
||||
-- GENERIC reader_settings KV (podcast order override, plan D5: key
|
||||
-- podcast_order:<feed_id>). The pre-existing reader_settings queries are
|
||||
-- hard-keyed to 'reader_settings'; these two are the generic form.
|
||||
-- ============================================================================
|
||||
|
||||
-- name: GetUserSettingByKey :one
|
||||
SELECT setting_value FROM reader_settings
|
||||
WHERE user_id = $1 AND setting_key = $2;
|
||||
|
||||
-- name: UpsertUserSettingByKey :exec
|
||||
INSERT INTO reader_settings (user_id, setting_key, setting_value)
|
||||
VALUES ($1, $2, $3)
|
||||
ON CONFLICT (user_id, setting_key)
|
||||
DO UPDATE SET setting_value = EXCLUDED.setting_value, updated_at = NOW();
|
||||
|
||||
-- name: DeleteUserSettingByKey :exec
|
||||
DELETE FROM reader_settings WHERE user_id = $1 AND setting_key = $2;
|
||||
|
||||
-- name: GetPodcastEpisodeProgress :many
|
||||
-- Per-user progress for a feed's episodes (row badges + the Phase-2
|
||||
-- continue fields). reading_progress is UNIQUE per (media_item, user).
|
||||
SELECT rp.media_item_id,
|
||||
rp.percentage,
|
||||
rp.last_read_at
|
||||
FROM reading_progress rp
|
||||
JOIN media_items mi ON mi.id = rp.media_item_id
|
||||
WHERE rp.user_id = $1
|
||||
AND mi.source_feed_id = $2
|
||||
AND mi.source = 'podcast_episode';
|
||||
|
||||
-- name: GetPodcastContinueForUser :many
|
||||
-- Continue aggregation (plan 2.5): the caller's IN-PROGRESS episodes
|
||||
-- (0 < pct < finishedThreshold) across ALL subscribed feeds, most
|
||||
-- recent first. Go groups per feed — the first row per feed IS the
|
||||
-- continue target.
|
||||
SELECT mi.source_feed_id,
|
||||
mi.id,
|
||||
mi.title,
|
||||
mi.feed_position,
|
||||
rp.percentage,
|
||||
rp.last_read_at
|
||||
FROM reading_progress rp
|
||||
JOIN media_items mi ON mi.id = rp.media_item_id
|
||||
WHERE rp.user_id = $1
|
||||
AND mi.source = 'podcast_episode'
|
||||
AND rp.percentage > 0
|
||||
AND rp.percentage < 0.995
|
||||
ORDER BY rp.last_read_at DESC;
|
||||
@@ -178,6 +178,16 @@ func verifyFunctions(ctx context.Context, db *pgxpool.Pool, expectedFunctions []
|
||||
return nil
|
||||
}
|
||||
|
||||
// schemaMigrationIdentity is the synthetic identity the schema
|
||||
// migration transaction declares for RLS (plan appendix A11). The
|
||||
// schema's dedup DO block SELECTs/DELETEs media_items, and with the
|
||||
// visibility policies live (FORCE applies to the owner) that SELECT
|
||||
// needs app.user_id set even in this maintenance tx. The identity has
|
||||
// NO library_visibility rows, so the policy grants it full visibility
|
||||
// — appropriate for the server's own schema pass, and unreachable by
|
||||
// requests (this tx only ever runs at startup, on the raw pool).
|
||||
const schemaMigrationIdentity = "00000000-0000-0000-0000-000000000001"
|
||||
|
||||
// executeSchema runs the entire schema.sql in a single transaction
|
||||
func executeSchema(ctx context.Context, db *pgxpool.Pool) error {
|
||||
tx, err := db.Begin(ctx)
|
||||
@@ -186,6 +196,12 @@ func executeSchema(ctx context.Context, db *pgxpool.Pool) error {
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
// Declare the migration identity BEFORE the file executes (A11):
|
||||
// policy'd tables are queried by schema.sql itself on re-runs.
|
||||
if _, err := tx.Exec(ctx, `SELECT set_config('app.user_id', $1, true)`, schemaMigrationIdentity); err != nil {
|
||||
return fmt.Errorf("schema execution failed (declare migration identity): %w", err)
|
||||
}
|
||||
|
||||
_, err = tx.Exec(ctx, SchemaFile)
|
||||
if err != nil {
|
||||
log.Printf("HINT: Run manually to debug: psql -h localhost -U postgres -d bookhoard -f database/schema/schema.sql")
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
// ScopedDB wraps a pgx pool so every sqlc statement runs inside a
|
||||
// transaction that declares the acting user via the LOCAL
|
||||
// `app.user_id` GUC — the one thing the RLS policies
|
||||
// (database/schema/schema.sql) need to filter media rows per user.
|
||||
//
|
||||
// Fail-loud lives in the ENGINE, not here: a context with no identity
|
||||
// simply declares nothing, and any RLS-governed statement (media_items)
|
||||
// then raises on the unset GUC — while ungoverned tables (auth lookups,
|
||||
// settings, sessions) keep working, as they have no policy. That is the
|
||||
// plan-C property: the database is the one door everything crosses; the
|
||||
// decorator is only the identity declarer.
|
||||
//
|
||||
// SEMANTICS PRESERVED: every sqlc call was already a single
|
||||
// auto-committed statement; one transaction per call keeps per-call
|
||||
// atomicity identical. pgx v5 rows are bound to their transaction, so
|
||||
// the transaction ends when the ROWS end: commit-on-Close for :many
|
||||
// results, commit-on-Scan for :one results, plain commit for Exec.
|
||||
//
|
||||
// Rollout plan: ~/.opencode/plan/server-rls-library-visibility.md.
|
||||
package database
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"log"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgconn"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
|
||||
"bookhoard/internal/rls"
|
||||
)
|
||||
|
||||
// Compile-pin: ScopedDB is a drop-in DBTX for database.New.
|
||||
var _ DBTX = (*ScopedDB)(nil)
|
||||
|
||||
// ScopedDB implements DBTX over a pool, declaring the acting user for
|
||||
// the RLS policies on every statement. Fallback (test harness ONLY —
|
||||
// see NewSeedDB) supplies an identity when the context carries none.
|
||||
type ScopedDB struct {
|
||||
Pool *pgxpool.Pool
|
||||
Fallback *pgtype.UUID
|
||||
}
|
||||
|
||||
// NewScopedDB is the constructor used at the single Queries
|
||||
// construction site (cmd/server/main.go).
|
||||
func NewScopedDB(pool *pgxpool.Pool) *ScopedDB {
|
||||
return &ScopedDB{Pool: pool}
|
||||
}
|
||||
|
||||
// NewSeedDB returns a ScopedDB for TEST-HARNESS seeding/verification:
|
||||
// when the context carries no identity it falls back to
|
||||
// rls.SeedIdentity, so direct media reads/writes from helpers work
|
||||
// without impersonating a real user. Never use in product wiring.
|
||||
func NewSeedDB(pool *pgxpool.Pool) *ScopedDB {
|
||||
seed := rls.SeedIdentity
|
||||
return &ScopedDB{Pool: pool, Fallback: &seed}
|
||||
}
|
||||
|
||||
const zeroUUID = "00000000-0000-0000-0000-000000000000"
|
||||
|
||||
// begin opens the per-statement transaction. If the context carries a
|
||||
// user identity, the GUC is declared (LOCAL); if not, NOTHING is
|
||||
// declared and the ENGINE decides: RLS-governed tables (media_items)
|
||||
// raise on the unset GUC — fail-loud at the database, the layer plan C
|
||||
// exists to enforce at — while RLS-ungoverned tables (users, settings,
|
||||
// sessions, auth lookups pre-bridge) simply work, as they have no
|
||||
// policy to evaluate. The decorator therefore never refuses: refusing
|
||||
// here would re-create a convention gate in code, one more door future
|
||||
// paths could forget — the engine is the one door everything crosses.
|
||||
func (s *ScopedDB) begin(ctx context.Context) (pgx.Tx, error) {
|
||||
tx, err := s.Pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
uid, ok := rls.FromContext(ctx)
|
||||
if (!ok || !uid.Valid || uid.String() == zeroUUID) && s.Fallback != nil {
|
||||
uid, ok = *s.Fallback, true
|
||||
}
|
||||
if ok && uid.Valid && uid.String() != zeroUUID {
|
||||
if err := rls.DeclareUser(ctx, tx, uid); err != nil {
|
||||
_ = tx.Rollback(ctx)
|
||||
return nil, fmt.Errorf("rls: declare user: %w", err)
|
||||
}
|
||||
}
|
||||
return tx, nil
|
||||
}
|
||||
|
||||
func (s *ScopedDB) Exec(ctx context.Context, sql string, args ...any) (pgconn.CommandTag, error) {
|
||||
tx, err := s.begin(ctx)
|
||||
if err != nil {
|
||||
return pgconn.CommandTag{}, err
|
||||
}
|
||||
// Rollback after a successful Commit is a no-op (ErrTxClosed ignored):
|
||||
// this defer only acts on the error paths.
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
tag, err := tx.Exec(ctx, sql, args...)
|
||||
if err != nil {
|
||||
return pgconn.CommandTag{}, err
|
||||
}
|
||||
return tag, tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (s *ScopedDB) Query(ctx context.Context, sql string, args ...any) (pgx.Rows, error) {
|
||||
tx, err := s.begin(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows, err := tx.Query(ctx, sql, args...)
|
||||
if err != nil {
|
||||
_ = tx.Rollback(ctx)
|
||||
return nil, err
|
||||
}
|
||||
return &scopedRows{Rows: rows, tx: tx, ctx: ctx}, nil
|
||||
}
|
||||
|
||||
// scopedRows commits the transaction when the rows close — committing
|
||||
// earlier would kill the rows mid-read. sqlc's :many path fully
|
||||
// consumes rows then Close()s. pgx v5 Rows.Close returns nothing, so a
|
||||
// commit error is logged, not propagated (a commit failure on a
|
||||
// read-only transaction is log-worthy only).
|
||||
type scopedRows struct {
|
||||
pgx.Rows
|
||||
tx pgx.Tx
|
||||
ctx context.Context
|
||||
}
|
||||
|
||||
func (r *scopedRows) Close() {
|
||||
r.Rows.Close()
|
||||
if err := r.tx.Commit(r.ctx); err != nil {
|
||||
log.Printf("rls: tx commit after rows: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// scopedRow terminates the transaction on EVERY Scan path for sqlc's
|
||||
// :one results (pgx.Row is just { Scan(dest ...) error }).
|
||||
type scopedRow struct {
|
||||
row pgx.Row
|
||||
tx pgx.Tx
|
||||
ctx context.Context
|
||||
}
|
||||
|
||||
func (r scopedRow) Scan(dest ...any) error {
|
||||
err := r.row.Scan(dest...)
|
||||
if err == nil {
|
||||
return r.tx.Commit(r.ctx)
|
||||
}
|
||||
_ = r.tx.Rollback(r.ctx) // best effort; the tx ends either way
|
||||
return err
|
||||
}
|
||||
|
||||
type errRow struct{ err error }
|
||||
|
||||
func (r errRow) Scan(_ ...any) error { return r.err }
|
||||
|
||||
func (s *ScopedDB) QueryRow(ctx context.Context, sql string, args ...any) pgx.Row {
|
||||
tx, err := s.begin(ctx)
|
||||
if err != nil {
|
||||
return errRow{err}
|
||||
}
|
||||
return scopedRow{row: tx.QueryRow(ctx, sql, args...), tx: tx, ctx: ctx}
|
||||
}
|
||||
@@ -82,6 +82,7 @@ var SettingDefaults = []SettingDefault{
|
||||
{Key: "sync_queue_batch_size", Value: "50", Type: SettingTypeInt, Min: "1", Max: "10000", RequiresRestart: true, Category: "sync", Group: "Sync Queue", Description: "Maximum items processed per sync queue flush"},
|
||||
{Key: "worker_pool_size", Value: "3", Type: SettingTypeInt, Min: "1", Max: "100", RequiresRestart: true, Category: "performance", Group: "Worker Pool", Description: "Number of background worker goroutines"},
|
||||
{Key: "worker_queue_cap", Value: "100", Type: SettingTypeInt, Min: "1", Max: "10000", RequiresRestart: true, Category: "performance", Group: "Worker Pool", Description: "Background worker job queue capacity"},
|
||||
{Key: "podcast_poll_interval_seconds", Value: "3600", Type: SettingTypeInt, Min: "300", Max: "604800", Category: "podcasts", Group: "Podcasts", Description: "How often to poll podcast feeds for new episodes (seconds; complete feeds back off)"},
|
||||
}
|
||||
|
||||
// defaultBy indexes SettingDefaults by key for O(1) lookup.
|
||||
@@ -188,6 +189,13 @@ func (r *SettingsRegistry) ScanPollInterval() time.Duration {
|
||||
return time.Duration(r.getInt("scan_poll_interval_seconds")) * time.Second
|
||||
}
|
||||
|
||||
// PodcastPollInterval is the feed-poller tick (live setting; the poller
|
||||
// re-reads it every cycle — writes go through the settings API path,
|
||||
// AGENTS.md gotcha #2).
|
||||
func (r *SettingsRegistry) PodcastPollInterval() time.Duration {
|
||||
return time.Duration(r.getInt("podcast_poll_interval_seconds")) * time.Second
|
||||
}
|
||||
|
||||
// AutoScanEnabled reports whether auto-scanning is on.
|
||||
func (r *SettingsRegistry) AutoScanEnabled() bool { return r.getBool("auto_scan_enabled") }
|
||||
|
||||
|
||||
@@ -3,7 +3,6 @@ package handlers
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/utils"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
@@ -102,7 +101,7 @@ func (h *AnalyticsHandler) GetReadingStats(c *echo.Context) error {
|
||||
// "today") never appear.
|
||||
endTime = endTime.AddDate(0, 0, 1).Add(-time.Nanosecond)
|
||||
|
||||
ctx := context.Background()
|
||||
ctx := c.Request().Context()
|
||||
|
||||
history, err := h.db.GetUserReadingHistory(ctx, database.GetUserReadingHistoryParams{
|
||||
UserID: user.ID,
|
||||
@@ -220,7 +219,7 @@ func (h *AnalyticsHandler) calculateMostActiveDay(history []database.GetUserRead
|
||||
func (h *AnalyticsHandler) GetDeviceUsage(c *echo.Context) error {
|
||||
user := c.Get("user").(database.Users)
|
||||
|
||||
ctx := context.Background()
|
||||
ctx := c.Request().Context()
|
||||
|
||||
usage, err := h.db.GetUserDeviceUsage(ctx, user.ID)
|
||||
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
|
||||
@@ -268,7 +267,7 @@ func (h *AnalyticsHandler) GetPopularBooks(c *echo.Context) error {
|
||||
limitInt = parsedLimit
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx := c.Request().Context()
|
||||
|
||||
books, err := h.db.GetPopularBooks(ctx, database.GetPopularBooksParams{
|
||||
UserID: user.ID,
|
||||
|
||||
+19
-17
@@ -10,6 +10,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"html"
|
||||
"html/template"
|
||||
"math"
|
||||
"net/http"
|
||||
@@ -335,13 +336,9 @@ func (h *AuthHandler) Register(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
html := fmt.Sprintf(`<div class="text-green-500">Registration successful! Redirecting...</div>
|
||||
<script>
|
||||
localStorage.setItem('token', '%s');
|
||||
localStorage.setItem('refreshToken', '%s');
|
||||
localStorage.setItem('user', JSON.stringify(%s));
|
||||
window.location.href = '/dashboard';
|
||||
</script>`, accessToken, refreshToken, fmt.Sprintf(`{"id":"%s","email":"%s","username":"%s"}`, uuid.UUID(user.ID.Bytes).String(), user.Email, user.Username))
|
||||
// CSP: see the login handler — data attributes, not inline scripts.
|
||||
html := fmt.Sprintf(`<div class="text-green-500" data-auth-token="%s" data-redirect="/dashboard">Registration successful! Redirecting...</div>`,
|
||||
accessToken)
|
||||
return c.HTML(http.StatusCreated, html)
|
||||
}
|
||||
|
||||
@@ -368,6 +365,10 @@ window.location.href = '/dashboard';
|
||||
}
|
||||
|
||||
// Login handles POST /api/auth/login
|
||||
// invalidCredentialsHTML is the htmx error partial for failed logins;
|
||||
// web/src/auth_forms.ts swaps 4xx/5xx HTML responses into the form target.
|
||||
const invalidCredentialsHTML = `<div class="text-red-500">Invalid email/username or password.</div>`
|
||||
|
||||
func (h *AuthHandler) Login(c *echo.Context) error {
|
||||
login := c.FormValue("login")
|
||||
password := c.FormValue("password")
|
||||
@@ -424,7 +425,7 @@ func (h *AuthHandler) Login(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
|
||||
return c.HTML(http.StatusUnauthorized, invalidCredentialsHTML)
|
||||
}
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
|
||||
}
|
||||
@@ -440,7 +441,7 @@ func (h *AuthHandler) Login(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
|
||||
return c.HTML(http.StatusUnauthorized, invalidCredentialsHTML)
|
||||
}
|
||||
return c.JSON(http.StatusUnauthorized, map[string]string{"error": "invalid credentials"})
|
||||
}
|
||||
@@ -481,17 +482,18 @@ func (h *AuthHandler) Login(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
// CSP: hash-mode script-src refuses inline scripts, so the htmx
|
||||
// partial carries data attributes instead; web/src/auth_forms.ts
|
||||
// stashes the token and navigates on htmx:afterSwap. The redirect
|
||||
// target is path-only (open-redirect hygiene) and the old
|
||||
// 'user'/'refreshToken' localStorage writes are gone — nothing
|
||||
// ever read them.
|
||||
redirect := c.FormValue("redirect")
|
||||
if redirect == "" {
|
||||
if redirect == "" || !strings.HasPrefix(redirect, "/") {
|
||||
redirect = "/dashboard"
|
||||
}
|
||||
html := fmt.Sprintf(`<div class="text-green-500">Login successful! Redirecting...</div>
|
||||
<script>
|
||||
localStorage.setItem('token', '%s');
|
||||
localStorage.setItem('refreshToken', '%s');
|
||||
localStorage.setItem('user', JSON.stringify(%s));
|
||||
window.location.href = '%s';
|
||||
</script>`, accessToken, refreshToken, fmt.Sprintf(`{"id":"%s","email":"%s","username":"%s","first_name":"%s","last_name":"%s"}`, uuid.UUID(user.ID.Bytes).String(), user.Email, user.Username, user.FirstName.String, user.LastName.String), redirect)
|
||||
html := fmt.Sprintf(`<div class="text-green-500" data-auth-token="%s" data-redirect="%s">Login successful! Redirecting...</div>`,
|
||||
accessToken, html.EscapeString(redirect))
|
||||
return c.HTML(http.StatusOK, html)
|
||||
}
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
// BookInfo is THE canonical wire+template card shape: every surface that
|
||||
// lists a book card as JSON (collections detail, series, preview,
|
||||
// dashboard sections, search) or renders a templ card (BookCard in
|
||||
// dashboard.templ, BooksGrid, BrowseDetail) carries exactly these seven
|
||||
// dashboard.templ, BooksGrid, BrowseDetail) carries exactly these nine
|
||||
// keys and no others — pinned by docs/contract_pins.md.
|
||||
//
|
||||
// H2 doctrine: this struct is constructed in EXACTLY ONE place —
|
||||
@@ -34,6 +34,15 @@ type BookInfo struct {
|
||||
// comic vs manga rides the LIBRARY type (format_group only knows
|
||||
// comic_archive — manga_type is unreliable across sources).
|
||||
Badge string `json:"badge"`
|
||||
// Viewer-scoped extras (2026-10-27 search-progress batch): nil
|
||||
// percentage = the requesting user never opened the book, so cards
|
||||
// render no progress bar and no completed badge. Adapters that
|
||||
// don't load viewer progress leave it nil; the search-unified
|
||||
// adapter is the only populated feed today. LibraryName rides the
|
||||
// same deal (unified rows carry it as a column; other feeds leave
|
||||
// it empty rather than pay a join).
|
||||
ViewerPercentage *float64 `json:"viewer_percentage"`
|
||||
LibraryName string `json:"library_name"`
|
||||
}
|
||||
|
||||
// LibraryTypeMap builds libraryID→library-type-name for badge resolution
|
||||
@@ -85,6 +94,8 @@ func buildBookCard(src cardSource) BookInfo {
|
||||
CoverImagePath: utils.ResolveMediaURL(src.LibraryID, src.CoverPath),
|
||||
FormatGroup: src.FormatGroup,
|
||||
Badge: MediaBadgeKind(src.FormatGroup, src.LibraryType),
|
||||
ViewerPercentage: src.ViewerProgress,
|
||||
LibraryName: src.LibraryName,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,6 +113,10 @@ type cardSource struct {
|
||||
CoverPath pgtype.Text
|
||||
FormatGroup string
|
||||
LibraryType string
|
||||
// Viewer-scoped extras: nil progress = no data loaded for the
|
||||
// requesting user (see BookInfo.ViewerPercentage).
|
||||
ViewerProgress *float64
|
||||
LibraryName string
|
||||
}
|
||||
|
||||
// BookCardsFromMediaItems builds cards from full media rows
|
||||
@@ -167,10 +182,17 @@ func BookCardsFromCollectionItemRows(rows []database.GetCollectionItemsRow, libT
|
||||
|
||||
// BookCardsFromUnified builds cards from search-unified rows. Those
|
||||
// rows carry the library type name as a SQL column, so this adapter
|
||||
// needs no LibraryTypeMap query at all.
|
||||
// needs no LibraryTypeMap query at all — and the viewer's own
|
||||
// progress (the 2026-10-27 search-progress batch), which no other
|
||||
// feed loads.
|
||||
func BookCardsFromUnified(rows []database.SearchMediaItemsUnifiedRow) []BookInfo {
|
||||
cards := make([]BookInfo, 0, len(rows))
|
||||
for _, r := range rows {
|
||||
var viewer *float64
|
||||
if r.ViewerPercentage.Valid {
|
||||
pct := r.ViewerPercentage.Float64
|
||||
viewer = &pct
|
||||
}
|
||||
cards = append(cards, buildBookCard(cardSource{
|
||||
ID: uuid.UUID(r.ID.Bytes),
|
||||
Title: r.Title,
|
||||
@@ -179,6 +201,8 @@ func BookCardsFromUnified(rows []database.SearchMediaItemsUnifiedRow) []BookInfo
|
||||
CoverPath: r.CoverImagePath,
|
||||
FormatGroup: r.FormatGroup,
|
||||
LibraryType: r.LibraryTypeName,
|
||||
ViewerProgress: viewer,
|
||||
LibraryName: r.LibraryName,
|
||||
}))
|
||||
}
|
||||
return cards
|
||||
|
||||
@@ -496,7 +496,8 @@ func (h *CollectionHandler) DeleteCollection(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
html := `<div class="text-green-500">Collection deleted! Redirecting...</div><script>window.location.href = '/collections';</script>`
|
||||
// CSP: no inline scripts in partials — auth_forms.ts navigates.
|
||||
html := `<div class="text-green-500" data-redirect="/collections">Collection deleted! Redirecting...</div>`
|
||||
return c.HTML(http.StatusOK, html)
|
||||
}
|
||||
|
||||
|
||||
@@ -80,7 +80,7 @@ func (h *ConflictHandler) GetConflictsData(c *echo.Context) ([]ConflictDetailRes
|
||||
status = "unresolved"
|
||||
}
|
||||
|
||||
ctx := context.Background()
|
||||
ctx := c.Request().Context()
|
||||
|
||||
var conflicts interface{}
|
||||
var err error
|
||||
@@ -175,7 +175,7 @@ func (h *ConflictHandler) GetConflict(c *echo.Context) error {
|
||||
}
|
||||
|
||||
conflictUUID := pgtype.UUID{Bytes: [16]byte(conflictID), Valid: true}
|
||||
conflict, err := h.db.GetSyncConflict(context.Background(), conflictUUID)
|
||||
conflict, err := h.db.GetSyncConflict(c.Request().Context(), conflictUUID)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return echo.NewHTTPError(http.StatusNotFound, "conflict not found")
|
||||
@@ -187,7 +187,7 @@ func (h *ConflictHandler) GetConflict(c *echo.Context) error {
|
||||
return echo.NewHTTPError(http.StatusForbidden, "access denied")
|
||||
}
|
||||
|
||||
mediaItem, err := h.db.GetMediaItem(context.Background(), conflict.MediaItemID)
|
||||
mediaItem, err := h.db.GetMediaItem(c.Request().Context(), conflict.MediaItemID)
|
||||
if err != nil {
|
||||
return echo.NewHTTPError(http.StatusInternalServerError, "failed to get media item")
|
||||
}
|
||||
@@ -223,7 +223,7 @@ func (h *ConflictHandler) ResolveConflict(c *echo.Context) error {
|
||||
}
|
||||
|
||||
conflictUUID := pgtype.UUID{Bytes: [16]byte(conflictID), Valid: true}
|
||||
conflict, err := h.db.GetSyncConflict(context.Background(), conflictUUID)
|
||||
conflict, err := h.db.GetSyncConflict(c.Request().Context(), conflictUUID)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return echo.NewHTTPError(http.StatusNotFound, "conflict not found")
|
||||
@@ -263,13 +263,13 @@ func (h *ConflictHandler) ResolveConflict(c *echo.Context) error {
|
||||
}
|
||||
|
||||
if conflict.ConflictType == "progress" {
|
||||
if err := h.applyProgressResolution(conflict.MediaItemID, conflict.UserID, winnerSource, winnerData); err == nil {
|
||||
if err := h.applyProgressResolution(c.Request().Context(), conflict.MediaItemID, conflict.UserID, winnerSource, winnerData); err == nil {
|
||||
appliedTo["progress"] = true
|
||||
}
|
||||
}
|
||||
|
||||
if conflict.ConflictType == "annotation_highlight" || conflict.ConflictType == "annotation_bookmark" || conflict.ConflictType == "annotation_note" {
|
||||
if err := h.applyAnnotationResolution(conflict.MediaItemID, conflict.UserID, winnerData, conflict.ConflictType); err == nil {
|
||||
if err := h.applyAnnotationResolution(c.Request().Context(), conflict.MediaItemID, conflict.UserID, winnerData, conflict.ConflictType); err == nil {
|
||||
appliedTo["annotations"] = true
|
||||
}
|
||||
}
|
||||
@@ -283,7 +283,7 @@ func (h *ConflictHandler) ResolveConflict(c *echo.Context) error {
|
||||
|
||||
resolutionDataJSON, _ := json.Marshal(resolutionData)
|
||||
|
||||
_, err = h.db.ResolveSyncConflict(context.Background(), database.ResolveSyncConflictParams{
|
||||
_, err = h.db.ResolveSyncConflict(c.Request().Context(), database.ResolveSyncConflictParams{
|
||||
ID: conflictUUID,
|
||||
ResolutionStatus: pgtype.Text{String: "user_resolved", Valid: true},
|
||||
ResolutionData: resolutionDataJSON,
|
||||
@@ -321,7 +321,7 @@ func (h *ConflictHandler) ResolveConflict(c *echo.Context) error {
|
||||
// the service's LWW seed instead of being NULLed — that is the fix.
|
||||
// chapter_progress keeps its stored value instead of being overwritten
|
||||
// with the book-level percentage.
|
||||
func (h *ConflictHandler) applyProgressResolution(mediaItemID pgtype.UUID, userID pgtype.UUID, winnerSource string, data map[string]interface{}) error {
|
||||
func (h *ConflictHandler) applyProgressResolution(ctx context.Context, mediaItemID pgtype.UUID, userID pgtype.UUID, winnerSource string, data map[string]interface{}) error {
|
||||
req := wsync.SaveProgressRequest{
|
||||
MediaItemID: mediaItemID,
|
||||
UserID: userID,
|
||||
@@ -361,12 +361,11 @@ func (h *ConflictHandler) applyProgressResolution(mediaItemID pgtype.UUID, userI
|
||||
req.TotalPages = &tp
|
||||
}
|
||||
|
||||
_, err := h.progressSvc.SaveProgress(context.Background(), req)
|
||||
_, err := h.progressSvc.SaveProgress(ctx, req)
|
||||
return err
|
||||
}
|
||||
|
||||
func (h *ConflictHandler) applyAnnotationResolution(mediaItemID pgtype.UUID, userID pgtype.UUID, winnerData map[string]interface{}, conflictType string) error {
|
||||
ctx := context.Background()
|
||||
func (h *ConflictHandler) applyAnnotationResolution(ctx context.Context, mediaItemID pgtype.UUID, userID pgtype.UUID, winnerData map[string]interface{}, conflictType string) error {
|
||||
|
||||
dedupKey, _ := winnerData["dedup_key"].(string)
|
||||
if dedupKey == "" {
|
||||
@@ -530,7 +529,7 @@ func (h *ConflictHandler) DeleteConflict(c *echo.Context) error {
|
||||
}
|
||||
|
||||
conflictUUID := pgtype.UUID{Bytes: [16]byte(conflictID), Valid: true}
|
||||
conflict, err := h.db.GetSyncConflict(context.Background(), conflictUUID)
|
||||
conflict, err := h.db.GetSyncConflict(c.Request().Context(), conflictUUID)
|
||||
if err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return echo.NewHTTPError(http.StatusNotFound, "conflict not found")
|
||||
@@ -542,7 +541,7 @@ func (h *ConflictHandler) DeleteConflict(c *echo.Context) error {
|
||||
return echo.NewHTTPError(http.StatusForbidden, "access denied")
|
||||
}
|
||||
|
||||
if err := h.db.DeleteSyncConflict(context.Background(), conflictUUID); err != nil {
|
||||
if err := h.db.DeleteSyncConflict(c.Request().Context(), conflictUUID); err != nil {
|
||||
return echo.NewHTTPError(http.StatusInternalServerError, "failed to delete conflict")
|
||||
}
|
||||
|
||||
@@ -552,7 +551,7 @@ func (h *ConflictHandler) DeleteConflict(c *echo.Context) error {
|
||||
func (h *ConflictHandler) DismissAllResolved(c *echo.Context) error {
|
||||
user := c.Get("user").(database.Users)
|
||||
|
||||
conflicts, err := h.db.ListConflictsByUser(context.Background(), user.ID)
|
||||
conflicts, err := h.db.ListConflictsByUser(c.Request().Context(), user.ID)
|
||||
if err != nil {
|
||||
return echo.NewHTTPError(http.StatusInternalServerError, "failed to list conflicts")
|
||||
}
|
||||
@@ -560,7 +559,7 @@ func (h *ConflictHandler) DismissAllResolved(c *echo.Context) error {
|
||||
deleted := 0
|
||||
for _, conflict := range conflicts {
|
||||
if conflict.ResolutionStatus.String == "user_resolved" || conflict.ResolutionStatus.String == "bulk_resolved" {
|
||||
if err := h.db.DeleteSyncConflict(context.Background(), conflict.ID); err == nil {
|
||||
if err := h.db.DeleteSyncConflict(c.Request().Context(), conflict.ID); err == nil {
|
||||
deleted++
|
||||
}
|
||||
}
|
||||
@@ -620,7 +619,7 @@ func (h *ConflictHandler) BulkResolveConflicts(c *echo.Context) error {
|
||||
}
|
||||
|
||||
conflictUUID := pgtype.UUID{Bytes: [16]byte(conflictID), Valid: true}
|
||||
conflict, err := h.db.GetSyncConflict(context.Background(), conflictUUID)
|
||||
conflict, err := h.db.GetSyncConflict(c.Request().Context(), conflictUUID)
|
||||
if err != nil {
|
||||
results = append(results, ConflictResult{
|
||||
ConflictID: conflictIDStr,
|
||||
@@ -702,7 +701,7 @@ func (h *ConflictHandler) BulkResolveConflicts(c *echo.Context) error {
|
||||
continue
|
||||
}
|
||||
|
||||
if err := h.applyProgressResolution(conflict.MediaItemID, conflict.UserID, winningSource, winnerData); err != nil {
|
||||
if err := h.applyProgressResolution(c.Request().Context(), conflict.MediaItemID, conflict.UserID, winningSource, winnerData); err != nil {
|
||||
results = append(results, ConflictResult{
|
||||
ConflictID: conflictIDStr,
|
||||
Status: "error",
|
||||
@@ -719,7 +718,7 @@ func (h *ConflictHandler) BulkResolveConflicts(c *echo.Context) error {
|
||||
}
|
||||
resolutionDataJSON, _ := json.Marshal(resolutionData)
|
||||
|
||||
_, err = h.db.ResolveSyncConflict(context.Background(), database.ResolveSyncConflictParams{
|
||||
_, err = h.db.ResolveSyncConflict(c.Request().Context(), database.ResolveSyncConflictParams{
|
||||
ID: conflictUUID,
|
||||
ResolutionStatus: pgtype.Text{String: "bulk_resolved", Valid: true},
|
||||
ResolutionData: resolutionDataJSON,
|
||||
@@ -817,7 +816,7 @@ func (h *ConflictHandler) BulkDismissConflicts(c *echo.Context) error {
|
||||
}
|
||||
|
||||
conflictUUID := pgtype.UUID{Bytes: [16]byte(conflictID), Valid: true}
|
||||
conflict, err := h.db.GetSyncConflict(context.Background(), conflictUUID)
|
||||
conflict, err := h.db.GetSyncConflict(c.Request().Context(), conflictUUID)
|
||||
if err != nil {
|
||||
results = append(results, ConflictResult{
|
||||
ConflictID: conflictIDStr,
|
||||
@@ -838,7 +837,7 @@ func (h *ConflictHandler) BulkDismissConflicts(c *echo.Context) error {
|
||||
continue
|
||||
}
|
||||
|
||||
if err := h.db.DeleteSyncConflict(context.Background(), conflictUUID); err != nil {
|
||||
if err := h.db.DeleteSyncConflict(c.Request().Context(), conflictUUID); err != nil {
|
||||
results = append(results, ConflictResult{
|
||||
ConflictID: conflictIDStr,
|
||||
Status: "error",
|
||||
|
||||
@@ -433,6 +433,12 @@ func (h *DeviceHandler) DeleteDevice(c *echo.Context) error {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "access denied"})
|
||||
}
|
||||
|
||||
// Detach reading history first: the history FK to devices is NO
|
||||
// ACTION on purpose (rows are real captured stats); NULLing the
|
||||
// device association keeps the stats while freeing the device.
|
||||
if err := h.db.DetachReadingHistoryFromDevice(c.Request().Context(), pgDeviceID); err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to detach device history"})
|
||||
}
|
||||
if err := h.db.DeleteDevice(c.Request().Context(), pgDeviceID); err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to delete device"})
|
||||
}
|
||||
@@ -499,7 +505,10 @@ func (h *DeviceHandler) RegenerateDeviceToken(c *echo.Context) error {
|
||||
|
||||
// Handle HTMX requests - return HTML with page reload script
|
||||
if c.Request().Header.Get("HX-Request") == "true" {
|
||||
html := `<div class="text-green-500">Token regenerated! Reloading ... </div><script>window.location.reload();`
|
||||
// CSP: no inline scripts in partials; HX-Refresh makes htmx
|
||||
// reload the page natively (the old script tag was also unclosed).
|
||||
c.Response().Header().Set("HX-Refresh", "true")
|
||||
html := `<div class="text-green-500">Token regenerated! Reloading ... </div>`
|
||||
return c.HTML(http.StatusOK, html)
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
|
||||
@@ -1,12 +1,15 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/labstack/echo/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/rls"
|
||||
"bookhoard/internal/services"
|
||||
)
|
||||
|
||||
@@ -49,12 +52,17 @@ func (h *JobsHandler) CreateJob(c *echo.Context) error {
|
||||
// Add database to params
|
||||
req.Params["db"] = h.db
|
||||
|
||||
// RLS plan §6: the job's Context carries the requesting admin's
|
||||
// identity — governed reads inside the job would raise otherwise.
|
||||
userUUID, _ := uuid.Parse(c.Get("user_id").(string))
|
||||
|
||||
// Create job
|
||||
job := &services.Job{
|
||||
ID: uuid.New().String(),
|
||||
Type: jobType,
|
||||
Params: req.Params,
|
||||
Status: services.JobStatusPending,
|
||||
Context: rls.WithUser(context.Background(), pgtype.UUID{Bytes: userUUID, Valid: true}),
|
||||
}
|
||||
|
||||
// Enqueue job
|
||||
|
||||
@@ -1542,6 +1542,13 @@ func (mh *MediaHandler) UpdateMediaItem(c *echo.Context) error {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "media item not found"})
|
||||
}
|
||||
|
||||
// Edit segregation (plan D10, task 1.8): podcast episodes are
|
||||
// fetcher-owned (D9 — the sole writer); the generic edit box would
|
||||
// clobber feed metadata that the next refresh overwrites anyway.
|
||||
if existing.Source == "podcast_episode" {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "podcast episodes are managed by their feed and cannot be edited here"})
|
||||
}
|
||||
|
||||
coverPath := existing.CoverImagePath.String
|
||||
coverUploaded := false
|
||||
|
||||
@@ -1801,6 +1808,57 @@ func (mh *MediaHandler) RescanMediaItem(c *echo.Context) error {
|
||||
return c.JSON(http.StatusOK, item)
|
||||
}
|
||||
|
||||
// WriteMetadataToFile handles POST /api/media-items/:id/write-metadata
|
||||
// (admin only). Persists the item's CURRENT database metadata into the
|
||||
// artifact on disk — embedded OPF for EPUBs, a Calibre-style sidecar
|
||||
// for formats that can't carry metadata (2026-10-27 feature) — then
|
||||
// re-extracts from the file so the scan-derived columns agree with
|
||||
// what was written. metadata_overrides are NOT reset: after the write,
|
||||
// file and DB agree, and the overrides keep stale sidecars from
|
||||
// clobbering future rescans.
|
||||
func (mh *MediaHandler) WriteMetadataToFile(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
|
||||
if user.Role != "admin" {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "admin access required"})
|
||||
}
|
||||
|
||||
mediaID := c.Param("id")
|
||||
mediaUUID, err := uuid.Parse(mediaID)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid media item id"})
|
||||
}
|
||||
|
||||
if _, err := mh.db.GetMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true}); err != nil {
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "media item not found"})
|
||||
}
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
|
||||
}
|
||||
|
||||
writer := services.NewMetadataWriter(mh.db)
|
||||
result, err := writer.WriteMetadataToFile(c.Request().Context(), mediaUUID)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
|
||||
}
|
||||
|
||||
// Re-extract so the scan pipeline sees the file we just wrote. A
|
||||
// rescan failure doesn't undo the write — it surfaces separately.
|
||||
rescanStatus := "ok"
|
||||
scanner := services.NewMediaScanner(mh.db)
|
||||
if err := scanner.RescanMediaItem(c.Request().Context(), pgtype.UUID{Bytes: mediaUUID, Valid: true}, false); err != nil {
|
||||
rescanStatus = "error: " + err.Error()
|
||||
}
|
||||
defer scanner.Close()
|
||||
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"write": result,
|
||||
"rescan": rescanStatus,
|
||||
"metadata_target": string(result.Target),
|
||||
"fields_written": result.FieldsWritten,
|
||||
})
|
||||
}
|
||||
|
||||
// DeleteMediaItem handles DELETE /api/media-items/:id (admin only)
|
||||
func (mh *MediaHandler) DeleteMediaItem(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/opds"
|
||||
@@ -18,6 +17,24 @@ import (
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
// opdsVisibleLibraries is GetUserVisibleLibraries minus podcasts-type
|
||||
// libraries (plan D16): feeds are not OPDS inventory day one — they
|
||||
// surface via the /podcasts endpoints only.
|
||||
func opdsVisibleLibraries(ctx context.Context, q *database.Queries, userID pgtype.UUID) ([]database.GetUserVisibleLibrariesRow, error) {
|
||||
libs, err := q.GetUserVisibleLibraries(ctx, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := make([]database.GetUserVisibleLibrariesRow, 0, len(libs))
|
||||
for _, lib := range libs {
|
||||
if lib.TypeName == services.LibraryTypePodcasts {
|
||||
continue
|
||||
}
|
||||
out = append(out, lib)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
type OPDSHandler struct {
|
||||
db *database.Queries
|
||||
libraryService *services.LibraryService
|
||||
@@ -209,7 +226,7 @@ func (h *OPDSHandler) GetDeviceCatalog(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to get libraries"))
|
||||
}
|
||||
@@ -318,7 +335,7 @@ func (h *OPDSHandler) serveAllBooksCatalog(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to get libraries"))
|
||||
}
|
||||
@@ -570,7 +587,7 @@ func (h *OPDSHandler) GetLibraryCatalog(c *echo.Context) error {
|
||||
|
||||
// The library must be visible to the device owner.
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to get libraries"))
|
||||
}
|
||||
@@ -677,7 +694,7 @@ func (h *OPDSHandler) SearchDeviceCatalog(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.XML(http.StatusInternalServerError, opds.NewErrorFeed("Failed to get libraries"))
|
||||
}
|
||||
@@ -818,7 +835,7 @@ func (h *OPDSHandler) DownloadBook(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get libraries"})
|
||||
}
|
||||
@@ -974,7 +991,7 @@ func (h *OPDSHandler) GetCoverImage(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get libraries"})
|
||||
}
|
||||
@@ -1115,7 +1132,7 @@ func (h *OPDSHandler) ListFormats(c *echo.Context) error {
|
||||
|
||||
// Get user's visible libraries
|
||||
userID := device.UserID.Bytes
|
||||
libraries, err := h.db.GetUserVisibleLibraries(c.Request().Context(), pgtype.UUID{Bytes: userID, Valid: true})
|
||||
libraries, err := opdsVisibleLibraries(c.Request().Context(), h.db, pgtype.UUID{Bytes: userID, Valid: true})
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "failed to get libraries"})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,610 @@
|
||||
// Podcast handlers (plan §8): JSON API over the RLS-governed podcast
|
||||
// surfaces. Handlers parse requests and call services; visibility is
|
||||
// the engine's job (D20) — Go checks only explicit-argument
|
||||
// authorization (target-library rules, admin role) and subscription
|
||||
// lookups (podcast_subscriptions is ungoverned BY DESIGN: it IS the
|
||||
// read policy's lookup table).
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"sort"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/services"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
type PodcastHandler struct {
|
||||
db *database.Queries
|
||||
podcasts *services.PodcastService
|
||||
worker *services.PodcastDownloadWorker
|
||||
|
||||
// episodeListRenderer renders the refreshed episode-list partial
|
||||
// for htmx action responses (download/refresh/order/policy).
|
||||
// Injected by the router at wiring time — the view assembly lives
|
||||
// with the other templ view code (handlers must not import
|
||||
// templates: templates imports handlers for MediaDetail). When nil
|
||||
// (tests, API-only), htmx callers fall back to JSON + HX-Trigger.
|
||||
episodeListRenderer EpisodeListRenderer
|
||||
}
|
||||
|
||||
// EpisodeListRenderer renders the post-action episode list for an htmx
|
||||
// request. includePolicySelect additionally out-of-band-swaps the
|
||||
// auto-download select (Download All flips the caller's policy — the
|
||||
// dropdown must reflect it immediately, issue #3).
|
||||
type EpisodeListRenderer func(c *echo.Context, feedID string, includePolicySelect bool) error
|
||||
|
||||
// SetEpisodeListRenderer wires the renderer (router-only; product wiring).
|
||||
func (h *PodcastHandler) SetEpisodeListRenderer(r EpisodeListRenderer) {
|
||||
h.episodeListRenderer = r
|
||||
}
|
||||
|
||||
// hxEpisodeList answers an htmx action with the refreshed partial when
|
||||
// a renderer is wired; otherwise falls back to 204 + HX-Trigger.
|
||||
func (h *PodcastHandler) hxEpisodeList(c *echo.Context, feedID string, includePolicySelect bool) error {
|
||||
if h.episodeListRenderer != nil {
|
||||
return h.episodeListRenderer(c, feedID, includePolicySelect)
|
||||
}
|
||||
return hxAccepted(c, "")
|
||||
}
|
||||
|
||||
// flipSubscriptionToAll implements issue #3: Download All (button or
|
||||
// add-feed checkbox) sets the CALLER's subscription policy to 'all' —
|
||||
// per-user, never global (files are shared D6, policies are not). The
|
||||
// dropdown reflects it via the renderer's oob swap; the user can turn
|
||||
// it down afterwards.
|
||||
func (h *PodcastHandler) flipSubscriptionToAll(ctx context.Context, userID, feedID pgtype.UUID) {
|
||||
sub, err := h.db.GetPodcastSubscription(ctx, database.GetPodcastSubscriptionParams{
|
||||
UserID: userID, FeedID: feedID,
|
||||
})
|
||||
if err != nil || sub.AutoDownload == "all" {
|
||||
return // not subscribed (admin pass-through) or already all
|
||||
}
|
||||
_, _ = h.db.UpdatePodcastSubscription(ctx, database.UpdatePodcastSubscriptionParams{
|
||||
UserID: userID,
|
||||
FeedID: feedID,
|
||||
DisplayName: sub.DisplayName,
|
||||
AutoDownload: "all",
|
||||
AutoDownloadCount: pgtype.Int4{}, // all needs no count
|
||||
})
|
||||
}
|
||||
|
||||
func NewPodcastHandler(db *database.Queries, svc *services.PodcastService, worker *services.PodcastDownloadWorker) *PodcastHandler {
|
||||
return &PodcastHandler{db: db, podcasts: svc, worker: worker}
|
||||
}
|
||||
|
||||
// ---- request/response shapes ----
|
||||
|
||||
type AddPodcastFeedRequest struct {
|
||||
LibraryID string `json:"library_id" form:"library_id" validate:"required"`
|
||||
URL string `json:"url" form:"url" validate:"required"`
|
||||
DownloadAll bool `json:"download_all" form:"download_all"`
|
||||
}
|
||||
|
||||
type UpdatePodcastSubscriptionRequest struct {
|
||||
DisplayName *string `json:"display_name" form:"display_name"`
|
||||
AutoDownload *string `json:"auto_download" form:"auto_download" validate:"omitempty,oneof=off all newest"`
|
||||
AutoDownloadCount *int `json:"auto_download_count" form:"auto_download_count"`
|
||||
}
|
||||
|
||||
type DownloadEpisodesRequest struct {
|
||||
EpisodeIDs []string `json:"episode_ids"`
|
||||
}
|
||||
|
||||
// hxAccepted answers an htmx request with 204 + the podcast:changed
|
||||
// trigger — the episode-list partial re-fetches itself (its hx-trigger
|
||||
// listens for the event; polling covers download progress). JSON
|
||||
// callers keep the JSON body. HX-Redirect/HX-Trigger headers are the
|
||||
// sanctioned Go-handler response channels (CSP discipline).
|
||||
func hxAccepted(c *echo.Context, redirect string) error {
|
||||
if redirect != "" {
|
||||
c.Response().Header().Set("HX-Redirect", redirect)
|
||||
return c.NoContent(http.StatusNoContent)
|
||||
}
|
||||
c.Response().Header().Set("HX-Trigger", "podcast:changed")
|
||||
return c.NoContent(http.StatusNoContent)
|
||||
}
|
||||
|
||||
func isHXRequest(c *echo.Context) bool {
|
||||
return c.Request().Header.Get("HX-Request") == "true"
|
||||
}
|
||||
|
||||
// podcastErrorResponse maps service errors (incl. PodcastError statuses)
|
||||
// onto HTTP responses.
|
||||
func podcastErrorResponse(c *echo.Context, err error) error {
|
||||
if pe, ok := err.(*services.PodcastError); ok {
|
||||
return c.JSON(pe.Status, map[string]string{"error": pe.Message})
|
||||
}
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": err.Error()})
|
||||
}
|
||||
|
||||
// podcastFeedSummaries is the list payload: subscription + feed metadata
|
||||
// + the facet values the /podcasts page filters on (computed here —
|
||||
// feed counts are tens, not thousands; no facet SQL needed).
|
||||
type podcastFeedSummary struct {
|
||||
Feed
|
||||
SubscribedAt string `json:"subscribed_at"`
|
||||
EpisodeTotal int `json:"episode_total"`
|
||||
DownloadedN int `json:"episode_downloaded"`
|
||||
UndownloadedN int `json:"episode_undownloaded"`
|
||||
NewEpisodes bool `json:"new_episodes"`
|
||||
}
|
||||
|
||||
// AddFeed POST /api/podcasts/feeds
|
||||
func (h *PodcastHandler) AddFeed(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
var req AddPodcastFeedRequest
|
||||
if err := c.Bind(&req); err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
|
||||
}
|
||||
if err := c.Validate(&req); err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
|
||||
}
|
||||
libraryID, err := parseUUID(req.LibraryID)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid library id"})
|
||||
}
|
||||
|
||||
res, err := h.podcasts.AddFeed(c.Request().Context(), user.ID, user.Role == "admin", libraryID, req.URL)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if req.DownloadAll && res.Feed != nil {
|
||||
if _, err := h.worker.QueueFeedDownloads(c.Request().Context(), res.Feed.ID); err != nil {
|
||||
// subscription stands; download-all failure surfaces via the
|
||||
// episode list states.
|
||||
}
|
||||
// Issue #3: the checkbox means "everything, now AND going
|
||||
// forward" — reflect it in the caller's subscription policy.
|
||||
h.flipSubscriptionToAll(c.Request().Context(), user.ID, res.Feed.ID)
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
// Post-redirect-get: land on the new feed's page. Fetch errors
|
||||
// (subscription stands) surface via the flash query param.
|
||||
if res.FeedError() != "" {
|
||||
return hxAccepted(c, "/podcasts/"+uuidString(res.Feed.ID)+"?fetch_error=1")
|
||||
}
|
||||
return hxAccepted(c, "/podcasts/"+uuidString(res.Feed.ID))
|
||||
}
|
||||
return c.JSON(http.StatusCreated, res)
|
||||
}
|
||||
|
||||
// ListFeeds GET /api/podcasts/feeds — the caller's subscriptions.
|
||||
func (h *PodcastHandler) ListFeeds(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
ctx := c.Request().Context()
|
||||
|
||||
feeds, err := h.db.ListMyPodcastFeeds(ctx, user.ID)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
|
||||
out := make([]podcastFeedSummary, 0, len(feeds))
|
||||
for i := range feeds {
|
||||
row := &feeds[i]
|
||||
total, _ := h.db.CountPodcastEpisodesForFeed(ctx, row.ID)
|
||||
downloaded, _ := h.db.CountDownloadedEpisodes(ctx, row.ID)
|
||||
undownloaded, _ := h.db.CountUndownloadedEpisodes(ctx, row.ID)
|
||||
out = append(out, podcastFeedSummary{
|
||||
Feed: Feed{
|
||||
ID: uuidString(row.ID), LibraryID: uuidString(row.LibraryID),
|
||||
FeedUrl: row.FeedUrl, PodcastGuid: row.PodcastGuid.String,
|
||||
Title: row.Title, Description: row.Description.String,
|
||||
Host: row.Host.String, Publisher: row.Publisher.String,
|
||||
Language: row.Language.String, Tags: row.Tags,
|
||||
Explicit: row.Explicit, ArtworkPath: row.ArtworkPath.String,
|
||||
FeedType: row.FeedType.String, Medium: row.Medium.String,
|
||||
IsComplete: row.IsComplete, LastFetchedAt: timeString(row.LastFetchedAt),
|
||||
},
|
||||
SubscribedAt: timeString(row.SubscribedAt),
|
||||
EpisodeTotal: int(total),
|
||||
DownloadedN: int(downloaded),
|
||||
UndownloadedN: int(undownloaded),
|
||||
// "New episodes" (D18): the feed released something the user
|
||||
// has not pulled — undownloaded rows exist beyond the last
|
||||
// downloaded position.
|
||||
NewEpisodes: undownloaded > 0,
|
||||
})
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"feeds": out,
|
||||
"facets": facetValuesFor(out),
|
||||
})
|
||||
}
|
||||
|
||||
// GetFeed GET /api/podcasts/feeds/:id — detail: metadata, episodes with
|
||||
// download state, gap summary. RLS answers 404 for non-subscribers
|
||||
// (existence leak) — no Go-level visibility plumbing.
|
||||
func (h *PodcastHandler) GetFeed(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
ctx := c.Request().Context()
|
||||
|
||||
feed, err := h.db.GetPodcastFeed(ctx, feedID)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "feed not found"})
|
||||
}
|
||||
sub, subFound := h.db.GetPodcastSubscription(ctx, database.GetPodcastSubscriptionParams{UserID: user.ID, FeedID: feedID})
|
||||
eps, err := h.db.ListPodcastEpisodes(ctx, feedID)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
|
||||
episodeViews := make([]PodcastEpisodeView, 0, len(eps))
|
||||
for i := range eps {
|
||||
episodeViews = append(episodeViews, episodeView(&eps[i]))
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"feed": feedWithSubscription(&feed, sub, subFound == nil),
|
||||
"episodes": episodeViews,
|
||||
"gap_count": countGaps(episodeViews),
|
||||
"undownloaded": len(episodeViews) - countDownloaded(episodeViews),
|
||||
"has_library_access": true, // RLS already gated the row; kept for client convenience
|
||||
})
|
||||
}
|
||||
|
||||
// UpdateSubscription PUT /api/podcasts/feeds/:id/subscription — the
|
||||
// only user-writable podcast state (D9): display name + auto-download.
|
||||
func (h *PodcastHandler) UpdateSubscription(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
var req UpdatePodcastSubscriptionRequest
|
||||
if err := c.Bind(&req); err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
|
||||
}
|
||||
|
||||
ctx := c.Request().Context()
|
||||
sub, err := h.db.GetPodcastSubscription(ctx, database.GetPodcastSubscriptionParams{UserID: user.ID, FeedID: feedID})
|
||||
if err != nil {
|
||||
// Not subscribed (or feed invisible): 404 semantics.
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "not subscribed"})
|
||||
}
|
||||
|
||||
displayName := sub.DisplayName
|
||||
if req.DisplayName != nil {
|
||||
displayName = pgtypeText(*req.DisplayName)
|
||||
}
|
||||
autoDownload := sub.AutoDownload
|
||||
if req.AutoDownload != nil {
|
||||
autoDownload = *req.AutoDownload
|
||||
}
|
||||
count := sub.AutoDownloadCount
|
||||
if req.AutoDownloadCount != nil {
|
||||
if *req.AutoDownloadCount < 0 || *req.AutoDownloadCount > 100 {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "auto_download_count out of range"})
|
||||
}
|
||||
count = pgtype.Int4{Int32: int32(*req.AutoDownloadCount), Valid: *req.AutoDownloadCount > 0}
|
||||
}
|
||||
if autoDownload == "newest" && !count.Valid {
|
||||
count = pgtype.Int4{Int32: 3, Valid: true} // sane default for newest-N
|
||||
}
|
||||
|
||||
updated, err := h.db.UpdatePodcastSubscription(ctx, database.UpdatePodcastSubscriptionParams{
|
||||
UserID: user.ID,
|
||||
FeedID: feedID,
|
||||
DisplayName: displayName,
|
||||
AutoDownload: autoDownload,
|
||||
AutoDownloadCount: count,
|
||||
})
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
// Policy changes feed the auto-download union on the next poll.
|
||||
_, _ = h.worker.ApplyAutoPolicies(ctx, feedID)
|
||||
return h.hxEpisodeList(c, feedID.String(), true)
|
||||
}
|
||||
return c.JSON(http.StatusOK, updated)
|
||||
}
|
||||
|
||||
// DeleteSubscription DELETE /api/podcasts/feeds/:id/subscription —
|
||||
// unsubscribe NEVER deletes files/episodes (D6; last unsubscribe
|
||||
// retains them so re-subscribing must not re-download).
|
||||
func (h *PodcastHandler) DeleteSubscription(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
rows, err := h.db.DeletePodcastSubscription(c.Request().Context(), database.DeletePodcastSubscriptionParams{UserID: user.ID, FeedID: feedID})
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if rows == 0 {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "not subscribed"})
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
return hxAccepted(c, "/podcasts")
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]string{"message": "unsubscribed (files retained)"})
|
||||
}
|
||||
|
||||
// RefreshFeed POST /api/podcasts/feeds/:id/refresh — manual poll.
|
||||
func (h *PodcastHandler) RefreshFeed(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
_ = user
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
if err := h.podcasts.RefreshFeed(c.Request().Context(), feedID); err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
return h.hxEpisodeList(c, feedID.String(), false)
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]string{"message": "refreshed"})
|
||||
}
|
||||
|
||||
// DownloadEpisodes POST /api/podcasts/feeds/:id/episodes/:eid/download
|
||||
// (+ bulk via episode_ids[]; Download All = all non-downloaded). Retry
|
||||
// is the same endpoint (idempotent state machine, plan §7).
|
||||
func (h *PodcastHandler) DownloadEpisodes(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
_ = user
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
ctx := c.Request().Context()
|
||||
|
||||
// The feed must be in the caller's scope; RLS answers 404.
|
||||
if _, err := h.db.GetPodcastFeed(ctx, feedID); err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "feed not found"})
|
||||
}
|
||||
|
||||
// Bulk form?
|
||||
var bulk DownloadEpisodesRequest
|
||||
_ = c.Bind(&bulk)
|
||||
if len(bulk.EpisodeIDs) > 0 {
|
||||
queued := 0
|
||||
for _, raw := range bulk.EpisodeIDs {
|
||||
epID, err := parseUUID(raw)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if err := h.worker.QueueEpisode(ctx, epID, feedID); err == nil {
|
||||
queued++
|
||||
}
|
||||
}
|
||||
return c.JSON(http.StatusAccepted, map[string]interface{}{"queued": queued})
|
||||
}
|
||||
|
||||
// Single-episode path (:eid).
|
||||
eidRaw := c.Param("eid")
|
||||
if eidRaw == "all" {
|
||||
// Download All affordance: exactly the non-downloaded set (pin).
|
||||
queued, err := h.worker.QueueFeedDownloads(ctx, feedID)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
// Download All flips the caller's policy to all (issue #3)
|
||||
// — the oob select keeps the dropdown honest.
|
||||
h.flipSubscriptionToAll(ctx, user.ID, feedID)
|
||||
return h.hxEpisodeList(c, feedID.String(), true)
|
||||
}
|
||||
return c.JSON(http.StatusAccepted, map[string]interface{}{"queued": queued})
|
||||
}
|
||||
epID, err := parseUUID(eidRaw)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid episode id"})
|
||||
}
|
||||
if err := h.worker.QueueEpisode(ctx, epID, feedID); err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
return h.hxEpisodeList(c, feedID.String(), false)
|
||||
}
|
||||
return c.JSON(http.StatusAccepted, map[string]string{"message": "queued"})
|
||||
}
|
||||
|
||||
// Search GET /api/podcasts/search?q= — subscription-scoped feed +
|
||||
// episode search (D14): covers un-downloaded and historical episodes
|
||||
// (rows are never deleted when a feed shrinks).
|
||||
func (h *PodcastHandler) Search(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
q := strings.TrimSpace(c.QueryParam("q"))
|
||||
ctx := c.Request().Context()
|
||||
|
||||
feeds, err := h.db.SearchMyPodcastFeeds(ctx, database.SearchMyPodcastFeedsParams{UserID: user.ID, SearchQuery: q})
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
eps, err := h.db.SearchMyPodcastEpisodes(ctx, q)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
episodeViews := make([]PodcastEpisodeView, 0, len(eps))
|
||||
for i := range eps {
|
||||
episodeViews = append(episodeViews, episodeView(&eps[i]))
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{"feeds": feeds, "episodes": episodeViews})
|
||||
}
|
||||
|
||||
// DeleteFeed DELETE /api/podcasts/feeds/:id — admin-only removal;
|
||||
// optional ?files=false keeps the episode files on disk.
|
||||
func (h *PodcastHandler) DeleteFeed(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
if user.Role != "admin" {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "admin only"})
|
||||
}
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
ctx := c.Request().Context()
|
||||
if _, err := h.db.GetPodcastFeed(ctx, feedID); err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "feed not found"})
|
||||
}
|
||||
|
||||
keepFiles := c.QueryParam("files") == "false"
|
||||
if !keepFiles {
|
||||
// File deletion is best-effort before the row cascade; episodes'
|
||||
// file_path values are relative to the download root.
|
||||
if eps, err := h.db.ListPodcastEpisodes(ctx, feedID); err == nil {
|
||||
if feed, err := h.db.GetPodcastFeed(ctx, feedID); err == nil {
|
||||
h.podcasts.DeleteFeedFiles(ctx, &feed, eps)
|
||||
}
|
||||
}
|
||||
}
|
||||
if rows, err := h.db.DeletePodcastFeed(ctx, feedID); err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
} else if rows == 0 {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "feed not found"})
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
return hxAccepted(c, "/podcasts")
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]string{"message": "feed removed"})
|
||||
}
|
||||
|
||||
// ---- Sequence (plan §8 / Phase 2: D5 order override, D7 chaining) ----
|
||||
|
||||
// podcastOrderSettingKey is the per-user, per-feed order override key
|
||||
// in the reader_settings KV (D5).
|
||||
func podcastOrderSettingKey(feedID string) string {
|
||||
return "podcast_order:" + feedID
|
||||
}
|
||||
|
||||
// podcastOrderValid: native (feed position) | oldest | newest.
|
||||
func podcastOrderValid(o string) bool {
|
||||
return o == "native" || o == "oldest" || o == "newest"
|
||||
}
|
||||
|
||||
// GetFeedOrder reads the caller's effective order for a feed
|
||||
// (default: native — feed-native order is the D5 default).
|
||||
func (h *PodcastHandler) GetFeedOrder(c *echo.Context, feedID string) string {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
row, err := h.db.GetUserSettingByKey(c.Request().Context(), database.GetUserSettingByKeyParams{
|
||||
UserID: user.ID,
|
||||
SettingKey: podcastOrderSettingKey(feedID),
|
||||
})
|
||||
if err != nil {
|
||||
return "native"
|
||||
}
|
||||
var v struct {
|
||||
Order string `json:"order"`
|
||||
}
|
||||
if err := json.Unmarshal(row, &v); err != nil || !podcastOrderValid(v.Order) {
|
||||
return "native"
|
||||
}
|
||||
return v.Order
|
||||
}
|
||||
|
||||
// SetFeedOrder PUT /api/podcasts/feeds/:id/order — per-user override.
|
||||
// Deleting the override (order=native) removes the KV row so the
|
||||
// default and the explicit choice are indistinguishable.
|
||||
func (h *PodcastHandler) SetFeedOrder(c *echo.Context) error {
|
||||
user := MustGetAuthenticatedUser(c)
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
var req struct {
|
||||
Order string `json:"order" form:"order" validate:"required,oneof=native oldest newest"`
|
||||
}
|
||||
if err := c.Bind(&req); err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid request"})
|
||||
}
|
||||
if err := c.Validate(&req); err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": err.Error()})
|
||||
}
|
||||
ctx := c.Request().Context()
|
||||
if _, err := h.db.GetPodcastFeed(ctx, feedID); err != nil {
|
||||
return c.JSON(http.StatusNotFound, map[string]string{"error": "feed not found"})
|
||||
}
|
||||
key := podcastOrderSettingKey(feedID.String())
|
||||
if req.Order == "native" {
|
||||
_ = h.db.DeleteUserSettingByKey(ctx, database.DeleteUserSettingByKeyParams{
|
||||
UserID: user.ID, SettingKey: key,
|
||||
})
|
||||
} else {
|
||||
payload, _ := json.Marshal(map[string]string{"order": req.Order})
|
||||
if err := h.db.UpsertUserSettingByKey(ctx, database.UpsertUserSettingByKeyParams{
|
||||
UserID: user.ID, SettingKey: key, SettingValue: payload,
|
||||
}); err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
}
|
||||
if isHXRequest(c) {
|
||||
return h.hxEpisodeList(c, feedID.String(), false)
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]string{"order": req.Order})
|
||||
}
|
||||
|
||||
// GetFeedSequence GET /api/podcasts/feeds/:id/sequence — the effective
|
||||
// listening order (override applied, feed-native fallback), DOWNLOADED
|
||||
// episodes only, plus the gap list (D7/D8). This payload powers both
|
||||
// the web player's chain mode and (later, §15) the media3 queue.
|
||||
func (h *PodcastHandler) GetFeedSequence(c *echo.Context) error {
|
||||
feedID, err := parseUUID(c.Param("id"))
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "invalid feed id"})
|
||||
}
|
||||
ctx := c.Request().Context()
|
||||
|
||||
order := h.GetFeedOrder(c, feedID.String())
|
||||
eps, err := h.db.ListPodcastEpisodes(ctx, feedID)
|
||||
if err != nil {
|
||||
return podcastErrorResponse(c, err)
|
||||
}
|
||||
|
||||
views := make([]PodcastEpisodeView, 0, len(eps))
|
||||
for i := range eps {
|
||||
views = append(views, episodeView(&eps[i]))
|
||||
}
|
||||
sortSlice(views, func(a, b PodcastEpisodeView) bool {
|
||||
switch order {
|
||||
case "oldest":
|
||||
return episodeSortKey(a) < episodeSortKey(b)
|
||||
case "newest":
|
||||
return episodeSortKey(a) > episodeSortKey(b)
|
||||
default: // native: feed position (fetch-maintained)
|
||||
if a.FeedPosition != b.FeedPosition {
|
||||
return a.FeedPosition < b.FeedPosition
|
||||
}
|
||||
return a.Date < b.Date
|
||||
}
|
||||
})
|
||||
|
||||
sequence := make([]PodcastEpisodeView, 0, len(views))
|
||||
gaps := make([]PodcastEpisodeView, 0)
|
||||
for _, v := range views {
|
||||
if v.Downloaded {
|
||||
sequence = append(sequence, v)
|
||||
} else {
|
||||
gaps = append(gaps, v)
|
||||
}
|
||||
}
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"feed_id": feedID.String(),
|
||||
"order": order,
|
||||
"episodes": sequence,
|
||||
"gaps": gaps,
|
||||
})
|
||||
}
|
||||
|
||||
// episodeSortKey: chronological key for oldest/newest (date, then
|
||||
// native position as the same-day tiebreaker).
|
||||
func episodeSortKey(v PodcastEpisodeView) string {
|
||||
return v.Date + fmt.Sprintf("%06d", v.FeedPosition)
|
||||
}
|
||||
|
||||
// sortSlice: sort.Slice with a controller-friendly wrapper.
|
||||
func sortSlice[T any](s []T, less func(a, b T) bool) {
|
||||
sort.Slice(s, func(i, j int) bool { return less(s[i], s[j]) })
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
// Podcast view types + mappers shared by the JSON API and the SSR
|
||||
// partials (plan §8/§9). One shape per concept so the htmx partials and
|
||||
// the API can never drift.
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// Feed is the API-facing feed shape (subscription fields attached when
|
||||
// the caller has one).
|
||||
type Feed struct {
|
||||
ID string `json:"id"`
|
||||
LibraryID string `json:"library_id"`
|
||||
FeedUrl string `json:"feed_url"`
|
||||
PodcastGuid string `json:"podcast_guid,omitempty"`
|
||||
Title string `json:"title"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Host string `json:"host,omitempty"`
|
||||
Publisher string `json:"publisher,omitempty"`
|
||||
Language string `json:"language,omitempty"`
|
||||
Tags []string `json:"tags,omitempty"`
|
||||
Explicit bool `json:"explicit"`
|
||||
ArtworkPath string `json:"artwork_path,omitempty"`
|
||||
FeedType string `json:"feed_type,omitempty"`
|
||||
Medium string `json:"medium,omitempty"`
|
||||
IsComplete bool `json:"is_complete"`
|
||||
LastFetchedAt string `json:"last_fetched_at,omitempty"`
|
||||
|
||||
// Subscription-owned (rendered when present).
|
||||
DisplayName string `json:"display_name,omitempty"`
|
||||
AutoDownload string `json:"auto_download,omitempty"`
|
||||
AutoDownloadCount int `json:"auto_download_count,omitempty"`
|
||||
}
|
||||
|
||||
// PodcastEpisodeView is the episode-list row: media facts + the
|
||||
// download lifecycle the list renders inline (D8/D19).
|
||||
type PodcastEpisodeView struct {
|
||||
ID string `json:"id"`
|
||||
FeedID string `json:"feed_id"`
|
||||
Title string `json:"title"`
|
||||
Subtitle string `json:"subtitle,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Authors []string `json:"authors,omitempty"`
|
||||
Hosts []string `json:"hosts,omitempty"`
|
||||
Guests []string `json:"guests,omitempty"`
|
||||
Date string `json:"date_published,omitempty"`
|
||||
FeedPosition int `json:"feed_position"`
|
||||
Season int `json:"season_number,omitempty"`
|
||||
Episode int `json:"episode_number,omitempty"`
|
||||
Duration int `json:"declared_duration_seconds,omitempty"`
|
||||
MimeType string `json:"mime_type,omitempty"`
|
||||
CoverImage string `json:"cover_image_path,omitempty"`
|
||||
EnclosureURL string `json:"enclosure_url,omitempty"`
|
||||
|
||||
// Download lifecycle.
|
||||
DownloadState string `json:"download_state"`
|
||||
DownloadError string `json:"download_error,omitempty"`
|
||||
DownloadAttempts int `json:"download_attempts"`
|
||||
Downloaded bool `json:"downloaded"`
|
||||
FileSize int64 `json:"file_size,omitempty"`
|
||||
|
||||
// Progress (from the shared reading_progress machinery).
|
||||
ProgressPercent float64 `json:"progress_percent,omitempty"`
|
||||
}
|
||||
|
||||
// episodeView maps a media_items row (podcast episode) onto the view.
|
||||
func episodeView(ep *database.MediaItems) PodcastEpisodeView {
|
||||
v := PodcastEpisodeView{
|
||||
ID: uuidString(ep.ID),
|
||||
FeedID: uuidString(ep.SourceFeedID),
|
||||
Title: ep.Title,
|
||||
Subtitle: ep.Subtitle.String,
|
||||
Description: ep.Description.String,
|
||||
Authors: ep.Authors,
|
||||
Hosts: ep.Hosts,
|
||||
Guests: ep.Guests,
|
||||
FeedPosition: int(ep.FeedPosition.Int32),
|
||||
Season: int(ep.SeasonNumber.Int32),
|
||||
Episode: int(ep.EpisodeNumber.Int32),
|
||||
Duration: int(ep.DeclaredDurationSeconds.Int32),
|
||||
MimeType: ep.MimeType.String,
|
||||
CoverImage: ep.CoverImagePath.String,
|
||||
EnclosureURL: ep.EnclosureUrl.String,
|
||||
|
||||
DownloadState: ep.DownloadState,
|
||||
DownloadError: ep.DownloadError.String,
|
||||
DownloadAttempts: int(ep.DownloadAttempts),
|
||||
Downloaded: ep.DownloadState == "downloaded",
|
||||
FileSize: ep.FileSize.Int64,
|
||||
}
|
||||
if ep.DatePublished.Valid {
|
||||
v.Date = ep.DatePublished.Time.Format("2006-01-02")
|
||||
}
|
||||
if !ep.FileSize.Valid {
|
||||
v.FileSize = 0
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// feedWithSubscription merges the feed row + the caller's subscription
|
||||
// (empty sub → bare feed).
|
||||
func feedWithSubscription(feed *database.PodcastFeeds, sub database.PodcastSubscriptions, subFound bool) Feed {
|
||||
f := Feed{
|
||||
ID: uuidString(feed.ID),
|
||||
LibraryID: uuidString(feed.LibraryID),
|
||||
FeedUrl: feed.FeedUrl,
|
||||
PodcastGuid: feed.PodcastGuid.String,
|
||||
Title: feed.Title,
|
||||
Description: feed.Description.String,
|
||||
Host: feed.Host.String,
|
||||
Publisher: feed.Publisher.String,
|
||||
Language: feed.Language.String,
|
||||
Tags: feed.Tags,
|
||||
Explicit: feed.Explicit,
|
||||
ArtworkPath: feed.ArtworkPath.String,
|
||||
FeedType: feed.FeedType.String,
|
||||
Medium: feed.Medium.String,
|
||||
IsComplete: feed.IsComplete,
|
||||
LastFetchedAt: timeString(feed.LastFetchedAt),
|
||||
}
|
||||
if subFound {
|
||||
f.DisplayName = sub.DisplayName.String
|
||||
f.AutoDownload = sub.AutoDownload
|
||||
f.AutoDownloadCount = int(sub.AutoDownloadCount.Int32)
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
// countGaps counts un-downloaded episodes that have a downloaded
|
||||
// episode AFTER them in list order — the D8 "middle episodes missing"
|
||||
// definition that drives the gap banner.
|
||||
func countGaps(eps []PodcastEpisodeView) int {
|
||||
seenDownloaded := false
|
||||
gaps := 0
|
||||
for i := len(eps) - 1; i >= 0; i-- {
|
||||
if eps[i].Downloaded {
|
||||
seenDownloaded = true
|
||||
} else if seenDownloaded {
|
||||
gaps++
|
||||
}
|
||||
}
|
||||
return gaps
|
||||
}
|
||||
|
||||
func countDownloaded(eps []PodcastEpisodeView) int {
|
||||
n := 0
|
||||
for _, e := range eps {
|
||||
if e.Downloaded {
|
||||
n++
|
||||
}
|
||||
}
|
||||
return n
|
||||
}
|
||||
|
||||
// facetValuesFor derives the /podcasts facet bar values (D18) from the
|
||||
// subscription list — feed counts are small; no facet SQL.
|
||||
func facetValuesFor(feeds []podcastFeedSummary) map[string][]string {
|
||||
hosts := map[string]bool{}
|
||||
tags := map[string]bool{}
|
||||
langs := map[string]bool{}
|
||||
media := map[string]bool{}
|
||||
for _, f := range feeds {
|
||||
if f.Host != "" {
|
||||
hosts[f.Host] = true
|
||||
}
|
||||
for _, t := range f.Tags {
|
||||
tags[t] = true
|
||||
}
|
||||
if f.Language != "" {
|
||||
langs[f.Language] = true
|
||||
}
|
||||
if f.Medium != "" {
|
||||
media[f.Medium] = true
|
||||
}
|
||||
}
|
||||
return map[string][]string{
|
||||
"hosts": sortedKeys(hosts),
|
||||
"tags": sortedKeys(tags),
|
||||
"language": sortedKeys(langs),
|
||||
"medium": sortedKeys(media),
|
||||
}
|
||||
}
|
||||
|
||||
func sortedKeys(set map[string]bool) []string {
|
||||
out := make([]string, 0, len(set))
|
||||
for k := range set {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
// uuidString renders a pgtype.UUID (empty string when NULL).
|
||||
func uuidString(id pgtype.UUID) string {
|
||||
if !id.Valid {
|
||||
return ""
|
||||
}
|
||||
return uuid.UUID(id.Bytes).String()
|
||||
}
|
||||
|
||||
// timeString renders a timestamp (RFC3339; empty when NULL).
|
||||
func timeString(t pgtype.Timestamptz) string {
|
||||
if !t.Valid {
|
||||
return ""
|
||||
}
|
||||
return t.Time.Format(time.RFC3339)
|
||||
}
|
||||
|
||||
// pgtypeText (handlers-local) — service has its own.
|
||||
func pgtypeText(s string) pgtype.Text {
|
||||
if s == "" {
|
||||
return pgtype.Text{}
|
||||
}
|
||||
return pgtype.Text{String: s, Valid: true}
|
||||
}
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"bookhoard/internal/database"
|
||||
wsync "bookhoard/internal/sync"
|
||||
"bookhoard/internal/utils"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"strconv"
|
||||
@@ -46,7 +45,7 @@ func (h *Handler) GetProgressHistory(c *echo.Context) error {
|
||||
}
|
||||
}
|
||||
|
||||
history, err := h.db.GetReadingHistory(context.Background(), database.GetReadingHistoryParams{
|
||||
history, err := h.db.GetReadingHistory(c.Request().Context(), database.GetReadingHistoryParams{
|
||||
UserID: pgtype.UUID{Bytes: user.ID.Bytes, Valid: true},
|
||||
MediaItemID: pgtype.UUID{Bytes: mediaItemID, Valid: true},
|
||||
Limit: int32(limit),
|
||||
|
||||
@@ -8,6 +8,8 @@ import (
|
||||
"net/http"
|
||||
|
||||
"github.com/google/uuid"
|
||||
|
||||
"bookhoard/internal/rls"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
@@ -83,6 +85,11 @@ func (h *Handler) ScanLibrary(c *echo.Context) error {
|
||||
for _, folder := range libraryFolders {
|
||||
if folder.FolderPath == req.FolderPath {
|
||||
registered = true
|
||||
// Podcast download roots are feed-managed (invariant
|
||||
// 4): never scanner-ingestible, even scoped.
|
||||
if folder.Purpose == services.FolderPurposePodcast {
|
||||
return c.JSON(http.StatusBadRequest, map[string]string{"error": "podcast download roots are feed-managed and cannot be scanned"})
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
@@ -127,6 +134,11 @@ func (h *Handler) ScanLibrary(c *echo.Context) error {
|
||||
|
||||
jobID := uuid.New().String()
|
||||
|
||||
// RLS plan §6: the scan job's media reads run under the triggering
|
||||
// admin's identity. h.ctx is a handler-lifetime Background context
|
||||
// (outlives the request), so the identity is stamped explicitly —
|
||||
// a bare context makes governed reads raise (placeholder reverting
|
||||
// to '' on reused connections, or 42704 on fresh ones).
|
||||
job := &services.Job{
|
||||
ID: jobID,
|
||||
Type: services.JobTypeScan,
|
||||
@@ -140,7 +152,7 @@ func (h *Handler) ScanLibrary(c *echo.Context) error {
|
||||
"trigger": "admin",
|
||||
},
|
||||
Status: services.JobStatusPending,
|
||||
Context: h.ctx,
|
||||
Context: rls.WithUser(h.ctx, pgtype.UUID{Bytes: userUUID, Valid: true}),
|
||||
}
|
||||
|
||||
if err := h.worker.EnqueueJob(job); err != nil {
|
||||
|
||||
@@ -64,7 +64,20 @@ func (h *SeriesHandler) GetSeries(c *echo.Context) error {
|
||||
return c.JSON(http.StatusForbidden, map[string]string{"error": "library not accessible"})
|
||||
}
|
||||
|
||||
seriesList, total, err := h.seriesService.GetSeriesPage(c.Request().Context(), libUUID, visibleIDs, limit, offset)
|
||||
// Sort/filter params (same contract as the SSR /series page):
|
||||
// unknown values resolve to defaults inside the service.
|
||||
opts := services.SeriesPageOptions{
|
||||
Sort: c.QueryParam("sort"),
|
||||
Dir: c.QueryParam("dir"),
|
||||
Query: c.QueryParam("q"),
|
||||
Author: c.QueryParam("author"),
|
||||
Status: c.QueryParam("status"),
|
||||
}
|
||||
if u, ok := c.Get("user").(database.Users); ok {
|
||||
opts.ViewerUserID = u.ID
|
||||
}
|
||||
|
||||
seriesList, total, err := h.seriesService.GetSeriesPage(c.Request().Context(), libUUID, visibleIDs, limit, offset, opts)
|
||||
if err != nil {
|
||||
return c.JSON(http.StatusInternalServerError, map[string]string{"error": "Failed to load series"})
|
||||
}
|
||||
@@ -76,6 +89,8 @@ func (h *SeriesHandler) GetSeries(c *echo.Context) error {
|
||||
TotalInSeries int `json:"total_in_series"`
|
||||
CoverPaths []string `json:"cover_paths"`
|
||||
LastEntryAt string `json:"last_entry_at"`
|
||||
ReadCount int64 `json:"read_count"`
|
||||
LastReadAt string `json:"last_read_at"`
|
||||
}
|
||||
|
||||
response := make([]SeriesResponse, 0, len(seriesList))
|
||||
@@ -87,6 +102,8 @@ func (h *SeriesHandler) GetSeries(c *echo.Context) error {
|
||||
TotalInSeries: s.TotalInSeries,
|
||||
CoverPaths: s.CoverPaths,
|
||||
LastEntryAt: s.LastEntryAt,
|
||||
ReadCount: s.ReadCount,
|
||||
LastReadAt: s.LastReadAt,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -150,7 +167,7 @@ func (h *SeriesHandler) GetSeriesBooks(c *echo.Context) error {
|
||||
})
|
||||
}
|
||||
|
||||
func GetSeriesCardsData(ctx context.Context, db *database.Queries, libraryID pgtype.UUID, visibleLibraryIDs []pgtype.UUID, limit, offset int) ([]services.SeriesInfo, int, error) {
|
||||
func GetSeriesCardsData(ctx context.Context, db *database.Queries, libraryID pgtype.UUID, visibleLibraryIDs []pgtype.UUID, limit, offset int, opts services.SeriesPageOptions) ([]services.SeriesInfo, int, error) {
|
||||
svc := services.NewSeriesService(db)
|
||||
return svc.GetSeriesPage(ctx, libraryID, visibleLibraryIDs, limit, offset)
|
||||
return svc.GetSeriesPage(ctx, libraryID, visibleLibraryIDs, limit, offset, opts)
|
||||
}
|
||||
@@ -489,12 +489,18 @@ func (h *SidecarHandler) UpdateSystemConfiguration(c *echo.Context) error {
|
||||
defaultTimezone = tz
|
||||
}
|
||||
|
||||
// Render success message with updated form
|
||||
// Render success message with updated form. NOTE: this Go-source
|
||||
// copy is the htmx-swapped twin of the templ form in
|
||||
// templates/admin_settings.templ (handlers cannot import templates:
|
||||
// templates/utils.go imports handlers types — cycle). It MUST keep
|
||||
// the twin's functional attributes — id, hx-*, data-settings-save,
|
||||
// hx-disabled-elt — or save feedback silently dies after the first
|
||||
// swap. e2e/settings-feedback.spec.ts saves twice to pin this.
|
||||
return c.HTML(http.StatusOK, fmt.Sprintf(`
|
||||
<div class="mb-4 p-4 rounded-lg" style="background-color: var(--bg-secondary); border: 1px solid var(--accent);">
|
||||
<p style="color: var(--text-success);">✅ Settings saved successfully!</p>
|
||||
</div>
|
||||
<form id="settings-form" hx-put="/api/system/config" hx-target="#settings-form" hx-swap="outerHTML">
|
||||
<form id="settings-form" data-settings-save="settings" hx-put="/api/system/config" hx-target="#settings-form" hx-swap="outerHTML">
|
||||
<div class="card p-6 rounded-lg border" style="background-color: var(--bg-secondary); border-color: var(--border);">
|
||||
<h3 class="text-xl font-semibold mb-6" style="color: var(--text-primary)">Base URL</h3>
|
||||
|
||||
@@ -513,7 +519,7 @@ func (h *SidecarHandler) UpdateSystemConfiguration(c *echo.Context) error {
|
||||
</div>
|
||||
|
||||
<div class="mt-6 flex justify-end">
|
||||
<button type="submit" class="btn-primary px-6 py-2 rounded-lg font-medium">
|
||||
<button type="submit" class="btn-primary px-6 py-2 rounded-lg font-medium" hx-disabled-elt="this">
|
||||
Save Settings
|
||||
</button>
|
||||
</div>
|
||||
@@ -551,7 +557,7 @@ func (h *SidecarHandler) UpdateSystemConfiguration(c *echo.Context) error {
|
||||
<p class="text-sm mt-1" style="color: var(--text-secondary)">Default timezone for users who haven't set their own.</p>
|
||||
</div>
|
||||
<div class="mt-6 flex justify-end">
|
||||
<button type="submit" class="btn-primary px-6 py-2 rounded-lg font-medium">
|
||||
<button type="submit" class="btn-primary px-6 py-2 rounded-lg font-medium" hx-disabled-elt="this">
|
||||
Save Settings
|
||||
</button>
|
||||
</div>
|
||||
|
||||
@@ -11,6 +11,8 @@ import (
|
||||
|
||||
jwt "github.com/golang-jwt/jwt/v5"
|
||||
"github.com/google/uuid"
|
||||
|
||||
"bookhoard/internal/rls"
|
||||
"github.com/gorilla/websocket"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
@@ -201,14 +203,17 @@ func (h *WSHandler) getInitialState(userID string) map[string]interface{} {
|
||||
|
||||
pgUserID := pgtype.UUID{Bytes: userUUID, Valid: true}
|
||||
|
||||
mediaItems, err := h.db.GetUserMediaItemsForSync(context.Background(), pgUserID)
|
||||
// RLS plan §6: this read is the connection's OWN user's media —
|
||||
// declare that identity explicitly (context.Background() carries
|
||||
// nothing through ScopedDB, and media_items is RLS-governed).
|
||||
mediaItems, err := h.db.GetUserMediaItemsForSync(rls.WithUser(context.Background(), pgUserID), pgUserID)
|
||||
if err != nil {
|
||||
return map[string]interface{}{"error": "failed to fetch initial state"}
|
||||
}
|
||||
|
||||
progressMap := make(map[string]map[string]interface{})
|
||||
for _, item := range mediaItems {
|
||||
progress, err := h.db.GetUniversalProgress(context.Background(), database.GetUniversalProgressParams{
|
||||
progress, err := h.db.GetUniversalProgress(rls.WithUser(context.Background(), pgUserID), database.GetUniversalProgressParams{
|
||||
MediaItemID: item.ID,
|
||||
UserID: pgUserID,
|
||||
})
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
package middleware_test
|
||||
|
||||
import (
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Go-source CSP tripwire.
|
||||
//
|
||||
// The templ scan lives in templates/csp_source_test.go, but htmx partials
|
||||
// can also be emitted from Go handler strings — auth.go used to answer a
|
||||
// successful login with an inline <script>window.location…</script> that
|
||||
// hash-mode CSP silently refused. This test scans the non-test Go sources
|
||||
// for the same forbidden patterns: inline script tags and native on*=
|
||||
// handler attributes in emitted HTML. Partials that need to navigate or
|
||||
// stash state use data attributes + web/src/auth_forms.ts, or htmx's
|
||||
// HX-Redirect/HX-Refresh headers.
|
||||
|
||||
var goScriptTagRe = regexp.MustCompile(`<script`)
|
||||
|
||||
var goHandlerAttrRe = regexp.MustCompile(
|
||||
`on(click|error|submit|load|change|input|focus|blur|mouseover|keydown|keyup)\s*=\s*"`)
|
||||
|
||||
func TestGoSourcesAreCSPSafe(t *testing.T) {
|
||||
root := "../.."
|
||||
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.IsDir() {
|
||||
switch d.Name() {
|
||||
case "node_modules", "static", ".git", "testdata", "docs":
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
name := d.Name()
|
||||
if !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
|
||||
return nil
|
||||
}
|
||||
// Generated templ codegen is out of scope: its source of truth is
|
||||
// the .templ file (scanned by templates/csp_source_test.go), and
|
||||
// the codegen legitimately emits external <script src=…> tags.
|
||||
if strings.HasSuffix(name, "_templ.go") {
|
||||
return nil
|
||||
}
|
||||
raw, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
src := string(raw)
|
||||
for _, m := range goScriptTagRe.FindAllStringIndex(src, -1) {
|
||||
t.Errorf("%s:%d: literal <script> in Go source — hash-mode CSP refuses inline scripts in emitted partials; use data attributes + web/src/auth_forms.ts or HX-Redirect/HX-Refresh",
|
||||
path, 1+strings.Count(src[:m[0]], "\n"))
|
||||
}
|
||||
for _, m := range goHandlerAttrRe.FindAllStringIndex(src, -1) {
|
||||
t.Errorf("%s:%d: native on*= handler attribute in Go source — CSP hash-mode refuses it; use Alpine bindings or delegated listeners",
|
||||
path, 1+strings.Count(src[:m[0]], "\n"))
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package middleware
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/rls"
|
||||
"context"
|
||||
"net/http"
|
||||
"strconv"
|
||||
@@ -183,6 +184,13 @@ func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerF
|
||||
c.Set("device_ctx", ctx)
|
||||
c.Set("device_id", device.ID.Bytes)
|
||||
|
||||
// RLS bridge (plan §5.1/A9): device-authenticated requests
|
||||
// (KOReader/Kobo sync) read media as the DEVICE OWNER — echo's
|
||||
// c.Set is echo-local, so the stdlib request context needs the
|
||||
// identity explicitly or every governed read raises.
|
||||
c.SetRequest(c.Request().WithContext(context.WithValue(
|
||||
c.Request().Context(), rls.CtxKeyUserID, device.UserID)))
|
||||
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
@@ -9,7 +11,14 @@ import (
|
||||
// the rendered component by security_headers_test.go.
|
||||
const ScriptHashThemeFlashGuard = "'sha256-JYIVpFyKdLUtWizMha0v69f6yHds+GbJRBWjvpuECBY='"
|
||||
|
||||
// SecurityHeadersMiddleware adds security headers to all responses
|
||||
// SecurityHeadersMiddleware adds security headers to all responses.
|
||||
// It also forces HTML pages to revalidate (Cache-Control: no-cache):
|
||||
// SSR pages are dynamic, and heuristic browser caching of a PAGE let a
|
||||
// stale page reference a stale main.js stamp long after the bundle was
|
||||
// fixed — the "search dropdown still broken" report was exactly that.
|
||||
// /api, /static and /uploads keep their own (or absent) caching: the
|
||||
// stamped bundles are content-frozen by URL, media files are immutable
|
||||
// by path.
|
||||
func SecurityHeadersMiddleware() echo.MiddlewareFunc {
|
||||
return func(next echo.HandlerFunc) echo.HandlerFunc {
|
||||
return func(c *echo.Context) error {
|
||||
@@ -24,8 +33,8 @@ func SecurityHeadersMiddleware() echo.MiddlewareFunc {
|
||||
//
|
||||
// The inline script allow-list: the theme flash guard is the
|
||||
// ONLY inline script in the app, and its byte-stable content is
|
||||
// hashed into script-src (sha256 of the bytes between
|
||||
// <script> and </script> as rendered by
|
||||
// hashed into script-src (sha256 of the bytes between the
|
||||
// script tags as rendered by
|
||||
// templates.ThemeFlashGuard()). Editing the guard requires
|
||||
// recomputing the hash — enforced by
|
||||
// internal/middleware/security_headers_test.go, which renders
|
||||
@@ -39,6 +48,19 @@ func SecurityHeadersMiddleware() echo.MiddlewareFunc {
|
||||
c.Response().Header().Set("Referrer-Policy", "strict-origin-when-cross-origin")
|
||||
c.Response().Header().Set("Permissions-Policy", "geolocation=(), microphone=(), camera=()")
|
||||
|
||||
// Pages must always revalidate (see doc comment). The check
|
||||
// runs BEFORE next(): once a templ handler writes the body,
|
||||
// response headers are already on the wire.
|
||||
switch p := c.Request().URL.Path; {
|
||||
case strings.HasPrefix(p, "/api/"),
|
||||
strings.HasPrefix(p, "/static/"),
|
||||
strings.HasPrefix(p, "/uploads/"),
|
||||
p == "/health":
|
||||
// Not a page: keep native caching behavior.
|
||||
default:
|
||||
c.Response().Header().Set("Cache-Control", "no-cache")
|
||||
}
|
||||
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4,12 +4,16 @@ import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"regexp"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"bookhoard/internal/middleware"
|
||||
"bookhoard/templates"
|
||||
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
// TestThemeFlashGuardHashMatchesCSP pins the CSP script hash to the theme
|
||||
@@ -49,3 +53,38 @@ middleware.ScriptHashThemeFlashGuard and the CSP header string.`,
|
||||
t.Error("script-src must not re-introduce 'unsafe-inline' alongside the hash")
|
||||
}
|
||||
}
|
||||
|
||||
// HTML pages must always revalidate (Cache-Control: no-cache): a
|
||||
// heuristically cached PAGE referenced a stale main.js stamp and kept
|
||||
// the old bundle running long after a fix deployed (the 2026-10-27
|
||||
// "search dropdown still broken" report). Static/API/media routes keep
|
||||
// their native caching — stamped bundles are content-frozen by URL.
|
||||
func TestHTMLPagesRevalidate(t *testing.T) {
|
||||
mw := middleware.SecurityHeadersMiddleware()
|
||||
|
||||
render := func(path string) string {
|
||||
e := echo.New()
|
||||
req, _ := http.NewRequest("GET", path, nil)
|
||||
rec := httptest.NewRecorder()
|
||||
c := e.NewContext(req, rec)
|
||||
handler := mw(func(c *echo.Context) error {
|
||||
return c.HTML(http.StatusOK, "<html></html>")
|
||||
})
|
||||
if err := handler(c); err != nil {
|
||||
t.Fatalf("handler error: %v", err)
|
||||
}
|
||||
return rec.Header().Get("Cache-Control")
|
||||
}
|
||||
|
||||
if got := render("/bookshelf"); got != "no-cache" {
|
||||
t.Errorf("/bookshelf: Cache-Control = %q, want no-cache", got)
|
||||
}
|
||||
if got := render("/series"); got != "no-cache" {
|
||||
t.Errorf("/series: Cache-Control = %q, want no-cache", got)
|
||||
}
|
||||
for _, p := range []string{"/static/main.js", "/api/media-items", "/uploads/library-x/book.epub", "/health"} {
|
||||
if got := render(p); got == "no-cache" {
|
||||
t.Errorf("%s: must keep native caching, got no-cache", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
// Package rls carries the authenticated end user's identity through
|
||||
// the stdlib context so the database layer can declare it to Postgres
|
||||
// per transaction (the `app.user_id` GUC the RLS policies read — see
|
||||
// database/schema/schema.sql and
|
||||
// ~/.opencode/plan/server-rls-library-visibility.md).
|
||||
//
|
||||
// WHY A SEPARATE PACKAGE: Echo's c.Set/c.Get is echo-local storage and
|
||||
// never reaches the context.Context handed to database calls; the auth
|
||||
// middleware therefore bridges the user id into the request context
|
||||
// here, and ScopedDB (internal/database) reads it back. A query whose
|
||||
// context carries no identity is REFUSED (fail-loud) — a code path
|
||||
// that forgets the middleware breaks visibly, it never leaks.
|
||||
package rls
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
type ctxKey int
|
||||
|
||||
// CtxKeyUserID is the context key the auth middleware bridge sets and
|
||||
// ScopedDB reads. Typed key — no string-key collisions.
|
||||
const CtxKeyUserID ctxKey = iota
|
||||
|
||||
// FromContext returns the authenticated end user's UUID.
|
||||
// ok=false means the caller forgot the middleware bridge — the
|
||||
// decorator will refuse the query loudly.
|
||||
func FromContext(ctx context.Context) (pgtype.UUID, bool) {
|
||||
uid, ok := ctx.Value(CtxKeyUserID).(pgtype.UUID)
|
||||
if !ok || !uid.Valid {
|
||||
return pgtype.UUID{}, false
|
||||
}
|
||||
return uid, true
|
||||
}
|
||||
|
||||
// WithUser re-attaches an identity to a (possibly detached) context.
|
||||
// Background jobs MUST capture the acting user's UUID from the request
|
||||
// context BEFORE `go func` detaches — context.Background() carries
|
||||
// nothing — and rebuild their job context with this.
|
||||
func WithUser(ctx context.Context, uid pgtype.UUID) context.Context {
|
||||
return context.WithValue(ctx, CtxKeyUserID, uid)
|
||||
}
|
||||
|
||||
// SeedIdentity is the harness seeding identity (plan §5): a valid UUID
|
||||
// with NO library_visibility rows, so RLS grants it full visibility.
|
||||
// It exists so test seeding and direct DB verification can read and
|
||||
// write governed tables without impersonating a real user. Product
|
||||
// code never runs as it.
|
||||
var SeedIdentity = pgtype.UUID{Bytes: [16]byte(uuid.MustParse("00000000-0000-0000-0000-000000000002")), Valid: true}
|
||||
|
||||
// DeclareUser sets the RLS identity GUC inside an ALREADY-OPEN
|
||||
// transaction. The `true` argument is is_local: transaction-scoped,
|
||||
// evaporating at COMMIT/ROLLBACK — NEVER session-scoped on a pooled
|
||||
// connection (cross-request identity bleed). For the manual-tx paths
|
||||
// the ScopedDB decorator cannot wrap.
|
||||
func DeclareUser(ctx context.Context, tx pgx.Tx, uid pgtype.UUID) error {
|
||||
_, err := tx.Exec(ctx, `SELECT set_config('app.user_id', $1, true)`, uid.String())
|
||||
return err
|
||||
}
|
||||
@@ -149,6 +149,36 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
}
|
||||
offset := (page - 1) * perSeriesPage
|
||||
|
||||
// Series-page state (URL is the source of truth — plain GET form,
|
||||
// no htmx on this page). Unknown values resolve to defaults in the
|
||||
// service's registries, so hand-edited URLs stay safe. The URL
|
||||
// carries the BASE sort key + direction; opts.Sort gets the
|
||||
// composed registry key (full keys win on re-resolve). With no
|
||||
// sort param at all, state stays empty so default URLs remain
|
||||
// clean (the toolbar treats empty as recent/desc).
|
||||
rawSort := c.QueryParam("sort")
|
||||
sortKey, sortDir := services.ResolveSeriesSort(rawSort, c.QueryParam("dir"))
|
||||
state := templates.SeriesPageState{
|
||||
Sort: services.SeriesSortBase(sortKey),
|
||||
Dir: sortDir,
|
||||
Query: c.QueryParam("q"),
|
||||
Status: c.QueryParam("status"),
|
||||
}
|
||||
if rawSort == "" {
|
||||
state.Sort = ""
|
||||
state.Dir = ""
|
||||
}
|
||||
opts := services.SeriesPageOptions{
|
||||
Sort: sortKey,
|
||||
Dir: sortDir,
|
||||
Query: state.Query,
|
||||
Author: c.QueryParam("author"), // API/deep-link back-compat; the toolbar's multi-field q covers authors
|
||||
Status: state.Status,
|
||||
}
|
||||
if viewerID, vidErr := uuid.Parse(user.ID); vidErr == nil {
|
||||
opts.ViewerUserID = uuidToPGType(viewerID)
|
||||
}
|
||||
|
||||
var seriesCards []templates.SeriesCardData
|
||||
totalPages := 1
|
||||
|
||||
@@ -162,7 +192,7 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
} else if libRes.LibUUID.Valid && !visibleSet[libRes.LibraryID] {
|
||||
errorMsg = "Library not accessible"
|
||||
} else {
|
||||
seriesList, total, err := handlers.GetSeriesCardsData(c.Request().Context(), cfg.Queries, libRes.LibUUID, visibleIDs, perSeriesPage, offset)
|
||||
seriesList, total, err := handlers.GetSeriesCardsData(c.Request().Context(), cfg.Queries, libRes.LibUUID, visibleIDs, perSeriesPage, offset, opts)
|
||||
if err != nil {
|
||||
log.Printf("GetSeriesCardsData failed: %v", err)
|
||||
errorMsg = "Error loading series"
|
||||
@@ -183,6 +213,7 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
BookCount: s.BookCount,
|
||||
TotalInSeries: s.TotalInSeries,
|
||||
CoverPaths: covers,
|
||||
ReadCount: s.ReadCount,
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -194,7 +225,7 @@ func registerFrontendRoutes(cfg *Config) {
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
err = templates.Series(user, seriesCards, libData, libraryID, totalPages, page, errorMsg).Render(c.Request().Context(), &buf)
|
||||
err = templates.Series(user, seriesCards, libData, libraryID, totalPages, page, state, errorMsg).Render(c.Request().Context(), &buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -1953,6 +1984,7 @@ func (cfg *Config) GetHealth(c *echo.Context) error {
|
||||
return c.JSON(http.StatusOK, map[string]interface{}{
|
||||
"status": "healthy",
|
||||
"version": version.Version,
|
||||
"commit": version.Commit,
|
||||
"database": "connected",
|
||||
"scan": map[string]interface{}{
|
||||
"scan_in_progress": cfg.Worker.HasActiveScans(),
|
||||
|
||||
@@ -55,6 +55,14 @@ func getTemplateUserWithTheme(c *echo.Context, cfg *Config) (templates.User, err
|
||||
token = cookie.Value
|
||||
}
|
||||
|
||||
// Sidebar /podcasts gate (plan D13): >=1 visible podcasts library.
|
||||
// One indexed COUNT per page render — the same class of per-request
|
||||
// lookup the live-user guard already does.
|
||||
showPodcasts := false
|
||||
if n, err := cfg.Queries.CountPodcastLibrariesForUser(c.Request().Context(), uuidToPGType(userUUID)); err == nil && n > 0 {
|
||||
showPodcasts = true
|
||||
}
|
||||
|
||||
return templates.User{
|
||||
ID: userID,
|
||||
Email: userEmail,
|
||||
@@ -68,6 +76,7 @@ func getTemplateUserWithTheme(c *echo.Context, cfg *Config) (templates.User, err
|
||||
CreatedAt: userDB.CreatedAt.Time,
|
||||
DefaultAudioSpeed: float64(userDB.DefaultAudioSpeed),
|
||||
KeepScreenAwake: userDB.KeepScreenAwake,
|
||||
ShowPodcastsNav: showPodcasts,
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
||||
@@ -74,6 +74,7 @@ func registerMediaRoutes(cfg *Config) {
|
||||
admin.POST("/media-items", cfg.MediaHandler.CreateMediaItem)
|
||||
admin.PUT("/media-items/:id", cfg.MediaHandler.UpdateMediaItem)
|
||||
admin.POST("/media-items/:id/rescan", cfg.MediaHandler.RescanMediaItem)
|
||||
admin.POST("/media-items/:id/write-metadata", cfg.MediaHandler.WriteMetadataToFile)
|
||||
admin.DELETE("/media-items/:id", cfg.MediaHandler.DeleteMediaItem)
|
||||
admin.POST("/media-items/purge-archived", cfg.MediaHandler.PurgeArchivedMediaItems)
|
||||
admin.POST("/media-items/:id/unarchive", cfg.MediaHandler.UnarchiveMediaItem)
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
package router
|
||||
|
||||
// Podcast routes (plan §8). All under /api/podcasts with the JWT
|
||||
// middleware; scoping is RLS's job (D20) — handlers add no visibility
|
||||
// plumbing.
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"net/http"
|
||||
|
||||
"bookhoard/templates"
|
||||
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
func registerPodcastRoutes(cfg *Config) {
|
||||
g := cfg.Echo.Group("/api/podcasts", createJWTMiddleware(cfg))
|
||||
|
||||
// Episode-list renderer (issue #1): htmx actions answer with the
|
||||
// refreshed partial directly — no event indirection to miss. The
|
||||
// view assembly lives here (templates) while the action logic lives
|
||||
// in the handler; this closure is the sanctioned seam.
|
||||
cfg.PodcastHandler.SetEpisodeListRenderer(func(c *echo.Context, feedID string, includePolicySelect bool) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
feedUUID, err := parsePodcastUUID(feedID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := buildPodcastDetailData(c, cfg, user, feedUUID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
render := templates.PodcastEpisodeList
|
||||
if templates.HasActiveDownloads(data.Episodes) {
|
||||
render = templates.PodcastEpisodeListPolling
|
||||
}
|
||||
if err := render(*data).Render(c.Request().Context(), &buf); err != nil {
|
||||
return err
|
||||
}
|
||||
if includePolicySelect {
|
||||
// Issue #3: out-of-band swap of the auto-download select —
|
||||
// the dropdown reflects the flipped policy immediately.
|
||||
if err := templates.PodcastAutoDownloadSelectOOB(data.Feed).Render(c.Request().Context(), &buf); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
g.POST("/feeds", cfg.PodcastHandler.AddFeed)
|
||||
g.GET("/feeds", cfg.PodcastHandler.ListFeeds)
|
||||
g.GET("/search", cfg.PodcastHandler.Search)
|
||||
|
||||
// :id-scoped (RLS answers 404 for feeds outside the caller's scope).
|
||||
feeds := g.Group("/feeds/:id")
|
||||
feeds.GET("", cfg.PodcastHandler.GetFeed)
|
||||
feeds.GET("/sequence", cfg.PodcastHandler.GetFeedSequence)
|
||||
feeds.PUT("/order", cfg.PodcastHandler.SetFeedOrder)
|
||||
feeds.PUT("/subscription", cfg.PodcastHandler.UpdateSubscription)
|
||||
feeds.DELETE("/subscription", cfg.PodcastHandler.DeleteSubscription)
|
||||
feeds.POST("/refresh", cfg.PodcastHandler.RefreshFeed)
|
||||
feeds.POST("/episodes/:eid/download", cfg.PodcastHandler.DownloadEpisodes)
|
||||
feeds.DELETE("", cfg.PodcastHandler.DeleteFeed) // admin-only (handler checks role)
|
||||
}
|
||||
@@ -0,0 +1,467 @@
|
||||
package router
|
||||
|
||||
// Podcast SSR pages (plan task 1.9): /podcasts (subscriptions grid +
|
||||
// add-feed wizard) and /podcasts/:id (feed detail with the inline
|
||||
// download lifecycle). Rendering ownership: ALL content is
|
||||
// templ-rendered server data; htmx swaps partials; the sanctioned JS is
|
||||
// the modal toggle + event delegation in web/src/podcasts.ts.
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/utils"
|
||||
"bookhoard/templates"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/labstack/echo/v5"
|
||||
)
|
||||
|
||||
func registerPodcastPages(cfg *Config) {
|
||||
pages := cfg.Echo.Group("", createJWTMiddleware(cfg))
|
||||
|
||||
pages.GET("/podcasts", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
data, err := buildPodcastsPageData(c, cfg)
|
||||
if err != nil {
|
||||
data = &templates.PodcastsPageData{ErrorMsg: "Could not load subscriptions"}
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := templates.PodcastsPage(user, *data, "/podcasts").Render(c.Request().Context(), &buf); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
pages.GET("/podcasts/:id", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
fid, ferr := parsePodcastUUID(c.Param("id"))
|
||||
if ferr != nil {
|
||||
return c.NoContent(http.StatusNotFound)
|
||||
}
|
||||
data, err := buildPodcastDetailData(c, cfg, user, fid)
|
||||
if err != nil {
|
||||
var buf bytes.Buffer
|
||||
_ = templates.ErrorPage("Feed not found", "not_found").Render(c.Request().Context(), &buf)
|
||||
return c.HTML(http.StatusNotFound, buf.String())
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := templates.PodcastDetailPage(user, *data).Render(c.Request().Context(), &buf); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
|
||||
// Episode list partial — the htmx refresh unit (poll while
|
||||
// downloads are active; podcast:changed events swap it too).
|
||||
pages.GET("/podcasts/:id/episodes", func(c *echo.Context) error {
|
||||
user, err := getTemplateUserWithTheme(c, cfg)
|
||||
if err != nil {
|
||||
return renderErrorPage(c, "Error loading user", "user_load_error")
|
||||
}
|
||||
fid, ferr := parsePodcastUUID(c.Param("id"))
|
||||
if ferr != nil {
|
||||
return c.NoContent(http.StatusNotFound)
|
||||
}
|
||||
data, err := buildPodcastDetailData(c, cfg, user, fid)
|
||||
if err != nil {
|
||||
return c.NoContent(http.StatusNotFound)
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if templates.HasActiveDownloads(data.Episodes) {
|
||||
err = templates.PodcastEpisodeListPolling(*data).Render(c.Request().Context(), &buf)
|
||||
} else {
|
||||
err = templates.PodcastEpisodeList(*data).Render(c.Request().Context(), &buf)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.HTML(http.StatusOK, buf.String())
|
||||
})
|
||||
}
|
||||
|
||||
// buildPodcastsPageData assembles the /podcasts page: subscription
|
||||
// cards, facet values, server-side filters (q/host/tag/language).
|
||||
func buildPodcastsPageData(c *echo.Context, cfg *Config) (*templates.PodcastsPageData, error) {
|
||||
user := handlers.MustGetAuthenticatedUser(c)
|
||||
ctx := c.Request().Context()
|
||||
|
||||
libs, err := cfg.Queries.GetPodcastsLibraryForUser(ctx, user.ID)
|
||||
data := &templates.PodcastsPageData{HasLibrary: len(libs) > 0}
|
||||
if err != nil || len(libs) == 0 {
|
||||
return data, err
|
||||
}
|
||||
for _, lib := range libs {
|
||||
data.Libraries = append(data.Libraries, templates.LibraryData{
|
||||
ID: uuid.UUID(lib.ID.Bytes).String(),
|
||||
Name: lib.Name,
|
||||
TypeName: "podcasts",
|
||||
})
|
||||
}
|
||||
|
||||
feeds, err := cfg.Queries.ListMyPodcastFeeds(ctx, user.ID)
|
||||
if err != nil {
|
||||
return data, err
|
||||
}
|
||||
// Continue aggregation (plan 2.5): first in-progress episode per
|
||||
// feed (the query is already most-recent-first).
|
||||
continueByFeed := make(map[string]*database.GetPodcastContinueForUserRow)
|
||||
if continueRows, err := cfg.Queries.GetPodcastContinueForUser(ctx, user.ID); err == nil {
|
||||
for i := range continueRows {
|
||||
row := &continueRows[i]
|
||||
key := uuid.UUID(row.SourceFeedID.Bytes).String()
|
||||
if _, ok := continueByFeed[key]; !ok {
|
||||
continueByFeed[key] = row
|
||||
}
|
||||
}
|
||||
}
|
||||
for i := range feeds {
|
||||
row := &feeds[i]
|
||||
total, _ := cfg.Queries.CountPodcastEpisodesForFeed(ctx, row.ID)
|
||||
downloaded, _ := cfg.Queries.CountDownloadedEpisodes(ctx, row.ID)
|
||||
data.Feeds = append(data.Feeds, templates.PodcastFeedCard{
|
||||
ID: uuid.UUID(row.ID.Bytes).String(),
|
||||
LibraryID: uuid.UUID(row.LibraryID.Bytes).String(),
|
||||
Title: row.Title,
|
||||
DisplayName: row.DisplayName.String,
|
||||
Description: firstLine(row.Description.String, 220),
|
||||
Host: row.Host.String,
|
||||
Publisher: row.Publisher.String,
|
||||
Language: row.Language.String,
|
||||
Tags: row.Tags,
|
||||
Explicit: row.Explicit,
|
||||
ArtworkPath: row.ArtworkPath.String,
|
||||
FeedType: row.FeedType.String,
|
||||
Medium: row.Medium.String,
|
||||
IsComplete: row.IsComplete,
|
||||
FeedURL: row.FeedUrl,
|
||||
LastFetched: pgTimeDisplay(row.LastFetchedAt, utils.PgTextString(user.Timezone)),
|
||||
SubscribedAt: pgTimeDisplay(row.SubscribedAt, utils.PgTextString(user.Timezone)),
|
||||
EpisodeTotal: int(total),
|
||||
DownloadedN: int(downloaded),
|
||||
UndownloadedN: int(total) - int(downloaded),
|
||||
NewEpisodes: total-downloaded > 0,
|
||||
AutoDownload: row.AutoDownload,
|
||||
AutoDownloadCount: int(row.AutoDownloadCount.Int32),
|
||||
Subscribed: true,
|
||||
})
|
||||
if cont := continueByFeed[uuid.UUID(row.ID.Bytes).String()]; cont != nil {
|
||||
card := &data.Feeds[len(data.Feeds)-1]
|
||||
card.HasContinue = true
|
||||
card.ContinueTitle = cont.Title
|
||||
card.ContinueEpisode = int(cont.FeedPosition.Int32)
|
||||
card.ContinuePercent = cont.Percentage.Float64
|
||||
card.ContinueLastRead = cont.LastReadAt.Time
|
||||
}
|
||||
}
|
||||
|
||||
// Facet values from the unfiltered subscription list (D18).
|
||||
allData := *data
|
||||
data.Facets = facetValuesFromCards(allData.Feeds)
|
||||
|
||||
// Server-side filters.
|
||||
q := strings.ToLower(strings.TrimSpace(c.QueryParam("q")))
|
||||
host := c.QueryParam("host")
|
||||
tag := c.QueryParam("tag")
|
||||
lang := c.QueryParam("language")
|
||||
|
||||
// Episode-level matches (issue #8 / D14): a query also hits episode
|
||||
// titles + descriptions — the feed surfaces with a match count.
|
||||
// SearchMyPodcastEpisodes is subscription-scoped via RLS (the
|
||||
// caller's identity). Only consult it when a query is present.
|
||||
matchesByFeed := make(map[string]int)
|
||||
if q != "" {
|
||||
if eps, err := cfg.Queries.SearchMyPodcastEpisodes(ctx, q); err == nil {
|
||||
for i := range eps {
|
||||
key := uuid.UUID(eps[i].SourceFeedID.Bytes).String()
|
||||
matchesByFeed[key]++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
filtered := data.Feeds[:0:0]
|
||||
for _, f := range data.Feeds {
|
||||
if q != "" {
|
||||
hay := strings.ToLower(f.Title + " " + f.DisplayName + " " + f.Host + " " + f.Publisher + " " + f.Description)
|
||||
matched := matchesByFeed[f.ID]
|
||||
if !strings.Contains(hay, q) && matched == 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
if host != "" && f.Host != host {
|
||||
continue
|
||||
}
|
||||
if lang != "" && !strings.EqualFold(f.Language, lang) {
|
||||
continue
|
||||
}
|
||||
if tag != "" {
|
||||
found := false
|
||||
for _, t := range f.Tags {
|
||||
if strings.EqualFold(t, tag) {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
continue
|
||||
}
|
||||
}
|
||||
filtered = append(filtered, f)
|
||||
}
|
||||
if q != "" {
|
||||
for i := range filtered {
|
||||
filtered[i].EpisodeMatches = matchesByFeed[filtered[i].ID]
|
||||
}
|
||||
}
|
||||
data.Feeds = filtered
|
||||
data.Query = q
|
||||
data.ContinueFeeds = templates.PodcastContinueFeeds(data.Feeds)
|
||||
return data, nil
|
||||
}
|
||||
|
||||
// buildPodcastDetailData assembles the feed-detail page + episode list
|
||||
// partial. RLS scopes the lookup (non-subscribers get sqlc.ErrNoRows →
|
||||
// 404 semantics — no Go visibility plumbing, D20).
|
||||
func buildPodcastDetailData(c *echo.Context, cfg *Config, user templates.User, feedID pgtype.UUID) (*templates.PodcastDetailData, error) {
|
||||
actor := handlers.MustGetAuthenticatedUser(c)
|
||||
ctx := c.Request().Context()
|
||||
|
||||
feed, err := cfg.Queries.GetPodcastFeed(ctx, feedID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
sub, _ := cfg.Queries.GetPodcastSubscription(ctx, database.GetPodcastSubscriptionParams{
|
||||
UserID: actor.ID, FeedID: feedID,
|
||||
})
|
||||
eps, err := cfg.Queries.ListPodcastEpisodes(ctx, feedID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
progressRows, _ := cfg.Queries.GetPodcastEpisodeProgress(ctx, database.GetPodcastEpisodeProgressParams{
|
||||
UserID: actor.ID, SourceFeedID: feedID,
|
||||
})
|
||||
progressByItem := make(map[string]struct {
|
||||
Pct float64
|
||||
LastRead pgtype.Timestamptz
|
||||
}, len(progressRows))
|
||||
for _, p := range progressRows {
|
||||
progressByItem[uuid.UUID(p.MediaItemID.Bytes).String()] = struct {
|
||||
Pct float64
|
||||
LastRead pgtype.Timestamptz
|
||||
}{p.Percentage.Float64, p.LastReadAt}
|
||||
}
|
||||
|
||||
total, _ := cfg.Queries.CountPodcastEpisodesForFeed(ctx, feedID)
|
||||
downloaded, _ := cfg.Queries.CountDownloadedEpisodes(ctx, feedID)
|
||||
subscriberCount, _ := cfg.Queries.CountPodcastFeedSubscribers(ctx, feedID)
|
||||
|
||||
card := templates.PodcastFeedCard{
|
||||
ID: uuid.UUID(feed.ID.Bytes).String(),
|
||||
LibraryID: uuid.UUID(feed.LibraryID.Bytes).String(),
|
||||
Title: feed.Title,
|
||||
DisplayName: sub.DisplayName.String,
|
||||
Description: feed.Description.String,
|
||||
Host: feed.Host.String,
|
||||
Publisher: feed.Publisher.String,
|
||||
Language: feed.Language.String,
|
||||
Tags: feed.Tags,
|
||||
Explicit: feed.Explicit,
|
||||
ArtworkPath: feed.ArtworkPath.String,
|
||||
FeedType: feed.FeedType.String,
|
||||
Medium: feed.Medium.String,
|
||||
IsComplete: feed.IsComplete,
|
||||
FeedURL: feed.FeedUrl,
|
||||
LastFetched: pgTimeDisplay(feed.LastFetchedAt, user.Timezone),
|
||||
EpisodeTotal: int(total),
|
||||
DownloadedN: int(downloaded),
|
||||
UndownloadedN: int(total) - int(downloaded),
|
||||
NewEpisodes: total-downloaded > 0,
|
||||
AutoDownload: sub.AutoDownload,
|
||||
AutoDownloadCount: int(sub.AutoDownloadCount.Int32),
|
||||
Subscribed: sub.ID.Valid,
|
||||
}
|
||||
|
||||
rows := make([]templates.PodcastEpisodeRow, 0, len(eps))
|
||||
gaps := 0
|
||||
seenDownloadedFrom := false // walk reversed: gap = undownloaded with a downloaded LATER in order
|
||||
for i := len(eps) - 1; i >= 0; i-- {
|
||||
ep := &eps[i]
|
||||
downloadedEp := ep.DownloadState == "downloaded"
|
||||
if !downloadedEp && seenDownloadedFrom {
|
||||
gaps++
|
||||
}
|
||||
if downloadedEp {
|
||||
seenDownloadedFrom = true
|
||||
}
|
||||
}
|
||||
// Forward pass for the actual rows (native order — the list keeps
|
||||
// feed order; the ORDER selector drives the sequence endpoint, and
|
||||
// the order control re-renders this list via the override applied
|
||||
// below).
|
||||
order := podcastOrderForFeed(cfg, c, feedID.String())
|
||||
for i := range eps {
|
||||
ep := &eps[i]
|
||||
rowID := uuid.UUID(ep.ID.Bytes).String()
|
||||
p := progressByItem[rowID]
|
||||
dateDisplay := ""
|
||||
if ep.DatePublished.Valid {
|
||||
dateDisplay = ep.DatePublished.Time.In(utils.UserLocation(user.Timezone)).Format("Jan 2, 2006")
|
||||
}
|
||||
rows = append(rows, templates.PodcastEpisodeRow{
|
||||
ID: uuid.UUID(ep.ID.Bytes).String(),
|
||||
FeedID: uuid.UUID(ep.SourceFeedID.Bytes).String(),
|
||||
Title: ep.Title,
|
||||
Description: firstLine(ep.Description.String, 200),
|
||||
Date: dateDisplay,
|
||||
Position: int(ep.FeedPosition.Int32),
|
||||
Duration: templates.FormatEpisodeDuration(int(ep.DeclaredDurationSeconds.Int32)),
|
||||
Size: templates.FormatEpisodeSize(ep.FileSize.Int64),
|
||||
PlayURL: "/readers/" + uuid.UUID(ep.ID.Bytes).String(),
|
||||
ListenPct: p.Pct,
|
||||
Downloaded: ep.DownloadState == "downloaded",
|
||||
DownloadState: ep.DownloadState,
|
||||
DownloadError: ep.DownloadError.String,
|
||||
})
|
||||
}
|
||||
// Order override reshapes the LIST too (the sequence endpoint is the
|
||||
// strict downloaded-only view; the list shows everything in the
|
||||
// chosen order so pending episodes stay visible).
|
||||
if order == "oldest" || order == "newest" {
|
||||
sort.SliceStable(rows, func(a, b int) bool {
|
||||
ka := rows[a].Date + itoaPad(rows[a].Position)
|
||||
kb := rows[b].Date + itoaPad(rows[b].Position)
|
||||
if order == "newest" {
|
||||
return ka > kb
|
||||
}
|
||||
return ka < kb
|
||||
})
|
||||
}
|
||||
|
||||
detail := &templates.PodcastDetailData{
|
||||
Feed: card,
|
||||
Episodes: rows,
|
||||
GapCount: gaps,
|
||||
Order: order,
|
||||
IsAdmin: actor.Role == "admin",
|
||||
IsSerial: feed.FeedType.String == "serial" || feed.Medium.String == "audiobook",
|
||||
SubscriberCount: int(subscriberCount),
|
||||
}
|
||||
|
||||
// Live byte progress for in-flight downloads (issue #1): read the
|
||||
// worker's snapshot; rows not actively downloading render states.
|
||||
if cfg.PodcastWorker != nil {
|
||||
for i := range detail.Episodes {
|
||||
row := &detail.Episodes[i]
|
||||
if row.DownloadState != "downloading" {
|
||||
continue
|
||||
}
|
||||
if recv, tot, ok := cfg.PodcastWorker.EpisodeProgress(pgtype.UUID{Bytes: uuid.MustParse(row.ID), Valid: true}); ok {
|
||||
row.ProgressKnown = true
|
||||
row.ProgressReceived = recv
|
||||
row.ProgressTotal = tot
|
||||
if tot > 0 {
|
||||
row.ProgressPct = float64(recv) / float64(tot) * 100
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return detail, nil
|
||||
}
|
||||
|
||||
// podcastOrderForFeed reads the caller's order override (D5).
|
||||
func podcastOrderForFeed(cfg *Config, c *echo.Context, feedID string) string {
|
||||
user := handlers.MustGetAuthenticatedUser(c)
|
||||
row, err := cfg.Queries.GetUserSettingByKey(c.Request().Context(), database.GetUserSettingByKeyParams{
|
||||
UserID: user.ID,
|
||||
SettingKey: "podcast_order:" + feedID,
|
||||
})
|
||||
if err != nil {
|
||||
return "native"
|
||||
}
|
||||
var v struct {
|
||||
Order string `json:"order"`
|
||||
}
|
||||
if err := json.Unmarshal(row, &v); err != nil {
|
||||
return "native"
|
||||
}
|
||||
switch v.Order {
|
||||
case "oldest", "newest":
|
||||
return v.Order
|
||||
}
|
||||
return "native"
|
||||
}
|
||||
|
||||
// facetValuesFromCards derives the filter bar values.
|
||||
func facetValuesFromCards(feeds []templates.PodcastFeedCard) map[string][]string {
|
||||
hosts, tags, langs := map[string]bool{}, map[string]bool{}, map[string]bool{}
|
||||
for _, f := range feeds {
|
||||
if f.Host != "" {
|
||||
hosts[f.Host] = true
|
||||
}
|
||||
for _, t := range f.Tags {
|
||||
tags[t] = true
|
||||
}
|
||||
if f.Language != "" {
|
||||
langs[f.Language] = true
|
||||
}
|
||||
}
|
||||
return map[string][]string{
|
||||
"hosts": sortedSet(hosts),
|
||||
"tags": sortedSet(tags),
|
||||
"language": sortedSet(langs),
|
||||
}
|
||||
}
|
||||
|
||||
func sortedSet(set map[string]bool) []string {
|
||||
out := make([]string, 0, len(set))
|
||||
for k := range set {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func firstLine(s string, max int) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if i := strings.IndexAny(s, "\n\r"); i > 0 {
|
||||
s = s[:i]
|
||||
}
|
||||
if len(s) > max {
|
||||
s = s[:max] + "…"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func pgTimeDisplay(t pgtype.Timestamptz, tz string) string {
|
||||
if !t.Valid {
|
||||
return ""
|
||||
}
|
||||
return utils.FormatTimeInTimezone(t.Time, tz)
|
||||
}
|
||||
|
||||
func itoaPad(n int) string {
|
||||
return fmt.Sprintf("%06d", n)
|
||||
}
|
||||
|
||||
|
||||
// parsePodcastUUID parses an id path param (handlers.parseUUID is
|
||||
// package-private).
|
||||
func parsePodcastUUID(s string) (pgtype.UUID, error) {
|
||||
id, err := uuid.Parse(strings.TrimSpace(s))
|
||||
if err != nil {
|
||||
return pgtype.UUID{}, err
|
||||
}
|
||||
return pgtype.UUID{Bytes: id, Valid: true}, nil
|
||||
}
|
||||
@@ -1,13 +1,17 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/services"
|
||||
"bookhoard/internal/sync"
|
||||
"bookhoard/internal/utils"
|
||||
"bookhoard/templates"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"sort"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
@@ -102,6 +106,18 @@ func registerReaderRoutes(cfg *Config) {
|
||||
TreatAsComic: handlers.ShouldTreatAsComic(libraryWithType.TypeName, mediaItem.FormatGroup),
|
||||
KeepScreenAwake: user.KeepScreenAwake,
|
||||
}
|
||||
// Podcast chaining (plan D7): episodes render the audio reader
|
||||
// with sequence context — the neighboring DOWNLOADED episodes in
|
||||
// the caller's effective order (override applied). RLS already
|
||||
// scoped the episode read; the subscription predicate covers the
|
||||
// neighbors the same way. ?single=1 escapes chaining (the "play
|
||||
// single episode" affordance).
|
||||
if mediaItem.Source == "podcast_episode" && mediaItem.SourceFeedID.Valid && c.QueryParam("single") != "1" {
|
||||
userPG := uuidToPGType(userUUID)
|
||||
if seq := buildPodcastSequenceContext(c, cfg, &mediaItem, userPG, libraryWithType.TypeName); seq != nil {
|
||||
metadata.PodcastSequence = seq
|
||||
}
|
||||
}
|
||||
// Render template
|
||||
var buf bytes.Buffer
|
||||
// Audiobooks get the audio player page — the foliate document
|
||||
@@ -144,3 +160,96 @@ func registerReaderRoutes(cfg *Config) {
|
||||
reader.GET("/settings", cfg.ReaderHandler.GetSettings)
|
||||
reader.PUT("/settings", cfg.ReaderHandler.UpdateSettings)
|
||||
}
|
||||
|
||||
// buildPodcastSequenceContext computes the episode chain payload (plan
|
||||
// D7): effective order — the D5 override applied, feed-native fallback —
|
||||
// DOWNLOADED episodes only (gaps never enter the queue), the current
|
||||
// episode's index, and prev/next reader URLs. RLS scopes the episode
|
||||
// reads; non-subscribers never reach this path in the first place.
|
||||
func buildPodcastSequenceContext(c *echo.Context, cfg *Config, item *database.MediaItems, userID pgtype.UUID, libraryType string) *templates.PodcastSequenceContext {
|
||||
ctx := c.Request().Context()
|
||||
eps, err := cfg.Queries.ListPodcastEpisodes(ctx, item.SourceFeedID)
|
||||
if err != nil || len(eps) == 0 {
|
||||
return nil
|
||||
}
|
||||
feed, err := cfg.Queries.GetPodcastFeed(ctx, item.SourceFeedID)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Effective order override (D5): reader_settings KV.
|
||||
order := "native"
|
||||
if row, err := cfg.Queries.GetUserSettingByKey(ctx, database.GetUserSettingByKeyParams{
|
||||
UserID: userID,
|
||||
SettingKey: "podcast_order:" + uuid.UUID(item.SourceFeedID.Bytes).String(),
|
||||
}); err == nil {
|
||||
var v struct {
|
||||
Order string `json:"order"`
|
||||
}
|
||||
if json.Unmarshal(row, &v) == nil && (v.Order == "oldest" || v.Order == "newest") {
|
||||
order = v.Order
|
||||
}
|
||||
}
|
||||
|
||||
type seqEp struct {
|
||||
id string
|
||||
title string
|
||||
date string
|
||||
pos int32
|
||||
}
|
||||
ordered := make([]seqEp, 0, len(eps))
|
||||
currentID := uuid.UUID(item.ID.Bytes).String()
|
||||
for i := range eps {
|
||||
ep := eps[i]
|
||||
if ep.DownloadState != "downloaded" {
|
||||
continue // un-downloaded episodes are skipped, not tracked (invariant 6)
|
||||
}
|
||||
date := ""
|
||||
if ep.DatePublished.Valid {
|
||||
date = ep.DatePublished.Time.Format("2006-01-02")
|
||||
}
|
||||
ordered = append(ordered, seqEp{
|
||||
id: uuid.UUID(ep.ID.Bytes).String(),
|
||||
title: ep.Title,
|
||||
date: date,
|
||||
pos: ep.FeedPosition.Int32,
|
||||
})
|
||||
}
|
||||
sort.SliceStable(ordered, func(a, b int) bool {
|
||||
ka := ordered[a].date + fmt.Sprintf("%06d", ordered[a].pos)
|
||||
kb := ordered[b].date + fmt.Sprintf("%06d", ordered[b].pos)
|
||||
if order == "newest" {
|
||||
return ka > kb
|
||||
}
|
||||
if order == "oldest" {
|
||||
return ka < kb
|
||||
}
|
||||
return ordered[a].pos < ordered[b].pos
|
||||
})
|
||||
|
||||
idx := -1
|
||||
for i, ep := range ordered {
|
||||
if ep.id == currentID {
|
||||
idx = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if idx < 0 {
|
||||
return nil // current episode not downloaded (or not in sequence)
|
||||
}
|
||||
|
||||
seq := &templates.PodcastSequenceContext{
|
||||
FeedURL: "/podcasts/" + uuid.UUID(item.SourceFeedID.Bytes).String(),
|
||||
FeedTitle: feed.Title,
|
||||
EpisodeIndex: idx + 1,
|
||||
IndexTotal: len(ordered),
|
||||
}
|
||||
if idx > 0 {
|
||||
seq.PrevURL = "/readers/" + ordered[idx-1].id
|
||||
}
|
||||
if idx < len(ordered)-1 {
|
||||
seq.NextURL = "/readers/" + ordered[idx+1].id
|
||||
seq.NextTitle = ordered[idx+1].title
|
||||
}
|
||||
return seq
|
||||
}
|
||||
@@ -6,14 +6,19 @@ import (
|
||||
"bookhoard/internal/handlers"
|
||||
"bookhoard/internal/middleware"
|
||||
ratelimit "bookhoard/internal/middleware"
|
||||
"bookhoard/internal/rls"
|
||||
"bookhoard/internal/services"
|
||||
"bookhoard/internal/sync"
|
||||
"bookhoard/templates"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"log"
|
||||
"mime"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -75,6 +80,9 @@ type Config struct {
|
||||
SidecarHandler *handlers.SidecarHandler
|
||||
ReaderHandler *handlers.ReaderHandler
|
||||
LibraryService *services.LibraryService
|
||||
PodcastHandler *handlers.PodcastHandler
|
||||
PodcastService *services.PodcastService
|
||||
PodcastWorker *services.PodcastDownloadWorker
|
||||
}
|
||||
|
||||
// getBaseURL returns the configured base URL from the database, falling back to
|
||||
@@ -151,6 +159,12 @@ func createOptionalAuthMiddleware(cfg *Config) echo.MiddlewareFunc {
|
||||
Timezone: liveUser.Timezone,
|
||||
Theme: liveUser.Theme, // SSR body class; see the required middleware
|
||||
})
|
||||
// RLS bridge: echo's c.Set is echo-local and never reaches
|
||||
// the stdlib context.Context the database layer receives —
|
||||
// ScopedDB refuses media queries without this (plan §5.1).
|
||||
c.SetRequest(c.Request().WithContext(context.WithValue(
|
||||
c.Request().Context(), rls.CtxKeyUserID,
|
||||
pgtype.UUID{Bytes: userUUID, Valid: true})))
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
@@ -218,6 +232,12 @@ func createJWTMiddleware(cfg *Config) echo.MiddlewareFunc {
|
||||
// admin/docs theme flash.
|
||||
Theme: liveUser.Theme,
|
||||
})
|
||||
// RLS bridge (same as the JWT path above): echo-local c.Set
|
||||
// never reaches the stdlib context.Context the database
|
||||
// layer receives — ScopedDB refuses media queries without it.
|
||||
c.SetRequest(c.Request().WithContext(context.WithValue(
|
||||
c.Request().Context(), rls.CtxKeyUserID,
|
||||
pgtype.UUID{Bytes: userUUID, Valid: true})))
|
||||
return nil
|
||||
},
|
||||
ErrorHandler: func(c *echo.Context, err error) error {
|
||||
@@ -327,12 +347,14 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
|
||||
registerConflictRoutes(cfg)
|
||||
registerAnalyticsRoutes(cfg)
|
||||
registerQueueRoutes(cfg)
|
||||
registerPodcastRoutes(cfg)
|
||||
registerJobRoutes(cfg)
|
||||
registerFiltersRoutes(cfg)
|
||||
registerOPDSRoutes(cfg)
|
||||
registerReaderRoutes(cfg)
|
||||
registerWebSocketRoutes(cfg)
|
||||
registerFrontendRoutes(cfg)
|
||||
registerPodcastPages(cfg)
|
||||
registerDocumentationRoutes(cfg)
|
||||
|
||||
// Static bundles are served unhashed (reader.js, index.js), so the
|
||||
@@ -349,6 +371,19 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
|
||||
}
|
||||
})
|
||||
|
||||
// Bundle stamps: ?v=<content-hash> per bundle, computed at startup —
|
||||
// a rebuild that changes a bundle's bytes changes its cache key with
|
||||
// no manual stamp bump (see templates/bundles.go; the manual ?v=N
|
||||
// stamp is how a fixed search dropdown stayed broken for a deploy).
|
||||
for _, bundle := range []string{"main.js", "reader.js"} {
|
||||
if data, err := os.ReadFile(filepath.Join("web/static", bundle)); err == nil {
|
||||
sum := sha256.Sum256(data)
|
||||
templates.SetBundleVersion(bundle, hex.EncodeToString(sum[:])[:8])
|
||||
} else {
|
||||
log.Printf("bundle %s not found for stamping: %v", bundle, err)
|
||||
}
|
||||
}
|
||||
|
||||
e.Static("/static", "web/static")
|
||||
|
||||
// Catch-all 404 handler - must be last
|
||||
@@ -364,6 +399,14 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
|
||||
// Start background tasks (queue processor and connection cleanup)
|
||||
scannerHandler.StartBackgroundTasks()
|
||||
|
||||
// Podcast subsystem background loops: sequential download worker +
|
||||
// feed poller (plan §7). The poller re-reads its interval setting
|
||||
// every tick; both exit with the server context.
|
||||
if cfg.PodcastWorker != nil && cfg.PodcastService != nil {
|
||||
cfg.PodcastWorker.Start(context.Background())
|
||||
services.NewPodcastPoller(cfg.Queries, cfg.PodcastService, cfg.PodcastWorker, cfg.Settings).Start(context.Background())
|
||||
}
|
||||
|
||||
// Start watch mode for all libraries (after 2 second delay for DB)
|
||||
go func() {
|
||||
time.Sleep(2 * time.Second)
|
||||
@@ -397,7 +440,15 @@ func RegisterRoutes(cfg *Config) *handlers.Handler {
|
||||
// startup scans for disk I/O.
|
||||
go func() {
|
||||
time.Sleep(30 * time.Second)
|
||||
services.NewHashBackfillService(cfg.Queries).Run(context.Background())
|
||||
// RLS plan §6: this self-heal sweeps ALL media — run it as the
|
||||
// first admin (the scanner's documented fallback identity);
|
||||
// a bare Background context makes governed reads raise.
|
||||
adminID, err := cfg.Queries.GetFirstAdmin(context.Background())
|
||||
if err != nil {
|
||||
log.Printf("[HASH-BACKFILL] no admin found, skipping backfill: %v", err)
|
||||
return
|
||||
}
|
||||
services.NewHashBackfillService(cfg.Queries).Run(rls.WithUser(context.Background(), adminID))
|
||||
}()
|
||||
|
||||
// Register progress routes with actual handler
|
||||
|
||||
@@ -90,9 +90,15 @@ func handleSearchHTML(c *echo.Context, cfg *Config) error {
|
||||
bookInfoList = handlers.MarkActiveConflicts(c.Request().Context(), cfg.Queries, user.ID, bookInfoList)
|
||||
// Render using BooksGrid template (or BookPickerGrid for collection picker)
|
||||
var buf bytes.Buffer
|
||||
if c.QueryParam("show_checkbox") == "true" {
|
||||
switch {
|
||||
case c.QueryParam("show_checkbox") == "true":
|
||||
err = templates.BookPickerGrid(bookInfoList).Render(c.Request().Context(), &buf)
|
||||
} else {
|
||||
case c.QueryParam("layout") == "dropdown":
|
||||
// Header-search dropdown: compact templ rows (rendering
|
||||
// ownership) carrying the viewer's progress/completed overlay.
|
||||
// Replaces the old innerHTML string renderer in search.ts.
|
||||
err = templates.SearchResults(bookInfoList, c.QueryParam("q")).Render(c.Request().Context(), &buf)
|
||||
default:
|
||||
err = templates.BooksGrid(bookInfoList, limit, offset, totalCount, libraryID).Render(c.Request().Context(), &buf)
|
||||
}
|
||||
if err != nil {
|
||||
|
||||
@@ -29,6 +29,12 @@ const (
|
||||
LibraryTypeComics = "comics"
|
||||
LibraryTypeManga = "manga"
|
||||
LibraryTypeAudiobooks = "audiobooks"
|
||||
// Podcasts libraries are feed containers (RSS subscribe+download),
|
||||
// not FS-scanned media: AllowedExtensions is deliberately empty —
|
||||
// the scanner skips these folders wholesale (podcast downloads are
|
||||
// written by the fetcher's download worker, never discovered by
|
||||
// the walker). See podcast_service.go.
|
||||
LibraryTypePodcasts = "podcasts"
|
||||
)
|
||||
|
||||
var AllowedExtensions = map[string][]string{
|
||||
@@ -43,6 +49,9 @@ var AllowedExtensions = map[string][]string{
|
||||
// browser-playable lossless/next-gen containers join for Batch B
|
||||
// metadata parity (.wma deliberately absent: browsers can't play it).
|
||||
LibraryTypeAudiobooks: {".m4b", ".m4a", ".mp3", ".flac", ".zip", ".opus", ".ogg", ".oga", ".aac", ".wav"},
|
||||
// Empty BY DESIGN: nothing in a podcasts library is scanner-facing.
|
||||
// The scanner additionally skips these folders outright.
|
||||
LibraryTypePodcasts: {},
|
||||
}
|
||||
|
||||
var MimeTypes = map[string]string{
|
||||
@@ -149,10 +158,39 @@ func (s *LibraryService) DeleteLibrary(ctx context.Context, libraryID pgtype.UUI
|
||||
}
|
||||
|
||||
// AddLibraryFolder adds a folder to a library
|
||||
// Folder purposes (library_folders.purpose): 'scan' roots feed the FS
|
||||
// walker; 'podcast' roots are download targets written only by the
|
||||
// podcast download worker.
|
||||
const (
|
||||
FolderPurposeScan = "scan"
|
||||
FolderPurposePodcast = "podcast"
|
||||
)
|
||||
|
||||
// AddLibraryFolder registers a folder against a library. The purpose is
|
||||
// derived from the library type: podcasts libraries register their
|
||||
// download root (purpose='podcast', exactly ONE folder — feeds lay out
|
||||
// <download-root>/<feed-slug>/...); every other type registers a
|
||||
// scanner root (purpose='scan').
|
||||
func (s *LibraryService) AddLibraryFolder(ctx context.Context, libraryID pgtype.UUID, folderPath string) (*database.LibraryFolders, error) {
|
||||
purpose := FolderPurposeScan
|
||||
|
||||
lib, err := s.db.GetLibrary(ctx, libraryID)
|
||||
if err == nil && lib.TypeName == LibraryTypePodcasts {
|
||||
// Podcasts libraries own exactly one folder: the download root.
|
||||
existing, err := s.db.GetLibraryFolders(ctx, libraryID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(existing) > 0 {
|
||||
return nil, fmt.Errorf("podcasts libraries have a single download root; %s is already registered", existing[0].FolderPath)
|
||||
}
|
||||
purpose = FolderPurposePodcast
|
||||
}
|
||||
|
||||
folder, err := s.db.AddLibraryFolder(ctx, database.AddLibraryFolderParams{
|
||||
LibraryID: libraryID,
|
||||
FolderPath: folderPath,
|
||||
Purpose: purpose,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -43,6 +43,8 @@ import (
|
||||
epub "github.com/ArcadiaLin/go-epub"
|
||||
"github.com/fsnotify/fsnotify"
|
||||
"github.com/google/uuid"
|
||||
|
||||
"bookhoard/internal/rls"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/nwaples/rardecode"
|
||||
@@ -502,6 +504,12 @@ func (s *MediaScanner) GetStats() (int, int, int) {
|
||||
return s.totalFiles, s.newItems, s.errors
|
||||
}
|
||||
|
||||
// FolderCount reports the active root-folder count (test/audit helper:
|
||||
// the podcast-exclusion pin asserts it stays 0 for podcasts roots).
|
||||
func (s *MediaScanner) FolderCount() int {
|
||||
return len(s.folders)
|
||||
}
|
||||
|
||||
// SetFolders configures the scanner's folders and (optionally) sets up an
|
||||
// fsnotify watcher over the full directory tree.
|
||||
//
|
||||
@@ -510,9 +518,34 @@ func (s *MediaScanner) GetStats() (int, int, int) {
|
||||
// watcher (and thus no fd/inotify watches) is allocated — the watcher is never
|
||||
// read by scan jobs and previously leaked one watcher per job.
|
||||
func (s *MediaScanner) SetFolders(folders []string, watch bool) error {
|
||||
s.folders = folders
|
||||
// Podcasts libraries are feed-managed (plan: podcast-integration.md
|
||||
// task 1.2, invariant 4): their download roots are written by the
|
||||
// download worker, never discovered by the walker — including
|
||||
// scoped single-folder scans (0f1d0abb), which all funnel through
|
||||
// here. Filter them out before anything else touches the list.
|
||||
scannable := folders[:0:0]
|
||||
for _, folder := range folders {
|
||||
lib, err := s.db.GetLibraryByFolder(context.Background(), folder)
|
||||
if err != nil {
|
||||
// Unresolvable folders keep the old behavior (scan walk
|
||||
// skips them per-file; watch adds them harmlessly).
|
||||
scannable = append(scannable, folder)
|
||||
continue
|
||||
}
|
||||
libType, err := s.db.GetLibraryType(context.Background(), lib.LibraryTypeID)
|
||||
if err != nil {
|
||||
scannable = append(scannable, folder)
|
||||
continue
|
||||
}
|
||||
if libType.Name == LibraryTypePodcasts {
|
||||
fmt.Printf("Scanner: skipping %s (podcasts library %q — feed-managed, not scanned)\n", folder, lib.Name)
|
||||
continue
|
||||
}
|
||||
scannable = append(scannable, folder)
|
||||
}
|
||||
folders = scannable
|
||||
|
||||
// Always close any previously-owned watcher so reconfiguration doesn't leak.
|
||||
s.folders = folders // Always close any previously-owned watcher so reconfiguration doesn't leak.
|
||||
if s.watcher != nil {
|
||||
if err := s.watcher.Close(); err != nil {
|
||||
fmt.Printf("Warning: failed to close old watcher during folder reconfiguration: %v\n", err)
|
||||
@@ -522,6 +555,12 @@ func (s *MediaScanner) SetFolders(folders []string, watch bool) error {
|
||||
|
||||
// Create + populate a fresh watcher only when the caller intends to read events.
|
||||
if watch {
|
||||
if len(folders) == 0 {
|
||||
// All requested folders filtered (e.g. a podcasts library:
|
||||
// nothing scannable, nothing to watch). Fail loudly at the
|
||||
// API boundary instead of panicking on folders[0] below.
|
||||
return fmt.Errorf("no scannable folders (podcasts libraries are feed-managed and cannot be scanned)")
|
||||
}
|
||||
watcher, err := fsnotify.NewWatcher()
|
||||
if err != nil {
|
||||
// Return an error instead of panicking so a failed watcher can't
|
||||
@@ -621,17 +660,23 @@ func (s *MediaScanner) enqueueLibraryScan(rootFolder, trigger string) {
|
||||
folderPaths[i] = f.FolderPath
|
||||
}
|
||||
|
||||
// The acting admin: the library owner, or the first admin as the
|
||||
// documented fallback (existing pattern). Used BOTH for the job's
|
||||
// UserID and — since RLS (plan §6) — as the identity the job's
|
||||
// media reads run under via its Context.
|
||||
adminIDStr := ""
|
||||
if libRow.CreatedByAdminID.Valid {
|
||||
adminIDStr = uuid.UUID(libRow.CreatedByAdminID.Bytes).String()
|
||||
adminUUID := libRow.CreatedByAdminID
|
||||
if adminUUID.Valid {
|
||||
adminIDStr = uuid.UUID(adminUUID.Bytes).String()
|
||||
}
|
||||
if adminIDStr == "" {
|
||||
if !adminUUID.Valid {
|
||||
fmt.Printf("[MTIME-POLL] Library has no owner, falling back to first admin\n")
|
||||
fallbackAdmin, err := s.db.GetFirstAdmin(context.Background())
|
||||
if err != nil {
|
||||
fmt.Printf("[MTIME-POLL] Warning: no admin found in database, skipping scan\n")
|
||||
return
|
||||
}
|
||||
adminUUID = fallbackAdmin
|
||||
adminIDStr = uuid.UUID(fallbackAdmin.Bytes).String()
|
||||
}
|
||||
|
||||
@@ -650,7 +695,7 @@ func (s *MediaScanner) enqueueLibraryScan(rootFolder, trigger string) {
|
||||
Type: JobTypeScan,
|
||||
Status: JobStatusPending,
|
||||
UserID: adminIDStr,
|
||||
Context: context.Background(),
|
||||
Context: rls.WithUser(context.Background(), adminUUID),
|
||||
Params: map[string]any{
|
||||
"library_id": libraryIDStr,
|
||||
"folders": folderPaths,
|
||||
@@ -667,6 +712,13 @@ func (s *MediaScanner) enqueueLibraryScan(rootFolder, trigger string) {
|
||||
}
|
||||
|
||||
func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
// RLS plan §6: a scan is server maintenance over library media —
|
||||
// self-declare the scanner's configured admin identity so internal
|
||||
// reads work regardless of the caller's context (worker jobs carry
|
||||
// one; test/direct callers may not). No-op when unconfigured.
|
||||
if s.adminID.Valid {
|
||||
ctx = rls.WithUser(ctx, s.adminID)
|
||||
}
|
||||
// Cancellation is checked at every stage boundary and inside the
|
||||
// walk callback: the in-flight file finishes, then the scan aborts
|
||||
// and reports context.Canceled (processJob classifies that as a
|
||||
@@ -831,18 +883,47 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
// retention window (ARCHIVE_RETENTION_DAYS; 0 = manual purge only).
|
||||
// Every branch logs - the previous hard-delete cleanup failed silently
|
||||
// and left orphaned rows undetected.
|
||||
// Group this scan's folders by owning library: the reconciliation is
|
||||
// per LIBRARY, comparing every library item against the union of all
|
||||
// its folders' walks. The old per-folder loop listed the whole
|
||||
// library's items against each single folder's walk, so with two
|
||||
// folders every item living under the other folder was flagged
|
||||
// missing on every scan (prod: 743/743 audiobooks stuck at
|
||||
// "missing (1/2)", unarchivable only because the walk kept clearing
|
||||
// them) — and with three or more folders earlier iterations' marks
|
||||
// crossed later ones' walks and archived real content mid-pass.
|
||||
scannedSet := make(map[string]bool, len(s.folders))
|
||||
for _, scanned := range s.folders {
|
||||
scannedSet[scanned] = true
|
||||
}
|
||||
type archiveScope struct {
|
||||
libraryID pgtype.UUID
|
||||
name string
|
||||
folders []string
|
||||
}
|
||||
scopes := make([]archiveScope, 0, len(s.folders))
|
||||
scopeIndex := make(map[pgtype.UUID]int, len(s.folders))
|
||||
for _, folder := range s.folders {
|
||||
lib, err := s.db.GetLibraryByFolder(ctx, folder)
|
||||
if err != nil {
|
||||
fmt.Printf("[ARCHIVE] Warning: no library found for folder %s, skipping it in archive pass: %v\n", folder, err)
|
||||
continue
|
||||
}
|
||||
idx, seen := scopeIndex[lib.LibraryID]
|
||||
if !seen {
|
||||
idx = len(scopes)
|
||||
scopeIndex[lib.LibraryID] = idx
|
||||
scopes = append(scopes, archiveScope{libraryID: lib.LibraryID, name: lib.Name})
|
||||
}
|
||||
scopes[idx].folders = append(scopes[idx].folders, folder)
|
||||
}
|
||||
|
||||
for _, scope := range scopes {
|
||||
if ctx.Err() != nil {
|
||||
fmt.Printf("[ARCHIVE] Scan cancelled, stopping archive pass\n")
|
||||
return ctx.Err()
|
||||
}
|
||||
|
||||
lib, err := s.db.GetLibraryByFolder(ctx, folder)
|
||||
if err != nil {
|
||||
fmt.Printf("[ARCHIVE] Warning: no library found for folder %s, skipping archive pass: %v\n", folder, err)
|
||||
continue
|
||||
}
|
||||
libraryID := lib.LibraryID
|
||||
libraryID := scope.libraryID
|
||||
|
||||
// Partial-scope guard (per-folder scans, 2026-10): the
|
||||
// reconciliation compares EVERY library item against this scan's
|
||||
@@ -855,10 +936,6 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
fmt.Printf("[ARCHIVE] Warning: failed to list folders for library, skipping archive pass: %v\n", regErr)
|
||||
continue
|
||||
}
|
||||
scannedSet := make(map[string]bool, len(s.folders))
|
||||
for _, scanned := range s.folders {
|
||||
scannedSet[scanned] = true
|
||||
}
|
||||
covered := 0
|
||||
for _, rf := range registeredFolders {
|
||||
if scannedSet[rf.FolderPath] {
|
||||
@@ -867,7 +944,7 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
}
|
||||
if covered < len(registeredFolders) {
|
||||
fmt.Printf("[ARCHIVE] Skipping reconciliation for %s: scan covered %d of %d registered folders\n",
|
||||
lib.Name, covered, len(registeredFolders))
|
||||
scope.name, covered, len(registeredFolders))
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -877,8 +954,11 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
continue
|
||||
}
|
||||
|
||||
// Build set of scanned file paths for this folder
|
||||
// Build the set of scanned paths across ALL of this library's
|
||||
// folders.
|
||||
scannedPaths := make(map[string]bool)
|
||||
walkFailed := false
|
||||
for _, folder := range scope.folders {
|
||||
if err := filepath.WalkDir(folder, func(path string, d os.DirEntry, walkErr error) error {
|
||||
if walkErr != nil {
|
||||
return walkErr
|
||||
@@ -889,14 +969,24 @@ func (s *MediaScanner) ScanFolders(ctx context.Context) error {
|
||||
return nil
|
||||
}); err != nil {
|
||||
fmt.Printf("[ARCHIVE] Warning: failed to walk directory %s, skipping archive pass: %v\n", folder, err)
|
||||
continue // Skip to next folder to avoid false archivals
|
||||
walkFailed = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if walkFailed {
|
||||
continue // Skip this library to avoid false archivals
|
||||
}
|
||||
|
||||
// Folder-book roots count as present: the item's file_path is the
|
||||
// DIRECTORY, which the file walk above never emits.
|
||||
// DIRECTORY, which the file walk above never emits. Attribution is
|
||||
// boundary-safe — a sibling folder whose name merely extends this
|
||||
// one's must not contribute its roots here.
|
||||
for root := range s.audioBookRoots {
|
||||
if strings.HasPrefix(root, folder) {
|
||||
for _, folder := range scope.folders {
|
||||
if folderContains(folder, root) {
|
||||
scannedPaths[s.getRelativePath(root)] = true
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -954,7 +1044,7 @@ func (s *MediaScanner) isScannableFile(path string) bool {
|
||||
// Find which folder this file belongs to
|
||||
var folder string
|
||||
for _, f := range s.folders {
|
||||
if strings.HasPrefix(path, f) {
|
||||
if folderContains(f, path) {
|
||||
folder = f
|
||||
break
|
||||
}
|
||||
@@ -1117,7 +1207,7 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
|
||||
folderLibrary := s.folderLibrary
|
||||
s.scanStateMu.RUnlock()
|
||||
for _, folder := range s.folders {
|
||||
if !strings.HasPrefix(path, folder) {
|
||||
if !folderContains(folder, path) {
|
||||
continue
|
||||
}
|
||||
if libID, cached := folderLibrary[folder]; cached {
|
||||
@@ -1260,7 +1350,7 @@ func (s *MediaScanner) processMediaFile(ctx context.Context, path string) (bool,
|
||||
// Use the root folder that contains this file
|
||||
var rootFolder string
|
||||
for _, folder := range s.folders {
|
||||
if strings.HasPrefix(path, folder) {
|
||||
if folderContains(folder, path) {
|
||||
rootFolder = folder
|
||||
break
|
||||
}
|
||||
@@ -1773,23 +1863,24 @@ func applyTitleDerivedSeries(metadata *MediaMetadata) {
|
||||
|
||||
// extractCalibreSidecar checks for and parses a Calibre metadata.opf sidecar file
|
||||
func (s *MediaScanner) extractCalibreSidecar(path string) *MediaMetadata {
|
||||
// Get directory of media file
|
||||
// Per-file sidecar FIRST (<basename>.opf, the writer's multi-book
|
||||
// safe name), then Calibre's dir-scoped metadata.opf.
|
||||
dir := filepath.Dir(path)
|
||||
opfPath := filepath.Join(dir, "metadata.opf")
|
||||
|
||||
// Check if sidecar exists
|
||||
if _, err := os.Stat(opfPath); os.IsNotExist(err) {
|
||||
return nil // No sidecar, not an error
|
||||
base := strings.TrimSuffix(filepath.Base(path), filepath.Ext(path))
|
||||
for _, name := range []string{base + ".opf", "metadata.opf"} {
|
||||
opfPath := filepath.Join(dir, name)
|
||||
if _, err := os.Stat(opfPath); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
// Parse sidecar
|
||||
metadata, err := s.parseCalibreMetadataOPF(opfPath)
|
||||
if err != nil {
|
||||
fmt.Printf("Warning: failed to parse Calibre metadata.opf: %v\n", err)
|
||||
return nil // Parsing failed, fall back to embedded
|
||||
continue // fall back to the next candidate / embedded
|
||||
}
|
||||
|
||||
return metadata
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// applyEmbeddedCoverFallback extracts a cover from the media file itself when
|
||||
@@ -4051,7 +4142,7 @@ func (s *MediaScanner) updateMediaItemWithMetadata(ctx context.Context, existing
|
||||
// could clear series/author that a sidecar didn't carry (Libation
|
||||
// writes "series": null while the folder layout knows the series).
|
||||
for _, folder := range s.folders {
|
||||
if strings.HasPrefix(path, folder) {
|
||||
if folderContains(folder, path) {
|
||||
if folderMetadata := s.extractFolderStructureMetadata(path, folder); folderMetadata != nil {
|
||||
if metadata.Title == "" && folderMetadata.Title != "" {
|
||||
metadata.Title = folderMetadata.Title
|
||||
@@ -4732,7 +4823,7 @@ func (s *MediaScanner) performInitialScan(ctx context.Context) {
|
||||
Type: JobTypeScan,
|
||||
Status: JobStatusPending,
|
||||
UserID: adminIDStr,
|
||||
Context: context.Background(),
|
||||
Context: rls.WithUser(ctx, s.adminID), // RLS plan §6: startup scans run as the configured admin
|
||||
Params: map[string]any{
|
||||
"library_id": libraryIDStr,
|
||||
"folders": folderPaths,
|
||||
@@ -5060,13 +5151,27 @@ func (s *MediaScanner) extractHashInfo(filePath string) (*HashInfo, *FormatInfo,
|
||||
return hashInfo, formatInfo, nil
|
||||
}
|
||||
|
||||
// folderContains reports whether path is folder itself or lies beneath it
|
||||
// at a directory boundary. This is the only path-correct containment test:
|
||||
// a bare strings.HasPrefix(path, folder) also matches string-prefix
|
||||
// siblings ("/app/audiobooks" claims "/app/audiobookssd/..."), which once
|
||||
// sliced a stray "sd/" into every stored relative path for that library
|
||||
// and resolved those files against the wrong root (folder ownership,
|
||||
// library attribution, extension allowlisting, cover 404s).
|
||||
func folderContains(folder, path string) bool {
|
||||
folder = filepath.Clean(folder)
|
||||
return path == folder || strings.HasPrefix(path, folder+string(filepath.Separator))
|
||||
}
|
||||
|
||||
func (s *MediaScanner) getRelativePath(absolutePath string) string {
|
||||
// Get the base folder paths from scanner
|
||||
for _, baseFolder := range s.folders {
|
||||
// Check if path is within this base folder
|
||||
if relPath, ok := strings.CutPrefix(absolutePath, baseFolder); ok {
|
||||
return strings.TrimPrefix(relPath, "/")
|
||||
// Check if path is within this base folder — boundary-aware, so a
|
||||
// sibling whose name merely extends ours can never be cut.
|
||||
if !folderContains(baseFolder, absolutePath) {
|
||||
continue
|
||||
}
|
||||
return strings.TrimPrefix(absolutePath[len(filepath.Clean(baseFolder)):], "/")
|
||||
}
|
||||
// Fallback: if no match, return as-is (shouldn't happen)
|
||||
return absolutePath
|
||||
|
||||
@@ -178,7 +178,7 @@ func (s *MediaScanner) maybeProcessAudioBook(ctx context.Context, path string) (
|
||||
// so any scannable audio file belongs to one.
|
||||
rootFolder := ""
|
||||
for _, folder := range s.folders {
|
||||
if strings.HasPrefix(path, folder) {
|
||||
if folderContains(folder, path) {
|
||||
rootFolder = folder
|
||||
break
|
||||
}
|
||||
@@ -731,7 +731,7 @@ func (s *MediaScanner) rescanMultiFileAudioBook(ctx context.Context, item databa
|
||||
|
||||
rootFolder := ""
|
||||
for _, folder := range s.folders {
|
||||
if strings.HasPrefix(fullPath, folder) {
|
||||
if folderContains(folder, fullPath) {
|
||||
rootFolder = folder
|
||||
break
|
||||
}
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
package services
|
||||
|
||||
import "testing"
|
||||
|
||||
// Regression tests for directory-boundary folder matching.
|
||||
//
|
||||
// getRelativePath (and the folder-membership checks across the scanner)
|
||||
// once used bare string-prefix matching, so a registered folder whose path
|
||||
// was a string prefix of a sibling's ("/app/audiobooks" vs
|
||||
// "/app/audiobookssd") claimed the sibling's files: relative paths were
|
||||
// cut mid-component into "sd/..."-style garbage, stored that way, and
|
||||
// resolved against the wrong root — covers 404ing while the bytes sat one
|
||||
// directory away. folderContains pins the rule: a folder only ever claims
|
||||
// itself or paths beneath it at a real separator boundary, regardless of
|
||||
// names, registration order, or trailing slashes.
|
||||
|
||||
func TestFolderContains(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
folder string
|
||||
path string
|
||||
want bool
|
||||
}{
|
||||
{"exact root", "/app/books", "/app/books", true},
|
||||
{"direct child", "/app/books", "/app/books/x.epub", true},
|
||||
{"deep child", "/app/books", "/app/books/a/b/c.epub", true},
|
||||
{"sibling extension", "/app/books", "/app/books2", false},
|
||||
{"sibling extension child", "/app/books", "/app/books2/x.epub", false},
|
||||
{"mid-word sibling", "/data/lib", "/data/library/x.epub", false},
|
||||
{"suffix without separator", "/app/books", "/app/books-backup/x.epub", false},
|
||||
{"unrelated", "/app/books", "/opt/other/x.epub", false},
|
||||
{"prefix inside path segment", "/app/books", "/app/series/books/x.epub", false},
|
||||
{"trailing slash in folder", "/app/books/", "/app/books/x.epub", true},
|
||||
{"trailing slash exact", "/app/books/", "/app/books", true},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := folderContains(tt.folder, tt.path); got != tt.want {
|
||||
t.Errorf("folderContains(%q, %q) = %v, want %v", tt.folder, tt.path, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetRelativePathFolderBoundary(t *testing.T) {
|
||||
// The prod trap: /app/audiobooks registered before /app/audiobookssd.
|
||||
// String-prefix cutting produced "sd/Brian Jacques/..." for every file
|
||||
// in the second folder; the boundary rule must make that impossible.
|
||||
newScanner := func(folders []string) *MediaScanner {
|
||||
s := NewMediaScanner(nil)
|
||||
s.folders = folders // SetFolders requires a live DB; tests only need path relativization
|
||||
return s
|
||||
}
|
||||
|
||||
siblingFolders := []string{"/app/audiobooks", "/app/audiobookssd"}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
folders []string
|
||||
path string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "sibling-prefix folder keeps its full name in relative path",
|
||||
folders: siblingFolders,
|
||||
path: "/app/audiobookssd/Brian Jacques/Redwall/Book 01 - Redwall/file.m4b",
|
||||
want: "Brian Jacques/Redwall/Book 01 - Redwall/file.m4b",
|
||||
},
|
||||
{
|
||||
name: "first registered folder unaffected",
|
||||
folders: siblingFolders,
|
||||
path: "/app/audiobooks/Author/Series/file.mp3",
|
||||
want: "Author/Series/file.mp3",
|
||||
},
|
||||
{
|
||||
name: "exact root yields empty path",
|
||||
folders: siblingFolders,
|
||||
path: "/app/audiobookssd",
|
||||
want: "",
|
||||
},
|
||||
{
|
||||
name: "order independence: sibling registered first",
|
||||
folders: []string{"/app/audiobookssd", "/app/audiobooks"},
|
||||
path: "/app/audiobookssd/Brian Jacques/Redwall/Book 01 - Redwall/file.m4b",
|
||||
want: "Brian Jacques/Redwall/Book 01 - Redwall/file.m4b",
|
||||
},
|
||||
{
|
||||
name: "order independence: first folder still correct when second registered first",
|
||||
folders: []string{"/app/audiobookssd", "/app/audiobooks"},
|
||||
path: "/app/audiobooks/Author/Series/file.mp3",
|
||||
want: "Author/Series/file.mp3",
|
||||
},
|
||||
{
|
||||
name: "trailing slash on registered folders",
|
||||
folders: []string{"/app/audiobooks/", "/app/audiobookssd/"},
|
||||
path: "/app/audiobookssd/Brian Jacques/file.m4b",
|
||||
want: "Brian Jacques/file.m4b",
|
||||
},
|
||||
{
|
||||
name: "mid-word sibling folders",
|
||||
folders: []string{"/data/lib", "/data/library"},
|
||||
path: "/data/library/author/book.epub",
|
||||
want: "author/book.epub",
|
||||
},
|
||||
{
|
||||
name: "author/series/title layout preserved under true root",
|
||||
folders: siblingFolders,
|
||||
path: "/app/audiobookssd/Brian Jacques/Redwall/Book 14 - The Taggerung/Taggerung.mp3",
|
||||
want: "Brian Jacques/Redwall/Book 14 - The Taggerung/Taggerung.mp3",
|
||||
},
|
||||
{
|
||||
name: "path outside every folder returned as-is",
|
||||
folders: siblingFolders,
|
||||
path: "/app/audiobooksBackup/x.mp3",
|
||||
want: "/app/audiobooksBackup/x.mp3",
|
||||
},
|
||||
{
|
||||
name: "unrelated path returned as-is",
|
||||
folders: siblingFolders,
|
||||
path: "/elsewhere/x.mp3",
|
||||
want: "/elsewhere/x.mp3",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
s := newScanner(tt.folders)
|
||||
if got := s.getRelativePath(tt.path); got != tt.want {
|
||||
t.Errorf("getRelativePath(%q) with folders %v = %q, want %q", tt.path, tt.folders, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,947 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// MetadataWriter persists the item's CURRENT database metadata back into
|
||||
// the artifact on disk, so the library stays portable (Calibre, phone
|
||||
// players, other servers). 2026-10-27 feature (user request): embed into
|
||||
// formats that carry metadata, write a Calibre-style sidecar for those
|
||||
// that don't.
|
||||
//
|
||||
// Safety contract:
|
||||
// - temp file + atomic rename; the original is NEVER truncated on
|
||||
// failure (a failed write must not destroy a book)
|
||||
// - the write mirrors what the scanner's read chain parses
|
||||
// (parseOPFContent), so a write + rescan round-trips
|
||||
// - metadata_overrides are left untouched: after a successful write
|
||||
// the file and the DB agree, and the overrides keep stale sidecars
|
||||
// from clobbering later rescans
|
||||
//
|
||||
// Consequences that are inherent to editing files in place (documented
|
||||
// in docs/user/calibre-integration.md): the mtime change makes the
|
||||
// watcher re-extract (wanted), and the SHA-256 changes (dedup/move
|
||||
// detection re-keys; Kobo device sidecars re-key on next sync).
|
||||
|
||||
// MetadataWriteTarget names where a format's metadata landed.
|
||||
type MetadataWriteTarget string
|
||||
|
||||
const (
|
||||
// MetadataTargetEmbeddedOPF: the OPF inside the EPUB was rewritten.
|
||||
MetadataTargetEmbeddedOPF MetadataWriteTarget = "embedded-opf"
|
||||
// MetadataTargetSidecarOPF: a Calibre-style sidecar was written next
|
||||
// to the file (formats that can't carry metadata: PDF, comics,
|
||||
// folder books).
|
||||
MetadataTargetSidecarOPF MetadataWriteTarget = "sidecar-opf"
|
||||
// MetadataTargetAudioTags: the container's own tags were rewritten
|
||||
// via ffmpeg stream-copy (single-file MP3/FLAC/M4A/M4B — 5b).
|
||||
MetadataTargetAudioTags MetadataWriteTarget = "audio-tags"
|
||||
// MetadataTargetComicInfo: ComicInfo.xml inside the CBZ was
|
||||
// injected/replaced (5c). CBR/CB7 stay on the sidecar path — RAR
|
||||
// and 7z archives can't be rewritten in place.
|
||||
MetadataTargetComicInfo MetadataWriteTarget = "comicinfo"
|
||||
)
|
||||
|
||||
// audioTagExts is the single-file set the tag writer claims; multi-file
|
||||
// (folder) audio stays on the sidecar path.
|
||||
var audioTagExts = map[string]bool{
|
||||
".mp3": true, ".flac": true, ".m4a": true, ".m4b": true,
|
||||
}
|
||||
|
||||
// MetadataWriteResult reports what the write did, for the UI toast and
|
||||
// the response contract.
|
||||
type MetadataWriteResult struct {
|
||||
Target MetadataWriteTarget `json:"target"`
|
||||
Path string `json:"path"` // absolute path of the artifact written
|
||||
Sidecar bool `json:"sidecar"`
|
||||
FieldsWritten []string `json:"fields_written"`
|
||||
}
|
||||
|
||||
// MetadataWriteTargetFor maps a file extension to the write target.
|
||||
// The UI mirrors this to label the button; keep both in lockstep.
|
||||
func MetadataWriteTargetFor(ext string) MetadataWriteTarget {
|
||||
ext = strings.ToLower(strings.TrimSpace(ext))
|
||||
switch {
|
||||
case ext == ".epub" || ext == ".kepub":
|
||||
return MetadataTargetEmbeddedOPF
|
||||
case audioTagExts[ext]:
|
||||
return MetadataTargetAudioTags
|
||||
case ext == ".cbz":
|
||||
return MetadataTargetComicInfo
|
||||
default:
|
||||
// CBR/CB7 (no in-place rewrite) and every other format take
|
||||
// the sidecar.
|
||||
return MetadataTargetSidecarOPF
|
||||
}
|
||||
}
|
||||
|
||||
// MetadataWriter carries the dependencies for file-writing operations.
|
||||
type MetadataWriter struct {
|
||||
db *database.Queries
|
||||
libraries *LibraryService
|
||||
}
|
||||
|
||||
func NewMetadataWriter(db *database.Queries) *MetadataWriter {
|
||||
return &MetadataWriter{db: db, libraries: NewLibraryService(db)}
|
||||
}
|
||||
|
||||
// WriteMetadataToFile dispatches on the item's format and persists the
|
||||
// current DB values into the artifact on disk.
|
||||
func (w *MetadataWriter) WriteMetadataToFile(ctx context.Context, itemID uuid.UUID) (*MetadataWriteResult, error) {
|
||||
item, err := w.db.GetMediaItem(ctx, pgtype.UUID{Bytes: itemID, Valid: true})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("media item not found: %w", err)
|
||||
}
|
||||
|
||||
// Edit segregation (plan D10/D9, task 1.8): podcast episodes are
|
||||
// feed-managed; writing their metadata into the file would be
|
||||
// clobbered by the next download/refresh and violates the
|
||||
// fetcher's sole-writer rule.
|
||||
if item.Source == "podcast_episode" {
|
||||
return nil, fmt.Errorf("podcast episodes are managed by their feed; metadata write-to-file is not available")
|
||||
}
|
||||
|
||||
absPath, err := w.libraries.ResolveMediaPath(ctx, item.LibraryID, item.FilePath)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("cannot resolve library folder: %w", err)
|
||||
}
|
||||
if _, err := os.Stat(absPath); err != nil {
|
||||
return nil, fmt.Errorf("media file missing on disk: %w", err)
|
||||
}
|
||||
|
||||
switch MetadataWriteTargetFor(filepath.Ext(absPath)) {
|
||||
case MetadataTargetEmbeddedOPF:
|
||||
path, err := w.writeEPUBMetadata(absPath, &item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MetadataWriteResult{
|
||||
Target: MetadataTargetEmbeddedOPF,
|
||||
Path: path,
|
||||
FieldsWritten: metadataFieldNames(&item),
|
||||
}, nil
|
||||
case MetadataTargetAudioTags:
|
||||
path, err := w.writeAudioTags(absPath, &item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MetadataWriteResult{
|
||||
Target: MetadataTargetAudioTags,
|
||||
Path: path,
|
||||
FieldsWritten: audioFieldNames(&item, filepath.Ext(absPath)),
|
||||
}, nil
|
||||
case MetadataTargetComicInfo:
|
||||
path, err := w.writeCBZComicInfo(absPath, &item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MetadataWriteResult{
|
||||
Target: MetadataTargetComicInfo,
|
||||
Path: path,
|
||||
FieldsWritten: comicFieldNames(&item),
|
||||
}, nil
|
||||
default:
|
||||
path, err := w.writeSidecarOPF(ctx, absPath, &item)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &MetadataWriteResult{
|
||||
Target: MetadataTargetSidecarOPF,
|
||||
Path: path,
|
||||
Sidecar: true,
|
||||
FieldsWritten: metadataFieldNames(&item),
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
// writeSidecarOPF writes a Calibre-style OPF next to the media file.
|
||||
// Naming: `metadata.opf` (Calibre's own convention) when the item is
|
||||
// the only book in its directory; otherwise `<basename>.opf` so one
|
||||
// book's sidecar can never poison its dir-mates' scans. The scanner
|
||||
// reads both (basename first).
|
||||
func (w *MetadataWriter) writeSidecarOPF(ctx context.Context, absPath string, item *database.MediaItems) (string, error) {
|
||||
dir := filepath.Dir(absPath)
|
||||
base := strings.TrimSuffix(filepath.Base(absPath), filepath.Ext(absPath))
|
||||
|
||||
others, err := w.countSiblingBooks(ctx, item)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("cannot check sibling books: %w", err)
|
||||
}
|
||||
name := "metadata.opf"
|
||||
if others > 0 {
|
||||
name = base + ".opf"
|
||||
}
|
||||
sidecarPath := filepath.Join(dir, name)
|
||||
|
||||
doc := buildSidecarOPFXML(item)
|
||||
if err := atomicWrite(sidecarPath, []byte(doc)); err != nil {
|
||||
return "", fmt.Errorf("failed to write sidecar: %w", err)
|
||||
}
|
||||
return sidecarPath, nil
|
||||
}
|
||||
|
||||
// countSiblingBooks counts OTHER non-archived items that live in the
|
||||
// same directory of the same library — the guard against writing a
|
||||
// dir-scoped metadata.opf over a multi-book folder.
|
||||
func (w *MetadataWriter) countSiblingBooks(ctx context.Context, item *database.MediaItems) (int, error) {
|
||||
dir := filepath.ToSlash(filepath.Dir(item.FilePath))
|
||||
siblings, err := w.db.CountOtherMediaItemsInDir(ctx, database.CountOtherMediaItemsInDirParams{
|
||||
LibraryID: item.LibraryID,
|
||||
ID: item.ID,
|
||||
DirPrefix: pgtype.Text{String: dir + "/", Valid: true},
|
||||
})
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return int(siblings), nil
|
||||
}
|
||||
|
||||
// writeEPUBMetadata splices a freshly generated <metadata> block into
|
||||
// the EPUB's existing OPF (manifest/spine/guide preserved untouched)
|
||||
// and rewrites the zip atomically: every other entry is copied
|
||||
// byte-identical, in original order with original compression — the
|
||||
// mimetype entry must stay first and STORED.
|
||||
func (w *MetadataWriter) writeEPUBMetadata(absPath string, item *database.MediaItems) (string, error) {
|
||||
reader, err := zip.OpenReader(absPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to open EPUB: %w", err)
|
||||
}
|
||||
defer reader.Close()
|
||||
|
||||
opfName := findOPFPathInZip(reader.File)
|
||||
if opfName == "" {
|
||||
return "", fmt.Errorf("EPUB has no OPF (container.xml missing rootfile)")
|
||||
}
|
||||
|
||||
newOPF, err := spliceOPFMetadata(reader.File, opfName, item)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
tmpPath := absPath + ".tmp-write"
|
||||
if err := rewriteEPUBZip(reader.File, opfName, newOPF, tmpPath); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", err
|
||||
}
|
||||
if err := os.Rename(tmpPath, absPath); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("failed to replace EPUB: %w", err)
|
||||
}
|
||||
return absPath, nil
|
||||
}
|
||||
|
||||
// spliceOPFMetadata returns the OPF with its <metadata> inner XML
|
||||
// replaced. The ORIGINAL opening tag (with its namespaces and
|
||||
// unique-identifier) is preserved so manifest/spine id references keep
|
||||
// resolving.
|
||||
func spliceOPFMetadata(entries []*zip.File, opfName string, item *database.MediaItems) ([]byte, error) {
|
||||
var opfEntry *zip.File
|
||||
for _, f := range entries {
|
||||
if f.Name == opfName {
|
||||
opfEntry = f
|
||||
break
|
||||
}
|
||||
}
|
||||
if opfEntry == nil {
|
||||
return nil, fmt.Errorf("OPF %q not found in EPUB", opfName)
|
||||
}
|
||||
rc, err := opfEntry.Open()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to open OPF: %w", err)
|
||||
}
|
||||
original, err := io.ReadAll(rc)
|
||||
closeErr := rc.Close()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to read OPF: %w", err)
|
||||
}
|
||||
if closeErr != nil {
|
||||
return nil, fmt.Errorf("failed to close OPF: %w", closeErr)
|
||||
}
|
||||
|
||||
metaStart := bytes.Index(original, []byte("<metadata"))
|
||||
if metaStart == -1 {
|
||||
return nil, fmt.Errorf("OPF has no <metadata> element")
|
||||
}
|
||||
openEnd := bytes.Index(original[metaStart:], []byte(">"))
|
||||
if openEnd == -1 {
|
||||
return nil, fmt.Errorf("OPF <metadata> tag malformed")
|
||||
}
|
||||
metaClose := bytes.Index(original[metaStart:], []byte("</metadata>"))
|
||||
if metaClose == -1 {
|
||||
return nil, fmt.Errorf("OPF has no </metadata> close")
|
||||
}
|
||||
|
||||
openTag := original[metaStart : metaStart+openEnd+1]
|
||||
inner := buildOPFMetadataInner(item) + originalIdentifiers(original[metaStart:metaStart+metaClose])
|
||||
|
||||
out := bytes.NewBuffer(nil)
|
||||
out.Write(original[:metaStart])
|
||||
out.Write(openTag)
|
||||
out.WriteString("\n")
|
||||
out.WriteString(inner)
|
||||
out.Write(original[metaStart+metaClose:])
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
|
||||
// originalIdentifiers extracts the old metadata's dc:identifier
|
||||
// elements verbatim. The package's unique-identifier attribute
|
||||
// references one of them, so replacing the metadata block must keep
|
||||
// them (generated fields come FIRST, so the parser's first-wins reads
|
||||
// the fresh ISBN; the originals remain for identifier stability).
|
||||
func originalIdentifiers(oldMetadata []byte) string {
|
||||
var b strings.Builder
|
||||
rest := oldMetadata
|
||||
for {
|
||||
start := bytes.Index(rest, []byte("<dc:identifier"))
|
||||
if start == -1 {
|
||||
break
|
||||
}
|
||||
end := bytes.Index(rest[start:], []byte("</dc:identifier>"))
|
||||
if end == -1 {
|
||||
break
|
||||
}
|
||||
end += start + len("</dc:identifier>")
|
||||
b.Write(rest[start:end])
|
||||
b.WriteString("\n")
|
||||
rest = rest[end:]
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// rewriteEPUBZip copies every entry to dst, substituting opfName's
|
||||
// content. Order, compression method and modes are preserved.
|
||||
func rewriteEPUBZip(entries []*zip.File, opfName string, newOPF []byte, dst string) error {
|
||||
return rewriteZipArchive(entries, opfName, newOPF, dst, "", nil)
|
||||
}
|
||||
|
||||
// rewriteZipArchive is the generic form: substitute name's content, and
|
||||
// (for the CBZ path, where ComicInfo.xml may not exist yet) append
|
||||
// extraName/extraContent as a final entry when the archive lacks the
|
||||
// substituted name.
|
||||
func rewriteZipArchive(entries []*zip.File, name string, content []byte, dst string, extraName string, extraContent []byte) error {
|
||||
out, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to create temp archive: %w", err)
|
||||
}
|
||||
writer := zip.NewWriter(out)
|
||||
substituted := false
|
||||
for _, f := range entries {
|
||||
header := f.FileHeader
|
||||
header.Flags &= ^uint16(0x8) // drop data-descriptor flag; sizes are known upfront
|
||||
var target io.Writer
|
||||
if f.Name == name {
|
||||
// Sizes/CRC recomputed by the writer for the new content.
|
||||
header.CompressedSize64 = 0
|
||||
header.UncompressedSize64 = 0
|
||||
header.CompressedSize = 0
|
||||
header.UncompressedSize = 0
|
||||
header.CRC32 = 0
|
||||
target, err = writer.CreateHeader(&header)
|
||||
if err != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to stage substituted entry: %w", err)
|
||||
}
|
||||
if _, err := target.Write(content); err != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to write substituted entry: %w", err)
|
||||
}
|
||||
substituted = true
|
||||
continue
|
||||
}
|
||||
target, err = writer.CreateHeader(&header)
|
||||
if err != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to stage entry %s: %w", f.Name, err)
|
||||
}
|
||||
rc, openErr := f.Open()
|
||||
if openErr != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to read entry %s: %w", f.Name, openErr)
|
||||
}
|
||||
if _, err := io.Copy(target, rc); err != nil {
|
||||
rc.Close()
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to copy entry %s: %w", f.Name, err)
|
||||
}
|
||||
if err := rc.Close(); err != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to finish entry %s: %w", f.Name, err)
|
||||
}
|
||||
}
|
||||
if !substituted && extraName != "" && extraContent != nil {
|
||||
target, cerr := writer.Create(extraName)
|
||||
if cerr != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to append entry %s: %w", extraName, cerr)
|
||||
}
|
||||
if _, werr := target.Write(extraContent); werr != nil {
|
||||
writer.Close()
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to write appended entry %s: %w", extraName, werr)
|
||||
}
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to finalize archive: %w", err)
|
||||
}
|
||||
if err := out.Sync(); err != nil {
|
||||
out.Close()
|
||||
return fmt.Errorf("failed to sync archive: %w", err)
|
||||
}
|
||||
return out.Close()
|
||||
}
|
||||
|
||||
// atomicWrite writes via temp file + rename in the SAME directory so a
|
||||
// crash or a full disk can never leave a truncated book (or sidecar)
|
||||
// behind.
|
||||
func atomicWrite(path string, data []byte) error {
|
||||
dir := filepath.Dir(path)
|
||||
tmp, err := os.CreateTemp(dir, ".bookhoard-write-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tmpPath := tmp.Name()
|
||||
// 0644: sidecars/rewritten files must be readable by host-side
|
||||
// tools sharing the volume (Calibre reads metadata.opf); a 0600
|
||||
// temp default would lock everyone but the server out.
|
||||
if err := tmp.Chmod(0o644); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmpPath)
|
||||
return err
|
||||
}
|
||||
if _, err := tmp.Write(data); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmpPath)
|
||||
return err
|
||||
}
|
||||
if err := tmp.Sync(); err != nil {
|
||||
tmp.Close()
|
||||
os.Remove(tmpPath)
|
||||
return err
|
||||
}
|
||||
if err := tmp.Close(); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return err
|
||||
}
|
||||
if err := os.Rename(tmpPath, path); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ---- audio tags (5b): ffmpeg stream-copy rewrite ----
|
||||
|
||||
// audioFieldNames lists the fields the audio tag path round-trips —
|
||||
// exactly the keys applyAudioTagFallbacks reads back (the ingestion
|
||||
// chain's tags rung). Fields the file format can't carry per our own
|
||||
// read chain (description, publisher, language, isbn, the full tag
|
||||
// list) are deliberately NOT written: "applicable and correct" means
|
||||
// nothing lands in a file the scanner would never find again.
|
||||
// Subtitle rides MP3/FLAC only (the mp4 muxer in our image drops
|
||||
// subtitle keys).
|
||||
func audioFieldNames(item *database.MediaItems, ext string) []string {
|
||||
fields := []string{"title"}
|
||||
if item.Subtitle.Valid && item.Subtitle.String != "" &&
|
||||
!strings.EqualFold(ext, ".m4a") && !strings.EqualFold(ext, ".m4b") {
|
||||
fields = append(fields, "subtitle")
|
||||
}
|
||||
if len(item.Authors) > 0 || (item.Author.Valid && item.Author.String != "") {
|
||||
fields = append(fields, "authors")
|
||||
}
|
||||
if len(item.Narrators) > 0 {
|
||||
fields = append(fields, "narrators")
|
||||
}
|
||||
if item.Series.Valid && item.Series.String != "" {
|
||||
fields = append(fields, "series")
|
||||
}
|
||||
if item.DatePublished.Valid {
|
||||
fields = append(fields, "date_published")
|
||||
}
|
||||
if len(item.Tags) > 0 {
|
||||
fields = append(fields, "genre")
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
// audioTagArgs maps the item's current values onto the container tag
|
||||
// keys the scanner's own fallback chain reads (media_scanner_audio.go):
|
||||
//
|
||||
// title → title + album (players want title; the scanner's
|
||||
// album rung names the book)
|
||||
// subtitle → subtitle (MP3/FLAC only — the mp4 muxer in
|
||||
// our image drops subtitle keys, and
|
||||
// fields we can't read back don't
|
||||
// get written)
|
||||
// authors → artist (first) + album_artist (joined)
|
||||
// narrators → composer (Audible's convention)
|
||||
// series → grouping (series-part rides the iTunes atom)
|
||||
// date → date; first tag → genre
|
||||
func audioTagArgs(item *database.MediaItems, ext string) [][2]string {
|
||||
var args [][2]string
|
||||
add := func(k, v string) {
|
||||
if strings.TrimSpace(v) != "" {
|
||||
args = append(args, [2]string{k, v})
|
||||
}
|
||||
}
|
||||
|
||||
title := strings.TrimSpace(item.Title)
|
||||
add("title", title)
|
||||
add("album", title)
|
||||
if item.Subtitle.Valid && !strings.EqualFold(ext, ".m4a") && !strings.EqualFold(ext, ".m4b") {
|
||||
add("subtitle", strings.TrimSpace(item.Subtitle.String))
|
||||
}
|
||||
|
||||
authors := item.Authors
|
||||
if len(authors) == 0 && item.Author.Valid {
|
||||
authors = []string{item.Author.String}
|
||||
}
|
||||
if len(authors) > 0 {
|
||||
add("artist", strings.TrimSpace(authors[0]))
|
||||
joined := make([]string, 0, len(authors))
|
||||
for _, a := range authors {
|
||||
a = strings.TrimSpace(a)
|
||||
if a != "" {
|
||||
joined = append(joined, a)
|
||||
}
|
||||
}
|
||||
add("album_artist", strings.Join(joined, " & "))
|
||||
}
|
||||
|
||||
if len(item.Narrators) > 0 {
|
||||
narrators := make([]string, 0, len(item.Narrators))
|
||||
for _, n := range item.Narrators {
|
||||
n = strings.TrimSpace(n)
|
||||
if n != "" {
|
||||
narrators = append(narrators, n)
|
||||
}
|
||||
}
|
||||
add("composer", strings.Join(narrators, ", "))
|
||||
}
|
||||
|
||||
if item.Series.Valid {
|
||||
add("grouping", strings.TrimSpace(item.Series.String))
|
||||
if item.SeriesNumber.Valid && item.SeriesNumber.Float64 > 0 {
|
||||
add("----:com.apple.itunes:series-part", strconvFormatFloat(item.SeriesNumber.Float64))
|
||||
}
|
||||
}
|
||||
|
||||
if item.DatePublished.Valid {
|
||||
add("date", item.DatePublished.Time.Format("2006-01-02"))
|
||||
}
|
||||
if len(item.Tags) > 0 && strings.TrimSpace(item.Tags[0]) != "" {
|
||||
add("genre", strings.TrimSpace(item.Tags[0]))
|
||||
}
|
||||
return args
|
||||
}
|
||||
|
||||
// probeDurationSeconds ffprobes a file's duration (0 on any failure).
|
||||
func probeDurationSeconds(path string) float64 {
|
||||
out, err := exec.Command("ffprobe", "-v", "error",
|
||||
"-show_entries", "format=duration",
|
||||
"-of", "default=noprint_wrappers=1:nokey=1", path).Output()
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
v, err := strconv.ParseFloat(strings.TrimSpace(string(out)), 64)
|
||||
if err != nil || v < 0 {
|
||||
return 0
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// writeAudioTags rewrites the container's tags via ffmpeg stream copy:
|
||||
// `-c copy` never re-encodes, `-map_metadata 0` preserves existing
|
||||
// global tags, and `-map_chapters 0` keeps the m4b chapter list
|
||||
// (chapters live in the header, not a stream). Attempt 1 copies every
|
||||
// stream; attempt 2 — for real-world files carrying text/subtitle/data
|
||||
// tracks the ipod muxer rejects ("Tag text incompatible with output
|
||||
// codec id ...") — maps audio + attached pictures only. Every attempt
|
||||
// is ffprobe-VALIDATED (duration within tolerance of the source)
|
||||
// BEFORE the atomic rename; a failure never touches the original.
|
||||
func (w *MetadataWriter) writeAudioTags(absPath string, item *database.MediaItems) (string, error) {
|
||||
info, err := os.Stat(absPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("media file missing on disk: %w", err)
|
||||
}
|
||||
if info.IsDir() {
|
||||
return "", fmt.Errorf("multi-file audio books take the sidecar path")
|
||||
}
|
||||
|
||||
srcDuration := probeDurationSeconds(absPath)
|
||||
|
||||
ext := strings.ToLower(filepath.Ext(absPath))
|
||||
tmpPath := absPath + ".tmp-write" + ext
|
||||
|
||||
// Shared args; each attempt appends its own stream mapping.
|
||||
base := []string{"-hide_banner", "-loglevel", "error", "-y",
|
||||
"-i", absPath,
|
||||
"-map_metadata", "0", "-map_chapters", "0", "-c", "copy",
|
||||
}
|
||||
for _, kv := range audioTagArgs(item, ext) {
|
||||
base = append(base, "-metadata", kv[0]+"="+kv[1])
|
||||
}
|
||||
|
||||
// Stream maps, best fidelity first.
|
||||
streamMaps := [][]string{
|
||||
{"-map", "0"}, // every stream (covers, chapters-as-streams)
|
||||
// Audio + attached pictures; drop text/subtitle/data tracks —
|
||||
// the ipod muxer refuses mov_text-style streams outright.
|
||||
{"-map", "0:a", "-map", "0:t?", "-dn", "-sn"},
|
||||
}
|
||||
|
||||
var lastErr error
|
||||
for i, streamMap := range streamMaps {
|
||||
args := append(append([]string{}, base...), streamMap...)
|
||||
args = append(args, tmpPath)
|
||||
if out, err := exec.Command("ffmpeg", args...).CombinedOutput(); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
lastErr = fmt.Errorf("ffmpeg tag rewrite failed (attempt %d): %s", i+1, strings.TrimSpace(string(out)))
|
||||
continue
|
||||
}
|
||||
|
||||
// Validation gate: the rewritten container must still hold the
|
||||
// same audio (duration within 1s or 0.5%, whichever is looser).
|
||||
outDuration := probeDurationSeconds(tmpPath)
|
||||
if outDuration <= 0 {
|
||||
os.Remove(tmpPath)
|
||||
lastErr = fmt.Errorf("rewritten audio failed validation (ffprobe read no duration)")
|
||||
continue
|
||||
}
|
||||
if srcDuration > 0 {
|
||||
tolerance := math.Max(1.0, srcDuration*0.005)
|
||||
if math.Abs(outDuration-srcDuration) > tolerance {
|
||||
os.Remove(tmpPath)
|
||||
lastErr = fmt.Errorf("rewritten audio failed validation (duration %.2fs → %.2fs)", srcDuration, outDuration)
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if err := os.Rename(tmpPath, absPath); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("failed to replace audio file: %w", err)
|
||||
}
|
||||
return absPath, nil
|
||||
}
|
||||
return "", lastErr
|
||||
}
|
||||
|
||||
// ---- CBZ ComicInfo.xml (5c) ----
|
||||
|
||||
// comicFieldNames lists the fields the CBZ write round-trips through
|
||||
// the scanner's ComicInfo mapping (media_scanner.go: mergeMetadata's
|
||||
// comic block). Fractional series numbers are omitted — the schema's
|
||||
// Number is an integer and truncating would round-trip a lie.
|
||||
func comicFieldNames(item *database.MediaItems) []string {
|
||||
fields := []string{"title"}
|
||||
if item.Series.Valid && item.Series.String != "" {
|
||||
fields = append(fields, "series")
|
||||
if item.SeriesNumber.Valid && item.SeriesNumber.Float64 > 0 &&
|
||||
item.SeriesNumber.Float64 == float64(int(item.SeriesNumber.Float64)) {
|
||||
fields = append(fields, "series_number")
|
||||
}
|
||||
if item.SeriesCount.Valid && item.SeriesCount.Int32 > 0 {
|
||||
fields = append(fields, "series_count")
|
||||
}
|
||||
}
|
||||
if len(item.Authors) > 0 || (item.Author.Valid && item.Author.String != "") {
|
||||
fields = append(fields, "authors")
|
||||
}
|
||||
if item.Description.Valid && item.Description.String != "" {
|
||||
fields = append(fields, "description")
|
||||
}
|
||||
if len(item.Publishers) > 0 || (item.Publisher.Valid && item.Publisher.String != "") {
|
||||
fields = append(fields, "publisher")
|
||||
}
|
||||
if item.Language.Valid && item.Language.String != "" {
|
||||
fields = append(fields, "language")
|
||||
}
|
||||
if item.DatePublished.Valid {
|
||||
fields = append(fields, "date_published")
|
||||
}
|
||||
if len(item.Tags) > 0 {
|
||||
fields = append(fields, "tags")
|
||||
}
|
||||
if item.WebUrl.Valid && item.WebUrl.String != "" {
|
||||
fields = append(fields, "web_url")
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
// buildComicInfoXML renders a ComicInfo.xml (schema v2.x) from the
|
||||
// item's current values — the mirror of the scanner's
|
||||
// extractMetadataFromArchive mapping. Empty values omit their element:
|
||||
// absent means "no data", never "delete".
|
||||
func buildComicInfoXML(item *database.MediaItems) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(xml.Header)
|
||||
b.WriteString(`<ComicInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">` + "\n")
|
||||
|
||||
tag := func(name, value string) {
|
||||
if strings.TrimSpace(value) != "" {
|
||||
b.WriteString("\t<" + name + ">" + escXML(value) + "</" + name + ">\n")
|
||||
}
|
||||
}
|
||||
tagInt := func(name string, v int) {
|
||||
if v > 0 {
|
||||
b.WriteString("\t<" + name + ">" + strconv.Itoa(v) + "</" + name + ">\n")
|
||||
}
|
||||
}
|
||||
|
||||
tag("Title", strings.TrimSpace(item.Title))
|
||||
tag("Series", strings.TrimSpace(item.Series.String))
|
||||
if item.SeriesNumber.Valid && item.SeriesNumber.Float64 > 0 &&
|
||||
item.SeriesNumber.Float64 == float64(int(item.SeriesNumber.Float64)) {
|
||||
tagInt("Number", int(item.SeriesNumber.Float64))
|
||||
}
|
||||
if item.SeriesCount.Valid {
|
||||
tagInt("Count", int(item.SeriesCount.Int32))
|
||||
}
|
||||
if item.Volume.Valid {
|
||||
tagInt("Volume", int(item.Volume.Int32))
|
||||
}
|
||||
|
||||
authors := item.Authors
|
||||
if len(authors) == 0 && item.Author.Valid {
|
||||
authors = []string{item.Author.String}
|
||||
}
|
||||
joinedAuthors := make([]string, 0, len(authors))
|
||||
for _, a := range authors {
|
||||
a = strings.TrimSpace(a)
|
||||
if a != "" {
|
||||
joinedAuthors = append(joinedAuthors, a)
|
||||
}
|
||||
}
|
||||
tag("Writer", strings.Join(joinedAuthors, ", "))
|
||||
|
||||
tag("Summary", item.Description.String)
|
||||
|
||||
publisher := ""
|
||||
if len(item.Publishers) > 0 {
|
||||
publisher = item.Publishers[0]
|
||||
} else if item.Publisher.Valid {
|
||||
publisher = item.Publisher.String
|
||||
}
|
||||
tag("Publisher", strings.TrimSpace(publisher))
|
||||
|
||||
if item.DatePublished.Valid {
|
||||
t := item.DatePublished.Time
|
||||
tagInt("Year", t.Year())
|
||||
tagInt("Month", int(t.Month()))
|
||||
tagInt("Day", t.Day())
|
||||
}
|
||||
|
||||
tag("LanguageISO", strings.TrimSpace(item.Language.String))
|
||||
if item.WebUrl.Valid {
|
||||
tag("Web", strings.TrimSpace(item.WebUrl.String))
|
||||
}
|
||||
|
||||
genre := strings.TrimSpace(item.Genre.String)
|
||||
if genre == "" && len(item.Tags) > 0 {
|
||||
genre = strings.TrimSpace(item.Tags[0])
|
||||
}
|
||||
tag("Genre", genre)
|
||||
if len(item.Tags) > 0 {
|
||||
cleanTags := make([]string, 0, len(item.Tags))
|
||||
for _, t := range item.Tags {
|
||||
t = strings.TrimSpace(t)
|
||||
if t != "" {
|
||||
cleanTags = append(cleanTags, t)
|
||||
}
|
||||
}
|
||||
tag("Tags", strings.Join(cleanTags, ", "))
|
||||
}
|
||||
|
||||
b.WriteString("</ComicInfo>\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// writeCBZComicInfo injects (or replaces) ComicInfo.xml inside the CBZ
|
||||
// — same atomic zip rewrite as the EPUB path, every other entry copied
|
||||
// byte-identical.
|
||||
func (w *MetadataWriter) writeCBZComicInfo(absPath string, item *database.MediaItems) (string, error) {
|
||||
reader, err := zip.OpenReader(absPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("failed to open CBZ: %w", err)
|
||||
}
|
||||
defer reader.Close()
|
||||
|
||||
newXML := buildComicInfoXML(item)
|
||||
|
||||
tmpPath := absPath + ".tmp-write"
|
||||
if err := rewriteZipArchive(reader.File, "ComicInfo.xml", []byte(newXML), tmpPath, "ComicInfo.xml", []byte(newXML)); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", err
|
||||
}
|
||||
if err := os.Rename(tmpPath, absPath); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("failed to replace CBZ: %w", err)
|
||||
}
|
||||
return absPath, nil
|
||||
}
|
||||
|
||||
// ---- OPF generation (mirrors parseOPFContent's read set) ----
|
||||
|
||||
// escXML escapes the five XML entities for element and attribute text.
|
||||
func escXML(s string) string {
|
||||
return strings.NewReplacer(
|
||||
"&", "&",
|
||||
"<", "<",
|
||||
">", ">",
|
||||
`"`, """,
|
||||
"'", "'",
|
||||
).Replace(s)
|
||||
}
|
||||
|
||||
// buildSidecarOPFXML renders a complete Calibre-compatible package
|
||||
// document for a sidecar.
|
||||
func buildSidecarOPFXML(item *database.MediaItems) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(xml.Header)
|
||||
b.WriteString(`<package version="2.0" xmlns="http://www.idpf.org/2007/opf" unique-identifier="bookhoard-uuid">` + "\n")
|
||||
b.WriteString(`<metadata xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">` + "\n")
|
||||
b.WriteString(buildOPFMetadataInner(item))
|
||||
b.WriteString("</metadata>\n</package>\n")
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// buildOPFMetadataInner renders the dc:* elements for the item's
|
||||
// current values — exactly the fields parseOPFContent reads back
|
||||
// (title/subtitle via title-type, creators, subjects, description,
|
||||
// publisher, date, language, isbn identifier, calibre:series metas).
|
||||
// Empty DB values simply omit their element: a sidecar absent-field
|
||||
// means "no data", never "delete this from the file".
|
||||
func buildOPFMetadataInner(item *database.MediaItems) string {
|
||||
var b strings.Builder
|
||||
|
||||
title := strings.TrimSpace(item.Title)
|
||||
if title != "" {
|
||||
b.WriteString("\t<dc:title id=\"tmain\">" + escXML(title) + "</dc:title>\n")
|
||||
b.WriteString("\t<meta refines=\"#tmain\" property=\"title-type\">main</meta>\n")
|
||||
}
|
||||
if item.Subtitle.Valid && strings.TrimSpace(item.Subtitle.String) != "" {
|
||||
b.WriteString("\t<dc:title id=\"tsub\">" + escXML(strings.TrimSpace(item.Subtitle.String)) + "</dc:title>\n")
|
||||
b.WriteString("\t<meta refines=\"#tsub\" property=\"title-type\">subtitle</meta>\n")
|
||||
}
|
||||
for _, a := range item.Authors {
|
||||
a = strings.TrimSpace(a)
|
||||
if a == "" {
|
||||
continue
|
||||
}
|
||||
b.WriteString("\t<dc:creator opf:role=\"aut\">" + escXML(a) + "</dc:creator>\n")
|
||||
}
|
||||
if len(item.Authors) == 0 && item.Author.Valid && strings.TrimSpace(item.Author.String) != "" {
|
||||
b.WriteString("\t<dc:creator opf:role=\"aut\">" + escXML(strings.TrimSpace(item.Author.String)) + "</dc:creator>\n")
|
||||
}
|
||||
if item.Description.Valid && strings.TrimSpace(item.Description.String) != "" {
|
||||
b.WriteString("\t<dc:description>" + escXML(item.Description.String) + "</dc:description>\n")
|
||||
}
|
||||
for _, p := range item.Publishers {
|
||||
p = strings.TrimSpace(p)
|
||||
if p == "" {
|
||||
continue
|
||||
}
|
||||
b.WriteString("\t<dc:publisher>" + escXML(p) + "</dc:publisher>\n")
|
||||
break // parser reads the first
|
||||
}
|
||||
if item.DatePublished.Valid {
|
||||
b.WriteString("\t<dc:date>" + item.DatePublished.Time.Format("2006-01-02") + "</dc:date>\n")
|
||||
}
|
||||
if item.Language.Valid && strings.TrimSpace(item.Language.String) != "" {
|
||||
b.WriteString("\t<dc:language>" + escXML(strings.TrimSpace(item.Language.String)) + "</dc:language>\n")
|
||||
}
|
||||
if item.Isbn.Valid && strings.TrimSpace(item.Isbn.String) != "" {
|
||||
b.WriteString("\t<dc:identifier opf:scheme=\"ISBN\">" + escXML(strings.TrimSpace(item.Isbn.String)) + "</dc:identifier>\n")
|
||||
}
|
||||
// The package's unique-identifier needs a matching element: the
|
||||
// scan's OPF uuid when known, else the item id (the parser files
|
||||
// urn:uuid under hash info, never metadata).
|
||||
uid := ""
|
||||
if item.OpfUuid.Valid {
|
||||
uid = strings.TrimSpace(item.OpfUuid.String)
|
||||
}
|
||||
if uid == "" && item.ID.Valid {
|
||||
uid = uuid.UUID(item.ID.Bytes).String()
|
||||
}
|
||||
if uid != "" {
|
||||
b.WriteString("\t<dc:identifier id=\"bookhoard-uuid\">urn:uuid:" + escXML(uid) + "</dc:identifier>\n")
|
||||
}
|
||||
for _, t := range item.Tags {
|
||||
t = strings.TrimSpace(t)
|
||||
if t == "" {
|
||||
continue
|
||||
}
|
||||
b.WriteString("\t<dc:subject>" + escXML(t) + "</dc:subject>\n")
|
||||
}
|
||||
if item.Series.Valid && strings.TrimSpace(item.Series.String) != "" {
|
||||
b.WriteString("\t<meta name=\"calibre:series\" content=\"" + escXML(item.Series.String) + "\"/>\n")
|
||||
if item.SeriesNumber.Valid && item.SeriesNumber.Float64 > 0 {
|
||||
b.WriteString("\t<meta name=\"calibre:series_index\" content=\"" + strconvFormatFloat(item.SeriesNumber.Float64) + "\"/>\n")
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// metadataFieldNames lists the DB fields a write mirrors — response
|
||||
// contract for the UI ("what got written").
|
||||
func metadataFieldNames(item *database.MediaItems) []string {
|
||||
fields := []string{"title"}
|
||||
if item.Subtitle.Valid && item.Subtitle.String != "" {
|
||||
fields = append(fields, "subtitle")
|
||||
}
|
||||
if (len(item.Authors) > 0) || (item.Author.Valid && item.Author.String != "") {
|
||||
fields = append(fields, "authors")
|
||||
}
|
||||
if item.Description.Valid && item.Description.String != "" {
|
||||
fields = append(fields, "description")
|
||||
}
|
||||
if len(item.Publishers) > 0 || (item.Publisher.Valid && item.Publisher.String != "") {
|
||||
fields = append(fields, "publisher")
|
||||
}
|
||||
if item.DatePublished.Valid {
|
||||
fields = append(fields, "date_published")
|
||||
}
|
||||
if item.Language.Valid && item.Language.String != "" {
|
||||
fields = append(fields, "language")
|
||||
}
|
||||
if item.Isbn.Valid && item.Isbn.String != "" {
|
||||
fields = append(fields, "isbn")
|
||||
}
|
||||
if len(item.Tags) > 0 {
|
||||
fields = append(fields, "tags")
|
||||
}
|
||||
if item.Series.Valid && item.Series.String != "" {
|
||||
fields = append(fields, "series")
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
// strconvFormatFloat renders series indexes without trailing noise
|
||||
// (3 → "3", 3.5 → "3.5").
|
||||
func strconvFormatFloat(f float64) string {
|
||||
return fmt.Sprintf("%g", f)
|
||||
}
|
||||
@@ -0,0 +1,352 @@
|
||||
package services
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"encoding/xml"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// Unit tests for the metadata writer's pure parts (2026-10-27): OPF
|
||||
// generation round-trips through the scanner's OWN parser
|
||||
// (parseOPFContent — write side and read side can never disagree), the
|
||||
// EPUB splice preserves manifest/spine, and the zip rewrite keeps
|
||||
// mimetype first + stored. The ffmpeg audio writer (5b) will extend
|
||||
// this file.
|
||||
|
||||
func fixtureItem() *database.MediaItems {
|
||||
pubDate, _ := time.Parse("2006-01-02", "1853-07-14")
|
||||
return &database.MediaItems{
|
||||
Title: `Progress & "Quotes" <etc>`,
|
||||
Subtitle: pgtype.Text{String: "A Subtitle", Valid: true},
|
||||
Authors: []string{"Ada Lovelace", "Grace Hopper"},
|
||||
Description: pgtype.Text{
|
||||
String: "A <b>description</b> with entities & such", Valid: true,
|
||||
},
|
||||
Publishers: []string{"Analytical Engine Press"},
|
||||
DatePublished: pgtype.Date{Time: pubDate, Valid: true},
|
||||
Language: pgtype.Text{String: "en", Valid: true},
|
||||
Isbn: pgtype.Text{String: "978-0-13-235088-4", Valid: true},
|
||||
Tags: []string{"Science", "Computing"},
|
||||
Series: pgtype.Text{String: "Scientific Memoirs", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 3, Valid: true},
|
||||
}
|
||||
}
|
||||
|
||||
// buildTestEPUB creates a minimal EPUB (mimetype + container + OPF with
|
||||
// manifest/spine) in memory.
|
||||
func buildTestEPUB(t *testing.T, opfContent string) []*zip.File {
|
||||
t.Helper()
|
||||
var buf bytes.Buffer
|
||||
w := zip.NewWriter(&buf)
|
||||
|
||||
mimetype, _ := w.CreateHeader(&zip.FileHeader{Name: "mimetype", Method: zip.Store})
|
||||
_, _ = mimetype.Write([]byte("application/epub+zip"))
|
||||
|
||||
container, _ := w.Create("META-INF/container.xml")
|
||||
_, _ = container.Write([]byte(`<?xml version="1.0"?><container version="1.0" xmlns="urn:oasis:names:tc:opendocument:xmlns:container"><rootfiles><rootfile full-path="OEBPS/content.opf" media-type="application/oebps-package+xml"/></rootfiles></container>`))
|
||||
|
||||
opf, _ := w.Create("OEBPS/content.opf")
|
||||
_, _ = opf.Write([]byte(opfContent))
|
||||
|
||||
chapter, _ := w.Create("OEBPS/chapter1.xhtml")
|
||||
_, _ = chapter.Write([]byte("<html><body>Chapter</body></html>"))
|
||||
|
||||
require.NoError(t, w.Close())
|
||||
reader, err := zip.NewReader(bytes.NewReader(buf.Bytes()), int64(buf.Len()))
|
||||
require.NoError(t, err)
|
||||
return reader.File
|
||||
}
|
||||
|
||||
const spliceTestOPF = `<?xml version="1.0" encoding="utf-8"?>
|
||||
<package version="3.0" xmlns="http://www.idpf.org/2007/opf" unique-identifier="pub-id">
|
||||
<metadata xmlns:dc="http://purl.org/dc/elements/1.1/">
|
||||
<dc:title id="t1">Old Title</dc:title>
|
||||
<dc:creator>Old Author</dc:creator>
|
||||
<dc:identifier id="pub-id">urn:uuid:1234</dc:identifier>
|
||||
</metadata>
|
||||
<manifest>
|
||||
<item id="ch1" href="chapter1.xhtml" media-type="application/xhtml+xml"/>
|
||||
</manifest>
|
||||
<spine>
|
||||
<itemref idref="ch1"/>
|
||||
</spine>
|
||||
</package>
|
||||
`
|
||||
|
||||
func TestMetadataWriteTargetFor(t *testing.T) {
|
||||
assert.Equal(t, MetadataTargetEmbeddedOPF, MetadataWriteTargetFor(".epub"))
|
||||
assert.Equal(t, MetadataTargetEmbeddedOPF, MetadataWriteTargetFor(".KEPUB"))
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(".pdf"))
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(""))
|
||||
}
|
||||
|
||||
func TestBuildOPFMetadataInnerRoundTrips(t *testing.T) {
|
||||
item := fixtureItem()
|
||||
xmlInner := buildOPFMetadataInner(item)
|
||||
|
||||
// Wrap in a full package and parse with the SCANNER's parser: the
|
||||
// write must survive the read.
|
||||
doc := `<?xml version="1.0"?><package xmlns="http://www.idpf.org/2007/opf"><metadata xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:opf="http://www.idpf.org/2007/opf">` +
|
||||
xmlInner + `</metadata></package>`
|
||||
md, err := parseOPFContent([]byte(doc))
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, "Progress & \"Quotes\" <etc>: A Subtitle", md.Title, "main title + calibre-style subtitle join")
|
||||
assert.Equal(t, "Ada Lovelace", md.Author, "first creator")
|
||||
assert.Equal(t, "A <b>description</b> with entities & such", md.Description, "entities unescaped on read")
|
||||
assert.Equal(t, "Analytical Engine Press", md.Publisher)
|
||||
assert.Equal(t, "en", md.Language)
|
||||
assert.Equal(t, "9780132350884", md.ISBN, "normalized on read")
|
||||
assert.Equal(t, "Scientific Memoirs", md.Series)
|
||||
assert.Equal(t, float64(3), md.SeriesNumber)
|
||||
assert.ElementsMatch(t, []string{"Science", "Computing"}, md.Tags)
|
||||
require.NotNil(t, md.PublishDate)
|
||||
assert.Equal(t, "1853-07-14", md.PublishDate.Format("2006-01-02"))
|
||||
}
|
||||
|
||||
func TestBuildOPFMetadataInnerOmitsEmptyFields(t *testing.T) {
|
||||
item := &database.MediaItems{Title: "Only A Title"}
|
||||
xmlInner := buildOPFMetadataInner(item)
|
||||
assert.NotContains(t, xmlInner, "dc:creator")
|
||||
assert.NotContains(t, xmlInner, "dc:description")
|
||||
assert.NotContains(t, xmlInner, "calibre:series")
|
||||
assert.NotContains(t, xmlInner, "dc:title id=\"tsub\"", "no subtitle element when unset")
|
||||
|
||||
// Singular-author fallback when the authors array is empty.
|
||||
item = &database.MediaItems{
|
||||
Title: "Fallback",
|
||||
Author: pgtype.Text{String: "Solo Author", Valid: true},
|
||||
}
|
||||
assert.Contains(t, buildOPFMetadataInner(item), "Solo Author")
|
||||
}
|
||||
|
||||
func TestSpliceOPFMetadataPreservesStructure(t *testing.T) {
|
||||
entries := buildTestEPUB(t, spliceTestOPF)
|
||||
item := fixtureItem()
|
||||
|
||||
newOPF, err := spliceOPFMetadata(entries, "OEBPS/content.opf", item)
|
||||
require.NoError(t, err)
|
||||
|
||||
// The old metadata values are gone…
|
||||
out := string(newOPF)
|
||||
assert.NotContains(t, out, "Old Title")
|
||||
assert.NotContains(t, out, "Old Author")
|
||||
// …the new values are in…
|
||||
assert.Contains(t, out, `Progress & "Quotes" <etc>`)
|
||||
assert.Contains(t, out, "Ada Lovelace")
|
||||
// …and manifest/spine/identifier survive untouched.
|
||||
assert.Contains(t, out, `<item id="ch1" href="chapter1.xhtml" media-type="application/xhtml+xml"/>`)
|
||||
assert.Contains(t, out, `<itemref idref="ch1"/>`)
|
||||
assert.Contains(t, out, `urn:uuid:1234`)
|
||||
assert.Contains(t, out, `unique-identifier="pub-id"`, "original opening tag preserved")
|
||||
|
||||
// The spliced OPF parses back with the scanner's parser.
|
||||
md, err := parseOPFContent(newOPF)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, item.Title+": A Subtitle", md.Title)
|
||||
}
|
||||
|
||||
func TestRewriteEPUBZipKeepsMimetypeFirstAndStored(t *testing.T) {
|
||||
entries := buildTestEPUB(t, spliceTestOPF)
|
||||
item := fixtureItem()
|
||||
newOPF, err := spliceOPFMetadata(entries, "OEBPS/content.opf", item)
|
||||
require.NoError(t, err)
|
||||
|
||||
tmp := filepath.Join(t.TempDir(), "book.epub")
|
||||
require.NoError(t, rewriteEPUBZip(entries, "OEBPS/content.opf", newOPF, tmp))
|
||||
|
||||
reader, err := zip.OpenReader(tmp)
|
||||
require.NoError(t, err)
|
||||
defer reader.Close()
|
||||
|
||||
require.NotEmpty(t, reader.File)
|
||||
assert.Equal(t, "mimetype", reader.File[0].Name, "mimetype must be the first entry")
|
||||
assert.Equal(t, zip.Store, reader.File[0].FileHeader.Method, "mimetype must be uncompressed")
|
||||
|
||||
var opfEntry *zip.File
|
||||
names := []string{}
|
||||
for _, f := range reader.File {
|
||||
names = append(names, f.Name)
|
||||
if f.Name == "OEBPS/content.opf" {
|
||||
opfEntry = f
|
||||
}
|
||||
}
|
||||
assert.ElementsMatch(t, []string{"mimetype", "META-INF/container.xml", "OEBPS/content.opf", "OEBPS/chapter1.xhtml"}, names)
|
||||
require.NotNil(t, opfEntry)
|
||||
rc, err := opfEntry.Open()
|
||||
require.NoError(t, err)
|
||||
content, err := io.ReadAll(rc)
|
||||
require.NoError(t, rc.Close())
|
||||
require.NoError(t, err)
|
||||
assert.Contains(t, string(content), "Ada Lovelace")
|
||||
}
|
||||
|
||||
func TestAtomicWriteReplacesContent(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
target := filepath.Join(dir, "metadata.opf")
|
||||
require.NoError(t, os.WriteFile(target, []byte("old"), 0o644))
|
||||
require.NoError(t, atomicWrite(target, []byte("new content")))
|
||||
got, err := os.ReadFile(target)
|
||||
require.NoError(t, err)
|
||||
assert.Equal(t, "new content", string(got))
|
||||
// No temp litter left behind.
|
||||
entries, err := os.ReadDir(dir)
|
||||
require.NoError(t, err)
|
||||
assert.Len(t, entries, 1)
|
||||
}
|
||||
|
||||
func TestAudioTagArgsMapsRoundTripKeys(t *testing.T) {
|
||||
// Every key here must have a reader in applyAudioTagFallbacks —
|
||||
// that's the round-trip contract with the scanner.
|
||||
item := &database.MediaItems{
|
||||
Title: "The Book",
|
||||
Subtitle: pgtype.Text{String: "The Sub", Valid: true},
|
||||
Authors: []string{"Ada Lovelace", "Grace Hopper"},
|
||||
Narrators: []string{"Simon Vance"},
|
||||
Series: pgtype.Text{String: "Memoirs", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 3.5, Valid: true},
|
||||
DatePublished: pgtype.Date{
|
||||
Time: dateOf("1953-04-16"),
|
||||
Valid: true,
|
||||
},
|
||||
Tags: []string{"Science"},
|
||||
}
|
||||
|
||||
got := map[string]string{}
|
||||
for _, kv := range audioTagArgs(item, ".mp3") {
|
||||
got[kv[0]] = kv[1]
|
||||
}
|
||||
assert.Equal(t, "The Book", got["title"])
|
||||
assert.Equal(t, "The Book", got["album"], "the scanner's album rung names the book")
|
||||
assert.Equal(t, "The Sub", got["subtitle"])
|
||||
assert.Equal(t, "Ada Lovelace", got["artist"], "first author")
|
||||
assert.Equal(t, "Ada Lovelace & Grace Hopper", got["album_artist"])
|
||||
assert.Equal(t, "Simon Vance", got["composer"], "narrators ride composer (Audible convention)")
|
||||
assert.Equal(t, "Memoirs", got["grouping"])
|
||||
assert.Equal(t, "3.5", got["----:com.apple.itunes:series-part"])
|
||||
assert.Equal(t, "1953-04-16", got["date"])
|
||||
assert.Equal(t, "Science", got["genre"])
|
||||
|
||||
// Empty fields emit nothing (a tag write must not blank a book).
|
||||
empty := audioTagArgs(&database.MediaItems{Title: "T"}, ".mp3")
|
||||
keys := map[string]bool{}
|
||||
for _, kv := range empty {
|
||||
keys[kv[0]] = true
|
||||
}
|
||||
assert.True(t, keys["title"])
|
||||
assert.False(t, keys["subtitle"])
|
||||
assert.False(t, keys["artist"])
|
||||
assert.False(t, keys["composer"])
|
||||
assert.False(t, keys["grouping"])
|
||||
assert.False(t, keys["genre"])
|
||||
|
||||
// M4B drops subtitle keys in this ffmpeg build — never emitted.
|
||||
m4b := audioTagArgs(item, ".m4b")
|
||||
for _, kv := range m4b {
|
||||
assert.NotEqual(t, "subtitle", kv[0], "mp4 muxer drops subtitle; don't write it")
|
||||
}
|
||||
}
|
||||
|
||||
// Single-file audio dispatch (5b): the four containers the ffmpeg tag
|
||||
// writer claims; folder/multi-file audio stays on the sidecar path.
|
||||
func TestAudioTagExts(t *testing.T) {
|
||||
for _, ext := range []string{".mp3", ".flac", ".m4a", ".m4b"} {
|
||||
assert.Equal(t, MetadataTargetAudioTags, MetadataWriteTargetFor(ext), ext)
|
||||
}
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(".pdf"))
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(".opus"))
|
||||
}
|
||||
|
||||
// CBZ dispatch (5c): ComicInfo injection claims CBZ; CBR/CB7 can't be
|
||||
// rewritten in place and stay on the sidecar path.
|
||||
func TestComicInfoTargetFor(t *testing.T) {
|
||||
assert.Equal(t, MetadataTargetComicInfo, MetadataWriteTargetFor(".cbz"))
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(".cbr"))
|
||||
assert.Equal(t, MetadataTargetSidecarOPF, MetadataWriteTargetFor(".cb7"))
|
||||
}
|
||||
|
||||
// buildComicInfoXML must round-trip through the scanner's OWN ComicInfo
|
||||
// struct (xml.Unmarshal — same parser extractMetadataFromArchive uses).
|
||||
func TestBuildComicInfoXMLRoundTrips(t *testing.T) {
|
||||
pubDate, _ := time.Parse("2006-01-02", "1987-03-10")
|
||||
item := &database.MediaItems{
|
||||
Title: "The Sad Comic <#1>",
|
||||
Series: pgtype.Text{String: "Watchmen", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 3, Valid: true},
|
||||
SeriesCount: pgtype.Int4{Int32: 12, Valid: true},
|
||||
Authors: []string{"Alan Moore", "Dave Gibbons"},
|
||||
Description: pgtype.Text{String: "Description & <entities>", Valid: true},
|
||||
Publishers: []string{"DC Comics"},
|
||||
DatePublished: pgtype.Date{
|
||||
Time: pubDate,
|
||||
Valid: true,
|
||||
},
|
||||
Language: pgtype.Text{String: "en", Valid: true},
|
||||
WebUrl: pgtype.Text{String: "https://example.com/watchmen", Valid: true},
|
||||
Tags: []string{"Superhero", "Drama"},
|
||||
}
|
||||
|
||||
parsed := ComicInfo{}
|
||||
require.NoError(t, xml.Unmarshal([]byte(buildComicInfoXML(item)), &parsed))
|
||||
|
||||
assert.Equal(t, "The Sad Comic <#1>", parsed.Title)
|
||||
assert.Equal(t, "Watchmen", parsed.Series)
|
||||
assert.Equal(t, 3, parsed.Number)
|
||||
assert.Equal(t, 12, parsed.Count)
|
||||
assert.Equal(t, "Alan Moore, Dave Gibbons", parsed.Writer)
|
||||
assert.Equal(t, "Description & <entities>", parsed.Summary)
|
||||
assert.Equal(t, "DC Comics", parsed.Publisher)
|
||||
assert.Equal(t, 1987, parsed.Year)
|
||||
assert.Equal(t, 3, parsed.Month)
|
||||
assert.Equal(t, 10, parsed.Day)
|
||||
assert.Equal(t, "en", parsed.LanguageISO)
|
||||
assert.Equal(t, "https://example.com/watchmen", parsed.Web)
|
||||
assert.Equal(t, "Superhero", parsed.Genre, "first tag doubles as genre")
|
||||
assert.Equal(t, "Superhero, Drama", parsed.Tags)
|
||||
}
|
||||
|
||||
func TestBuildComicInfoXMLSkipsFractionalNumber(t *testing.T) {
|
||||
// Number is an integer field; a fractional series number must not
|
||||
// round-trip truncated (a lie).
|
||||
item := &database.MediaItems{
|
||||
Title: "X",
|
||||
Series: pgtype.Text{String: "S", Valid: true},
|
||||
SeriesNumber: pgtype.Float8{Float64: 3.5, Valid: true},
|
||||
}
|
||||
parsed := ComicInfo{}
|
||||
require.NoError(t, xml.Unmarshal([]byte(buildComicInfoXML(item)), &parsed))
|
||||
assert.Equal(t, 0, parsed.Number, "fractional series number omitted, not truncated")
|
||||
assert.NotContains(t, buildComicInfoXML(item), "<Number>")
|
||||
}
|
||||
|
||||
// The demoted-genre contract in full (885d6140): a format word only
|
||||
// sinks to tags-only when it's ALONE. A blob where ANY candidate maps
|
||||
// to a shelf still wins the genre column; the demoted word never does.
|
||||
func TestPickSmartGenreMixedBlob(t *testing.T) {
|
||||
genre, all := pickSmartGenre([]string{"Audiobook, Science Fiction, Unabridged"})
|
||||
assert.Equal(t, "science fiction", genre, "a recognized shelf wins the column")
|
||||
assert.Contains(t, all, "Audiobook", "the demoted word is still a tag candidate")
|
||||
assert.Contains(t, all, "Unabridged")
|
||||
|
||||
// Format words alone → column empty, everything tags-only.
|
||||
genre, all = pickSmartGenre([]string{"Audiobook, Unabridged"})
|
||||
assert.Empty(t, genre, "demoted words alone never fill the genre column")
|
||||
assert.Contains(t, all, "Audiobook")
|
||||
assert.Contains(t, all, "Unabridged")
|
||||
}
|
||||
|
||||
func dateOf(s string) time.Time {
|
||||
tm, _ := time.Parse("2006-01-02", s)
|
||||
return tm
|
||||
}
|
||||
@@ -0,0 +1,206 @@
|
||||
// SSRF-guarded HTTP fetching for the podcast fetcher (plan §7).
|
||||
//
|
||||
// This is the code path that turns USER-SUPPLIED URLs into server
|
||||
// fetches — treat it as a security boundary, not a convenience wrapper:
|
||||
//
|
||||
// - https-only schemes (feed URL, enclosure, artwork, EVERY redirect hop)
|
||||
// - private/loopback/link-local/multicast/unspecified IP ranges blocked
|
||||
// at DIAL time via Dialer.Control — the check sees the RESOLVED
|
||||
// address, so a DNS name that rebinding-resolves to 127.0.0.1 (or a
|
||||
// redirect into the internal network) is refused at the socket, not
|
||||
// merely at URL-parse time
|
||||
// - hard response size caps on every body (feed XML, artwork, enclosures)
|
||||
//
|
||||
// Go's redirect follower re-dials through the same dialer per hop, so
|
||||
// the IP guard re-applies automatically; the scheme guard re-applies
|
||||
// via CheckRedirect.
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Response size caps (bytes). Feeds are XML; artwork is an image;
|
||||
// enclosures are audio episodes — the enclosure cap is deliberately the
|
||||
// large one (audiobook chapters run long) but still bounded.
|
||||
const (
|
||||
maxFeedSizeBytes = 25 << 20 // 25 MiB
|
||||
maxArtworkSizeBytes = 20 << 20 // 20 MiB
|
||||
maxEnclosureCapBytes = 1 << 30 // 1 GiB
|
||||
)
|
||||
|
||||
// ErrBlockedPrivateAddress marks SSRF-guard rejections (dial-time IP or
|
||||
// scheme checks). Handlers surface it as a validation failure, and the
|
||||
// download worker records it as the download_error text.
|
||||
var ErrBlockedPrivateAddress = errors.New("address blocked: private, loopback, or otherwise non-public")
|
||||
|
||||
// podcastHTTPClient builds the shared guard client. Timeout is generous
|
||||
// (podcast servers are slow) but bounded; enclosures stream through a
|
||||
// LimitReader so a lying Content-Length cannot exhaust disk.
|
||||
func podcastHTTPClient() *http.Client {
|
||||
dialer := &net.Dialer{
|
||||
Timeout: 15 * time.Second,
|
||||
Control: func(network, address string, _ syscall.RawConn) error {
|
||||
return guardDialAddress(address)
|
||||
},
|
||||
}
|
||||
transport := &http.Transport{
|
||||
DialContext: dialer.DialContext,
|
||||
TLSHandshakeTimeout: 15 * time.Second,
|
||||
ResponseHeaderTimeout: 60 * time.Second,
|
||||
// Enclosure downloads are long; idle conn reuse still applies.
|
||||
IdleConnTimeout: 90 * time.Second,
|
||||
TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12},
|
||||
Proxy: nil, // never route through a user-controllable proxy
|
||||
}
|
||||
return &http.Client{
|
||||
Transport: transport,
|
||||
Timeout: 15 * time.Minute, // outer bound (biggest enclosures on slow lines)
|
||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
||||
if len(via) >= 10 {
|
||||
return fmt.Errorf("stopped after 10 redirects")
|
||||
}
|
||||
if err := guardURLScheme(req.URL); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// guardDialAddress validates the RESOLVED host:port at dial time.
|
||||
func guardDialAddress(address string) error {
|
||||
host, _, err := net.SplitHostPort(address)
|
||||
if err != nil {
|
||||
return fmt.Errorf("dial guard: %w", err)
|
||||
}
|
||||
ip := net.ParseIP(host)
|
||||
if ip == nil {
|
||||
return fmt.Errorf("dial guard: unparseable address %q", host)
|
||||
}
|
||||
if !isPublicIP(ip) {
|
||||
return ErrBlockedPrivateAddress
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// guardURLScheme enforces https on any URL we are about to touch.
|
||||
func guardURLScheme(u *url.URL) error {
|
||||
if u == nil || u.Scheme != "https" {
|
||||
return fmt.Errorf("only https feed/enclosure URLs are supported")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// isPublicIP reports whether ip is a routable public unicast address.
|
||||
// Everything loopback/private/link-local/multicast/unspecified —
|
||||
// including IPv4-mapped IPv6 spellings of those — is rejected.
|
||||
func isPublicIP(ip net.IP) bool {
|
||||
if ip == nil {
|
||||
return false
|
||||
}
|
||||
// Normalize IPv4-mapped IPv6 (::ffff:10.0.0.1) to its v4 form so
|
||||
// the v4 range checks below actually fire.
|
||||
if ip4 := ip.To4(); ip4 != nil {
|
||||
ip = ip4
|
||||
}
|
||||
if ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() ||
|
||||
ip.IsLinkLocalMulticast() || ip.IsInterfaceLocalMulticast() ||
|
||||
ip.IsMulticast() || ip.IsUnspecified() {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// guardFetch performs a GET with the scheme/IP guards, conditional-GET
|
||||
// validators, and a hard body cap. Returns the body bytes, the FINAL
|
||||
// URL after redirects (new-feed-url / migration detection), validators
|
||||
// to persist, and whether the server answered 304 Not Modified.
|
||||
type fetchResult struct {
|
||||
Body []byte
|
||||
FinalURL string
|
||||
ETag string
|
||||
LastModified string
|
||||
NotModified bool
|
||||
StatusCode int
|
||||
}
|
||||
|
||||
// podcastHTTPClientFactory is the indirection TESTS use: the SSRF suite
|
||||
// exercises the REAL client; fetch-driven integration tests swap in a
|
||||
// client that trusts httptest's loopback TLS cert. Never reassign
|
||||
// outside tests (use SetPodcastHTTPClientForTests).
|
||||
var podcastHTTPClientFactory = podcastHTTPClient
|
||||
|
||||
// SetPodcastHTTPClientForTests swaps the fetch client. nil restores the
|
||||
// production client (SSRF guards intact).
|
||||
func SetPodcastHTTPClientForTests(c *http.Client) {
|
||||
if c == nil {
|
||||
podcastHTTPClientFactory = podcastHTTPClient
|
||||
return
|
||||
}
|
||||
podcastHTTPClientFactory = func() *http.Client { return c }
|
||||
}
|
||||
|
||||
func guardFetch(ctx context.Context, rawURL, etag, lastModified string, maxSize int64) (*fetchResult, error) {
|
||||
u, err := url.Parse(rawURL)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid URL: %w", err)
|
||||
}
|
||||
if err := guardURLScheme(u); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// A recognizable UA; some podcast hosts 403 empty/default agents.
|
||||
req.Header.Set("User-Agent", "Bookhoard/1.0 (podcast fetcher; +https://github.com/bookhoard)")
|
||||
req.Header.Set("Accept", "*/*")
|
||||
if etag != "" {
|
||||
req.Header.Set("If-None-Match", etag)
|
||||
}
|
||||
if lastModified != "" {
|
||||
req.Header.Set("If-Modified-Since", lastModified)
|
||||
}
|
||||
|
||||
resp, err := podcastHTTPClientFactory().Do(req)
|
||||
if err != nil {
|
||||
// Unwrap redirect/scheme guard errors so callers can classify.
|
||||
return nil, err
|
||||
}
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
|
||||
res := &fetchResult{
|
||||
FinalURL: resp.Request.URL.String(),
|
||||
ETag: resp.Header.Get("ETag"),
|
||||
LastModified: resp.Header.Get("Last-Modified"),
|
||||
StatusCode: resp.StatusCode,
|
||||
}
|
||||
if resp.StatusCode == http.StatusNotModified {
|
||||
res.NotModified = true
|
||||
return res, nil
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return res, fmt.Errorf("fetch %s: HTTP %d", rawURL, resp.StatusCode)
|
||||
}
|
||||
|
||||
body, err := io.ReadAll(io.LimitReader(resp.Body, maxSize+1))
|
||||
if err != nil {
|
||||
return res, fmt.Errorf("fetch %s: %w", rawURL, err)
|
||||
}
|
||||
if int64(len(body)) > maxSize {
|
||||
return res, fmt.Errorf("fetch %s: response exceeds %d byte cap", rawURL, maxSize)
|
||||
}
|
||||
res.Body = body
|
||||
return res, nil
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
// Podcast feed poller (plan §7 task 1.6): one periodic worker ticking
|
||||
// on the podcast_poll_interval_seconds system setting (re-read each
|
||||
// cycle — the settings API path keeps the registry cache honest),
|
||||
// polling every non-complete feed and applying subscriber auto-download
|
||||
// policies after each successful fetch (D6).
|
||||
//
|
||||
// RLS: detached work under the first-admin identity — the service's
|
||||
// adminCtx supplies it (invariant 12). Back-off: itunes:complete feeds
|
||||
// are skipped by the ticker (explicit refresh still works).
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
)
|
||||
|
||||
type PodcastPoller struct {
|
||||
db *database.Queries
|
||||
svc *PodcastService
|
||||
worker *PodcastDownloadWorker
|
||||
settings *database.SettingsRegistry
|
||||
}
|
||||
|
||||
func NewPodcastPoller(db *database.Queries, svc *PodcastService, worker *PodcastDownloadWorker, settings *database.SettingsRegistry) *PodcastPoller {
|
||||
return &PodcastPoller{db: db, svc: svc, worker: worker, settings: settings}
|
||||
}
|
||||
|
||||
// Start launches the poll loop; exits when ctx is cancelled.
|
||||
func (p *PodcastPoller) Start(ctx context.Context) {
|
||||
go func() {
|
||||
for {
|
||||
interval := p.settings.PodcastPollInterval()
|
||||
if interval < time.Minute {
|
||||
interval = time.Minute // floor against bad settings rows
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-time.After(interval):
|
||||
}
|
||||
p.tick(ctx)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// tick polls every non-complete feed once.
|
||||
func (p *PodcastPoller) tick(ctx context.Context) {
|
||||
actx, err := p.svc.adminCtx(ctx)
|
||||
if err != nil {
|
||||
log.Printf("[PODCAST-POLL] no acting identity, skipping tick: %v", err)
|
||||
return
|
||||
}
|
||||
feeds, err := p.db.ListPodcastFeedsForPolling(actx)
|
||||
if err != nil {
|
||||
log.Printf("[PODCAST-POLL] worklist: %v", err)
|
||||
return
|
||||
}
|
||||
for _, feed := range feeds {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
if feed.IsComplete {
|
||||
continue // back off ended feeds (explicit refresh still works)
|
||||
}
|
||||
if err := p.svc.PollFeed(actx, &feed); err != nil {
|
||||
log.Printf("[PODCAST-POLL] %s: %v", feed.Title, err)
|
||||
continue
|
||||
}
|
||||
// Fresh episodes → apply the subscriber policy union (auto-
|
||||
// download without waiting for the next tick).
|
||||
if _, err := p.worker.ApplyAutoPolicies(actx, feed.ID); err != nil {
|
||||
log.Printf("[PODCAST-POLL] auto-download policies for %s: %v", feed.Title, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,560 @@
|
||||
// RSS 2.0 + iTunes + Podcasting-2.0 feed parsing for the podcast
|
||||
// fetcher (plan podcast-integration.md §5/§7). Stdlib encoding/xml only
|
||||
// — no new deps.
|
||||
//
|
||||
// Namespace strategy: iTunes fields MUST be namespace-qualified in the
|
||||
// struct tags (the itunes URI) because their local names collide with
|
||||
// core RSS ("title", "subtitle", "author", "summary", "explicit",
|
||||
// "image", "duration"). Podcasting-2.0 fields (podcast:guid,
|
||||
// podcast:transcript, ...) have collision-free local names, so they
|
||||
// match on local name — encoding/xml matches Name.Local when the tag
|
||||
// carries no namespace, which also tolerates both declared namespace
|
||||
// URIs the ecosystem uses.
|
||||
//
|
||||
// Out of scope at parse time (plan §5): podcast:value, liveItem,
|
||||
// soundbite, alternateEnclosure, socialInteract, remoteItem/podroll,
|
||||
// locked/block, txt, chat. lastBuildDate/ttl are poll hints only.
|
||||
package services
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// itunesNS is the canonical iTunes podcast namespace URI.
|
||||
const itunesNS = "http://www.itunes.com/dtds/podcast-1.0.dtd"
|
||||
|
||||
// PodcastFeedXML is one parsed RSS document.
|
||||
type PodcastFeedXML struct {
|
||||
XMLName xml.Name `xml:"rss"`
|
||||
Channel PodcastChannelXML `xml:"channel"`
|
||||
}
|
||||
|
||||
type PodcastChannelXML struct {
|
||||
Title string `xml:"title"`
|
||||
Link string `xml:"link"`
|
||||
Language string `xml:"language"`
|
||||
Description string `xml:"description"`
|
||||
|
||||
// iTunes channel namespace.
|
||||
ITunesAuthor string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd author"`
|
||||
ITunesOwner struct {
|
||||
Name string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd name"`
|
||||
Email string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd email"`
|
||||
} `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd owner"`
|
||||
ITunesImage struct {
|
||||
Href string `xml:"href,attr"`
|
||||
} `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd image"`
|
||||
ITunesExplicit string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd explicit"`
|
||||
ITunesType string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd type"`
|
||||
ITunesCategory struct {
|
||||
Text string `xml:"text,attr"`
|
||||
Subcategory struct {
|
||||
Text string `xml:"text,attr"`
|
||||
} `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd category"`
|
||||
} `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd category"`
|
||||
ITunesNewFeedURL string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd new-feed-url"`
|
||||
ITunesComplete string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd complete"`
|
||||
|
||||
// Podcasting 2.0 (matched on local name; see file comment).
|
||||
PodcastGUID string `xml:"guid"`
|
||||
PodcastMedium string `xml:"medium"`
|
||||
|
||||
LastBuildDate string `xml:"lastBuildDate"`
|
||||
|
||||
Items []PodcastItemXML `xml:"item"`
|
||||
}
|
||||
|
||||
type PodcastItemXML struct {
|
||||
GUID string `xml:"guid"`
|
||||
Title string `xml:"title"`
|
||||
ITunesTitle string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd title"`
|
||||
Subtitle string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd subtitle"`
|
||||
// Description resolution order (plan §5): description >
|
||||
// itunes:summary > content:encoded.
|
||||
Description string `xml:"description"`
|
||||
ITunesSummary string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd summary"`
|
||||
ContentEncoded string `xml:"encoded"`
|
||||
|
||||
// Item author: itunes:author preferred, then RSS author, then dc:creator.
|
||||
ITunesAuthor string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd author"`
|
||||
Author string `xml:"author"`
|
||||
Creator string `xml:"http://purl.org/dc/elements/1.1/ creator"`
|
||||
|
||||
ITunesImage struct {
|
||||
Href string `xml:"href,attr"`
|
||||
} `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd image"`
|
||||
ITunesExplicit string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd explicit"`
|
||||
ITunesDuration string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd duration"`
|
||||
ITunesEpisode string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd episode"`
|
||||
ITunesSeason string `xml:"http://www.itunes.com/dtds/podcast-1.0.dtd season"`
|
||||
|
||||
PubDate string `xml:"pubDate"`
|
||||
|
||||
Enclosure struct {
|
||||
URL string `xml:"url,attr"`
|
||||
Length int64 `xml:"length,attr"`
|
||||
Type string `xml:"type,attr"`
|
||||
} `xml:"enclosure"`
|
||||
|
||||
// Podcasting 2.0: store-now, feature-later URLs.
|
||||
Transcript struct {
|
||||
URL string `xml:"url,attr"`
|
||||
} `xml:"transcript"`
|
||||
Chapters struct {
|
||||
URL string `xml:"url,attr"`
|
||||
} `xml:"chapters"`
|
||||
Persons []struct {
|
||||
Href string `xml:"href,attr"`
|
||||
Role string `xml:"role,attr"`
|
||||
} `xml:"person"`
|
||||
}
|
||||
|
||||
// ParsePodcastFeed decodes a feed document via a namespace-aware token
|
||||
// decoder (see decodePodcastFeed). Returns an error when the XML is
|
||||
// unparseable or the document is not an RSS channel.
|
||||
func ParsePodcastFeed(data []byte) (*PodcastFeedXML, error) {
|
||||
feed, err := decodePodcastFeed(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("feed parse: %w", err)
|
||||
}
|
||||
if feed.Channel.Title == "" && len(feed.Channel.Items) == 0 {
|
||||
return nil, fmt.Errorf("feed parse: not an RSS channel (no title, no items)")
|
||||
}
|
||||
return feed, nil
|
||||
}
|
||||
|
||||
// EpisodeTitle prefers itunes:title over the raw RSS title (plan §5).
|
||||
func (it *PodcastItemXML) EpisodeTitle() string {
|
||||
if t := strings.TrimSpace(it.ITunesTitle); t != "" {
|
||||
return t
|
||||
}
|
||||
return strings.TrimSpace(it.Title)
|
||||
}
|
||||
|
||||
// EpisodeDescription resolves the description fallback chain.
|
||||
func (it *PodcastItemXML) EpisodeDescription() string {
|
||||
for _, c := range []string{it.Description, it.ITunesSummary, it.ContentEncoded} {
|
||||
if t := strings.TrimSpace(c); t != "" {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// EpisodeAuthor resolves the author fallback chain.
|
||||
func (it *PodcastItemXML) EpisodeAuthor() string {
|
||||
for _, c := range []string{it.ITunesAuthor, it.Author, it.Creator} {
|
||||
if t := strings.TrimSpace(c); t != "" {
|
||||
return t
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// EpisodeExplicit maps the itunes:explicit spelling variants to bool.
|
||||
func (it *PodcastItemXML) EpisodeExplicit() bool {
|
||||
return parseExplicit(it.ITunesExplicit)
|
||||
}
|
||||
|
||||
func parseExplicit(v string) bool {
|
||||
switch strings.ToLower(strings.TrimSpace(v)) {
|
||||
case "true", "yes", "explicit", "1":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// HostsAndGuests splits podcast:person entries by role (plan §5).
|
||||
func (it *PodcastItemXML) HostsAndGuests() (hosts []string, guests []string) {
|
||||
for _, p := range it.Persons {
|
||||
name := p.Href
|
||||
if name == "" {
|
||||
continue
|
||||
}
|
||||
switch strings.ToLower(p.Role) {
|
||||
case "guest":
|
||||
guests = append(guests, name)
|
||||
default: // host and unlabeled roles surface as hosts
|
||||
hosts = append(hosts, name)
|
||||
}
|
||||
}
|
||||
return hosts, guests
|
||||
}
|
||||
|
||||
// parseITunesDuration accepts the three spellings the spec allows —
|
||||
// plain seconds, HH:MM:SS, MM:SS — plus a best-effort prose fallback
|
||||
// ("58 minutes"). Returns 0 for anything unparseable (the column stays
|
||||
// NULL; ffprobe reconciles after download anyway).
|
||||
func parseITunesDuration(v string) int {
|
||||
v = strings.TrimSpace(v)
|
||||
if v == "" {
|
||||
return 0
|
||||
}
|
||||
if secs, err := strconv.Atoi(v); err == nil {
|
||||
return secs
|
||||
}
|
||||
parts := strings.Split(v, ":")
|
||||
if len(parts) <= 3 {
|
||||
total := 0
|
||||
ok := true
|
||||
for _, p := range parts {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(p))
|
||||
if err != nil {
|
||||
ok = false
|
||||
break
|
||||
}
|
||||
total = total*60 + n
|
||||
}
|
||||
if ok {
|
||||
return total
|
||||
}
|
||||
}
|
||||
// Prose best-effort: "<N> minutes/hours/seconds".
|
||||
lower := strings.ToLower(v)
|
||||
fields := strings.Fields(lower)
|
||||
for i, f := range fields {
|
||||
if n, err := strconv.Atoi(f); err == nil && i+1 < len(fields) {
|
||||
switch {
|
||||
case strings.HasPrefix(fields[i+1], "hour"):
|
||||
return n * 3600
|
||||
case strings.HasPrefix(fields[i+1], "minute"):
|
||||
return n * 60
|
||||
case strings.HasPrefix(fields[i+1], "second"):
|
||||
return n
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
// parseFeedTime parses the RFC-822/1123 variants feeds actually emit.
|
||||
func parseFeedTime(v string) (t time.Time, ok bool) {
|
||||
v = strings.TrimSpace(v)
|
||||
if v == "" {
|
||||
return time.Time{}, false
|
||||
}
|
||||
layouts := []string{
|
||||
time.RFC1123Z,
|
||||
time.RFC1123,
|
||||
time.RFC822Z,
|
||||
time.RFC822,
|
||||
"Mon, 2 Jan 2006 15:04:05 -0700",
|
||||
"Mon, 2 Jan 2006 15:04:05 MST",
|
||||
"2 Jan 2006 15:04:05 -0700",
|
||||
"2 Jan 2006 15:04:05 MST",
|
||||
time.ANSIC,
|
||||
time.UnixDate,
|
||||
}
|
||||
for _, layout := range layouts {
|
||||
if t, err := time.Parse(layout, v); err == nil {
|
||||
return t, true
|
||||
}
|
||||
}
|
||||
return time.Time{}, false
|
||||
}
|
||||
|
||||
// ============================================================================
|
||||
// Namespace-aware decoding.
|
||||
//
|
||||
// WHY NOT encoding/xml: its field matching for UNQUALIFIED tags ignores
|
||||
// namespaces — a field tagged `xml:"title"` also matches
|
||||
// `<itunes:title>` (first field in declaration order wins, document
|
||||
// order clobbers), so the two collide unpredictably. RSS feeds stack
|
||||
// core RSS, iTunes and Podcasting-2.0 namespaces over the SAME local
|
||||
// names (title, author, description, guid), so a token-loop decoder
|
||||
// dispatching on (namespace, local) is the only deterministic shape.
|
||||
// It also tolerates BOTH podcast-namespace URIs the ecosystem uses and
|
||||
// arbitrary prefix spellings, because matching is on resolved
|
||||
// namespace, never on the prefix string.
|
||||
// ============================================================================
|
||||
|
||||
// decodePodcastFeed runs the token loop and fills the document struct.
|
||||
func decodePodcastFeed(data []byte) (*PodcastFeedXML, error) {
|
||||
dec := xml.NewDecoder(bytes.NewReader(data))
|
||||
dec.Strict = false // feeds in the wild carry stray entities; be liberal
|
||||
|
||||
feed := &PodcastFeedXML{}
|
||||
// Find the <rss> start element (tolerate comments/PIs/doctype).
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if se, ok := tok.(xml.StartElement); ok {
|
||||
if se.Name.Local == "rss" {
|
||||
break
|
||||
}
|
||||
// Some feeds wrap in nothing else; only rss is accepted.
|
||||
return nil, fmt.Errorf("unexpected root element %q", se.Name.Local)
|
||||
}
|
||||
}
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
if t.Name.Local == "channel" {
|
||||
if err := decodeChannel(dec, &feed.Channel); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err := skipElement(dec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
case xml.EndElement:
|
||||
// </rss> — done.
|
||||
return feed, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// decodeChannel walks the channel's children.
|
||||
func decodeChannel(dec *xml.Decoder, ch *PodcastChannelXML) error {
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
ns, local := t.Name.Space, t.Name.Local
|
||||
switch {
|
||||
case ns == "" && local == "title":
|
||||
ch.Title, err = elementText(dec)
|
||||
case ns == "" && local == "description":
|
||||
ch.Description, err = elementText(dec)
|
||||
case ns == "" && local == "language":
|
||||
ch.Language, err = elementText(dec)
|
||||
case ns == "" && local == "link":
|
||||
ch.Link, err = elementText(dec)
|
||||
case ns == "" && local == "lastBuildDate":
|
||||
ch.LastBuildDate, err = elementText(dec)
|
||||
case ns == itunesNS && local == "author":
|
||||
ch.ITunesAuthor, err = elementText(dec)
|
||||
case ns == itunesNS && local == "owner":
|
||||
err = decodeITunesOwner(dec, ch)
|
||||
case ns == itunesNS && local == "image":
|
||||
ch.ITunesImage.Href = attrValue(t.Attr, "href")
|
||||
err = skipElement(dec)
|
||||
case ns == itunesNS && local == "explicit":
|
||||
ch.ITunesExplicit, err = elementText(dec)
|
||||
case ns == itunesNS && local == "type":
|
||||
ch.ITunesType, err = elementText(dec)
|
||||
case ns == itunesNS && local == "new-feed-url":
|
||||
ch.ITunesNewFeedURL, err = elementText(dec)
|
||||
case ns == itunesNS && local == "complete":
|
||||
ch.ITunesComplete, err = elementText(dec)
|
||||
case ns == itunesNS && local == "category":
|
||||
ch.ITunesCategory.Text = attrValue(t.Attr, "text")
|
||||
err = decodeITunesCategory(dec, ch)
|
||||
case ns != "" && local == "guid":
|
||||
// podcast:guid (either declared URI); core RSS has no
|
||||
// channel-level guid, so any namespaced guid is it.
|
||||
ch.PodcastGUID, err = elementText(dec)
|
||||
case ns != "" && local == "medium":
|
||||
ch.PodcastMedium, err = elementText(dec)
|
||||
case ns == "" && local == "item":
|
||||
var item PodcastItemXML
|
||||
if err := decodeItem(dec, &item); err != nil {
|
||||
return err
|
||||
}
|
||||
ch.Items = append(ch.Items, item)
|
||||
continue
|
||||
default:
|
||||
err = skipElement(dec)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
case xml.EndElement:
|
||||
return nil // </channel>
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func decodeITunesOwner(dec *xml.Decoder, ch *PodcastChannelXML) error {
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
if t.Name.Space == itunesNS && t.Name.Local == "name" {
|
||||
if ch.ITunesOwner.Name, err = elementText(dec); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if t.Name.Space == itunesNS && t.Name.Local == "email" {
|
||||
if ch.ITunesOwner.Email, err = elementText(dec); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err := skipElement(dec); err != nil {
|
||||
return err
|
||||
}
|
||||
case xml.EndElement:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func decodeITunesCategory(dec *xml.Decoder, ch *PodcastChannelXML) error {
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
if t.Name.Space == itunesNS && t.Name.Local == "category" {
|
||||
ch.ITunesCategory.Subcategory.Text = attrValue(t.Attr, "text")
|
||||
}
|
||||
if err := skipElement(dec); err != nil {
|
||||
return err
|
||||
}
|
||||
case xml.EndElement:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// decodeItem walks one <item>.
|
||||
func decodeItem(dec *xml.Decoder, it *PodcastItemXML) error {
|
||||
for {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
ns, local := t.Name.Space, t.Name.Local
|
||||
switch {
|
||||
case ns == "" && local == "guid":
|
||||
it.GUID, err = elementText(dec)
|
||||
case ns == "" && local == "title":
|
||||
it.Title, err = elementText(dec)
|
||||
case ns == "" && local == "description":
|
||||
it.Description, err = elementText(dec)
|
||||
case ns == "" && local == "pubDate":
|
||||
it.PubDate, err = elementText(dec)
|
||||
case ns == "" && local == "author":
|
||||
it.Author, err = elementText(dec)
|
||||
case ns == "" && local == "enclosure":
|
||||
it.Enclosure.URL = attrValue(t.Attr, "url")
|
||||
it.Enclosure.Type = attrValue(t.Attr, "type")
|
||||
if raw := attrValue(t.Attr, "length"); raw != "" {
|
||||
it.Enclosure.Length, _ = strconv.ParseInt(raw, 10, 64)
|
||||
}
|
||||
err = skipElement(dec)
|
||||
case ns == itunesNS && local == "title":
|
||||
it.ITunesTitle, err = elementText(dec)
|
||||
case ns == itunesNS && local == "subtitle":
|
||||
it.Subtitle, err = elementText(dec)
|
||||
case ns == itunesNS && local == "summary":
|
||||
it.ITunesSummary, err = elementText(dec)
|
||||
case ns == itunesNS && local == "author":
|
||||
it.ITunesAuthor, err = elementText(dec)
|
||||
case ns == itunesNS && local == "explicit":
|
||||
it.ITunesExplicit, err = elementText(dec)
|
||||
case ns == itunesNS && local == "duration":
|
||||
it.ITunesDuration, err = elementText(dec)
|
||||
case ns == itunesNS && local == "episode":
|
||||
it.ITunesEpisode, err = elementText(dec)
|
||||
case ns == itunesNS && local == "season":
|
||||
it.ITunesSeason, err = elementText(dec)
|
||||
case ns == itunesNS && local == "image":
|
||||
it.ITunesImage.Href = attrValue(t.Attr, "href")
|
||||
err = skipElement(dec)
|
||||
case ns == "http://purl.org/dc/elements/1.1/" && local == "creator":
|
||||
it.Creator, err = elementText(dec)
|
||||
case ns == "http://purl.org/rss/1.0/modules/content/" && local == "encoded":
|
||||
it.ContentEncoded, err = elementText(dec)
|
||||
case ns != "" && local == "transcript":
|
||||
it.Transcript.URL = attrValue(t.Attr, "url")
|
||||
err = skipElement(dec)
|
||||
case ns != "" && local == "chapters":
|
||||
it.Chapters.URL = attrValue(t.Attr, "url")
|
||||
err = skipElement(dec)
|
||||
case ns != "" && local == "person":
|
||||
href, role := attrValue(t.Attr, "href"), attrValue(t.Attr, "role")
|
||||
it.Persons = append(it.Persons, struct {
|
||||
Href string `xml:"href,attr"`
|
||||
Role string `xml:"role,attr"`
|
||||
}{Href: href, Role: role})
|
||||
err = skipElement(dec)
|
||||
default:
|
||||
err = skipElement(dec)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
case xml.EndElement:
|
||||
return nil // </item>
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// elementText accumulates chardata until this element closes (leaf-ish
|
||||
// elements: CDATA sections come through as xml.CharData and are
|
||||
// included; child elements are skipped).
|
||||
func elementText(dec *xml.Decoder) (string, error) {
|
||||
var sb strings.Builder
|
||||
depth := 1
|
||||
for depth > 0 {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
switch t := tok.(type) {
|
||||
case xml.StartElement:
|
||||
depth++
|
||||
case xml.EndElement:
|
||||
depth--
|
||||
case xml.CharData:
|
||||
if depth == 1 {
|
||||
sb.Write(t)
|
||||
}
|
||||
}
|
||||
}
|
||||
return strings.TrimSpace(sb.String()), nil
|
||||
}
|
||||
|
||||
// skipElement consumes an element we don't model (depth-safe).
|
||||
func skipElement(dec *xml.Decoder) error {
|
||||
depth := 1
|
||||
for depth > 0 {
|
||||
tok, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch tok.(type) {
|
||||
case xml.StartElement:
|
||||
depth++
|
||||
case xml.EndElement:
|
||||
depth--
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// attrValue returns the named attribute (local-name match; the
|
||||
// attributes we read carry no namespace).
|
||||
func attrValue(attrs []xml.Attr, local string) string {
|
||||
for _, a := range attrs {
|
||||
if a.Name.Local == local {
|
||||
return strings.TrimSpace(a.Value)
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,303 @@
|
||||
// Unit tests for the podcast fetcher's pure logic (plan task 1.4/1.10):
|
||||
// RSS/iTunes/Podcasting-2.0 parsing, duration/pubDate spellings, slug/
|
||||
// path layout, and the SSRF guard (scheme + resolved-IP checks). No DB.
|
||||
package services
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
// pgUUID alias keeps the helper signatures short.
|
||||
type pgUUID = pgtype.UUID
|
||||
|
||||
// Full-fixture feed covering the three namespaces and the fallback
|
||||
// chains (itunes:title vs title; description vs itunes:summary vs
|
||||
// content:encoded; itunes:author vs author vs dc:creator).
|
||||
const podcastFixtureXML = `<?xml version="1.0" encoding="UTF-8"?>
|
||||
<rss version="2.0"
|
||||
xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
|
||||
xmlns:content="http://purl.org/rss/1.0/modules/content/"
|
||||
xmlns:dc="http://purl.org/dc/elements/1.1/"
|
||||
xmlns:podcast="https://podcastindex.org/namespace/1.0">
|
||||
<channel>
|
||||
<title>The Test Feed</title>
|
||||
<link>https://example.com/show</link>
|
||||
<language>en-us</language>
|
||||
<description><p>A feed about <b>testing</b>.</p></description>
|
||||
<itunes:author>Host Name</itunes:author>
|
||||
<itunes:owner>
|
||||
<itunes:name>Owner Name</itunes:name>
|
||||
<itunes:email>owner@example.com</itunes:email>
|
||||
</itunes:owner>
|
||||
<itunes:image href="https://cdn.example.com/art.jpg"/>
|
||||
<itunes:explicit>false</itunes:explicit>
|
||||
<itunes:type>serial</itunes:type>
|
||||
<itunes:category text="Fiction">
|
||||
<itunes:category text="Drama"/>
|
||||
</itunes:category>
|
||||
<podcast:guid>9b0241c2-xxxx-4xxx-yxxx-xxxxxxxxxxxx</podcast:guid>
|
||||
<podcast:medium>audiobook</podcast:medium>
|
||||
<lastBuildDate>Mon, 26 Oct 2026 08:00:00 +0000</lastBuildDate>
|
||||
<item>
|
||||
<guid isPermaLink="false">episode-guid-1</guid>
|
||||
<itunes:title>Episode One: The Beginning</itunes:title>
|
||||
<title>Ep 1</title>
|
||||
<itunes:subtitle>First chapter</itunes:subtitle>
|
||||
<description><![CDATA[<p>Chapter one text.</p>]]></description>
|
||||
<itunes:author>Narrator A</itunes:author>
|
||||
<itunes:duration>1:02:03</itunes:duration>
|
||||
<itunes:episode>1</itunes:episode>
|
||||
<itunes:season>2</itunes:season>
|
||||
<pubDate>Sun, 25 Oct 2026 12:00:00 +0000</pubDate>
|
||||
<enclosure url="https://cdn.example.com/ep1.mp3" length="52428800" type="audio/mpeg"/>
|
||||
<podcast:transcript url="https://cdn.example.com/ep1.json" type="application/json"/>
|
||||
<podcast:chapters url="https://cdn.example.com/ep1-chapters.json" type="application/json+chapters"/>
|
||||
<podcast:person href="Narrator A" role="host"/>
|
||||
<podcast:person href="Guest B" role="guest"/>
|
||||
</item>
|
||||
<item>
|
||||
<guid>episode-guid-2</guid>
|
||||
<title>Episode Two</title>
|
||||
<itunes:summary>Summary only description.</itunes:summary>
|
||||
<dc:creator>Writer C</dc:creator>
|
||||
<itunes:duration>5400</itunes:duration>
|
||||
<pubDate>Mon, 26 Oct 2026 12:00:00 GMT</pubDate>
|
||||
<enclosure url="https://cdn.example.com/ep2.m4a" length="1048576" type="audio/mp4"/>
|
||||
</item>
|
||||
</channel>
|
||||
</rss>`
|
||||
|
||||
func TestParsePodcastFeed(t *testing.T) {
|
||||
feed, err := ParsePodcastFeed([]byte(podcastFixtureXML))
|
||||
require.NoError(t, err)
|
||||
|
||||
ch := feed.Channel
|
||||
assert.Equal(t, "The Test Feed", ch.Title)
|
||||
assert.Equal(t, "en-us", ch.Language)
|
||||
assert.Equal(t, "Host Name", ch.ITunesAuthor)
|
||||
assert.Equal(t, "Owner Name", ch.ITunesOwner.Name)
|
||||
assert.Equal(t, "owner@example.com", ch.ITunesOwner.Email)
|
||||
assert.Equal(t, "https://cdn.example.com/art.jpg", ch.ITunesImage.Href)
|
||||
assert.Equal(t, "serial", ch.ITunesType)
|
||||
assert.Equal(t, "Fiction", ch.ITunesCategory.Text)
|
||||
assert.Equal(t, "Drama", ch.ITunesCategory.Subcategory.Text)
|
||||
assert.Equal(t, "9b0241c2-xxxx-4xxx-yxxx-xxxxxxxxxxxx", ch.PodcastGUID)
|
||||
assert.Equal(t, "audiobook", ch.PodcastMedium)
|
||||
assert.False(t, parseExplicit(ch.ITunesExplicit))
|
||||
|
||||
buildAt, ok := parseFeedTime(ch.LastBuildDate)
|
||||
require.True(t, ok)
|
||||
assert.Equal(t, 2026, buildAt.Year())
|
||||
|
||||
require.Len(t, ch.Items, 2)
|
||||
|
||||
// Item 1: itunes:title preferred, description chain, hosts/guests.
|
||||
it1 := &ch.Items[0]
|
||||
assert.Equal(t, "Episode One: The Beginning", it1.EpisodeTitle())
|
||||
assert.Equal(t, "First chapter", it1.Subtitle)
|
||||
assert.Equal(t, "<p>Chapter one text.</p>", it1.EpisodeDescription()) // raw HTML; stripping happens at storage time
|
||||
assert.Equal(t, "Narrator A", it1.EpisodeAuthor())
|
||||
assert.Equal(t, 3723, parseITunesDuration(it1.ITunesDuration)) // 1:02:03
|
||||
assert.Equal(t, 2, intMust(it1.ITunesSeason))
|
||||
assert.Equal(t, 1, intMust(it1.ITunesEpisode))
|
||||
assert.Equal(t, "https://cdn.example.com/ep1.mp3", it1.Enclosure.URL)
|
||||
assert.Equal(t, int64(52428800), it1.Enclosure.Length)
|
||||
assert.Equal(t, "audio/mpeg", it1.Enclosure.Type)
|
||||
assert.Equal(t, "https://cdn.example.com/ep1.json", it1.Transcript.URL)
|
||||
assert.Equal(t, "https://cdn.example.com/ep1-chapters.json", it1.Chapters.URL)
|
||||
hosts, guests := it1.HostsAndGuests()
|
||||
assert.Equal(t, []string{"Narrator A"}, hosts)
|
||||
assert.Equal(t, []string{"Guest B"}, guests)
|
||||
assert.Equal(t, "2026-10-25", mustDate(it1.PubDate))
|
||||
|
||||
// Item 2: fallback chains — plain title, itunes:summary, dc:creator.
|
||||
it2 := &ch.Items[1]
|
||||
assert.Equal(t, "Episode Two", it2.EpisodeTitle())
|
||||
assert.Equal(t, "Summary only description.", it2.EpisodeDescription())
|
||||
assert.Equal(t, "Writer C", it2.EpisodeAuthor())
|
||||
assert.Equal(t, 5400, parseITunesDuration(it2.ITunesDuration)) // seconds
|
||||
assert.Equal(t, "https://cdn.example.com/ep2.m4a", it2.Enclosure.URL)
|
||||
}
|
||||
|
||||
func intMust(s string) int {
|
||||
n, _ := parseSeasonOrEpisode(s)
|
||||
return n
|
||||
}
|
||||
|
||||
func parseSeasonOrEpisode(s string) (int, bool) {
|
||||
if s == "" {
|
||||
return 0, false
|
||||
}
|
||||
n := 0
|
||||
for _, r := range s {
|
||||
if r < '0' || r > '9' {
|
||||
return 0, false
|
||||
}
|
||||
n = n*10 + int(r-'0')
|
||||
}
|
||||
return n, true
|
||||
}
|
||||
|
||||
func mustDate(pubDate string) string {
|
||||
if t, ok := parseFeedTime(pubDate); ok {
|
||||
return t.UTC().Format("2006-01-02")
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func TestParseITunesDuration(t *testing.T) {
|
||||
assert.Equal(t, 0, parseITunesDuration(""))
|
||||
assert.Equal(t, 90, parseITunesDuration("90"))
|
||||
assert.Equal(t, 3723, parseITunesDuration("1:02:03"))
|
||||
assert.Equal(t, 123, parseITunesDuration("2:03"))
|
||||
assert.Equal(t, 3600, parseITunesDuration("1 hour"))
|
||||
assert.Equal(t, 120, parseITunesDuration("2 minutes"))
|
||||
assert.Equal(t, 0, parseITunesDuration("soonish"))
|
||||
}
|
||||
|
||||
func TestParseFeedTimeVariants(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
in string
|
||||
year int
|
||||
}{
|
||||
{"Mon, 26 Oct 2026 08:00:00 +0000", 2026},
|
||||
{"Mon, 26 Oct 2026 08:00:00 GMT", 2026},
|
||||
{"26 Oct 2026 08:00:00 +0000", 2026},
|
||||
} {
|
||||
tm, ok := parseFeedTime(tc.in)
|
||||
require.True(t, ok, "parse %q", tc.in)
|
||||
assert.Equal(t, tc.year, tm.Year())
|
||||
}
|
||||
_, ok := parseFeedTime("not a date")
|
||||
assert.False(t, ok)
|
||||
}
|
||||
|
||||
func TestEpisodeExplicitSpellings(t *testing.T) {
|
||||
assert.True(t, parseExplicit("true"))
|
||||
assert.True(t, parseExplicit("yes"))
|
||||
assert.True(t, parseExplicit("explicit"))
|
||||
assert.False(t, parseExplicit("false"))
|
||||
assert.False(t, parseExplicit("clean"))
|
||||
assert.False(t, parseExplicit(""))
|
||||
}
|
||||
|
||||
func TestSlugifyAndPaths(t *testing.T) {
|
||||
assert.Equal(t, "the-test-feed", slugify("The Test Feed!", 60))
|
||||
assert.Equal(t, "a-b-c", slugify(" A -- B // C ", 60))
|
||||
assert.Equal(t, "abcdefghij", slugify("ABCDEFGHIJ", 10))
|
||||
|
||||
slug := feedSlug("My Feed", testUUID("11111111-1111-1111-1111-111111111111"))
|
||||
assert.Equal(t, "my-feed-11111111", slug)
|
||||
|
||||
rel := episodeRelPath(slug, 42, "Episode: The Test", ".mp3")
|
||||
assert.Equal(t, "my-feed-11111111/0042-episode-the-test.mp3", rel)
|
||||
|
||||
assert.Equal(t, "my-feed-11111111/artwork.jpg", artworkRelPath(slug, ".jpg"))
|
||||
}
|
||||
|
||||
func TestEnclosureExtension(t *testing.T) {
|
||||
assert.Equal(t, ".mp3", enclosureExtension("https://x.test/a/b/ep.mp3?token=1", ""))
|
||||
assert.Equal(t, ".m4a", enclosureExtension("https://x.test/ep", "audio/mp4"))
|
||||
assert.Equal(t, ".flac", enclosureExtension("https://x.test/ep", "audio/flac"))
|
||||
assert.Equal(t, ".mp3", enclosureExtension("https://x.test/ep", "weird/type"))
|
||||
}
|
||||
|
||||
func TestStripHTML(t *testing.T) {
|
||||
assert.Equal(t, "hello world", stripHTML("<p>hello <b>world</b></p>"))
|
||||
assert.Equal(t, "a b", stripHTML("a\n\n b"))
|
||||
assert.Equal(t, "", stripHTML(""))
|
||||
}
|
||||
|
||||
func TestIsPublicIP(t *testing.T) {
|
||||
blocked := []string{
|
||||
"127.0.0.1", "10.0.0.5", "172.16.0.9", "192.168.1.1",
|
||||
"169.254.169.254", // cloud metadata endpoint
|
||||
"0.0.0.0", "::1", "fe80::1", "fd00::5", "::ffff:127.0.0.1",
|
||||
"::ffff:10.0.0.1", "224.0.0.1",
|
||||
}
|
||||
for _, s := range blocked {
|
||||
assert.False(t, isPublicIP(net.ParseIP(s)), "%s must be blocked", s)
|
||||
}
|
||||
allowed := []string{
|
||||
"93.184.216.34", "1.1.1.1", "2606:4700:4700::1111",
|
||||
}
|
||||
for _, s := range allowed {
|
||||
assert.True(t, isPublicIP(net.ParseIP(s)), "%s must be allowed", s)
|
||||
}
|
||||
assert.False(t, isPublicIP(nil))
|
||||
}
|
||||
|
||||
func TestGuardURLScheme(t *testing.T) {
|
||||
u, _ := url.Parse("http://example.com/feed.xml")
|
||||
assert.Error(t, guardURLScheme(u))
|
||||
u, _ = url.Parse("ftp://example.com/feed.xml")
|
||||
assert.Error(t, guardURLScheme(u))
|
||||
u, _ = url.Parse("https://example.com/feed.xml")
|
||||
assert.NoError(t, guardURLScheme(u))
|
||||
assert.Error(t, guardURLScheme(nil))
|
||||
}
|
||||
|
||||
// The REAL client (production factory) must refuse loopback targets —
|
||||
// this is the SSRF boundary; httptest servers live on 127.0.0.1.
|
||||
func TestGuardFetchBlocksLoopback(t *testing.T) {
|
||||
srv := newInsecureFeedServer(t, podcastFixtureXML)
|
||||
defer srv.Close()
|
||||
_, err := guardFetch(t.Context(), srv.URL, "", "", maxFeedSizeBytes)
|
||||
require.Error(t, err)
|
||||
assert.True(t, strings.Contains(err.Error(), "blocked") ||
|
||||
strings.Contains(err.Error(), "https"), "expected an SSRF/scheme rejection, got: %v", err)
|
||||
}
|
||||
|
||||
func TestGuardFetchRejectsPlainHTTP(t *testing.T) {
|
||||
// A public-looking http:// URL: scheme guard fires before any dial.
|
||||
_, err := guardFetch(t.Context(), "http://example.com/feed.xml", "", "", maxFeedSizeBytes)
|
||||
require.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "https")
|
||||
}
|
||||
|
||||
func TestClassifyFetchError(t *testing.T) {
|
||||
err := classifyFetchError(ErrBlockedPrivateAddress)
|
||||
assert.Contains(t, err.Error(), "blocked")
|
||||
someErr := &url.Error{Op: "Get", Err: errors.New("boom")}
|
||||
assert.Equal(t, someErr.Error(), classifyFetchError(someErr).Error())
|
||||
}
|
||||
|
||||
// testUUID builds a pgtype.UUID from a string (test helper).
|
||||
func testUUID(s string) pgUUID {
|
||||
u, _ := uuid.Parse(s)
|
||||
return pgUUID{Bytes: u, Valid: true}
|
||||
}
|
||||
|
||||
// newInsecureFeedServer serves body over httptest TLS (loopback) —
|
||||
// used to prove the production client REFUSES it.
|
||||
func newInsecureFeedServer(t *testing.T, body string) *httptest.Server {
|
||||
t.Helper()
|
||||
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/rss+xml")
|
||||
_, _ = w.Write([]byte(body))
|
||||
}))
|
||||
t.Cleanup(srv.Close)
|
||||
return srv
|
||||
}
|
||||
|
||||
// Apple Podcast launcher schemes normalize to the https feed they wrap.
|
||||
func TestNormalizeFeedURL(t *testing.T) {
|
||||
assert.Equal(t, "https://x.test/feed.xml", normalizeFeedURL("itpc://x.test/feed.xml"))
|
||||
assert.Equal(t, "https://x.test/feed.xml", normalizeFeedURL("podcasts://x.test/feed.xml"))
|
||||
assert.Equal(t, "https://x.test/feed.xml", normalizeFeedURL("feed://x.test/feed.xml"))
|
||||
assert.Equal(t, "https://x.test/feed.xml", normalizeFeedURL("ITPC://x.test/feed.xml"), "scheme match is case-insensitive")
|
||||
assert.Equal(t, "https://x.test/feed.xml", normalizeFeedURL("https://x.test/feed.xml"))
|
||||
assert.Equal(t, "http://x.test/feed.xml", normalizeFeedURL("http://x.test/feed.xml"), "http stays — the guard rejects it downstream")
|
||||
assert.Equal(t, "", normalizeFeedURL(" "))
|
||||
}
|
||||
@@ -0,0 +1,785 @@
|
||||
// Podcast fetcher (plan podcast-integration.md §7): fetch → parse →
|
||||
// upsert-without-delete. The fetcher is the SOLE writer of feed/episode
|
||||
// metadata (D9) — user-facing podcast writes are limited to
|
||||
// podcast_subscriptions rows and the reader_settings order override.
|
||||
//
|
||||
// RLS identity (invariant 12): fetches run detached from any request,
|
||||
// so every governed statement runs under the FIRST-ADMIN identity
|
||||
// (scanner fallback precedent). With the D20 admin escape that identity
|
||||
// sees and updates every feed/episode row. NEVER a bypass flag, never
|
||||
// the bootstrap superuser.
|
||||
//
|
||||
// Upsert semantics: episodes are created/updated METADATA-ONLY, keyed
|
||||
// on (source_feed_id, episode_guid); rows for items that vanish from
|
||||
// the XML are NEVER deleted (history accumulates — D14 search covers
|
||||
// gone episodes). Download state is owned by the download worker; the
|
||||
// upsert cannot clobber it (guarded in the query itself).
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/rls"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// PodcastError is a status-carrying service error: handlers map it to
|
||||
// the HTTP response directly (services stay echo-free).
|
||||
type PodcastError struct {
|
||||
Status int
|
||||
Message string
|
||||
}
|
||||
|
||||
func (e *PodcastError) Error() string { return e.Message }
|
||||
|
||||
func errNotFound(msg string) error { return &PodcastError{Status: 404, Message: msg} }
|
||||
func errForbidden(msg string) error { return &PodcastError{Status: 403, Message: msg} }
|
||||
func errBadRequest(msg string) error { return &PodcastError{Status: 400, Message: msg} }
|
||||
|
||||
// PodcastService carries the fetcher/worker dependencies. One instance
|
||||
// per server (handlers + poller share it so the per-feed locks dedupe).
|
||||
type PodcastService struct {
|
||||
db *database.Queries
|
||||
|
||||
adminMu sync.Mutex
|
||||
adminID pgtype.UUID
|
||||
adminOnce bool
|
||||
|
||||
// Per-feed mutexes: a poll and a download for the same feed must
|
||||
// not race (e.g. both writing the episode row). Keyed by feed UUID.
|
||||
feedLocksMu sync.Mutex
|
||||
feedLocks map[string]*sync.Mutex
|
||||
}
|
||||
|
||||
func NewPodcastService(db *database.Queries) *PodcastService {
|
||||
return &PodcastService{
|
||||
db: db,
|
||||
feedLocks: make(map[string]*sync.Mutex),
|
||||
}
|
||||
}
|
||||
|
||||
// feedLock returns the per-feed mutex (creates on first use).
|
||||
func (s *PodcastService) feedLock(feedID string) *sync.Mutex {
|
||||
s.feedLocksMu.Lock()
|
||||
defer s.feedLocksMu.Unlock()
|
||||
if m, ok := s.feedLocks[feedID]; ok {
|
||||
return m
|
||||
}
|
||||
m := &sync.Mutex{}
|
||||
s.feedLocks[feedID] = m
|
||||
return m
|
||||
}
|
||||
|
||||
// adminIdentity resolves the acting identity for detached podcast work:
|
||||
// the first admin, resolved once per service lifetime. Errors surface
|
||||
// to the caller (no admin → no fetch — same contract as the scanner).
|
||||
func (s *PodcastService) adminIdentity() (pgtype.UUID, error) {
|
||||
s.adminMu.Lock()
|
||||
defer s.adminMu.Unlock()
|
||||
if !s.adminOnce && !s.adminID.Valid {
|
||||
id, err := s.db.GetFirstAdmin(context.Background())
|
||||
if err != nil {
|
||||
return pgtype.UUID{}, fmt.Errorf("podcasts: no admin identity available for detached work: %w", err)
|
||||
}
|
||||
s.adminID = id
|
||||
s.adminOnce = true
|
||||
}
|
||||
if !s.adminID.Valid {
|
||||
return pgtype.UUID{}, fmt.Errorf("podcasts: no admin identity available for detached work")
|
||||
}
|
||||
return s.adminID, nil
|
||||
}
|
||||
|
||||
// adminCtx wraps ctx with the first-admin identity (invariant 12).
|
||||
func (s *PodcastService) adminCtx(ctx context.Context) (context.Context, error) {
|
||||
id, err := s.adminIdentity()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return rls.WithUser(ctx, id), nil
|
||||
}
|
||||
|
||||
// AddFeedResult carries what the add-feed flow did (the handler turns
|
||||
// it into the response / serial feed hint).
|
||||
type AddFeedResult struct {
|
||||
Feed *database.PodcastFeeds
|
||||
// Created is true when this call created the feed row; false when
|
||||
// an existing row was reused (URL dedupe or guid migration match).
|
||||
Created bool
|
||||
// Subscribed is true when the requesting user's subscription was
|
||||
// created by this call (false = already subscribed).
|
||||
Subscribed bool
|
||||
// EpisodeCount is the number of episode rows the fetch upserted.
|
||||
EpisodeCount int
|
||||
// FetchError, when set, means the feed row exists but the fetch
|
||||
// failed — the subscription stands, the error surfaces to the user.
|
||||
FetchError string
|
||||
}
|
||||
|
||||
// FeedError returns the fetch failure text ("" when the fetch was fine
|
||||
// or there is no feed row).
|
||||
func (r *AddFeedResult) FeedError() string {
|
||||
return r.FetchError
|
||||
}
|
||||
|
||||
// AddFeed implements POST /api/podcasts/feeds (plan §8): fetch+parse the
|
||||
// user-supplied URL, create-or-reuse the feed row, upsert metadata-only
|
||||
// episodes, subscribe the creator. Authorization on explicit arguments
|
||||
// (D20 scoping model): the target library must be visible to the actor
|
||||
// and, for non-admins, allow member feeds.
|
||||
func (s *PodcastService) AddFeed(ctx context.Context, userID pgtype.UUID, isAdmin bool, libraryID pgtype.UUID, rawFeedURL string) (*AddFeedResult, error) {
|
||||
// ---- explicit-argument authorization (the ONLY Go-level gate) ----
|
||||
lib, err := s.db.GetLibrary(ctx, libraryID)
|
||||
if err != nil {
|
||||
// Invisible-or-nonexistent: 404 semantics (existence leak).
|
||||
return nil, errNotFound("library not found")
|
||||
}
|
||||
if !isAdmin {
|
||||
if lib.TypeName != LibraryTypePodcasts {
|
||||
return nil, errNotFound("library not found")
|
||||
}
|
||||
if !lib.AllowMemberFeeds {
|
||||
return nil, errForbidden("this library does not allow members to add feeds")
|
||||
}
|
||||
}
|
||||
|
||||
feedURL := normalizeFeedURL(rawFeedURL)
|
||||
if feedURL == "" {
|
||||
return nil, errBadRequest("feed URL is required")
|
||||
}
|
||||
|
||||
// The fetch/upsert runs under the first-admin identity (detached-
|
||||
// equivalent); the request context never carries special powers into
|
||||
// feed writes.
|
||||
actx, err := s.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
res := &AddFeedResult{}
|
||||
|
||||
// ---- fetch + parse (SSRF-guarded; plan §7) ----
|
||||
fetched, err := guardFetch(ctx, feedURL, "", "", maxFeedSizeBytes)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not fetch feed: %w", classifyFetchError(err))
|
||||
}
|
||||
parsed, err := ParsePodcastFeed(fetched.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not parse feed: %w", err)
|
||||
}
|
||||
|
||||
// Follow itunes:new-feed-url once at add time so the row is born
|
||||
// pointing at the publisher's current home (migration semantics).
|
||||
finalURL := fetched.FinalURL
|
||||
if neu := strings.TrimSpace(parsed.Channel.ITunesNewFeedURL); neu != "" && neu != finalURL {
|
||||
if retried, err := guardFetch(ctx, neu, "", "", maxFeedSizeBytes); err == nil {
|
||||
if reparsed, err := ParsePodcastFeed(retried.Body); err == nil {
|
||||
parsed = reparsed
|
||||
finalURL = retried.FinalURL
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
title := parsed.Channel.Title
|
||||
if title == "" {
|
||||
title = "Untitled feed"
|
||||
}
|
||||
|
||||
// ---- create-or-reuse (UNIQUE(library_id, feed_url) + guid dedupe) ----
|
||||
feed, err := s.db.CreatePodcastFeed(actx, database.CreatePodcastFeedParams{
|
||||
LibraryID: libraryID,
|
||||
FeedUrl: finalURL,
|
||||
Title: truncate(title, 500),
|
||||
CreatedByAdminID: pgtype.UUID{}, // fetcher owns feed metadata; creator attribution is not user metadata
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("could not create feed: %w", err)
|
||||
}
|
||||
res.Feed = &feed
|
||||
res.Created = true
|
||||
|
||||
// Guid migration dedupe: a row with the same canonical podcast_guid
|
||||
// under a different URL reuses THAT row (do not fork the feed).
|
||||
if guid := strings.TrimSpace(parsed.Channel.PodcastGUID); guid != "" {
|
||||
if feed.PodcastGuid.String == "" || feed.PodcastGuid.String != guid {
|
||||
if existing, err := s.db.GetPodcastFeedByGuid(actx, database.GetPodcastFeedByGuidParams{
|
||||
LibraryID: feed.LibraryID,
|
||||
PodcastGuid: pgtypeText(guid),
|
||||
}); err == nil && existing.ID != feed.ID {
|
||||
// Attach the subscriber to the canonical row; drop the
|
||||
// freshly created stub (nothing subscribed to it yet).
|
||||
_, _ = s.db.DeletePodcastFeed(actx, feed.ID)
|
||||
feed = existing
|
||||
res.Feed = &existing
|
||||
// Re-point the canonical row at the fetched URL if it moved.
|
||||
if feedUrlForComparison(existing.FeedUrl) != feedUrlForComparison(finalURL) {
|
||||
_ = s.db.UpdatePodcastFeedRedirect(actx, database.UpdatePodcastFeedRedirectParams{
|
||||
ID: existing.ID,
|
||||
FeedUrl: finalURL,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if _, err := s.db.SubscribePodcastFeed(actx, database.SubscribePodcastFeedParams{
|
||||
UserID: userID,
|
||||
FeedID: feed.ID,
|
||||
}); err == nil {
|
||||
res.Subscribed = true
|
||||
}
|
||||
|
||||
// ---- metadata + episodes ----
|
||||
if err := s.applyFetch(actx, &feed, parsed, fetched); err != nil {
|
||||
// Feed exists + subscribed, but the metadata pass failed: not
|
||||
// fatal for the request; the poller reconciles later.
|
||||
res.FetchError = err.Error()
|
||||
return res, nil
|
||||
}
|
||||
|
||||
updated, err := s.db.GetPodcastFeed(actx, feed.ID)
|
||||
if err == nil {
|
||||
res.Feed = &updated
|
||||
}
|
||||
return res, nil
|
||||
}
|
||||
|
||||
// RefreshFeed implements POST /api/podcasts/feeds/:id/refresh: a manual
|
||||
// poll. The per-feed lock serializes against poller ticks and downloads.
|
||||
func (s *PodcastService) RefreshFeed(ctx context.Context, feedID pgtype.UUID) error {
|
||||
actx, err := s.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
feed, err := s.db.GetPodcastFeed(actx, feedID)
|
||||
if err != nil {
|
||||
return errNotFound("feed not found")
|
||||
}
|
||||
return s.PollFeed(actx, &feed)
|
||||
}
|
||||
|
||||
// PollFeed fetches + reconciles ONE feed (caller supplies the acting
|
||||
// identity in ctx). Safe to call concurrently per feed — the per-feed
|
||||
// lock serializes poll/download.
|
||||
func (s *PodcastService) PollFeed(ctx context.Context, feed *database.PodcastFeeds) error {
|
||||
lock := s.feedLock(feed.ID.String())
|
||||
lock.Lock()
|
||||
defer lock.Unlock()
|
||||
|
||||
// Complete feeds back off: they are fetched only on explicit
|
||||
// refresh (the poller skips them entirely — this path is manual).
|
||||
fetched, err := guardFetch(ctx, feed.FeedUrl, feed.Etag.String, feed.LastModified.String, maxFeedSizeBytes)
|
||||
if err != nil {
|
||||
if feed.LastFetchedAt.Valid {
|
||||
// A previously-good feed with a transient failure: keep the
|
||||
// old state; the poller retries next tick.
|
||||
return fmt.Errorf("fetch %s: %w", feed.Title, classifyFetchError(err))
|
||||
}
|
||||
return fmt.Errorf("fetch %s: %w", feed.Title, classifyFetchError(err))
|
||||
}
|
||||
if fetched.NotModified {
|
||||
_, _ = s.db.UpdatePodcastFeedFetch(ctx, database.UpdatePodcastFeedFetchParams{
|
||||
ID: feed.ID,
|
||||
Etag: pgtypeText(feed.Etag.String),
|
||||
LastModified: pgtypeText(feed.LastModified.String),
|
||||
LastBuildDate: feed.LastBuildDate,
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
parsed, err := ParsePodcastFeed(fetched.Body)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse %s: %w", feed.Title, err)
|
||||
}
|
||||
|
||||
// itunes:new-feed-url: the publisher MOVED the feed. Rewrite the
|
||||
// stored URL (row, subscriptions and episodes stay put) and refetch
|
||||
// the new location once so this poll applies its content.
|
||||
if neu := strings.TrimSpace(parsed.Channel.ITunesNewFeedURL); neu != "" && neu != feed.FeedUrl {
|
||||
_ = s.db.UpdatePodcastFeedRedirect(ctx, database.UpdatePodcastFeedRedirectParams{ID: feed.ID, FeedUrl: neu})
|
||||
if retried, err := guardFetch(ctx, neu, "", "", maxFeedSizeBytes); err == nil {
|
||||
if reparsed, err := ParsePodcastFeed(retried.Body); err == nil {
|
||||
parsed = reparsed
|
||||
fetched = retried
|
||||
}
|
||||
}
|
||||
} else if fetched.FinalURL != "" && feedUrlForComparison(fetched.FinalURL) != feedUrlForComparison(feed.FeedUrl) {
|
||||
// A permanent HTTP redirect chain moved the feed; persist the
|
||||
// final URL so conditional validators stop bouncing.
|
||||
_ = s.db.UpdatePodcastFeedRedirect(ctx, database.UpdatePodcastFeedRedirectParams{ID: feed.ID, FeedUrl: fetched.FinalURL})
|
||||
}
|
||||
|
||||
if err := s.applyFetch(ctx, feed, parsed, fetched); err != nil {
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// applyFetch writes channel metadata, upserts episodes metadata-only,
|
||||
// touches validators, and downloads artwork. Write-after-success rule
|
||||
// (invariant 7): failures here never clear existing rows.
|
||||
func (s *PodcastService) applyFetch(ctx context.Context, feed *database.PodcastFeeds, parsed *PodcastFeedXML, fetched *fetchResult) error {
|
||||
// --- channel metadata (fetcher-owned) ---
|
||||
title := parsed.Channel.Title
|
||||
if title == "" {
|
||||
title = feed.Title // never blank out a title on a broken doc
|
||||
}
|
||||
desc := stripHTML(parsed.Channel.Description)
|
||||
|
||||
host := parsed.Channel.ITunesAuthor
|
||||
publisher := parsed.Channel.ITunesOwner.Name
|
||||
|
||||
tagSet := map[string]bool{}
|
||||
if cat := strings.TrimSpace(parsed.Channel.ITunesCategory.Text); cat != "" {
|
||||
tagSet[cat] = true
|
||||
}
|
||||
if sub := strings.TrimSpace(parsed.Channel.ITunesCategory.Subcategory.Text); sub != "" {
|
||||
tagSet[sub] = true
|
||||
}
|
||||
tags := sortedStringSet(tagSet)
|
||||
|
||||
feedType := strings.TrimSpace(parsed.Channel.ITunesType)
|
||||
if feedType != "serial" && feedType != "episodic" {
|
||||
feedType = "" // unknown → column stays NULL
|
||||
}
|
||||
medium := strings.TrimSpace(parsed.Channel.PodcastMedium)
|
||||
|
||||
if _, err := s.db.UpdatePodcastFeedMetadata(ctx, database.UpdatePodcastFeedMetadataParams{
|
||||
ID: feed.ID,
|
||||
Title: truncate(title, 500),
|
||||
Description: pgtypeText(desc),
|
||||
Host: pgtypeText(truncate(host, 500)),
|
||||
Publisher: pgtypeText(truncate(publisher, 500)),
|
||||
OwnerName: pgtypeText(truncate(parsed.Channel.ITunesOwner.Name, 500)),
|
||||
OwnerEmail: pgtypeText(truncate(parsed.Channel.ITunesOwner.Email, 500)),
|
||||
Language: pgtypeText(truncate(parsed.Channel.Language, 50)),
|
||||
Tags: tags,
|
||||
Explicit: parseExplicit(parsed.Channel.ITunesExplicit),
|
||||
FeedType: pgtypeText(feedType),
|
||||
Medium: pgtypeText(truncate(medium, 20)),
|
||||
IsComplete: parseExplicit(parsed.Channel.ITunesComplete),
|
||||
PodcastGuid: pgtypeText(strings.TrimSpace(parsed.Channel.PodcastGUID)),
|
||||
}); err != nil {
|
||||
return fmt.Errorf("feed metadata update: %w", err)
|
||||
}
|
||||
// itunes:complete flips the poller's back-off flag (complete feeds
|
||||
// poll rarely); explicit refresh still works.
|
||||
|
||||
// --- episodes (metadata-only upsert, positions from THIS parse) ---
|
||||
root, err := s.downloadRootAbs(ctx, feed)
|
||||
if err != nil {
|
||||
return fmt.Errorf("download root: %w", err)
|
||||
}
|
||||
feedSlug := feedSlug(title, feed.ID)
|
||||
|
||||
count := 0
|
||||
for i := range parsed.Channel.Items {
|
||||
if err := s.upsertEpisode(ctx, feed, feedSlug, &parsed.Channel.Items[i], i+1); err != nil {
|
||||
// One bad item must not sink the batch.
|
||||
log.Printf("[PODCAST] episode upsert failed for feed %s item %d: %v", feed.ID, i+1, err)
|
||||
continue
|
||||
}
|
||||
count++
|
||||
}
|
||||
|
||||
// --- validators + fetch timestamp ---
|
||||
var lastBuildDate pgtype.Timestamptz
|
||||
if t, ok := parseFeedTime(parsed.Channel.LastBuildDate); ok {
|
||||
lastBuildDate = pgtypeTimestamptz(t)
|
||||
}
|
||||
if _, err := s.db.UpdatePodcastFeedFetch(ctx, database.UpdatePodcastFeedFetchParams{
|
||||
ID: feed.ID,
|
||||
Etag: pgtypeText(fetched.ETag),
|
||||
LastModified: pgtypeText(fetched.LastModified),
|
||||
LastBuildDate: lastBuildDate,
|
||||
}); err != nil {
|
||||
return fmt.Errorf("fetch touch: %w", err)
|
||||
}
|
||||
|
||||
// --- artwork (best-effort; feed renders fine without it) ---
|
||||
s.refreshArtwork(ctx, feed, root, parsed)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// upsertEpisode maps one XML item onto a media_items row (plan §5 field
|
||||
// inventory). Positions are 1-based, renumbered per fetch; file_path is
|
||||
// the PLANNED final layout — the upsert only applies it to rows that
|
||||
// are not yet downloaded (query-side guard); the abs root resolves at
|
||||
// download time via the library folders.
|
||||
func (s *PodcastService) upsertEpisode(ctx context.Context, feed *database.PodcastFeeds, feedSlug string, item *PodcastItemXML, position int) error {
|
||||
title := item.EpisodeTitle()
|
||||
if title == "" {
|
||||
title = fmt.Sprintf("Episode %d", position)
|
||||
}
|
||||
guid := strings.TrimSpace(item.GUID)
|
||||
if guid == "" {
|
||||
// GUID-less feeds exist; fall back to a stable identity from the
|
||||
// enclosure URL (the next-best immutable per-episode key).
|
||||
if item.Enclosure.URL != "" {
|
||||
guid = "enc:" + item.Enclosure.URL
|
||||
} else {
|
||||
return fmt.Errorf("item %q has neither guid nor enclosure; skipping", title)
|
||||
}
|
||||
}
|
||||
|
||||
ext := enclosureExtension(item.Enclosure.URL, item.Enclosure.Type)
|
||||
relPath := episodeRelPath(feedSlug, position, title, ext)
|
||||
|
||||
var published pgtype.Date
|
||||
if t, ok := parseFeedTime(item.PubDate); ok {
|
||||
published = pgtypeDate(t)
|
||||
}
|
||||
|
||||
authors := []string{}
|
||||
if a := item.EpisodeAuthor(); a != "" {
|
||||
authors = append(authors, truncate(a, 255))
|
||||
}
|
||||
hosts, guests := item.HostsAndGuests()
|
||||
|
||||
_, err := s.db.UpsertPodcastEpisode(ctx, database.UpsertPodcastEpisodeParams{
|
||||
LibraryID: feed.LibraryID,
|
||||
Title: truncate(title, 255),
|
||||
Subtitle: pgtypeText(truncate(item.Subtitle, 500)),
|
||||
Description: pgtypeText(stripHTML(item.EpisodeDescription())),
|
||||
Authors: authors,
|
||||
AuthorsSearch: lowerSearchSlice(authors),
|
||||
FilePath: relPath,
|
||||
FileSize: pgtypeInt8(item.Enclosure.Length),
|
||||
MimeType: pgtypeText(truncate(item.Enclosure.Type, 100)),
|
||||
CoverImagePath: pgtypeText(artworkRelPath(feedSlug, artworkExtFromURL(item.ITunesImage.Href))),
|
||||
Explicit: item.EpisodeExplicit(),
|
||||
DatePublished: published,
|
||||
Language: pgtypeText(truncate(feed.Language.String, 10)),
|
||||
Tags: feed.Tags,
|
||||
Genre: pgtypeText(strings.Join(feed.Tags, ", ")),
|
||||
SourceFeedID: feed.ID,
|
||||
EpisodeGuid: pgtypeText(truncate(guid, 500)),
|
||||
FeedPosition: pgtypeInt4(int32(position)),
|
||||
SeasonNumber: pgtypeInt4FromStr(item.ITunesSeason),
|
||||
EpisodeNumber: pgtypeInt4FromStr(item.ITunesEpisode),
|
||||
EnclosureUrl: pgtypeText(item.Enclosure.URL),
|
||||
DeclaredDurationSeconds: pgtypeInt4(int32(parseITunesDuration(item.ITunesDuration))),
|
||||
TranscriptUrl: pgtypeText(item.Transcript.URL),
|
||||
ChaptersUrl: pgtypeText(item.Chapters.URL),
|
||||
Hosts: truncateSlice(hosts, 255),
|
||||
Guests: truncateSlice(guests, 255),
|
||||
})
|
||||
return err
|
||||
}
|
||||
|
||||
// refreshArtwork downloads the channel image once per changed URL.
|
||||
// Artwork is feed-level: episodes point at the same relative path.
|
||||
func (s *PodcastService) refreshArtwork(ctx context.Context, feed *database.PodcastFeeds, downloadRoot string, parsed *PodcastFeedXML) {
|
||||
imgURL := strings.TrimSpace(parsed.Channel.ITunesImage.Href)
|
||||
if imgURL == "" {
|
||||
return
|
||||
}
|
||||
target := filepath.Join(downloadRoot, artworkRelPath(feedSlug(feed.Title, feed.ID), artworkExtFromURL(imgURL)))
|
||||
if feed.ArtworkPath.Valid {
|
||||
if existing := filepath.Join(downloadRoot, feed.ArtworkPath.String); existing == target {
|
||||
if _, err := os.Stat(existing); err == nil {
|
||||
return // unchanged
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||
return
|
||||
}
|
||||
res, err := guardFetch(ctx, imgURL, "", "", maxArtworkSizeBytes)
|
||||
if err != nil || len(res.Body) == 0 {
|
||||
log.Printf("[PODCAST] artwork fetch failed for feed %s: %v", feed.ID, err)
|
||||
return
|
||||
}
|
||||
if err := os.WriteFile(target, res.Body, 0o644); err != nil {
|
||||
log.Printf("[PODCAST] artwork write failed for feed %s: %v", feed.ID, err)
|
||||
return
|
||||
}
|
||||
rel, err := filepath.Rel(downloadRoot, target)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_ = s.db.UpdatePodcastFeedArtwork(ctx, database.UpdatePodcastFeedArtworkParams{
|
||||
ID: feed.ID,
|
||||
ArtworkPath: pgtypeText(rel),
|
||||
})
|
||||
}
|
||||
|
||||
// DeleteFeedFiles removes a feed's episode files + artwork from the
|
||||
// download root (admin feed-removal path; ?files=false skips it).
|
||||
// Best-effort per file — the row cascade proceeds regardless.
|
||||
func (s *PodcastService) DeleteFeedFiles(ctx context.Context, feed *database.PodcastFeeds, episodes []database.MediaItems) {
|
||||
root, err := s.downloadRootAbs(ctx, feed)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
for _, ep := range episodes {
|
||||
if ep.FilePath != "" {
|
||||
_ = os.Remove(filepath.Join(root, ep.FilePath))
|
||||
// Embedded-cover sibling (extractEmbeddedAudioCover convention).
|
||||
base := filepath.Join(root, strings.TrimSuffix(ep.FilePath, filepath.Ext(ep.FilePath)))
|
||||
_ = os.Remove(base + ".cover.jpg")
|
||||
}
|
||||
}
|
||||
if feed.ArtworkPath.Valid {
|
||||
_ = os.Remove(filepath.Join(root, feed.ArtworkPath.String))
|
||||
}
|
||||
// The feed directory itself: remove when empty (other feeds never
|
||||
// share a slug — the feed-id suffix guarantees that).
|
||||
_ = os.Remove(filepath.Join(root, feedSlug(feed.Title, feed.ID)))
|
||||
}
|
||||
|
||||
// downloadRootAbs resolves the podcasts library's single download root
|
||||
// (purpose='podcast') as an absolute filesystem path.
|
||||
func (s *PodcastService) downloadRootAbs(ctx context.Context, feed *database.PodcastFeeds) (string, error) {
|
||||
folders, err := s.db.GetPodcastLibraryFolder(ctx, feed.LibraryID)
|
||||
if err != nil || len(folders) == 0 {
|
||||
return "", fmt.Errorf("library %s has no download root registered", feed.LibraryID)
|
||||
}
|
||||
return folders[0].FolderPath, nil
|
||||
}
|
||||
|
||||
// episodeRelPath is the pinned episode layout (plan §6):
|
||||
// <feed-slug>/<feed_position:04d>-<slug>.<ext> — relative to the
|
||||
// download root. getFullFilePath/ServeFile/Range streaming resolve it
|
||||
// unchanged.
|
||||
func episodeRelPath(feedSlug string, position int, title, ext string) string {
|
||||
return fmt.Sprintf("%s/%04d-%s%s", feedSlug, position, slugify(title, 80), ext)
|
||||
}
|
||||
|
||||
// artworkRelPath is the feed-level artwork location (feed-level, so
|
||||
// every episode row can point at the same file).
|
||||
func artworkRelPath(feedSlug, ext string) string {
|
||||
return fmt.Sprintf("%s/artwork%s", feedSlug, ext)
|
||||
}
|
||||
|
||||
// feedSlug derives the stable per-feed directory name: slugged title +
|
||||
// short feed-id suffix so two identically-titled feeds never collide.
|
||||
func feedSlug(title string, feedID pgtype.UUID) string {
|
||||
base := slugify(title, 60)
|
||||
suffix := "feed"
|
||||
if feedID.Valid {
|
||||
s := feedID.String()
|
||||
if len(s) >= 8 {
|
||||
suffix = s[:8]
|
||||
}
|
||||
}
|
||||
if base == "" {
|
||||
base = "feed"
|
||||
}
|
||||
return base + "-" + suffix
|
||||
}
|
||||
|
||||
var slugStrip = regexp.MustCompile(`[^a-z0-9]+`)
|
||||
|
||||
// slugify lowercases, strips non-alphanumerics to dashes, and clamps
|
||||
// the length (file-name-safe; unicode letters drop — file layout is
|
||||
// an internal key, never display text).
|
||||
func slugify(s string, maxLen int) string {
|
||||
s = strings.ToLower(strings.TrimSpace(s))
|
||||
s = slugStrip.ReplaceAllString(s, "-")
|
||||
s = strings.Trim(s, "-")
|
||||
if len(s) > maxLen {
|
||||
s = strings.TrimRight(s[:maxLen], "-")
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// enclosureExtension picks a file extension from the URL, then the
|
||||
// MIME type, then a safe default.
|
||||
func enclosureExtension(rawURL, mimeType string) string {
|
||||
if u, err := urlParse(rawURL); err == nil {
|
||||
if ext := strings.ToLower(filepath.Ext(u.Path)); ext == ".mp3" || ext == ".m4a" || ext == ".m4b" ||
|
||||
ext == ".flac" || ext == ".ogg" || ext == ".oga" || ext == ".opus" || ext == ".wav" || ext == ".aac" {
|
||||
return ext
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case strings.Contains(mimeType, "mp4") || strings.Contains(mimeType, "m4a"):
|
||||
return ".m4a"
|
||||
case strings.Contains(mimeType, "mpeg") || strings.Contains(mimeType, "mp3"):
|
||||
return ".mp3"
|
||||
case strings.Contains(mimeType, "flac"):
|
||||
return ".flac"
|
||||
case strings.Contains(mimeType, "ogg"):
|
||||
return ".ogg"
|
||||
case strings.Contains(mimeType, "wav"):
|
||||
return ".wav"
|
||||
}
|
||||
return ".mp3"
|
||||
}
|
||||
|
||||
// artworkExtFromURL keeps the source extension for artwork (jpg/png/webp).
|
||||
func artworkExtFromURL(rawURL string) string {
|
||||
if u, err := urlParse(rawURL); err == nil {
|
||||
switch strings.ToLower(filepath.Ext(u.Path)) {
|
||||
case ".png":
|
||||
return ".png"
|
||||
case ".webp":
|
||||
return ".webp"
|
||||
case ".gif":
|
||||
return ".gif"
|
||||
}
|
||||
}
|
||||
return ".jpg"
|
||||
}
|
||||
|
||||
// stripHTML removes tags and collapses whitespace — feed descriptions
|
||||
// are HTML; storage wants text (the TUI-ish surfaces render it plain).
|
||||
func stripHTML(s string) string {
|
||||
if s == "" {
|
||||
return ""
|
||||
}
|
||||
var b strings.Builder
|
||||
inTag := false
|
||||
for _, r := range s {
|
||||
switch {
|
||||
case r == '<':
|
||||
inTag = true
|
||||
case r == '>':
|
||||
inTag = false
|
||||
case !inTag:
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
out := b.String()
|
||||
// Collapse runs of whitespace.
|
||||
out = strings.Join(strings.Fields(out), " ")
|
||||
return strings.TrimSpace(out)
|
||||
}
|
||||
|
||||
// truncate clamps a string to max runes (VARCHAR safety at the DB edge).
|
||||
func truncate(s string, max int) string {
|
||||
s = strings.TrimSpace(s)
|
||||
runes := []rune(s)
|
||||
if len(runes) <= max {
|
||||
return s
|
||||
}
|
||||
return string(runes[:max])
|
||||
}
|
||||
|
||||
// truncateSlice clamps each element and drops empties.
|
||||
func truncateSlice(in []string, max int) []string {
|
||||
out := make([]string, 0, len(in))
|
||||
for _, s := range in {
|
||||
if t := truncate(s, max); t != "" {
|
||||
out = append(out, t)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// lowerSearchSlice mirrors authors into authors_search (scanner convention).
|
||||
func lowerSearchSlice(in []string) []string {
|
||||
out := make([]string, 0, len(in))
|
||||
for _, s := range in {
|
||||
out = append(out, strings.ToLower(s))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// sortedStringSet → deterministic TEXT[] writes.
|
||||
func sortedStringSet(set map[string]bool) []string {
|
||||
out := make([]string, 0, len(set))
|
||||
for k := range set {
|
||||
if strings.TrimSpace(k) == "" {
|
||||
continue
|
||||
}
|
||||
out = append(out, strings.TrimSpace(k))
|
||||
}
|
||||
// tiny insertion sort — tag sets are 1–5 entries
|
||||
for i := 1; i < len(out); i++ {
|
||||
for j := i; j > 0 && out[j] < out[j-1]; j-- {
|
||||
out[j], out[j-1] = out[j-1], out[j]
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// classifyFetchError maps guard/HTTP errors to user-facing text (the
|
||||
// add-feed dialog and the download_error column both render it).
|
||||
// normalizeFeedURL accepts the launcher-scheme spellings podcast
|
||||
// pages hand out alongside plain RSS (issue follow-up: "one says RSS,
|
||||
// the other says Apple Podcast") and rewrites them to the https URL
|
||||
// they wrap: itpc:// and podcasts:// are Apple's subscribe-launcher
|
||||
// aliases for the SAME feed document; feed:// is the generic form.
|
||||
// Anything else keeps its scheme — guardFetch enforces https.
|
||||
func normalizeFeedURL(raw string) string {
|
||||
u := strings.TrimSpace(raw)
|
||||
lowered := strings.ToLower(u)
|
||||
for _, prefix := range []string{"itpc://", "podcasts://", "feed://"} {
|
||||
if strings.HasPrefix(lowered, prefix) {
|
||||
return "https://" + u[len(prefix):]
|
||||
}
|
||||
}
|
||||
return u
|
||||
}
|
||||
|
||||
func classifyFetchError(err error) error {
|
||||
if errors.Is(err, ErrBlockedPrivateAddress) {
|
||||
return fmt.Errorf("blocked: private/loopback addresses are not fetchable")
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
// urlParse local helper so podcast files don't each import net/url.
|
||||
func urlParse(raw string) (*url.URL, error) {
|
||||
return url.Parse(raw)
|
||||
}
|
||||
|
||||
// ---- pgtype helpers shared by the podcast files ----
|
||||
|
||||
func pgtypeText(s string) pgtype.Text {
|
||||
if strings.TrimSpace(s) == "" {
|
||||
return pgtype.Text{}
|
||||
}
|
||||
return pgtype.Text{String: s, Valid: true}
|
||||
}
|
||||
|
||||
func pgtypeInt4(v int32) pgtype.Int4 {
|
||||
if v == 0 {
|
||||
return pgtype.Int4{}
|
||||
}
|
||||
return pgtype.Int4{Int32: v, Valid: true}
|
||||
}
|
||||
|
||||
func pgtypeInt4FromStr(s string) pgtype.Int4 {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(s))
|
||||
if err != nil || n <= 0 {
|
||||
return pgtype.Int4{}
|
||||
}
|
||||
return pgtype.Int4{Int32: int32(n), Valid: true}
|
||||
}
|
||||
|
||||
func pgtypeInt8(v int64) pgtype.Int8 {
|
||||
if v <= 0 {
|
||||
return pgtype.Int8{}
|
||||
}
|
||||
return pgtype.Int8{Int64: v, Valid: true}
|
||||
}
|
||||
|
||||
func pgtypeTimestamptz(t time.Time) pgtype.Timestamptz {
|
||||
return pgtype.Timestamptz{Time: t, Valid: true}
|
||||
}
|
||||
|
||||
func pgtypeDate(t time.Time) pgtype.Date {
|
||||
return pgtype.Date{Time: t.UTC().Truncate(24 * time.Hour), Valid: true}
|
||||
}
|
||||
|
||||
// feedUrlForComparison normalizes trailing slashes for the redirect check.
|
||||
func feedUrlForComparison(u string) string {
|
||||
return strings.TrimRight(strings.TrimSpace(u), "/")
|
||||
}
|
||||
@@ -0,0 +1,627 @@
|
||||
// Podcast download worker (plan §7 / D6 / D8 / D19): ONE sequential
|
||||
// worker (day one) driving the episode state machine
|
||||
//
|
||||
// pending → queued → downloading → downloaded
|
||||
// ↘ failed (+ download_error)
|
||||
//
|
||||
// queued rows are the DB-backed worklist — the state is persisted so
|
||||
// the episode list ALWAYS renders truth (no in-memory queue to lose).
|
||||
//
|
||||
// Idempotency: a download request for an episode whose file already
|
||||
// exists and probes cleanly re-marks it downloaded without re-fetching
|
||||
// (crash-between-move-and-update reconciliation); a downloaded episode
|
||||
// re-request is a full no-op.
|
||||
//
|
||||
// Retry semantics: manual retry (same endpoint as download) resets
|
||||
// download_attempts; auto-download policies re-queue failed episodes on
|
||||
// later polls up to maxAutoDownloadAttempts, then stay failed until a
|
||||
// human retries.
|
||||
//
|
||||
// RLS: the worker runs detached — every statement executes under the
|
||||
// first-admin identity (invariant 12), same contract as the fetcher.
|
||||
package services
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"bookhoard/internal/database"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
// maxAutoDownloadAttempts caps POLICY-driven re-queues of a failing
|
||||
// episode; a human retry always resets the budget.
|
||||
const maxAutoDownloadAttempts = 5
|
||||
|
||||
// PodcastDownloadWorker processes queued episode downloads.
|
||||
type PodcastDownloadWorker struct {
|
||||
db *database.Queries
|
||||
svc *PodcastService
|
||||
wake chan struct{}
|
||||
|
||||
// Live byte-progress slots for in-flight downloads (issue #1).
|
||||
// Keyed by episode UUID string; guarded by progMu.
|
||||
progMu sync.Mutex
|
||||
progress map[string]*episodeProgress
|
||||
}
|
||||
|
||||
func NewPodcastDownloadWorker(db *database.Queries, svc *PodcastService) *PodcastDownloadWorker {
|
||||
return &PodcastDownloadWorker{
|
||||
db: db,
|
||||
svc: svc,
|
||||
wake: make(chan struct{}, 1),
|
||||
progress: make(map[string]*episodeProgress),
|
||||
}
|
||||
}
|
||||
|
||||
// Wake nudges the loop (new work queued).
|
||||
func (w *PodcastDownloadWorker) Wake() {
|
||||
select {
|
||||
case w.wake <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
|
||||
// Start launches the loop; it exits when ctx is cancelled. Called once
|
||||
// from route registration (scanner background-tasks precedent).
|
||||
func (w *PodcastDownloadWorker) Start(ctx context.Context) {
|
||||
go func() {
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-w.wake:
|
||||
case <-time.After(5 * time.Second):
|
||||
// Poll fallback: a queued row written by another process
|
||||
// (or a crashed worker's leftovers) still gets picked up.
|
||||
}
|
||||
w.drain(ctx)
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// drain processes queued episodes until the queue is empty.
|
||||
func (w *PodcastDownloadWorker) drain(ctx context.Context) {
|
||||
for {
|
||||
if ctx.Err() != nil {
|
||||
return
|
||||
}
|
||||
acted, err := w.processNext(ctx)
|
||||
if err != nil {
|
||||
log.Printf("[PODCAST-DL] worker step: %v", err)
|
||||
return
|
||||
}
|
||||
if !acted {
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// QueueEpisode marks one episode queued (manual download / retry path).
|
||||
// Idempotent by design: downloaded → no-op. The per-feed lock
|
||||
// serializes against polls rewriting the same row.
|
||||
func (w *PodcastDownloadWorker) QueueEpisode(ctx context.Context, episodeID, feedID pgtype.UUID) error {
|
||||
actx, err := w.svc.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
lock := w.svc.feedLock(feedID.String())
|
||||
lock.Lock()
|
||||
defer lock.Unlock()
|
||||
|
||||
ep, err := w.db.GetPodcastEpisode(actx, episodeID)
|
||||
if err != nil {
|
||||
return errNotFound("episode not found")
|
||||
}
|
||||
if ep.DownloadState == "downloaded" {
|
||||
return nil // idempotent no-op
|
||||
}
|
||||
// Manual path resets the attempt budget (SetPodcastEpisodePending)
|
||||
// then queues — a human retry always overrides auto-retry capping.
|
||||
if err := w.db.SetPodcastEpisodePending(actx, episodeID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := w.db.SetPodcastEpisodeDownloadQueued(actx, episodeID); err != nil {
|
||||
return err
|
||||
}
|
||||
w.Wake()
|
||||
return nil
|
||||
}
|
||||
|
||||
// QueueFeedDownloads implements "Download All" (D8): queue exactly the
|
||||
// non-downloaded set — pending AND failed rows (retry is the same path).
|
||||
func (w *PodcastDownloadWorker) QueueFeedDownloads(ctx context.Context, feedID pgtype.UUID) (int, error) {
|
||||
actx, err := w.svc.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
lock := w.svc.feedLock(feedID.String())
|
||||
lock.Lock()
|
||||
defer lock.Unlock()
|
||||
|
||||
eps, err := w.db.ListUndownloadedEpisodes(actx, feedID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
queued := 0
|
||||
for _, ep := range eps {
|
||||
if _, err := w.db.SetPodcastEpisodeDownloadQueued(actx, ep.ID); err == nil {
|
||||
queued++
|
||||
}
|
||||
}
|
||||
w.Wake()
|
||||
return queued, nil
|
||||
}
|
||||
|
||||
// ApplyAutoPolicies unions subscriber policies for a feed (D6: most
|
||||
// aggressive wins for what ARRIVES — deletion stays admin-only) and
|
||||
// queues the target set, respecting the auto-retry attempt cap.
|
||||
func (w *PodcastDownloadWorker) ApplyAutoPolicies(ctx context.Context, feedID pgtype.UUID) (int, error) {
|
||||
actx, err := w.svc.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
subs, err := w.db.ListFeedSubscribers(actx, feedID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
wantAll := false
|
||||
newestN := 0
|
||||
for _, sub := range subs {
|
||||
switch sub.AutoDownload {
|
||||
case "all":
|
||||
wantAll = true
|
||||
case "newest":
|
||||
if n := int(sub.AutoDownloadCount.Int32); n > newestN {
|
||||
newestN = n
|
||||
}
|
||||
}
|
||||
}
|
||||
if !wantAll && newestN == 0 {
|
||||
return 0, nil // nobody's policy asks for anything
|
||||
}
|
||||
|
||||
lock := w.svc.feedLock(feedID.String())
|
||||
lock.Lock()
|
||||
defer lock.Unlock()
|
||||
|
||||
eps, err := w.db.ListUndownloadedEpisodes(actx, feedID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// ListUndownloadedEpisodes is native-order (oldest first); "newest
|
||||
// N" takes the TAIL (highest positions = latest releases).
|
||||
targets := eps
|
||||
if !wantAll && newestN > 0 && len(eps) > newestN {
|
||||
targets = eps[len(eps)-newestN:]
|
||||
}
|
||||
|
||||
queued := 0
|
||||
for _, ep := range targets {
|
||||
// Auto-retry budget: only re-queue within the cap. A failed
|
||||
// episode past the cap waits for a human (its row renders the
|
||||
// reason + retry button — D8/D19).
|
||||
if ep.DownloadState == "failed" && int(ep.DownloadAttempts) >= maxAutoDownloadAttempts {
|
||||
continue
|
||||
}
|
||||
if _, err := w.db.SetPodcastEpisodeDownloadQueued(actx, ep.ID); err == nil {
|
||||
queued++
|
||||
}
|
||||
}
|
||||
if queued > 0 {
|
||||
w.Wake()
|
||||
}
|
||||
return queued, nil
|
||||
}
|
||||
|
||||
// processNext claims and processes one queued episode. Returns false
|
||||
// when the queue was empty.
|
||||
func (w *PodcastDownloadWorker) processNext(ctx context.Context) (bool, error) {
|
||||
actx, err := w.svc.adminCtx(ctx)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
ep, err := w.db.GetNextQueuedEpisode(actx)
|
||||
if err != nil {
|
||||
return false, nil // empty queue (or read failure — logged upstream)
|
||||
}
|
||||
|
||||
_ = w.db.SetPodcastEpisodeDownloading(actx, ep.ID)
|
||||
|
||||
if err := w.runPipeline(actx, ep); err != nil {
|
||||
// failed/rejected: state + reason persist; metadata/file facts
|
||||
// untouched (invariant 7 — write-after-success only).
|
||||
_ = w.db.MarkPodcastEpisodeFailed(actx, database.MarkPodcastEpisodeFailedParams{
|
||||
ID: ep.ID,
|
||||
DownloadError: pgtypeText(err.Error()),
|
||||
Column3: false, // auto failure: never resets the attempt budget
|
||||
})
|
||||
log.Printf("[PODCAST-DL] episode %s failed: %v", ep.ID, err)
|
||||
}
|
||||
return true, nil
|
||||
}
|
||||
|
||||
// runPipeline: fetch enclosure (SSRF/size guards) → temp file → MIME/
|
||||
// size validation → ffprobe reconcile → move into place → SUCCESS-only
|
||||
// row update.
|
||||
func (w *PodcastDownloadWorker) runPipeline(ctx context.Context, ep database.MediaItems) error {
|
||||
if !ep.EnclosureUrl.Valid || ep.EnclosureUrl.String == "" {
|
||||
return fmt.Errorf("episode has no enclosure URL")
|
||||
}
|
||||
if !ep.SourceFeedID.Valid {
|
||||
return fmt.Errorf("episode has no feed reference")
|
||||
}
|
||||
|
||||
feed, err := w.db.GetPodcastFeed(ctx, ep.SourceFeedID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("feed lookup: %w", errNotFound("feed not found"))
|
||||
}
|
||||
root, err := w.svc.downloadRootAbs(ctx, &feed)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Declared-size pre-check: reject hopeless downloads before
|
||||
// transferring a byte (validation rejections render like failures,
|
||||
// with their reason — D8).
|
||||
if ep.FileSize.Valid && ep.FileSize.Int64 > maxEnclosureCapBytes {
|
||||
return fmt.Errorf("rejected: enclosure declares %d bytes (cap %d)", ep.FileSize.Int64, maxEnclosureCapBytes)
|
||||
}
|
||||
|
||||
feedDir := filepath.Join(root, filepath.Dir(ep.FilePath))
|
||||
finalPath := filepath.Join(root, ep.FilePath)
|
||||
if err := os.MkdirAll(feedDir, 0o755); err != nil {
|
||||
return fmt.Errorf("download dir: %w", err)
|
||||
}
|
||||
|
||||
// Idempotency reconciliation: file already on disk and probes?
|
||||
if _, err := os.Stat(finalPath); err == nil {
|
||||
if payload := probeEpisodeAudio(finalPath); payload != nil {
|
||||
return w.markDownloaded(ctx, ep, &feed, finalPath, payload)
|
||||
}
|
||||
// Unprobeable stray: fall through and re-fetch over it.
|
||||
}
|
||||
|
||||
// ---- fetch to temp (streamed with byte-progress tracking) ----
|
||||
// Enclosures stream straight to disk through a counting loop — no
|
||||
// whole-body buffering — so the episode list can render a real
|
||||
// progress bar while the worker runs (issue #1).
|
||||
tmpPath := finalPath + ".part-" + randomSuffix()
|
||||
cleanup := func() { _ = os.Remove(tmpPath) }
|
||||
|
||||
received, total, fetchErr := w.streamEnclosure(ctx, ep, tmpPath)
|
||||
w.endProgress(ep.ID) // terminal either way; the row state is the truth
|
||||
if fetchErr != nil {
|
||||
cleanup()
|
||||
return fetchErr
|
||||
}
|
||||
_, _ = received, total
|
||||
|
||||
// ---- validation ----
|
||||
info, err := os.Stat(tmpPath)
|
||||
if err != nil || info.Size() == 0 {
|
||||
cleanup()
|
||||
return fmt.Errorf("rejected: empty download")
|
||||
}
|
||||
if info.Size() > maxEnclosureCapBytes {
|
||||
cleanup()
|
||||
return fmt.Errorf("rejected: enclosure exceeds %d byte cap", maxEnclosureCapBytes)
|
||||
}
|
||||
if !looksLikeAudioFile(tmpPath) {
|
||||
cleanup()
|
||||
return fmt.Errorf("rejected: downloaded file does not look like audio media")
|
||||
}
|
||||
|
||||
// ---- ffprobe reconcile + embedded cover (scanner machinery reuse) ----
|
||||
payload := probeEpisodeAudio(tmpPath)
|
||||
|
||||
// ---- move into place (crash-safe: rename is atomic on-same-fs) ----
|
||||
if err := os.Rename(tmpPath, finalPath); err != nil {
|
||||
cleanup()
|
||||
return fmt.Errorf("finalize: %w", err)
|
||||
}
|
||||
|
||||
return w.markDownloaded(ctx, ep, &feed, finalPath, payload)
|
||||
}
|
||||
|
||||
// ---- byte-progress tracking (issue #1) ----
|
||||
// In-memory, deliberately NOT persisted: a downloading row renders a
|
||||
// bar from this snapshot while the worker holds it; the DB row's state
|
||||
// remains the durable truth (queued/downloading/downloaded).
|
||||
|
||||
type episodeProgress struct {
|
||||
Received int64
|
||||
Total int64 // 0 = unknown (no Content-Length, no declared length)
|
||||
}
|
||||
|
||||
// beginProgress registers a download slot (total 0 = unknown until the
|
||||
// response's Content-Length arrives).
|
||||
func (w *PodcastDownloadWorker) beginProgress(episodeID pgtype.UUID, total int64) {
|
||||
w.progMu.Lock()
|
||||
defer w.progMu.Unlock()
|
||||
w.progress[episodeID.String()] = &episodeProgress{Total: total}
|
||||
}
|
||||
|
||||
// addProgress accumulates streamed bytes.
|
||||
func (w *PodcastDownloadWorker) addProgress(episodeID pgtype.UUID, n int64) {
|
||||
w.progMu.Lock()
|
||||
defer w.progMu.Unlock()
|
||||
if p, ok := w.progress[episodeID.String()]; ok {
|
||||
p.Received += n
|
||||
if p.Total == 0 {
|
||||
// Total unknown at start; keep received-only (bar renders
|
||||
// indeterminate).
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// setProgressTotal records the authoritative total once known.
|
||||
func (w *PodcastDownloadWorker) setProgressTotal(episodeID pgtype.UUID, total int64) {
|
||||
w.progMu.Lock()
|
||||
defer w.progMu.Unlock()
|
||||
if p, ok := w.progress[episodeID.String()]; ok && total > 0 {
|
||||
p.Total = total
|
||||
}
|
||||
}
|
||||
|
||||
// endProgress drops the slot (terminal state reached — the row itself
|
||||
// now tells the story).
|
||||
func (w *PodcastDownloadWorker) endProgress(episodeID pgtype.UUID) {
|
||||
w.progMu.Lock()
|
||||
defer w.progMu.Unlock()
|
||||
delete(w.progress, episodeID.String())
|
||||
}
|
||||
|
||||
// EpisodeProgress reports the live byte counts for a downloading
|
||||
// episode (ok=false when nothing is in flight for it).
|
||||
func (w *PodcastDownloadWorker) EpisodeProgress(episodeID pgtype.UUID) (received, total int64, ok bool) {
|
||||
w.progMu.Lock()
|
||||
defer w.progMu.Unlock()
|
||||
p, ok := w.progress[episodeID.String()]
|
||||
if !ok {
|
||||
return 0, 0, false
|
||||
}
|
||||
return p.Received, p.Total, true
|
||||
}
|
||||
|
||||
// streamEnclosure performs the SSRF-guarded enclosure download straight
|
||||
// to tmpPath while updating the progress slot. Returns bytes received
|
||||
// and the authoritative total (0 when unknown). Scheme and size guards
|
||||
// match guardFetch; redirects re-dial through the same client so the
|
||||
// IP guard re-applies per hop.
|
||||
func (w *PodcastDownloadWorker) streamEnclosure(ctx context.Context, ep database.MediaItems, tmpPath string) (int64, int64, error) {
|
||||
u, err := url.Parse(ep.EnclosureUrl.String)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("download failed: invalid enclosure URL: %w", err)
|
||||
}
|
||||
if err := guardURLScheme(u); err != nil {
|
||||
return 0, 0, fmt.Errorf("download failed: %w", err)
|
||||
}
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, ep.EnclosureUrl.String, nil)
|
||||
if err != nil {
|
||||
return 0, 0, fmt.Errorf("download failed: %w", err)
|
||||
}
|
||||
req.Header.Set("User-Agent", "Bookhoard/1.0 (podcast fetcher; +https://github.com/bookhoard)")
|
||||
|
||||
// Declared total (feed's enclosure length) seeds the bar until the
|
||||
// response's Content-Length replaces it.
|
||||
declared := int64(0)
|
||||
if ep.FileSize.Valid {
|
||||
declared = ep.FileSize.Int64
|
||||
}
|
||||
w.beginProgress(ep.ID, declared)
|
||||
|
||||
resp, err := podcastHTTPClientFactory().Do(req)
|
||||
if err != nil {
|
||||
return 0, declared, fmt.Errorf("download failed: %w", classifyFetchError(err))
|
||||
}
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return 0, declared, fmt.Errorf("download failed: HTTP %d for enclosure", resp.StatusCode)
|
||||
}
|
||||
|
||||
total := resp.ContentLength // -1 when unknown
|
||||
if total < 0 {
|
||||
total = declared
|
||||
}
|
||||
w.setProgressTotal(ep.ID, total)
|
||||
|
||||
out, err := os.Create(tmpPath)
|
||||
if err != nil {
|
||||
return 0, total, fmt.Errorf("write: %w", err)
|
||||
}
|
||||
defer func() { _ = out.Close() }()
|
||||
|
||||
buf := make([]byte, 64<<10)
|
||||
var received int64
|
||||
for {
|
||||
n, readErr := resp.Body.Read(buf)
|
||||
if n > 0 {
|
||||
received += int64(n)
|
||||
if received > maxEnclosureCapBytes {
|
||||
return received, total, fmt.Errorf("rejected: enclosure exceeds %d byte cap", maxEnclosureCapBytes)
|
||||
}
|
||||
w.addProgress(ep.ID, int64(n))
|
||||
if _, werr := out.Write(buf[:n]); werr != nil {
|
||||
return received, total, fmt.Errorf("write: %w", werr)
|
||||
}
|
||||
}
|
||||
if readErr == io.EOF {
|
||||
break
|
||||
}
|
||||
if readErr != nil {
|
||||
return received, total, fmt.Errorf("download failed: %w", readErr)
|
||||
}
|
||||
}
|
||||
return received, total, nil
|
||||
}
|
||||
|
||||
// markDownloaded is the SUCCESS-ONLY row write (invariant 7): the file
|
||||
// is on disk and (when ffprobe is available) probed before this runs.
|
||||
func (w *PodcastDownloadWorker) markDownloaded(ctx context.Context, ep database.MediaItems, feed *database.PodcastFeeds, absPath string, payload *AudioMetadataPayload) error {
|
||||
info, err := os.Stat(absPath)
|
||||
if err != nil {
|
||||
return fmt.Errorf("post-move stat: %w", err)
|
||||
}
|
||||
|
||||
// Embedded cover extraction (optional enrichment; scanner precedent).
|
||||
coverRel := pgtype.Text{}
|
||||
base := strings.TrimSuffix(absPath, filepath.Ext(absPath))
|
||||
if coverAbs := extractEmbeddedAudioCover(absPath, base); coverAbs != "" {
|
||||
if root, err := w.svc.downloadRootAbs(ctx, feed); err == nil {
|
||||
if rel, err := filepath.Rel(root, coverAbs); err == nil {
|
||||
coverRel = pgtypeText(rel)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
var chapterCount pgtype.Int4
|
||||
if payload != nil && len(payload.Chapters) > 0 {
|
||||
chapterCount = pgtypeInt4(int32(len(payload.Chapters)))
|
||||
}
|
||||
|
||||
_, err = w.db.MarkPodcastEpisodeDownloaded(ctx, database.MarkPodcastEpisodeDownloadedParams{
|
||||
ID: ep.ID,
|
||||
FilePath: ep.FilePath,
|
||||
FileSize: pgtypeInt8(info.Size()),
|
||||
MimeType: pgtypeText(strings.TrimSpace(pickMimeType(ep, absPath))),
|
||||
CoverImagePath: coverRel,
|
||||
AudioMetadata: audioPayloadJSONB(payload),
|
||||
ChapterCount: chapterCount,
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("mark downloaded: %w", err)
|
||||
}
|
||||
log.Printf("[PODCAST-DL] downloaded %q (%s) for feed %s", ep.Title, ep.FilePath, feed.Title)
|
||||
return nil
|
||||
}
|
||||
|
||||
// probeEpisodeAudio shells out to ffprobe and builds the audio_metadata
|
||||
// payload (scanner machinery reuse; nil when ffprobe is absent — the
|
||||
// episode still downloads, facts degrade to declared values).
|
||||
func probeEpisodeAudio(path string) *AudioMetadataPayload {
|
||||
report := runFFProbe(path)
|
||||
if report == nil {
|
||||
return nil
|
||||
}
|
||||
payload := &AudioMetadataPayload{}
|
||||
if report.Format.Duration > 0 {
|
||||
payload.DurationSeconds = float64(report.Format.Duration)
|
||||
}
|
||||
if report.Format.BitRate > 0 {
|
||||
payload.BitRate = int64(report.Format.BitRate)
|
||||
}
|
||||
if len(report.Streams) > 0 {
|
||||
profile := report.Streams[0].CodecName
|
||||
if p := report.Streams[0].Profile; p != "" && p != "unknown" {
|
||||
profile += " / " + p
|
||||
}
|
||||
payload.CodecProfile = profile
|
||||
}
|
||||
for _, ch := range report.Chapters {
|
||||
payload.Chapters = append(payload.Chapters, AudioChapter{
|
||||
Index: ch.ID,
|
||||
Title: ch.Tags.Title,
|
||||
Start: float64(ch.StartTime),
|
||||
End: float64(ch.EndTime),
|
||||
})
|
||||
}
|
||||
return payload
|
||||
}
|
||||
|
||||
// looksLikeAudioFile sniffs content: an audio container starts with
|
||||
// known magic bytes, or (ID3v1/v2, ADTS) has recognizable heads. HTML
|
||||
// error pages and JSON never pass.
|
||||
func looksLikeAudioFile(path string) bool {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return false
|
||||
}
|
||||
defer func() { _ = f.Close() }()
|
||||
head := make([]byte, 16)
|
||||
n, _ := io.ReadFull(f, head)
|
||||
if n < 4 {
|
||||
return false
|
||||
}
|
||||
h := head[:n]
|
||||
switch {
|
||||
// MP3: ID3 tag or MPEG frame sync.
|
||||
case string(h[:3]) == "ID3":
|
||||
return true
|
||||
case h[0] == 0xFF && h[1]&0xE0 == 0xE0:
|
||||
return true
|
||||
// MP4/M4A/M4B: ftyp box.
|
||||
case string(h[4:8]) == "ftyp":
|
||||
return true
|
||||
// FLAC
|
||||
case string(h[:4]) == "fLaC":
|
||||
return true
|
||||
// OGG/Opus
|
||||
case string(h[:4]) == "OggS":
|
||||
return true
|
||||
// WAV/RIFF
|
||||
case string(h[:4]) == "RIFF":
|
||||
return true
|
||||
// AAC ADTS (sync word)
|
||||
case h[0] == 0xFF && (h[1]&0xF6) == 0xF0:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// pickMimeType prefers the feed-declared type; falls back by extension.
|
||||
func pickMimeType(ep database.MediaItems, absPath string) string {
|
||||
if ep.MimeType.Valid && strings.HasPrefix(ep.MimeType.String, "audio/") {
|
||||
return ep.MimeType.String
|
||||
}
|
||||
switch strings.ToLower(filepath.Ext(absPath)) {
|
||||
case ".m4a", ".m4b":
|
||||
return "audio/mp4"
|
||||
case ".mp3":
|
||||
return "audio/mpeg"
|
||||
case ".flac":
|
||||
return "audio/flac"
|
||||
case ".ogg", ".oga", ".opus":
|
||||
return "audio/ogg"
|
||||
case ".wav":
|
||||
return "audio/wav"
|
||||
case ".aac":
|
||||
return "audio/aac"
|
||||
}
|
||||
return "audio/mpeg"
|
||||
}
|
||||
|
||||
// audioPayloadJSONB marshals the probed payload for the JSONB column
|
||||
// (nil payload → NULL: keep the column honest, never an empty object).
|
||||
func audioPayloadJSONB(p *AudioMetadataPayload) []byte {
|
||||
if p == nil {
|
||||
return nil
|
||||
}
|
||||
b, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return nil
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// randomSuffix disambiguates concurrent temp files (single worker, but
|
||||
// a crashed worker's stale .part files must never collide).
|
||||
func randomSuffix() string {
|
||||
sum := sha256.Sum256([]byte(time.Now().Format(time.RFC3339Nano) + fmt.Sprint(os.Getpid())))
|
||||
return hex.EncodeToString(sum[:4])
|
||||
}
|
||||
@@ -320,3 +320,40 @@ func TestSeriesNumberFloatOrderingValues(t *testing.T) {
|
||||
t.Fatal("1 must sort before 1.5")
|
||||
}
|
||||
}
|
||||
|
||||
// ResolveSeriesSort composes the toolbar's base key + direction into
|
||||
// the registry key the SQL cascade implements. "recent" is both a base
|
||||
// key AND a full registry key (meaning desc) — the composition must
|
||||
// run BEFORE the full-key passthrough, or Date Added can never sort
|
||||
// ascending (2026-10-27 regression the direction toggle exposed).
|
||||
func TestResolveSeriesSortComposition(t *testing.T) {
|
||||
cases := []struct {
|
||||
sort, dir string
|
||||
wantKey, wantD string
|
||||
}{
|
||||
// THE regression: Date Added + toggle up.
|
||||
{"recent", "asc", "recent_asc", "asc"},
|
||||
{"recent", "", "recent", "desc"},
|
||||
{"recent", "desc", "recent", "desc"},
|
||||
{"recent", "bogus", "recent", "desc"},
|
||||
// Full keys (back-compat URLs / JSON API) win as-is.
|
||||
{"name_desc", "", "name_desc", "desc"},
|
||||
{"name_asc", "desc", "name_asc", "asc"},
|
||||
// Base keys compose; default direction applies when dir absent.
|
||||
{"name", "", "name_asc", "asc"},
|
||||
{"name", "desc", "name_desc", "desc"},
|
||||
{"books", "", "books_desc", "desc"},
|
||||
{"books", "asc", "books_asc", "asc"},
|
||||
{"last_read", "asc", "last_read_asc", "asc"},
|
||||
// Unknown/empty → default.
|
||||
{"", "", "recent", "desc"},
|
||||
{"bogus", "asc", "recent", "desc"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
key, dir := ResolveSeriesSort(tc.sort, tc.dir)
|
||||
if key != tc.wantKey || dir != tc.wantD {
|
||||
t.Errorf("ResolveSeriesSort(%q, %q) = (%q, %q), want (%q, %q)",
|
||||
tc.sort, tc.dir, key, dir, tc.wantKey, tc.wantD)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"bookhoard/internal/database"
|
||||
"bookhoard/internal/utils"
|
||||
"context"
|
||||
"strings"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
@@ -16,6 +17,143 @@ type SeriesInfo struct {
|
||||
TotalInSeries int
|
||||
CoverPaths []string
|
||||
LastEntryAt string
|
||||
// Viewer-scoped progress aggregates (requesting user): books
|
||||
// finished (percentage >= 0.999) and the series' most recent read.
|
||||
ReadCount int64
|
||||
LastReadAt string
|
||||
}
|
||||
|
||||
// SeriesPageOptions carries the series page's sort/filter state. Zero
|
||||
// values mean defaults (recent, no filters). Extensible by design:
|
||||
// adding a sort = one entry in seriesSortKeys + a matching CASE arm in
|
||||
// queries.sql (GetDistinctSeries) + one <option> in templates/
|
||||
// series.templ; adding a filter = a param on this struct threaded into
|
||||
// the query's WHERE/HAVING. Unknown values resolve to defaults, so a
|
||||
// stale URL or hand-rolled API call can never reach the SQL with an
|
||||
// unhandled key.
|
||||
type SeriesPageOptions struct {
|
||||
Sort string // base toolbar key or full registry key
|
||||
Dir string // asc | desc ("" = the base key's default)
|
||||
Query string // multi-field: series name, author, or book title
|
||||
Author string // matches when ANY book's author matches
|
||||
Status string // "" | unstarted | reading | finished
|
||||
ViewerUserID pgtype.UUID
|
||||
}
|
||||
|
||||
// seriesSortKeys is the registry of sort= values the series page
|
||||
// understands.
|
||||
var seriesSortKeys = map[string]bool{
|
||||
"recent": true, // default: newest series first
|
||||
"recent_asc": true,
|
||||
"name_asc": true,
|
||||
"name_desc": true,
|
||||
"author_asc": true,
|
||||
"author_desc": true,
|
||||
"books_asc": true,
|
||||
"books_desc": true,
|
||||
"progress_asc": true,
|
||||
"progress_desc": true,
|
||||
"last_read_asc": true,
|
||||
"last_read_desc": true,
|
||||
}
|
||||
|
||||
// seriesStatusKeys is the registry of status= values. The three states
|
||||
// are mutually exclusive and exhaustive: a series is finished when
|
||||
// every owned book is, reading when any book sits between 0 and the
|
||||
// finish threshold, unstarted when no book has progress > 0.
|
||||
var seriesStatusKeys = map[string]bool{
|
||||
"unstarted": true,
|
||||
"reading": true,
|
||||
"finished": true,
|
||||
}
|
||||
|
||||
// seriesBaseSorts is the toolbar's base sort keys (the <select>) with
|
||||
// the direction each one STARTS in (the asc/desc toggle's initial
|
||||
// state). Composed with ResolveSeriesSort into the full registry keys
|
||||
// the SQL cascade implements.
|
||||
var seriesBaseSorts = map[string]string{
|
||||
"recent": "desc",
|
||||
"name": "asc",
|
||||
"author": "asc",
|
||||
"books": "desc",
|
||||
"progress": "desc",
|
||||
"last_read": "desc",
|
||||
}
|
||||
|
||||
// ResolveSeriesSort maps a sort value (either a FULL registry key like
|
||||
// "name_desc" — back-compat with old URLs and the JSON API — or a BASE
|
||||
// toolbar key composed with a direction) to the registry key the SQL
|
||||
// cascade implements, plus the EFFECTIVE direction (for the toggle's
|
||||
// arrow). Unknown values resolve to the default.
|
||||
func ResolveSeriesSort(sort, dir string) (string, string) {
|
||||
sort = strings.TrimSpace(strings.ToLower(sort))
|
||||
dir = strings.ToLower(strings.TrimSpace(dir))
|
||||
if dir != "asc" && dir != "desc" {
|
||||
dir = ""
|
||||
}
|
||||
// "recent" is BOTH a base toolbar key and a registry key (it means
|
||||
// desc). Handle it before the full-key check or dir=asc is silently
|
||||
// swallowed and the direction toggle can never flip Date Added.
|
||||
if sort == "recent" {
|
||||
if dir == "asc" {
|
||||
return "recent_asc", "asc"
|
||||
}
|
||||
return "recent", "desc"
|
||||
}
|
||||
// Full registry key straight from the URL wins as-is.
|
||||
if seriesSortKeys[sort] {
|
||||
return sort, seriesDirOf(sort)
|
||||
}
|
||||
base := sort
|
||||
if base == "" {
|
||||
base = "recent"
|
||||
}
|
||||
defaultDir, known := seriesBaseSorts[base]
|
||||
if !known {
|
||||
// Unknown base: the whole thing falls to the default — an
|
||||
// unhandled sort must not sneak a direction past the registry.
|
||||
return "recent", "desc"
|
||||
}
|
||||
if dir == "" {
|
||||
dir = defaultDir
|
||||
}
|
||||
if base == "recent" && dir == "desc" {
|
||||
return "recent", "desc"
|
||||
}
|
||||
return base + "_" + dir, dir
|
||||
}
|
||||
|
||||
// seriesDirOf derives the display direction of a full registry key.
|
||||
func seriesDirOf(key string) string {
|
||||
switch {
|
||||
case strings.HasSuffix(key, "_asc"):
|
||||
return "asc"
|
||||
case strings.HasSuffix(key, "_desc"):
|
||||
return "desc"
|
||||
case key == "recent":
|
||||
return "desc"
|
||||
default:
|
||||
if d, ok := seriesBaseSorts[key]; ok {
|
||||
return d
|
||||
}
|
||||
return "desc"
|
||||
}
|
||||
}
|
||||
|
||||
// SeriesSortBase strips the direction suffix from a full registry key
|
||||
// ("name_desc" → "name") for the toolbar's base-key <select>.
|
||||
func SeriesSortBase(key string) string {
|
||||
key = strings.TrimSuffix(strings.TrimSuffix(key, "_asc"), "_desc")
|
||||
if _, ok := seriesBaseSorts[key]; ok {
|
||||
return key
|
||||
}
|
||||
return "recent"
|
||||
}
|
||||
|
||||
// nargText makes an optional text param: empty string → SQL NULL so
|
||||
// the query's IS NULL guard skips the filter.
|
||||
func nargText(v string) pgtype.Text {
|
||||
return pgtype.Text{String: v, Valid: v != ""}
|
||||
}
|
||||
|
||||
type SeriesService struct {
|
||||
@@ -31,18 +169,35 @@ func NewSeriesService(db *database.Queries) *SeriesService {
|
||||
// Pass libraryID for a single-library page, or leave it invalid and pass
|
||||
// visibleLibraryIDs for an across-my-libraries page. Both filters compose:
|
||||
// the handler validates any explicit library_id against the visible set.
|
||||
func (s *SeriesService) GetSeriesPage(ctx context.Context, libraryID pgtype.UUID, visibleLibraryIDs []pgtype.UUID, limit, offset int) ([]SeriesInfo, int, error) {
|
||||
totalCount, err := s.db.GetDistinctSeriesCount(ctx, database.GetDistinctSeriesCountParams{
|
||||
func (s *SeriesService) GetSeriesPage(ctx context.Context, libraryID pgtype.UUID, visibleLibraryIDs []pgtype.UUID, limit, offset int, opts SeriesPageOptions) ([]SeriesInfo, int, error) {
|
||||
sortKey, _ := ResolveSeriesSort(opts.Sort, opts.Dir)
|
||||
sort := nargText(sortKey)
|
||||
status := nargText("")
|
||||
if seriesStatusKeys[opts.Status] {
|
||||
status = nargText(opts.Status)
|
||||
}
|
||||
|
||||
countParams := database.GetDistinctSeriesCountParams{
|
||||
ViewerUserID: opts.ViewerUserID,
|
||||
LibraryID: libraryID,
|
||||
VisibleLibraryIds: visibleLibraryIDs,
|
||||
})
|
||||
SeriesQuery: nargText(opts.Query),
|
||||
AuthorQuery: nargText(opts.Author),
|
||||
Status: status,
|
||||
}
|
||||
totalCount, err := s.db.GetDistinctSeriesCount(ctx, countParams)
|
||||
if err != nil {
|
||||
return nil, 0, err
|
||||
}
|
||||
|
||||
rows, err := s.db.GetDistinctSeries(ctx, database.GetDistinctSeriesParams{
|
||||
ViewerUserID: opts.ViewerUserID,
|
||||
LibraryID: libraryID,
|
||||
VisibleLibraryIds: visibleLibraryIDs,
|
||||
SeriesQuery: countParams.SeriesQuery,
|
||||
AuthorQuery: countParams.AuthorQuery,
|
||||
Status: status,
|
||||
Sort: sort,
|
||||
Limit: pgtype.Int4{Int32: int32(limit), Valid: true},
|
||||
Offset: pgtype.Int4{Int32: int32(offset), Valid: true},
|
||||
})
|
||||
@@ -76,6 +231,18 @@ func (s *SeriesService) GetSeriesPage(ctx context.Context, libraryID pgtype.UUID
|
||||
}
|
||||
}
|
||||
|
||||
lastRead := ""
|
||||
if row.LastReadAt != nil {
|
||||
switch v := row.LastReadAt.(type) {
|
||||
case pgtype.Timestamptz:
|
||||
if v.Valid {
|
||||
lastRead = v.Time.String()
|
||||
}
|
||||
case string:
|
||||
lastRead = v
|
||||
}
|
||||
}
|
||||
|
||||
if totalInSeries == 0 || totalInSeries < int(row.BookCount) {
|
||||
totalInSeries = int(row.BookCount)
|
||||
}
|
||||
@@ -87,6 +254,8 @@ func (s *SeriesService) GetSeriesPage(ctx context.Context, libraryID pgtype.UUID
|
||||
TotalInSeries: totalInSeries,
|
||||
CoverPaths: coverPaths,
|
||||
LastEntryAt: lastEntry,
|
||||
ReadCount: row.FinishedCount,
|
||||
LastReadAt: lastRead,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -296,6 +296,13 @@ func (s *StatisticsService) UserSummary(ctx context.Context, userID pgtype.UUID,
|
||||
entry.Sessions++
|
||||
summary.SessionCount++
|
||||
|
||||
// Podcast episodes (plan D15): sessions flow into time-spent
|
||||
// (the minutes/pages/sessions math above already ran), but an
|
||||
// episode is not a book — never enters BooksActive/BooksFinished.
|
||||
if strings.EqualFold(strings.TrimSpace(sess.LibraryTypeName.String), "podcasts") {
|
||||
continue
|
||||
}
|
||||
|
||||
mediaUUID, err := uuid.FromBytes(sess.MediaItemID.Bytes[0:16])
|
||||
if err == nil {
|
||||
key := mediaUUID.String()
|
||||
|
||||
@@ -8,3 +8,11 @@ package version
|
||||
// The release workflow passes the git tag (e.g. v1.5) as the APP_VERSION
|
||||
// Docker build arg, which the Dockerfile forwards to this flag.
|
||||
var Version = "dev"
|
||||
|
||||
// Commit is the git commit the binary was built from (short hash),
|
||||
// injected alongside Version via ldflags (GIT_COMMIT build arg). It
|
||||
// exists so "the UI I'm looking at" vs "the code in the repo" drift is
|
||||
// VISIBLE — the 2026-10-29 podcast debugging session burned an hour on
|
||||
// a stale container that predated the feature. "none" for unstampeded
|
||||
// local builds.
|
||||
var Commit = "none"
|
||||
Loaded 100 of 183 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user