Files

T

john-okeefe 1db89669fb docs: add device_identifier investigation to implementation plan

- Add Section 16.1: Codebase investigation results
- Document that device_identifier was added in Phase 1 (commit 3b2075f)
- Clarify it's for device management, not authentication
- Show active usage in device registration (line 37: validate:"required,min=1,max=255")
- Identify dead code: GetDeviceByIdentifier query exists but not called
- Confirm OPDS uses device.id for lookup (not device_identifier)
- Distinguish authentication (auth_token) from device identification (device_identifier)

2026-02-12 11:17:46 -05:00

48 KiB

Raw Blame History

Bookhoard Device Authentication & Sync Implementation Plan

Executive Summary

Current Problem:

OPDS routes are publicly accessible (security vulnerability)
Kobo sync requires custom authentication approach
KOReader documentation is incorrect about Basic Auth
No clear authentication strategy for different device types

Vision: Transform Bookhoard into a comprehensive Kindle-replacement ecosystem with seamless device sync, supporting both Kobo (native) and KOReader (via plugin) with full functionality including progress, highlights, notes, and bookmarks.

Authentication Strategy:

Kobo: API key in URL path (per-device, revocable, no jailbreak needed)
KOReader: Bearer token in Authorization header (per-device, revocable, via plugin)
OPDS: Both authentication methods supported

1. Current State Analysis

1.1 Authentication Landscape

Existing Infrastructure:

// DeviceAuthMiddleware - Currently only supports Bearer tokens
// File: internal/middleware/device_auth.go:37-108
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
    // Only checks Authorization: Bearer {token}
    // Validates against devices.auth_token
}

Routes Using Device Auth:

/api/sync/kobo/* - Kobo sync endpoints
/api/sync/koreader/* - KOReader sync endpoints
/opds/devices/* - OPDS catalog

Database Schema:

-- devices table has auth_token field for all device authentication
-- Can store revocable API keys for Kobo devices

1.2 Kobo Integration Status

Proven Approach (Komga):

Komga successfully handles Kobo sync without jailbreak using API keys in URL path
Configuration: api_endpoint=https://komga.example.com/kobo/{api_key}
Kobo firmware supports custom api_endpoint in Kobo eReader.conf
API key embedded in URL path works reliably with stock Kobo firmware

What's Currently Broken:

Current middleware expects Bearer token in Authorization header
Routes need to support API key in URL path parameter
Device auth tokens (API keys) already generated during registration

What's Working:

Kobo sync handler implementation exists and is functional
Endpoints registered: /api/sync/kobo/markup, /bookmark, /v1/initialization, etc.
Device auth tokens (API keys) already generated during registration
Can sync: progress, highlights, notes, bookmarks once auth is fixed

1.3 KOReader Integration Status

What's Broken:

Documentation says use "Basic Auth" - this is WRONG
KOReader doesn't support custom headers natively for sync
Bearer token can't be sent via standard KOReader settings

What's Working:

KOReader sync handler supports full feature set
Can sync: progress, bookmarks, highlights, notes, library metadata
Plugin architecture exists but no Bookhoard plugin yet

1.4 OPDS Status

Current:

Routes registered in internal/router/opds.go:11-12
✅ Already using DeviceAuthMiddleware (line 12: opds.Use(cfg.DeviceAuthMiddleware.Authenticate))
✅ Authentication is required - not publicly accessible
Routes: /opds/devices/:deviceId/* - all protected

Required:

✅ No changes needed for OPDS security
After Phase 1, OPDS will automatically support both auth methods:
- Kobo: API key in URL path (?token= or path parameter)
- KOReader: Bearer token via Authorization header

2. User Priorities (Ranked)

Priority 1: Functionality First

User Quote: "I want a Kindle system replacement, otherwise I could just use Booklore or KOReader"

Requirements:

✅ Full sync: progress, highlights, notes, bookmarks
✅ Works on both Kobo and KOReader
✅ Seamless user experience
✅ OPDS for wireless book delivery
✅ Two-way sync (device ↔ server)

Priority 2: Security Second (with documentation)

User Quote: "This is a self-hosted app so as long as the user is aware of the security implications in the documentation it should be fine"

Acceptable Trade-offs:

API key in URL for Kobo (per-device, revocable, documented security considerations)
Token-based for KOReader (more secure)
Users responsible for network security (VPN, local network, HTTPS)
Security implications clearly documented

Priority 3: Simple Setup

Goal: Minimal friction for users

Kobo Experience:

Register device in Bookhoard UI
Copy device API key from device management page
One line in Kobo eReader.conf with API key
Token can be regenerated if compromised
Works immediately with stock firmware

KOReader Experience:

Copy plugin files
Enter server URL and token
One-time setup
Background sync works automatically

3. Target State

3.1 Unified Authentication Architecture

┌─────────────────────────────────────────────────────────────────┐
│              DeviceAuthMiddleware (Enhanced)                   │
│  Unified authentication using revocable API keys:               │
│                                                               │
│  1. Authorization: Bearer {token}                             │
│     → Extract from Authorization header                        │
│     → Lookup device by auth_token                             │
│     → Used by: KOReader, OPDS apps, API clients              │
│                                                               │
│  2. URL Path Parameter: /api/sync/kobo/{token}                │
│     → Extract token from URL path (c.Param("token"))          │
│     → Lookup device by auth_token                             │
│     → Used by: Kobo e-readers (stock firmware, no jailbreak) │
│                                                               │
│  3. Query Parameter: ?token={token}                           │
│     → Extract from query string (c.QueryParam("token"))       │
│     → Lookup device by auth_token                             │
│     → Used by: OPDS catalog access                           │
│                                                               │
│  All methods result in: device context set in echo.Context    │
└─────────────────────────────────────────────────────────────────┘

Why This Architecture:

Simple: All methods use same auth_token lookup (no new database logic)
Secure: API keys are random UUIDs, revocable, can be regenerated
Universal: All devices get auth_tokens (existing field, no schema change)
Proven: Komga uses this exact approach for Kobo sync successfully
No Firmware Limitations: URL path parameter works with stock Kobo firmware (no jailbreak needed)
Revocable: API keys can be regenerated by users without device re-registration
Single Authentication Method: All device types use the same auth_token lookup, simplifying codebase


### 3.2 Feature Matrix

| Feature | Kobo (Stock Firmware) | KOReader (Plugin) |
|---------|---------------|-------------------|
| **Progress Sync** | ✅ Automatic (API key in URL) | ✅ Automatic (Bearer token) |
| **Highlights** | ✅ Native support | ✅ Via plugin API |
| **Bookmarks** | ✅ Native support | ✅ Via plugin API |
| **Notes** | ✅ Native support | ✅ Via plugin API |
| **OPDS** | ✅ Native support | ✅ Native OPDS support |
| **Setup Complexity** | Low (1 config line) | Medium (plugin install) |
| **Auth Method** | API key in URL path | Bearer token in header |
| **Security** | High (revocable tokens) | High (revocable tokens) |

### 3.3 User Flows

**Kobo User Journey:**
1. Open Bookhoard web UI
2. Navigate to Device Management → Add New Device
3. Enter device name and select "Kobo" as device type
4. Click Register Device
5. From device details page, copy full Kobo Sync URL with API key:
    ```
    http://YOUR_IP:8765/api/sync/kobo/YOUR_API_KEY
    ```
6. Connect Kobo to computer, open `.kobo/Kobo/Kobo eReader.conf`
7. Add/edit line: `api_endpoint=http://YOUR_IP:8765/api/sync/kobo/YOUR_API_KEY`
    - Paste the entire URL from step 5 (includes device API key)
8. Save and eject Kobo (restarts automatically)
9. Kobo syncs to Bookhoard instead of Kobo store
10. For OPDS: Add catalog URL with `?token=YOUR_API_KEY` parameter

**Important Note - Token Regeneration:**
- If token is compromised or lost, regenerate from device management page
- OLD token will immediately stop working
- User must update `api_endpoint` line with NEW token
- One-click copy button makes this easy

**KOReader User Journey:**
1. Install KOReader on device
2. Open Bookhoard web UI
3. Register device → copy API key
4. Approve device
5. Install Bookhoard plugin → paste API key
6. Configure sync settings
7. Read books → automatic sync
8. Add OPDS catalog with API key in header (via plugin)

---

## 4. Implementation Plan

### Phase 1: Enhanced Authentication (Week 1)
**Goal:** Make auth work for both Kobo (API key in URL) and KOReader (Bearer token)

**Tasks:**

1. **Update Routing for Kobo** (`internal/router/sync.go`)
    - [ ] Change Kobo route group from `/api/sync/kobo` to `/api/sync/kobo/:token`
    - [ ] Update all Kobo route handlers to use path parameter
    - [ ] Test that routes still work with Bearer token (backward compatibility)

2. **Update DeviceAuthMiddleware** (`internal/middleware/device_auth.go`)
    - [ ] Try Bearer token lookup first (existing behavior)
    - [ ] If no Bearer token, check for `token` path parameter (`c.Param("token")`)
    - [ ] Query database: `GetDeviceByAuthToken(token)` (already exists)
    - [ ] Set device context on successful auth
    - [ ] Return 401 if both methods fail

3. **Add Token Regeneration Backend** (`internal/handlers/devices.go`)
    - [ ] Add endpoint: `POST /api/devices/:id/regenerate-token`
    - [ ] Generate new random UUID for auth_token
    - [ ] Update device in database
    - [ ] Return new token to user
    - [ ] Require JWT auth (user must be logged in)

4. **Update Device Registration UI** (`templates/devices.templ`, `templates/partials/device-management.templ`)
   - [ ] **Show "Copy Full Sync URL" button for each device** (uses clipboard API)
   - [ ] For Kobo: Display complete URL: `http://IP:8765/api/sync/kobo/{DEVICE_TOKEN}`
   - [ ] **Add "Regenerate Token" button** (with confirmation dialog: "This will revoke current token. Continue?")
   - [ ] After regeneration, show toast: "Token regenerated - update Kobo config"
   - [ ] For KOReader: Show auth token in copyable input field
   - [ ] Display clear setup instructions per device type with full URL example

5. **Apply Middleware to OPDS** (`internal/router/opds.go`)
    - [ ] Verify OPDS routes have DeviceAuthMiddleware (already applied)
    - [ ] Add support for `?token=` query parameter for OPDS access
    - [ ] Update comments to reflect dual auth support

**Testing:**
- Kobo device sync with api-key url auth
- KOReader sync with Bearer token auth
- OPDS access with both auth methods

---

### Phase 2: Kobo Integration (Week 1-2)
**Goal:** Seamless Kobo experience with API key-based auth (stock firmware)

**Tasks:**

1. **Kobo Setup Documentation** (`docs/user/devices/kobo-setup.md`)
    - [ ] Show how to get API key from Bookhoard device management page
    - [ ] Step-by-step guide for editing `Kobo eReader.conf`
    - [ ] Include example config: `api_endpoint=http://192.168.1.100:8765/api/sync/kobo/{API_KEY}`
    - [ ] Security warning about API keys in logs
    - [ ] Network security recommendations (VPN, local network)
    - [ ] Instructions for regenerating token if compromised
    - **Note**: Documentation was updated on 2026-02-12 to reflect API key authentication
    - **Verification**: Confirm lines 37-53 don't reference entering serial number (device registration generates token automatically)

2. **Kobo Sync Validation** (`cmd/server/tests/kobo_test.go`)
    - [ ] Update tests to use API key in URL path
    - [ ] Test sync endpoints with `/api/sync/kobo/{token}/markup` format
    - [ ] Verify all features work: progress, highlights, bookmarks, notes
    - [ ] Test token regeneration doesn't break existing sync

3. **Kobo OPDS Integration**
    - [ ] Document OPDS URL format: `http://IP:8765/opds/devices/{DEVICE_ID}/catalog?token={API_KEY}`
    - [ ] Test OPDS browsing with API key in URL parameter
    - [ ] Test book downloads

**Success Criteria:**
- New Kobo device registered and syncing within 5 minutes
- All sync features working without jailbreak
- API key can be regenerated if compromised
- Security implications clearly documented
- Works with stock Kobo firmware (no modifications)

---

### Phase 3: KOReader Plugin Development (Week 2-4)
**Goal:** Full-featured KOReader plugin with Bearer token auth

**Tasks:**

1. **Plugin Architecture**
   - [ ] Create `koreader-plugin/bookhoard.koplugin/` directory
   - [ ] Implement `_meta.lua` (plugin metadata)
   - [ ] Implement `main.lua` (entry point)
   
2. **Core Plugin Features**
   - [ ] Settings UI: server URL, auth token, sync options
   - [ ] HTTP client with Bearer token authentication
   - [ ] Auto-sync on events:
     - Page turn (configurable: every N pages)
     - Bookmark added/deleted
     - Highlight created/deleted
     - Note added/edited
   - [ ] Manual sync button in KOReader menu
   - [ ] Sync status indicator
   - [ ] Error handling and retry logic

3. **Plugin API Integration**
   - [ ] Call `/api/sync/koreader/progress` with reading position
   - [ ] Call `/api/sync/koreader/bookmarks` with annotations
   - [ ] Call `/api/sync/koreader/metadata` to get server-side progress
   - [ ] Handle conflicts (server vs local progress)

4. **Plugin Documentation**
   - [ ] Installation guide (copy files, restart KOReader)
   - [ ] Configuration guide (get token from Bookhoard UI)
   - [ ] Usage instructions
   - [ ] Troubleshooting guide

**Success Criteria:**
- Plugin installed and configured in under 10 minutes
- Background sync works transparently
- All annotation types sync properly
- No Lua errors or crashes

---

### Phase 4: OPDS Security & Integration (Week 2-3)
**Goal:** Secure OPDS with unified device authentication

**Tasks:**

1. **Secure OPDS Routes** (`internal/router/opds.go`)
   - [ ] Apply `DeviceAuthMiddleware` to all OPDS endpoints
   - [ ] Ensure auth works with both api-key and Bearer methods
   - [ ] Update router comments

2. **Kobo OPDS Access**
   - [ ] Document OPDS URL with device ID
   - [ ] Test OPDS access via Kobo browser
   - [ ] Test book downloads

3. **KOReader OPDS Access**
   - [ ] Works natively via KOReader's OPDS support
   - [ ] Uses Bearer token in header
   - [ ] Document catalog URL format

4. **Test Coverage** (`cmd/server/tests/opds_test.go`)
   - [ ] Test unauthenticated access → expect 401
   - [ ] Test with valid Bearer token → expect 200
   - [ ] Test with valid api-key url → expect 200
   - [ ] Test book download with auth

---

### Phase 5: Testing & Documentation (Week 4)
**Goal:** Production-ready release with comprehensive docs

**Tasks:**

1. **End-to-End Testing**
   - [ ] Fresh Kobo setup: register → sync → highlights → OPDS
   - [ ] Fresh KOReader setup: install plugin → sync → highlights → OPDS
   - [ ] Cross-device sync: Kobo ↔ KOReader same book
   - [ ] Offline scenarios and conflict resolution
   - [ ] Error handling and recovery

2. **Security Documentation**
   - [ ] Security implications page
   - [ ] Network security recommendations
    - [ ] Comparison: API Key (URL path) vs Bearer token - both use revocable auth_token
   - [ ] Threat model for self-hosted users

3. **User Documentation**
   - [ ] Quick start guide (Kobo)
   - [ ] Quick start guide (KOReader)
   - [ ] Feature comparison table
   - [ ] FAQ and troubleshooting

4. **Release Preparation**
   - [ ] Version bump
   - [ ] Changelog
   - [ ] Migration guide (if needed)

---

## 5. Technical Decisions

### 5.1 Why API Key in URL Path for Kobo?

**Approach Chosen:** Per-device API key embedded in URL path: `/api/sync/kobo/{api_key}/...`

**Pros:**
- No jailbreak needed - works with stock Kobo firmware
- Proven approach - Komga uses this successfully
- API keys are random UUIDs (cryptographically secure)
- Tokens can be revoked and regenerated
- Single configuration line in Kobo config
- Simple token management (user can regenerate)
- Uses existing `auth_token` field (no schema change)

**Cons:**
- API key visible in Kobo logs and server logs
- URL-based authentication (less secure than headers)
- Key appears in Kobo config file (plain text)

**Mitigations:**
- Use HTTPS for sync (reverse proxy)
- Document that API keys are sensitive
- Recommend local network only (VPN for remote)
- Provide one-click token regeneration
- Acceptable security for self-hosted use (like Komga)

**Alternative Considered:**
- **Serial-based auth**: Rejected because it uses predictable serials that can't be revoked and is less secure
- **Kobo store token**: Rejected because it's managed by Kobo, not under our control
- **Custom HTTP headers**: Rejected because Kobo firmware doesn't support custom headers

**Authentication Strategy:**
- Kobo: API key in URL path (per-device, revocable)
- KOReader: Bearer token (per-device, revocable)
- OPDS: Both methods supported (API key via query parameter, bearer token via header)

### 5.2 Why Not Use Kobo's Token?

Kobo sends `Authorization: Bearer {kobo-store-token}` which is:
- Managed by Kobo (not under our control)
- May rotate/expire without warning
- Only valid for Kobo's servers
- Doesn't identify device in our system
- Can't be extracted from Kobo config without jailbreak

**Decision:** Use API key in URL path instead (proven by Komga). The API key in URL path provides per-device authentication that is revocable and works with stock Kobo firmware without jailbreak.

### 5.3 Token Regeneration Strategy

**Backend:**
- Endpoint: `POST /api/devices/:id/regenerate-token`
- Requires: User JWT auth (device owner)
- Action: Generate new UUID, update `devices.auth_token`
- Response: Return new token to user

**Frontend:**
- "Regenerate Token" button on device management page
- Confirmation dialog: "This will revoke current token. Continue?"
- One-click copy to clipboard
- Update display instructions with new token
- Toast notification: "Token regenerated successfully"

**Impact:**
- Old token immediately invalid
- Device must update config with new token
- Sync will fail until config updated
- Useful for: Lost tokens, compromised devices, periodic rotation

### 5.4 Why Bearer Token for KOReader?

**Pros:**
- Cryptographically secure (random UUID)
- Can be revoked/regenerated
- Standard authentication method
- Works with KOReader plugin architecture
- Plugin can store token securely

**Cons:**
- Requires plugin (can't use native settings)
- Token must be copy-pasted during setup
- User must manage token

**Trade-off Accepted:**
More secure but requires plugin installation. Justified because KOReader users are typically more technical and willing to install plugins.

**Note:** Same token regeneration as Kobo (reuses backend logic)

---

## 6. Files to Modify

### Core Authentication
- `internal/middleware/device_auth.go` - Add URL path token extraction
- `internal/router/sync.go` - Update Kobo routes to use path parameter
- `internal/router/opds.go` - Add ?token= query parameter support (already has middleware)

### Device Management
- `internal/handlers/devices.go` - Add regenerate token endpoint
- `internal/database/queries/queries.sql` - No changes needed (auth_token already exists)
- `templates/devices.templ` - Add copy token, regenerate token buttons
- `templates/partials/device-management.templ` - Kobo config instructions with API key

### Testing
- `cmd/server/tests/kobo_test.go` - Update for URL path token
- `cmd/server/tests/opds_test.go` - Add ?token= auth tests
- `cmd/server/tests/test_helpers.go` - Add token regeneration helpers

### Documentation
- `docs/user/devices/kobo-setup.md` - Rewrite for API key auth
- `docs/user/devices/koreader-setup.md` - Update for Bearer token
- `docs/user/devices/security.md` - New security considerations doc
- `docs/developer/api/kobo/` - Update API docs with path parameters

### New Files
- `koreader-plugin/bookhoard.koplugin/_meta.lua` - Plugin metadata (Phase 3)
- `koreader-plugin/bookhoard.koplugin/main.lua` - Plugin implementation (Phase 3)
- `docs/user/devices/security.md` - Security considerations

---

## 7. Success Metrics

**Functionality:**
- ✅ Kobo syncs progress, highlights, notes, bookmarks automatically
- ✅ KOReader syncs same features via plugin
- ✅ OPDS works on both platforms
- ✅ Cross-device sync works (read on Kobo, continue on KOReader)

**Usability:**
- ✅ Kobo setup time: < 5 minutes
- ✅ KOReader setup time: < 10 minutes
- ✅ No token management for Kobo users
- ✅ Clear documentation with security warnings

**Security:**
- ✅ OPDS no longer publicly accessible
- ✅ Auth required for all sync endpoints
- ✅ Security implications documented
- ✅ Network security recommendations provided

---

## 8. Future Enhancements (Out of Scope)

- Auto-discovery of devices on network
- QR code setup for mobile devices
- Background sync service (no plugin needed)
- Readwise/Joplin integration
- Calibre integration
- Mobile apps (iOS/Android)

---

## 9. Decisions Made (User Input)

Based on user feedback, the following decisions have been finalized:

### 9.1 KOReader Sync Trigger
**Decision:** Configurable with multiple options
- **Auto-sync**: Every N pages (configurable, default every 10 pages)
- **Auto-sync**: Every N minutes (configurable, default every 5 minutes)
- **Manual sync**: Always available via menu
- **Event-triggered**: On bookmark/highlight/note creation (immediate)

**Rationale:** User wants automatic sync for convenience but manual fallback for control.

### 9.2 Plugin Distribution
**Decision:** Separate repository under Bookhoard organization
- Repository: `github.com/bookhoard/koreader-plugin`
- Licensed under same terms as Bookhoard
- Versioned independently
- Referenced in main Bookhoard documentation

**Rationale:** Clean separation of concerns, easier plugin-specific issues/PRs.

### 9.3 Kobo Authentication
**Decision:** API key in URL path (per-device)
- No serial number needed
- Token entered in device registration (already exists)
- Configured in Kobo as: `api_endpoint=http://IP:8765/api/sync/kobo/{TOKEN}`
- Token can be regenerated by user (new endpoint)
- Stored in `devices.auth_token` field (no schema change)

**Rationale:** Simplest user experience (just copy token), most secure (revocable), proven to work (Komga).

### 9.4 Conflict Resolution
**Decision:** Use existing codebase strategy
- Already implemented in `internal/handlers/koreader.go`
- Uses timestamp-based last-write-wins
- Device priority can be configured per-device

**Rationale:** Don't reinvent the wheel, existing implementation is sufficient.

### 9.5 Implementation Order
**Decision:** Parallel implementation with separate sections
- Phase 2a: Kobo Integration
- Phase 2b: KOReader Plugin (runs concurrently)
- Separate milestones and deliverables
- Can ship Kobo support before plugin is ready

**Rationale:** Faster time-to-market for Kobo users, plugin can follow.

### 9.6 KOReader Device Identification Strategy

**Problem:** Multiple KOReader installations each register as separate devices, exhausting `max_devices` limit even though they represent the same physical device.

**Note:** This is about device MANAGEMENT, not authentication. Authentication uses `auth_token` (API keys) for all devices. Device identifier is for tracking multiple installations of the same physical device.

---

## 16.1 Codebase Investigation: device_identifier Field

**Investigation Date:** 2026-02-12

**Purpose:** Clarify the role and usage of `device_identifier` field in the devices table

### Git History Analysis

**Introduction (Commit 3b2075f, Phase 1 - "highlights and notes annotation system"):**
- Added as part of broader annotation system feature
- Original intent: Track physical device identity across software reinstalls
- Has existed since early project history (not legacy code)

### Current Codebase Usage

**Active Usage:**
```go
// internal/handlers/devices.go:37
DeviceIdentifier string `json:"device_identifier" validate:"required,min=1,max=255"`

Required field in device registration requests
Stored in devices.device_identifier (VARCHAR(255) UNIQUE NOT NULL)
Used in registration flow to track device identity

Dead Code:

-- internal/database/queries/queries.sql:731
-- name: GetDeviceByIdentifier :one
SELECT * FROM devices WHERE device_identifier = $1;

Function exists in generated database code (internal/database/querier.go:130)
NOT called anywhere in handlers or tests (0 references)
Can be considered for removal during code cleanup

OPDS Handler Behavior

Current Implementation (internal/handlers/opds.go:56-89):

deviceID := c.Param("deviceId")  // Extracts UUID from URL
deviceUUID, err := uuid.Parse(deviceID)
device, err := h.db.GetDevice(c.Request().Context(), pgtype.UUID{Bytes: deviceUUID, Valid: true})

Finding: OPDS handler uses device id (UUID) for lookup, NOT device_identifier

Authentication vs Device Identification

Authentication (what currently works):

auth_token field stores revocable API keys
Used by DeviceAuthMiddleware for all device authentication
Bearer token or URL path parameter lookup via GetDeviceByAuthToken

Device Identification (what this is about):

device_identifier field tracks physical device identity
Helps prevent duplicate device registrations for same physical device
NOT used for authentication (only for device management/registration)

Conclusion

No inconsistency found. The implementation plan's handling of device_identifier is accurate:

Field exists for device management purposes
Required in current registration flow
Auth uses auth_token only (no confusion)
Dead query (GetDeviceByIdentifier) can be removed during cleanup

Decision: Keep Section 9.6 - The KOReader Device Identification Strategy is valid for solving the max_devices exhaustion problem. The plan correctly distinguishes between device identification (management) and authentication (security).

Architecture:

Plugin generates unique device_id on first launch (stored in KOReader settings)
Plugin sends device_identifier during registration API call
Backend validates auth_token matches device record
Reinstalling KOReader reads same device_id from settings, reuses existing device record

Registration Flow Options:

Approach	User Experience	Complexity	Pros/Cons
A. Manual Link	User registers in Bookhoard, copies Device ID + Token	Low	✅ Simple server ❌ User copies two values
B. Self-Register	Plugin auto-registers, user approves in web UI	Medium	✅ One-time setup ⚠️ Requires approval endpoint
C. QR Code Bridge	Scan QR code with Device ID + Token	Low-Medium	✅ Very user-friendly ❌ Requires QR library

Recommendation: Option B (Self-Register) for best UX.

Backend Changes Required:

// Devices table - Add Device Identifier (Already Exists!)
-- devices.device_identifier already exists (models.go:67)
-- No schema change needed - just use it properly

// internal/handlers/devices.go - InitiateDeviceRegistration()
// Add optional pre_generated_device_id parameter:
DeviceIdentifier string `json:"device_identifier" validate:"omitempty,uuid"`

Plugin Changes Required:

-- First Run Detection:
-- On first launch:
if not G_Settings:hasSetting("bookhoard_device_id") then
    local device_id = uuid.generate()
    G_Settings:saveSetting("bookhoard_device_id", device_id)
    G_Settings:saveSetting("bookhoard_device_name", "KOReader on " .. Device.model)
    
    -- Trigger registration
    register_device()
end

-- Registration API Call:
-- POST /api/devices/register
{
    device_name: "KOReader on Kindle Paperwhite",
    device_type: "koreader",
    device_identifier: "550e8400-e29b..."  -- From plugin settings
}

Device Management UI:

Display device_identifier for each device
"Copy Device ID" button for KOReader setup
"Regenerate Token" doesn't change Device ID
Show "Linked Devices" count (e.g., "3 KOReader installations linked")

Sync Flow With Device ID:

-- Plugin Headers:
Authorization: Bearer {auth_token}
X-Bookhoard-Device-ID: {device_identifier}

Benefits:

Aspect	Current	With Device ID
Reinstalls	New device each time	Same device, update token only
max_devices	Counts installs, not devices	Accurate device count
Security	Token-only	Device ID + token (two-factor)
Troubleshooting	"Which KOReader is this?"	Clear device identification
Plugin Transfer	Manual token copy	Migrate settings file

Phase Integration: Implement Device ID in Phase 1 (alongside dual authentication). It's same amount of work but solves max_devices problem elegantly.

Questions for User:

Registration Workflow: Should KOReader plugin self-register (user approves in browser) or require manual registration first?
Device ID Persistence: Where does KOReader store settings?
- /mnt/us/koreader/settings.bookhoard.lua?
- System G_Settings?
- This determines if Device ID survives plugin updates
max_devices Behavior:
- Should each Device ID count as 1 device (recommended)?
- Or limit tokens per Device ID (more complex)?
Backward Compatibility:
- How to handle existing KOReader installations without Device ID?
- Force re-registration? Auto-migrate?
KOReader Plugin Repo:
- Does github.com/bookhoard/koreader-plugin exist?
- Should Device ID generation happen there, or in main Bookhoard repo?

10. Historical Context & Conversation Summary

Why This Plan Exists

This implementation plan emerged from a detailed analysis of the current authentication and sync architecture. Key discoveries from codebase review:

The Authentication Problem

Original Issue: The plan started with @fix-opds-device-authentication.md which identified:

OPDS routes were publicly accessible (security vulnerability)
Test GetDeviceCatalog_WithoutDeviceAuth expected 401 but got 404
DeviceAuthMiddleware was already applied to OPDS routes, but tests were failing

Root Cause Discovery:

Kobo devices send Authorization: Bearer {kobo-store-token} (Kobo's store token, not Bookhoard's)
Kobo cannot send custom Bearer tokens through eReader.conf
Current middleware only supports Bearer tokens in Authorization header
Solution: Use API key in URL path (proven by Komga)

The Sidecar File Red Herring

Initial Thought: Use .bookhoard.json sidecar file to pass device tokens to Kobo

Problem Discovered:

Sidecar handler exists (internal/handlers/sidecar.go) but routes are NOT registered
Kobo firmware is locked down - cannot read custom config files
No plugin architecture on Kobo (unlike KOReader)
Would require Kobo firmware modification (impossible for most users)

Decision: Abandon sidecar approach, use API key in URL path (proven by Komga)

API keys are revocable and regenerable
Works with stock Kobo firmware
Simple user configuration

KOReader Documentation Bug

Original Documentation: Claimed KOReader uses "Basic Auth" with username/password

Reality:

KOReader sync sends Authorization: Bearer {token} to sync servers
The official KOReader sync protocol uses MD5 hashed passwords
Bookhoard's implementation expects Bearer tokens in DeviceAuthMiddleware
KOReader cannot send custom headers through native settings

Solution: Create KOReader plugin that can send proper Bearer tokens

Booklore Analysis

Investigation: Analyzed how Booklore handles Kobo authentication

Booklore Approach:

Token embedded in URL path: /api/kobo/{token}/...
Works but token appears in logs
One token per user (not per device)
OPDS uses separate Basic Auth

Decision: Don't copy Booklore - less secure. Use device-specific auth with proper tokens for KOReader.

Plugin Feasibility Research

Concern: Is KOReader a "moving target" for plugin development?

Findings:

UI plugins are unstable (Issue #13942) - frequent breaking changes
Backend/sync plugins (like Wallabag) are stable
Wallabag2 plugin has minimal updates over years
HTTP client APIs don't change often

Decision: Proceed with plugin - sync plugins are low-risk

Technical Constraints Discovered

Kobo Firmware Limitations:
- Cannot send custom HTTP headers
- Cannot install plugins
- Settings in eReader.conf are limited to specific keys
- Solution: API key in URL path works with stock firmware
KOReader Capabilities:
- Can install plugins (copy files to /koreader/plugins/)
- Lua-based plugin architecture
- Can send custom HTTP requests
- Plugin can register itself
- Native OPDS client support
Security Trade-offs:
- API Key in URl: Which means it's stored in logs
- Bearer tokens: Cryptographically secure, requires setup
- User accepted security/usability trade-off for self-hosted use

Why Functionality Trumps Security

User Quote: "I want a Kindle system replacement, otherwise I could just use booklore or KOReader. My biggest concern is do any of these limit any of the syncing/downloading implementation I already have. Functionality and features are more important than anything else. Although security does come in at a close second."

Translation:

Self-hosted = trusted network environment
User understands and accepts risks
Willing to document security implications
Prioritizes working features over perfect security

Resulting Architecture:

Kobo: API key in URL path (revocable, simple)
KOReader: Bearer token (high security, requires plugin setup)
Both get full feature parity
Security implications clearly documented

11. Updated Implementation Timeline

Parallel Track Structure

Track A: Kobo Integration

Phase 1: Enhanced Auth (Week 1)
Phase 2a: Kobo Implementation (Week 1-2)
Phase 4: Kobo Testing (Week 3)

Track B: KOReader Plugin

Phase 1: Enhanced Auth (Week 1) [shared]
Phase 2b: Plugin Development (Week 2-4)
Phase 4: Plugin Testing (Week 4)

Track C: OPDS Security

Phase 3: OPDS Integration (Week 2-3)

Track D: Documentation

Phase 5: Final Docs & Release (Week 4)

Milestones

Milestone 1 (End of Week 2): Kobo Fully Functional

Kobo: API key in URL (works with stock firmware)
All sync features operational
OPDS access secured
Can ship to Kobo users

Milestone 2 (End of Week 4): KOReader Plugin Ready

Plugin released in separate repo
Full feature parity with Kobo
Documentation complete
Production release

12. Files & Components Reference

Critical Files for Implementation

Authentication Core:

internal/middleware/device_auth.go:37-108 - Main auth middleware
internal/database/queries/queries.sql - Device queries
internal/database/models.go - Device model (has device_identifier)

Routing:

internal/router/opds.go:11-12 - OPDS routes (need middleware)
internal/router/sync.go:17-50 - Sync routes
internal/router/device.go - Device management routes

Handlers:

internal/handlers/kobo.go:26-679 - Kobo sync implementation
internal/handlers/koreader.go:23-897 - KOReader sync implementation
internal/handlers/opds.go - OPDS handler

Tests:

cmd/server/tests/kobo_test.go - Kobo tests
cmd/server/tests/opds_test.go - OPDS tests
cmd/server/tests/test_helpers.go - Test utilities

Documentation (to update):

docs/user/devices/kobo-setup.md - Currently has wrong info
docs/user/devices/koreader-setup.md - Needs Bearer token info

Sidecar (reference only, not implementing):

internal/handlers/sidecar.go - Exists but not wired up
Routes not registered in router

13. Next Steps

Immediate Actions (When Ready to Proceed)

Review this plan - Ensure all decisions and context are captured
Create KOReader plugin repo - Set up github.com/bookhoard/koreader-plugin
Begin Phase 1 - Enhanced DeviceAuthMiddleware
Parallel development - Kobo and KOReader tracks

Success Criteria (Reiterated)

Functional:

✅ Kobo: Enter serial → register → sync works immediately
✅ KOReader: Install plugin → plugin registers → sync works
✅ Both: Full sync (progress, highlights, notes, bookmarks)
✅ OPDS: Authenticated access on both platforms
✅ Cross-device: Read on one, continue on another

Security:

✅ OPDS no longer publicly accessible (authenticated with API key or Bearer token)
✅ All sync endpoints require device authentication (API key or bearer token)
✅ Device authentication uses revocable API keys (Kobo/KOReader)
✅ Security implications documented
✅ Network security recommendations provided

User Experience:

✅ Kobo setup: < 5 minutes
✅ KOReader setup: < 10 minutes
✅ Clear documentation
✅ Working examples

Plan Status: Ready for implementation Last Updated: Based on conversation ending with user decisions Note: User indicated it's late and not proceeding tonight

Ready to implement when you are. This plan captures all our discussion and decisions.

16. Documentation Updates Required (Pre-Implementation Checklist)

Based on codebase analysis, the following documentation updates are needed:

koreader-setup.md (docs/user/devices/koreader-setup.md)

Current Issues:

Line 126: Says "Basic Auth" - should be "Bearer Token"
Lines 127-128: Reference username/password - should reference auth_token
Documentation states KOReader uses Basic Auth (incorrect)

Required Changes:

- 1. **Authentication Method**: Select "Basic Auth"
- 2. **Username**: Your Bookhoard email or username
- 3. **Password**: Your Bookhoard password
+ 1. **Authentication Method**: Bearer Token (API Key)
+ 2. **Auth Token**: Copy from Bookhoard Device Management page
+ 3. **Setup**: Plugin will include token in Authorization header automatically

kobo-setup.md (docs/user/devices/kobo-setup.md)

Current Status:

Updated on 2026-02-12 to reflect API key authentication
Shows full URL format with token

Verification Needed:

Confirm lines 37-53 (device registration) don't mention entering serial number
Verify registration flow describes automatic API key generation
Ensure "Copy Full Sync URL" button is documented

New Documentation: security.md

Required Content:

API key in URL path security considerations
Comparison: API key (URL) vs Bearer token (header)
Network security recommendations (HTTPS, VPN, local-only)
Token regeneration best practices
Risk mitigations for self-hosted deployments

10. Risk Assessment

Low Risk:

API key in URL path (well-understood pattern)
OPDS security (straightforward middleware application)
Kobo sync (already implemented, just fixing auth route)

Medium Risk:

KOReader plugin development (Lua learning curve, potential API changes)
Cross-device sync conflicts (need clear resolution strategy)

High Risk:

KOReader API instability (if they change plugin APIs frequently)
User adoption (may resist plugin installation)

Mitigations:

Start with Kobo to prove concept
Simple plugin architecture (minimize breaking changes impact)
Excellent documentation to reduce friction
Community feedback loop

11. Timeline Summary

Phase	Duration	Key Deliverable
Phase 1	Week 1	Enhanced auth middleware working
Phase 2	Week 1-2	Kobo fully functional
Phase 3	Week 2-4	KOReader plugin complete
Phase 4	Week 2-3	OPDS secured
Phase 5	Week 4	Testing & docs complete

Total Duration: 4 weeks

Next Steps:

User reviews and approves plan
Answer open questions (Section 9)
Begin Phase 1 implementation
Weekly check-ins on progress

Ready to proceed?

14. Authentication Strategy Confirmation

14.1 Kobo Authentication: API Key in URL Path

Decision: Use per-device API keys embedded in URL path (proven by Komga)