docs: add comprehensive device authentication implementation plan
Add detailed implementation plan covering: - Enhanced authentication middleware (Bearer + serial) - Kobo native sync with serial-based auth - KOReader plugin development plan - OPDS security hardening - Parallel implementation tracks - Complete historical context and decision rationale This plan documents the strategy to transform Bookhoard into a Kindle-replacement ecosystem with full sync support for both Kobo (native) and KOReader (via plugin) devices. Key decisions: - Kobo: Serial number authentication (simplest UX) - KOReader: Bearer token via plugin (most secure) - Plugin: Separate repository under Bookhoard org - Implementation: Parallel tracks for faster delivery
This commit is contained in:
@@ -0,0 +1,760 @@
|
||||
# Bookhoard Device Authentication & Sync Implementation Plan
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Current Problem:**
|
||||
- OPDS routes are publicly accessible (security vulnerability)
|
||||
- Kobo sync uses Bearer tokens but Kobo firmware can't send them (broken)
|
||||
- KOReader documentation is incorrect about Basic Auth
|
||||
- No clear authentication strategy for different device types
|
||||
- Sidecar implementation exists but isn't integrated
|
||||
|
||||
**Vision:**
|
||||
Transform Bookhoard into a comprehensive Kindle-replacement ecosystem with seamless device sync, supporting both Kobo (native) and KOReader (via plugin) with full functionality including progress, highlights, notes, and bookmarks.
|
||||
|
||||
---
|
||||
|
||||
## 1. Current State Analysis
|
||||
|
||||
### 1.1 Authentication Landscape
|
||||
|
||||
**Existing Infrastructure:**
|
||||
```go
|
||||
// DeviceAuthMiddleware - Currently only supports Bearer tokens
|
||||
// File: internal/middleware/device_auth.go:37-108
|
||||
func (m *DeviceAuthMiddleware) Authenticate(next echo.HandlerFunc) echo.HandlerFunc {
|
||||
// Only checks Authorization: Bearer {token}
|
||||
// Validates against devices.auth_token
|
||||
}
|
||||
```
|
||||
|
||||
**Routes Using Device Auth:**
|
||||
- `/api/sync/kobo/*` - Kobo sync endpoints
|
||||
- `/api/sync/koreader/*` - KOReader sync endpoints
|
||||
- `/opds/devices/*` - OPDS catalog (currently NOT protected, needs auth)
|
||||
|
||||
**Database Schema:**
|
||||
```sql
|
||||
-- devices table already has device_identifier field
|
||||
-- Can store serial numbers for Kobo devices
|
||||
```
|
||||
|
||||
### 1.2 Kobo Integration Status
|
||||
|
||||
**What's Broken:**
|
||||
- Kobo sends `Authorization: Bearer {kobo-store-token}` (wrong token)
|
||||
- Kobo sends `x-kobo-device: {"SerialNumber":"N123..."}` (what we need to use)
|
||||
- Current middleware rejects Kobo requests because token doesn't match
|
||||
|
||||
**What's Working:**
|
||||
- Kobo sync handler implementation exists
|
||||
- Endpoints registered: `/api/sync/kobo/markup`, `/bookmark`, `/v1/initialization`, etc.
|
||||
- Can sync progress, highlights, notes, bookmarks once auth works
|
||||
|
||||
### 1.3 KOReader Integration Status
|
||||
|
||||
**What's Broken:**
|
||||
- Documentation says use "Basic Auth" - this is WRONG
|
||||
- KOReader doesn't support custom headers natively for sync
|
||||
- Bearer token can't be sent via standard KOReader settings
|
||||
|
||||
**What's Working:**
|
||||
- KOReader sync handler supports full feature set
|
||||
- Can sync: progress, bookmarks, highlights, notes, library metadata
|
||||
- Plugin architecture exists but no Bookhoard plugin yet
|
||||
|
||||
### 1.4 OPDS Status
|
||||
|
||||
**Current:**
|
||||
- Routes registered in `internal/router/opds.go:11-12`
|
||||
- **NOT using DeviceAuthMiddleware** (security hole)
|
||||
- Publicly accessible - anyone can enumerate device UUIDs
|
||||
|
||||
**Required:**
|
||||
- Must require authentication
|
||||
- Needs to work with both Kobo (serial) and KOReader (Bearer token)
|
||||
|
||||
---
|
||||
|
||||
## 2. User Priorities (Ranked)
|
||||
|
||||
### Priority 1: Functionality First
|
||||
**User Quote:** "I want a Kindle system replacement, otherwise I could just use Booklore or KOReader"
|
||||
|
||||
**Requirements:**
|
||||
- ✅ Full sync: progress, highlights, notes, bookmarks
|
||||
- ✅ Works on both Kobo and KOReader
|
||||
- ✅ Seamless user experience
|
||||
- ✅ OPDS for wireless book delivery
|
||||
- ✅ Two-way sync (device ↔ server)
|
||||
|
||||
### Priority 2: Security Second (with documentation)
|
||||
**User Quote:** "This is a self-hosted app so as long as the user is aware of the security implications in the documentation it should be fine"
|
||||
|
||||
**Acceptable Trade-offs:**
|
||||
- Serial-based auth for Kobo (less secure but easier)
|
||||
- Security risks documented clearly
|
||||
- Token-based for KOReader (more secure)
|
||||
- Users responsible for network security (VPN, etc.)
|
||||
|
||||
### Priority 3: Simple Setup
|
||||
**Goal:** Minimal friction for users
|
||||
|
||||
**Kobo Experience:**
|
||||
- Enter serial number during registration
|
||||
- One line in `Kobo eReader.conf`
|
||||
- No token management
|
||||
- Works immediately
|
||||
|
||||
**KOReader Experience:**
|
||||
- Copy plugin files
|
||||
- Enter server URL and token
|
||||
- One-time setup
|
||||
- Background sync works automatically
|
||||
|
||||
---
|
||||
|
||||
## 3. Target State
|
||||
|
||||
### 3.1 Unified Authentication Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ DeviceAuthMiddleware │
|
||||
│ Enhanced to support multiple auth methods: │
|
||||
│ │
|
||||
│ 1. Authorization: Bearer {token} │
|
||||
│ → Lookup device by auth_token │
|
||||
│ → Used by: KOReader, OPDS apps │
|
||||
│ │
|
||||
│ 2. X-Kobo-Device: {"SerialNumber":"N123..."} │
|
||||
│ → Extract serial from JSON │
|
||||
│ → Lookup device by device_identifier │
|
||||
│ → Used by: Kobo e-readers │
|
||||
│ │
|
||||
│ Both paths result in: device context set in echo.Context │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### 3.2 Feature Matrix
|
||||
|
||||
| Feature | Kobo (Native) | KOReader (Plugin) |
|
||||
|---------|---------------|-------------------|
|
||||
| **Progress Sync** | ✅ Automatic | ✅ Automatic |
|
||||
| **Highlights** | ✅ Native support | ✅ Via plugin API |
|
||||
| **Bookmarks** | ✅ Native support | ✅ Via plugin API |
|
||||
| **Notes** | ✅ Native support | ✅ Via plugin API |
|
||||
| **OPDS** | ✅ Via NickelMenu | ✅ Native OPDS support |
|
||||
| **Setup Complexity** | Low (1 config line) | Medium (plugin install) |
|
||||
| **Auth Method** | Serial number | Bearer token |
|
||||
| **Security** | Medium (documented) | High |
|
||||
|
||||
### 3.3 User Flows
|
||||
|
||||
**Kobo User Journey:**
|
||||
1. Install NickelMenu (one-time, jailbreak)
|
||||
2. Open Bookhoard web UI
|
||||
3. Register device → enter serial number (from Settings)
|
||||
4. Approve device
|
||||
5. Edit `Kobo eReader.conf` → add sync URL
|
||||
6. Read books → automatic sync
|
||||
7. Access OPDS via NickelMenu
|
||||
|
||||
**KOReader User Journey:**
|
||||
1. Install KOReader on device
|
||||
2. Open Bookhoard web UI
|
||||
3. Register device → copy auth token
|
||||
4. Approve device
|
||||
5. Install Bookhoard plugin → paste token
|
||||
6. Configure sync settings
|
||||
7. Read books → automatic sync
|
||||
8. Add OPDS catalog → browse library
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Plan
|
||||
|
||||
### Phase 1: Enhanced Authentication (Week 1)
|
||||
**Goal:** Make auth work for both Kobo and KOReader
|
||||
|
||||
**Tasks:**
|
||||
|
||||
1. **Update DeviceAuthMiddleware** (`internal/middleware/device_auth.go`)
|
||||
- [ ] Try Bearer token lookup first (existing behavior)
|
||||
- [ ] If no Bearer token, check for `x-kobo-device` header
|
||||
- [ ] Parse JSON to extract `SerialNumber` field
|
||||
- [ ] Query database: `GetDeviceByDeviceIdentifier(serial)`
|
||||
- [ ] Set device context on successful auth
|
||||
- [ ] Return 401 if both methods fail
|
||||
|
||||
2. **Add Database Query** (`internal/database/queries/devices.sql`)
|
||||
- [ ] Add query: `GetDeviceByDeviceIdentifier`
|
||||
- [ ] Ensure index exists on `device_identifier` column
|
||||
|
||||
3. **Update Device Registration UI** (`templates/devices.templ`)
|
||||
- [ ] Add device type selector (Kobo/KOReader)
|
||||
- [ ] Show serial number field for Kobo (required)
|
||||
- [ ] Show serial number field for KOReader (optional, for fallback)
|
||||
- [ ] Generate/display auth token after approval (for KOReader)
|
||||
|
||||
4. **Apply Middleware to OPDS** (`internal/router/opds.go`)
|
||||
- [ ] Add `DeviceAuthMiddleware` to all `/opds/devices/*` routes
|
||||
- [ ] Update comments to reflect auth requirements
|
||||
|
||||
**Testing:**
|
||||
- Kobo device sync with serial auth
|
||||
- KOReader sync with Bearer token auth
|
||||
- OPDS access with both auth methods
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Kobo Integration (Week 1-2)
|
||||
**Goal:** Seamless Kobo experience with serial-based auth
|
||||
|
||||
**Tasks:**
|
||||
|
||||
1. **Kobo Setup Documentation** (`docs/user/devices/kobo-setup.md`)
|
||||
- [ ] Revise to reflect serial-based auth (not username/password)
|
||||
- [ ] Step-by-step guide for finding serial number
|
||||
- [ ] NickelMenu configuration for sync
|
||||
- [ ] Security warning about serial-based auth
|
||||
- [ ] Network security recommendations (VPN, local network)
|
||||
|
||||
2. **Kobo Sync Validation** (`cmd/server/tests/kobo_test.go`)
|
||||
- [ ] Update tests to use serial auth
|
||||
- [ ] Test sync endpoints with `x-kobo-device` header
|
||||
- [ ] Verify all features work: progress, highlights, bookmarks, notes
|
||||
|
||||
3. **Kobo OPDS Integration**
|
||||
- [ ] Document OPDS URL format
|
||||
- [ ] Test OPDS browsing with serial auth
|
||||
- [ ] Test book downloads
|
||||
|
||||
**Success Criteria:**
|
||||
- New Kobo device registered and syncing within 5 minutes
|
||||
- All sync features working without token management
|
||||
- Security implications clearly documented
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: KOReader Plugin Development (Week 2-4)
|
||||
**Goal:** Full-featured KOReader plugin with Bearer token auth
|
||||
|
||||
**Tasks:**
|
||||
|
||||
1. **Plugin Architecture**
|
||||
- [ ] Create `koreader-plugin/bookhoard.koplugin/` directory
|
||||
- [ ] Implement `_meta.lua` (plugin metadata)
|
||||
- [ ] Implement `main.lua` (entry point)
|
||||
|
||||
2. **Core Plugin Features**
|
||||
- [ ] Settings UI: server URL, auth token, sync options
|
||||
- [ ] HTTP client with Bearer token authentication
|
||||
- [ ] Auto-sync on events:
|
||||
- Page turn (configurable: every N pages)
|
||||
- Bookmark added/deleted
|
||||
- Highlight created/deleted
|
||||
- Note added/edited
|
||||
- [ ] Manual sync button in KOReader menu
|
||||
- [ ] Sync status indicator
|
||||
- [ ] Error handling and retry logic
|
||||
|
||||
3. **Plugin API Integration**
|
||||
- [ ] Call `/api/sync/koreader/progress` with reading position
|
||||
- [ ] Call `/api/sync/koreader/bookmarks` with annotations
|
||||
- [ ] Call `/api/sync/koreader/metadata` to get server-side progress
|
||||
- [ ] Handle conflicts (server vs local progress)
|
||||
|
||||
4. **Plugin Documentation**
|
||||
- [ ] Installation guide (copy files, restart KOReader)
|
||||
- [ ] Configuration guide (get token from Bookhoard UI)
|
||||
- [ ] Usage instructions
|
||||
- [ ] Troubleshooting guide
|
||||
|
||||
**Success Criteria:**
|
||||
- Plugin installed and configured in under 10 minutes
|
||||
- Background sync works transparently
|
||||
- All annotation types sync properly
|
||||
- No Lua errors or crashes
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: OPDS Security & Integration (Week 2-3)
|
||||
**Goal:** Secure OPDS with unified device authentication
|
||||
|
||||
**Tasks:**
|
||||
|
||||
1. **Secure OPDS Routes** (`internal/router/opds.go`)
|
||||
- [ ] Apply `DeviceAuthMiddleware` to all OPDS endpoints
|
||||
- [ ] Ensure auth works with both serial and Bearer methods
|
||||
- [ ] Update router comments
|
||||
|
||||
2. **Kobo OPDS Access**
|
||||
- [ ] Document OPDS URL with device ID
|
||||
- [ ] Test with NickelMenu OPDS browser
|
||||
- [ ] Test book downloads
|
||||
|
||||
3. **KOReader OPDS Access**
|
||||
- [ ] Works natively via KOReader's OPDS support
|
||||
- [ ] Uses Bearer token in header
|
||||
- [ ] Document catalog URL format
|
||||
|
||||
4. **Test Coverage** (`cmd/server/tests/opds_test.go`)
|
||||
- [ ] Test unauthenticated access → expect 401
|
||||
- [ ] Test with valid Bearer token → expect 200
|
||||
- [ ] Test with valid serial header → expect 200
|
||||
- [ ] Test book download with auth
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Testing & Documentation (Week 4)
|
||||
**Goal:** Production-ready release with comprehensive docs
|
||||
|
||||
**Tasks:**
|
||||
|
||||
1. **End-to-End Testing**
|
||||
- [ ] Fresh Kobo setup: register → sync → highlights → OPDS
|
||||
- [ ] Fresh KOReader setup: install plugin → sync → highlights → OPDS
|
||||
- [ ] Cross-device sync: Kobo ↔ KOReader same book
|
||||
- [ ] Offline scenarios and conflict resolution
|
||||
- [ ] Error handling and recovery
|
||||
|
||||
2. **Security Documentation**
|
||||
- [ ] Security implications page
|
||||
- [ ] Network security recommendations
|
||||
- [ ] Comparison: Bearer vs Serial auth
|
||||
- [ ] Threat model for self-hosted users
|
||||
|
||||
3. **User Documentation**
|
||||
- [ ] Quick start guide (Kobo)
|
||||
- [ ] Quick start guide (KOReader)
|
||||
- [ ] Feature comparison table
|
||||
- [ ] FAQ and troubleshooting
|
||||
|
||||
4. **Release Preparation**
|
||||
- [ ] Version bump
|
||||
- [ ] Changelog
|
||||
- [ ] Migration guide (if needed)
|
||||
|
||||
---
|
||||
|
||||
## 5. Technical Decisions
|
||||
|
||||
### 5.1 Why Serial-Based Auth for Kobo?
|
||||
|
||||
**Pros:**
|
||||
- No token management required
|
||||
- Works with unmodified Kobo firmware
|
||||
- Single configuration line
|
||||
- User can't lose/forget credentials
|
||||
|
||||
**Cons:**
|
||||
- Serial numbers are somewhat predictable
|
||||
- No token rotation possible
|
||||
- If leaked, can't be revoked (must delete device)
|
||||
|
||||
**Mitigations:**
|
||||
- Document security risks
|
||||
- Recommend VPN for remote access
|
||||
- Use HTTPS (reverse proxy)
|
||||
- Acceptable for self-hosted personal use
|
||||
|
||||
### 5.2 Why Bearer Token for KOReader?
|
||||
|
||||
**Pros:**
|
||||
- Cryptographically secure
|
||||
- Can be revoked/regenerated
|
||||
- Standard authentication method
|
||||
- Works with KOReader plugin architecture
|
||||
|
||||
**Cons:**
|
||||
- Requires plugin (can't use native settings)
|
||||
- Token must be copy-pasted during setup
|
||||
- User must manage token
|
||||
|
||||
**Trade-off Accepted:**
|
||||
More secure but requires plugin installation. Justified because KOReader users are typically more technical and willing to install plugins.
|
||||
|
||||
### 5.3 Why Not Use Kobo's Token?
|
||||
|
||||
Kobo sends `Authorization: Bearer {kobo-store-token}` which is:
|
||||
- Managed by Kobo (not under our control)
|
||||
- May rotate/expire without warning
|
||||
- Only valid for Kobo's servers
|
||||
- Doesn't identify the device in our system
|
||||
|
||||
**Decision:** Parse the `x-kobo-device` header for serial number instead.
|
||||
|
||||
---
|
||||
|
||||
## 6. Files to Modify
|
||||
|
||||
### Core Authentication
|
||||
- `internal/middleware/device_auth.go` - Enhance with serial support
|
||||
- `internal/database/queries/devices.sql` - Add serial lookup query
|
||||
- `internal/router/opds.go` - Apply middleware
|
||||
|
||||
### Device Management
|
||||
- `templates/devices.templ` - Update registration form
|
||||
- `internal/handlers/devices.go` - Handle serial input
|
||||
|
||||
### Testing
|
||||
- `cmd/server/tests/kobo_test.go` - Update for serial auth
|
||||
- `cmd/server/tests/opds_test.go` - Add auth tests
|
||||
- `cmd/server/tests/test_helpers.go` - Add serial auth helpers
|
||||
|
||||
### Documentation
|
||||
- `docs/user/devices/kobo-setup.md` - Rewrite for serial auth
|
||||
- `docs/user/devices/koreader-setup.md` - Update for Bearer token
|
||||
- `docs/user/security.md` - New security implications doc
|
||||
- `koreader-plugin/README.md` - Plugin documentation
|
||||
|
||||
### New Files
|
||||
- `koreader-plugin/bookhoard.koplugin/_meta.lua` - Plugin metadata
|
||||
- `koreader-plugin/bookhoard.koplugin/main.lua` - Plugin implementation
|
||||
- `docs/user/quickstart-kobo.md` - Quick start guide
|
||||
- `docs/user/quickstart-koreader.md` - Quick start guide
|
||||
|
||||
---
|
||||
|
||||
## 7. Success Metrics
|
||||
|
||||
**Functionality:**
|
||||
- ✅ Kobo syncs progress, highlights, notes, bookmarks automatically
|
||||
- ✅ KOReader syncs same features via plugin
|
||||
- ✅ OPDS works on both platforms
|
||||
- ✅ Cross-device sync works (read on Kobo, continue on KOReader)
|
||||
|
||||
**Usability:**
|
||||
- ✅ Kobo setup time: < 5 minutes
|
||||
- ✅ KOReader setup time: < 10 minutes
|
||||
- ✅ No token management for Kobo users
|
||||
- ✅ Clear documentation with security warnings
|
||||
|
||||
**Security:**
|
||||
- ✅ OPDS no longer publicly accessible
|
||||
- ✅ Auth required for all sync endpoints
|
||||
- ✅ Security implications documented
|
||||
- ✅ Network security recommendations provided
|
||||
|
||||
---
|
||||
|
||||
## 8. Future Enhancements (Out of Scope)
|
||||
|
||||
- Auto-discovery of devices on network
|
||||
- QR code setup for mobile devices
|
||||
- Background sync service (no plugin needed)
|
||||
- Readwise/Joplin integration
|
||||
- Calibre integration
|
||||
- Mobile apps (iOS/Android)
|
||||
|
||||
---
|
||||
|
||||
## 9. Decisions Made (User Input)
|
||||
|
||||
Based on user feedback, the following decisions have been finalized:
|
||||
|
||||
### 9.1 KOReader Sync Trigger
|
||||
**Decision:** Configurable with multiple options
|
||||
- **Auto-sync**: Every N pages (configurable, default every 10 pages)
|
||||
- **Auto-sync**: Every N minutes (configurable, default every 5 minutes)
|
||||
- **Manual sync**: Always available via menu
|
||||
- **Event-triggered**: On bookmark/highlight/note creation (immediate)
|
||||
|
||||
**Rationale:** User wants automatic sync for convenience but manual fallback for control.
|
||||
|
||||
### 9.2 Plugin Distribution
|
||||
**Decision:** Separate repository under Bookhoard organization
|
||||
- Repository: `github.com/bookhoard/koreader-plugin`
|
||||
- Licensed under same terms as Bookhoard
|
||||
- Versioned independently
|
||||
- Referenced in main Bookhoard documentation
|
||||
|
||||
**Rationale:** Clean separation of concerns, easier plugin-specific issues/PRs.
|
||||
|
||||
### 9.3 Kobo Authentication
|
||||
**Decision:** Serial number ONLY
|
||||
- No Bearer token fallback for Kobo
|
||||
- Serial entered during device registration
|
||||
- Stored in `devices.device_identifier`
|
||||
|
||||
**Rationale:** Simplest user experience, one auth method per platform.
|
||||
|
||||
### 9.4 Conflict Resolution
|
||||
**Decision:** Use existing codebase strategy
|
||||
- Already implemented in `internal/handlers/koreader.go`
|
||||
- Uses timestamp-based last-write-wins
|
||||
- Device priority can be configured per-device
|
||||
|
||||
**Rationale:** Don't reinvent the wheel, existing implementation is sufficient.
|
||||
|
||||
### 9.5 Implementation Order
|
||||
**Decision:** Parallel implementation with separate sections
|
||||
- Phase 2a: Kobo Integration
|
||||
- Phase 2b: KOReader Plugin (runs concurrently)
|
||||
- Separate milestones and deliverables
|
||||
- Can ship Kobo support before plugin is ready
|
||||
|
||||
**Rationale:** Faster time-to-market for Kobo users, plugin can follow.
|
||||
|
||||
---
|
||||
|
||||
## 10. Historical Context & Conversation Summary
|
||||
|
||||
### Why This Plan Exists
|
||||
|
||||
This implementation plan emerged from a detailed analysis of the current authentication and sync architecture. Key discoveries from codebase review:
|
||||
|
||||
#### The Authentication Problem
|
||||
|
||||
**Original Issue:** The plan started with `@fix-opds-device-authentication.md` which identified:
|
||||
- OPDS routes were publicly accessible (security vulnerability)
|
||||
- Test `GetDeviceCatalog_WithoutDeviceAuth` expected 401 but got 404
|
||||
- DeviceAuthMiddleware was already applied to OPDS routes, but tests were failing
|
||||
|
||||
**Root Cause Discovery:**
|
||||
- Kobo devices send `Authorization: Bearer {kobo-store-token}` (Kobo's store token, not Bookhoard's)
|
||||
- Kobo also sends `x-kobo-device: {"SerialNumber":"N123..."}` header
|
||||
- The middleware was rejecting requests because the Bearer token didn't match `devices.auth_token`
|
||||
- Kobo cannot send custom Bearer tokens through `eReader.conf`
|
||||
|
||||
#### The Sidecar File Red Herring
|
||||
|
||||
**Initial Thought:** Use `.bookhoard.json` sidecar file to pass device tokens to Kobo
|
||||
|
||||
**Problem Discovered:**
|
||||
- Sidecar handler exists (`internal/handlers/sidecar.go`) but routes are NOT registered
|
||||
- Kobo firmware is locked down - cannot read custom config files
|
||||
- No plugin architecture on Kobo (unlike KOReader)
|
||||
- Would require Kobo firmware modification (impossible for most users)
|
||||
|
||||
**Decision:** Abandon sidecar approach, use serial-based auth instead
|
||||
|
||||
#### KOReader Documentation Bug
|
||||
|
||||
**Original Documentation:** Claimed KOReader uses "Basic Auth" with username/password
|
||||
|
||||
**Reality:**
|
||||
- KOReader sync sends `Authorization: Bearer {token}` to sync servers
|
||||
- The official KOReader sync protocol uses MD5 hashed passwords
|
||||
- Bookhoard's implementation expects Bearer tokens in `DeviceAuthMiddleware`
|
||||
- KOReader cannot send custom headers through native settings
|
||||
|
||||
**Solution:** Create KOReader plugin that can send proper Bearer tokens
|
||||
|
||||
#### Booklore Analysis
|
||||
|
||||
**Investigation:** Analyzed how Booklore handles Kobo authentication
|
||||
|
||||
**Booklore Approach:**
|
||||
- Token embedded in URL path: `/api/kobo/{token}/...`
|
||||
- Works but token appears in logs
|
||||
- One token per user (not per device)
|
||||
- OPDS uses separate Basic Auth
|
||||
|
||||
**Decision:** Don't copy Booklore - less secure. Use device-specific auth with proper tokens for KOReader.
|
||||
|
||||
#### Plugin Feasibility Research
|
||||
|
||||
**Concern:** Is KOReader a "moving target" for plugin development?
|
||||
|
||||
**Findings:**
|
||||
- UI plugins are unstable (Issue #13942) - frequent breaking changes
|
||||
- Backend/sync plugins (like Wallabag) are stable
|
||||
- Wallabag2 plugin has minimal updates over years
|
||||
- HTTP client APIs don't change often
|
||||
|
||||
**Decision:** Proceed with plugin - sync plugins are low-risk
|
||||
|
||||
### Technical Constraints Discovered
|
||||
|
||||
1. **Kobo Firmware Limitations:**
|
||||
- Cannot send custom HTTP headers (except `x-kobo-device`)
|
||||
- Cannot install plugins
|
||||
- Settings in `eReader.conf` are limited
|
||||
- NickelMenu required for OPDS access (jailbreak)
|
||||
|
||||
2. **KOReader Capabilities:**
|
||||
- Can install plugins (copy files to `/koreader/plugins/`)
|
||||
- Lua-based plugin architecture
|
||||
- Can send custom HTTP requests
|
||||
- Can access device serial for identification
|
||||
- Native OPDS client support
|
||||
|
||||
3. **Security Trade-offs:**
|
||||
- Serial numbers: Somewhat predictable, easy for users
|
||||
- Bearer tokens: Cryptographically secure, requires setup
|
||||
- User accepted security/usability trade-off for self-hosted use
|
||||
|
||||
### Why Functionality Trumps Security
|
||||
|
||||
**User Quote:** "I want a Kindle system replacement, otherwise I could just use booklore or KOReader. My biggest concern is do any of these limit any of the syncing/downloading implementation I already have. Functionality and features are more important than anything else. Although security does come in at a close second."
|
||||
|
||||
**Translation:**
|
||||
- Self-hosted = trusted network environment
|
||||
- User understands and accepts risks
|
||||
- Willing to document security implications
|
||||
- Prioritizes working features over perfect security
|
||||
|
||||
**Resulting Architecture:**
|
||||
- Kobo: Serial auth (medium security, zero friction)
|
||||
- KOReader: Bearer token (high security, requires plugin setup)
|
||||
- Both get full feature parity
|
||||
- Security implications clearly documented
|
||||
|
||||
---
|
||||
|
||||
## 11. Updated Implementation Timeline
|
||||
|
||||
### Parallel Track Structure
|
||||
|
||||
**Track A: Kobo Integration**
|
||||
- Phase 1: Enhanced Auth (Week 1)
|
||||
- Phase 2a: Kobo Implementation (Week 1-2)
|
||||
- Phase 4: Kobo Testing (Week 3)
|
||||
|
||||
**Track B: KOReader Plugin**
|
||||
- Phase 1: Enhanced Auth (Week 1) [shared]
|
||||
- Phase 2b: Plugin Development (Week 2-4)
|
||||
- Phase 4: Plugin Testing (Week 4)
|
||||
|
||||
**Track C: OPDS Security**
|
||||
- Phase 3: OPDS Integration (Week 2-3)
|
||||
|
||||
**Track D: Documentation**
|
||||
- Phase 5: Final Docs & Release (Week 4)
|
||||
|
||||
### Milestones
|
||||
|
||||
**Milestone 1 (End of Week 2):** Kobo Fully Functional
|
||||
- Serial auth working
|
||||
- All sync features operational
|
||||
- OPDS access secured
|
||||
- Can ship to Kobo users
|
||||
|
||||
**Milestone 2 (End of Week 4):** KOReader Plugin Ready
|
||||
- Plugin released in separate repo
|
||||
- Full feature parity with Kobo
|
||||
- Documentation complete
|
||||
- Production release
|
||||
|
||||
---
|
||||
|
||||
## 12. Files & Components Reference
|
||||
|
||||
### Critical Files for Implementation
|
||||
|
||||
**Authentication Core:**
|
||||
- `internal/middleware/device_auth.go:37-108` - Main auth middleware
|
||||
- `internal/database/queries/devices.sql` - Device queries
|
||||
- `internal/database/models.go` - Device model (has device_identifier)
|
||||
|
||||
**Routing:**
|
||||
- `internal/router/opds.go:11-12` - OPDS routes (need middleware)
|
||||
- `internal/router/sync.go:17-50` - Sync routes
|
||||
- `internal/router/device.go` - Device management routes
|
||||
|
||||
**Handlers:**
|
||||
- `internal/handlers/kobo.go:26-679` - Kobo sync implementation
|
||||
- `internal/handlers/koreader.go:23-897` - KOReader sync implementation
|
||||
- `internal/handlers/opds.go` - OPDS handler
|
||||
|
||||
**Tests:**
|
||||
- `cmd/server/tests/kobo_test.go` - Kobo tests
|
||||
- `cmd/server/tests/opds_test.go` - OPDS tests
|
||||
- `cmd/server/tests/test_helpers.go` - Test utilities
|
||||
|
||||
**Documentation (to update):**
|
||||
- `docs/user/devices/kobo-setup.md` - Currently has wrong info
|
||||
- `docs/user/devices/koreader-setup.md` - Needs Bearer token info
|
||||
|
||||
**Sidecar (reference only, not implementing):**
|
||||
- `internal/handlers/sidecar.go` - Exists but not wired up
|
||||
- Routes not registered in router
|
||||
|
||||
---
|
||||
|
||||
## 13. Next Steps
|
||||
|
||||
### Immediate Actions (When Ready to Proceed)
|
||||
|
||||
1. **Review this plan** - Ensure all decisions and context are captured
|
||||
2. **Create KOReader plugin repo** - Set up `github.com/bookhoard/koreader-plugin`
|
||||
3. **Begin Phase 1** - Enhanced DeviceAuthMiddleware
|
||||
4. **Parallel development** - Kobo and KOReader tracks
|
||||
|
||||
### Success Criteria (Reiterated)
|
||||
|
||||
**Functional:**
|
||||
- ✅ Kobo: Enter serial → register → sync works immediately
|
||||
- ✅ KOReader: Install plugin → enter token → sync works
|
||||
- ✅ Both: Full sync (progress, highlights, notes, bookmarks)
|
||||
- ✅ OPDS: Authenticated access on both platforms
|
||||
- ✅ Cross-device: Read on one, continue on another
|
||||
|
||||
**Security:**
|
||||
- ✅ OPDS no longer public
|
||||
- ✅ All sync endpoints require auth
|
||||
- ✅ Security implications documented
|
||||
- ✅ Network security recommendations provided
|
||||
|
||||
**User Experience:**
|
||||
- ✅ Kobo setup: < 5 minutes
|
||||
- ✅ KOReader setup: < 10 minutes
|
||||
- ✅ Clear documentation
|
||||
- ✅ Working examples
|
||||
|
||||
---
|
||||
|
||||
**Plan Status:** Ready for implementation
|
||||
**Last Updated:** Based on conversation ending with user decisions
|
||||
**Note:** User indicated it's late and not proceeding tonight
|
||||
|
||||
**Ready to implement when you are. This plan captures all our discussion and decisions.**
|
||||
|
||||
---
|
||||
|
||||
## 10. Risk Assessment
|
||||
|
||||
**Low Risk:**
|
||||
- Serial auth implementation (well-understood pattern)
|
||||
- OPDS security (straightforward middleware application)
|
||||
- Kobo sync (already implemented, just fixing auth)
|
||||
|
||||
**Medium Risk:**
|
||||
- KOReader plugin development (Lua learning curve, potential API changes)
|
||||
- Cross-device sync conflicts (need clear resolution strategy)
|
||||
|
||||
**High Risk:**
|
||||
- KOReader API instability (if they change plugin APIs frequently)
|
||||
- User adoption (may resist plugin installation)
|
||||
|
||||
**Mitigations:**
|
||||
- Start with Kobo to prove concept
|
||||
- Simple plugin architecture (minimize breaking changes impact)
|
||||
- Excellent documentation to reduce friction
|
||||
- Community feedback loop
|
||||
|
||||
---
|
||||
|
||||
## 11. Timeline Summary
|
||||
|
||||
| Phase | Duration | Key Deliverable |
|
||||
|-------|----------|----------------|
|
||||
| Phase 1 | Week 1 | Enhanced auth middleware working |
|
||||
| Phase 2 | Week 1-2 | Kobo fully functional |
|
||||
| Phase 3 | Week 2-4 | KOReader plugin complete |
|
||||
| Phase 4 | Week 2-3 | OPDS secured |
|
||||
| Phase 5 | Week 4 | Testing & docs complete |
|
||||
|
||||
**Total Duration: 4 weeks**
|
||||
|
||||
---
|
||||
|
||||
**Next Steps:**
|
||||
1. User reviews and approves plan
|
||||
2. Answer open questions (Section 9)
|
||||
3. Begin Phase 1 implementation
|
||||
4. Weekly check-ins on progress
|
||||
|
||||
**Ready to proceed?**
|
||||
Reference in New Issue
Block a user