Bookhoard/bookhoard
0
0 Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases 70 Wiki Activity

docs(dev): websocket wire doc rewritten from source; endpoint doc ping claim fixed

Browse Source 
The Feb-era 700-line page documented client->server JSON messages the
server never acted on (handleMessage logs only), a wrong ping/pong
exchange, and missed every message type added since. Rewritten tight
(~100 lines), every claim pinned: dual auth (JWT ?token= or device
Bearer header), initial_state on connect under the connecting
identity's RLS scope, the full 12-type message catalog (including
collection_updated, found per-user-targeted in the collections handler
and defined as a raw string outside the sync constants), WS-frame
keepalive (30s server ping, 90s read deadline — browsers answer
automatically), and the backpressure contract clients must honor:
bounded per-connection queues skip messages when full, so the socket
is a hint and REST refetch + fresh initial_state is the gap-healer.
Fan-out documented precisely: sync/scan events instance-wide,
collection events per-user.

Also corrected the gated endpoint doc (api/websocket/protocol.md):
its 'client sends {type:ping}, server responds pong' section
contradicted the handler; replaced with the true keepalive story.
Route pin untouched — the drift gates stay green.
This commit is contained in:
John O'Keefe committed 2026-11-02 14:55:16 -05:00
1 parent 8a40349fa1
commit e40be80e78
2 files changed
+82 -692
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/developer/websocket-api.md
+74 -685
View File
@@ -1,700 +1,89 @@
# WebSocket API Documentation
<!-- Ground truth: internal/sync/websocket.go (ConnectionManager, message
constants, broadcast fan-out + drop-on-full backpressure),
internal/handlers/websocket.go (dual auth, initial_state on connect,
30s ping / 90s read deadline, handleMessage logs-only),
internal/router/sync.go (route group), internal/services/worker.go
(scan event emitters). Re-verify against these when the wire
changes. Rewritten 2026-11-02 from source; the Feb-era version
documented client->server messages that the server never acted on. -->
**Base URL**: `ws://localhost:8765/ws/sync`
**Protocol**: WebSocket (RFC 6455)
**Authentication**: Required (JWT token or device token)
# WebSocket API (real-time events)
---
One channel, one direction: `/ws/sync` is a **server → client push feed**
for sync and scan events. It is not a command channel — clients do all
reading and writing through the REST API, and the server logs (but does
not act on) anything a client sends. The one client obligation is the
standard WebSocket pong, below.
## Table of Contents
## Connecting
1. [Authentication](#authentication)
2. [Connection](#connection)
3. [Message Format](#message-format)
4. [Message Types](#message-types)
5. [Client Implementation Guide](#client-implementation-guide)
6. [Examples](#examples)
7. [Troubleshooting](#troubleshooting)
`WS /ws/sync` — two accepted identities:
---
| Method | How |
| --- | --- |
| User (JWT) | `?token=<jwt>` query parameter |
| Device (KOReader, Kobo) | `Authorization: Bearer <device-token>` header |
## Authentication
No token → `401`. On connect the server immediately sends one
**`initial_state`** message: the connecting identity's media items for
sync, read under that identity's own data visibility.
### Option 1: JWT Token (Web/Mobile Clients)
## Message envelope
Pass JWT token as query parameter:
```
ws://localhost:8765/ws/sync?token=<your_jwt_token>
```
**How to get JWT token**:
```bash
curl -X POST http://localhost:8765/api/auth/login \
-H "Content-Type: application/json" \
-d '{"email":"user@example.com","password":"password"}'
```
Response:
```json
{
"token": "eyJhbGciOiJIUzI1NiIs...",
"user": {
"id": "user-uuid",
"email": "user@example.com"
}
}
```
### Option 2: Device Token (Kobo, KOReader, etc.)
Pass device token in Authorization header:
```
ws://localhost:8765/ws/sync?token=any-value
Authorization: Bearer <device_token>
```
**How to get device token**:
Device tokens are generated when devices are registered via the API.
---
## Connection
### Step 1: Connect to WebSocket
**JavaScript Example**:
```javascript
const token = "your-jwt-token";
const ws = new WebSocket(`ws://localhost:8765/ws/sync?token=${token}`);
ws.onopen = () => {
console.log("Connected to Bookhoard WebSocket");
};
ws.onerror = (error) => {
console.error("WebSocket error:", error);
};
ws.onclose = (event) => {
console.log("WebSocket closed:", event.code, event.reason);
};
```
**Go Example**:
```go
import (
"github.com/gorilla/websocket"
)
url := "ws://localhost:8765/ws/sync?token=" + token
ws, _, err := websocket.DefaultDialer.Dial(url, nil)
if err != nil {
log.Fatal("Dial error:", err)
}
defer ws.Close()
```
**Python Example**:
```python
import websocket
token = "your-jwt-token"
url = f"ws://localhost:8765/ws/sync?token={token}"
def on_message(ws, message):
print(f"Received: {message}")
def on_error(ws, error):
print(f"Error: {error}")
def on_close(ws, close_status_code, close_msg):
print("Closed")
def on_open(ws):
print("Connected")
ws = websocket.WebSocketApp(
url,
on_open=on_open,
on_message=on_message,
on_error=on_error,
on_close=on_close
)
ws.run_forever()
```
### Step 2: Handle Initial State
Immediately after connecting, you'll receive the initial state:
```javascript
ws.onmessage = (event) => {
const message = JSON.parse(event.data);
if (message.type === "initial_state") {
console.log("Initial progress:", message.data.progress);
console.log("Connected devices:", message.data.devices);
// Store initial state
initialState = message.data;
}
};
```
### Step 3: Handle Real-time Updates
Listen for subsequent updates:
```javascript
ws.onmessage = (event) => {
const message = JSON.parse(event.data);
switch (message.type) {
case "progress_update":
handleProgressUpdate(message);
break;
case "annotation_update":
handleAnnotationUpdate(message);
break;
case "conflict":
handleConflict(message);
break;
case "heartbeat":
// Server ping, ignore
break;
default:
console.log("Unknown message type:", message.type);
}
};
function handleProgressUpdate(message) {
const { book_id, percentage } = message.data;
const source = message.source_device;
console.log(`Progress update for ${book_id}: ${percentage * 100}%`);
console.log(`Source: ${source.name} (${source.type})`);
// Update UI
updateProgressBar(book_id, percentage);
}
function handleAnnotationUpdate(message) {
const { book_id, annotation_type, data } = message.data;
const source = message.source_device;
console.log(`Annotation update for ${book_id}: ${annotation_type}`);
console.log(`Source: ${source.name}`);
// Update UI
showAnnotation(book_id, data);
}
function handleConflict(message) {
const { book_id, notification_type, conflict_id } = message.data;
console.log(`Conflict detected for ${book_id}: ${notification_type}`);
// Show conflict resolution UI
showConflictDialog(book_id, conflict_id);
}
```
---
## Message Format
All messages follow this structure:
```json
{
"type": "message_type",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
// Message-specific data
},
"source_device": {
"id": "device-uuid",
"name": "Device Name",
"type": "kobo|koreader|web|mobile"
}
}
```
**Fields**:
- `type` (string, required): Message type identifier
- `timestamp` (string, required): ISO 8601 timestamp
- `data` (object, required): Message payload
- `source_device` (object, optional): Device that sent the update (not present for initial_state)
---
## Message Types
### 1. initial_state
**When**: Immediately after connection
**Purpose**: Send current progress for all user's books
**Data Structure**:
```json
{
"type": "initial_state",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"progress": {
"<book_uuid>": {
"percentage": 0.5,
"current_page": 150,
"total_pages": 300,
"last_read": "2026-02-01T11:30:00Z"
}
},
"devices": {
"kobo": 2,
"koreader": 1,
"web": 3
}
}
}
```
**Fields**:
- `progress` (object): Map of book UUID → progress data
- `percentage` (number): 0.0 to 1.0
- `current_page` (number): Current page number
- `total_pages` (number): Total pages in book
- `last_read` (string): ISO 8601 timestamp of last read
- `devices` (object): Connection statistics by device type
### 2. progress_update
**When**: Any device syncs reading progress
**Purpose**: Notify all connected clients of progress change
**Data Structure**:
Every server message is a JSON object:
```json
{
"type": "progress_update",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"book_id": "book-uuid-1",
"percentage": 0.75
},
"source_device": {
"id": "device-uuid-1",
"name": "My Kobo Clara",
"type": "kobo"
}
"timestamp": "2026-11-02T12:00:00Z",
"data": { "...": "..." },
"source_device": { "id": "…", "name": "Living-room Kobo", "type": "kobo" }
}
```
**Fields**:
- `book_id` (string): UUID of book with updated progress
- `percentage` (number): New progress value (0.0 to 1.0)
- `source_device` (object): Device that sent the update
**Use Cases**:
- Update progress bar in real-time
- Sync reading position across devices
- Update "currently reading" lists
### 3. annotation_update
**When**: Any device syncs annotations, bookmarks, or highlights
**Purpose**: Share annotations across devices
**Data Structure**:
```json
{
"type": "annotation_update",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"book_id": "book-uuid-1",
"annotation_type": "bookmark",
"data": {
"page": 150,
"text": "Great quote on page 150",
"chapter": 5,
"created_at": "2026-02-01T12:00:00Z"
}
},
"source_device": {
"id": "device-uuid-1",
"name": "My Kobo Clara",
"type": "kobo"
}
}
```
**Fields**:
- `book_id` (string): UUID of book
- `annotation_type` (string): Type of annotation (bookmark, highlight, note)
- `data` (object): Annotation-specific data
- `page` (number): Page number
- `text` (string): Annotated text or note
- `chapter` (number): Chapter number (optional)
- `created_at` (string): ISO 8601 timestamp
- `source_device` (object): Device that created the annotation
**Use Cases**:
- Show bookmarks on all devices
- Share highlights between devices
- Display reading notes
### 4. conflict
**When**: Sync conflict is detected
**Purpose**: Notify user of conflicting updates
**Data Structure**:
```json
{
"type": "conflict",
"timestamp": "2026-02-01T12:00:00Z",
"data": {
"book_id": "book-uuid-1",
"notification_type": "progress_conflict",
"conflict_id": "conflict-uuid-1"
}
}
```
**Fields**:
- `book_id` (string): UUID of book with conflict
- `notification_type` (string): Type of conflict (progress_conflict, annotation_conflict)
- `conflict_id` (string): UUID of conflict record
**Use Cases**:
- Prompt user to resolve conflict
- Show conflict resolution UI
- Log conflict for manual review
### 5. heartbeat
**When**: Server sends ping every 30 seconds
**Purpose**: Keep connection alive
**Data Structure**:
```json
{
"type": "heartbeat",
"timestamp": "2026-02-01T12:00:00Z"
}
```
**Handling**: Clients should respond automatically (handled by WebSocket protocol)
---
## Client Implementation Guide
### Reconnection Strategy
Implement exponential backoff for reconnection:
```javascript
let reconnectAttempts = 0;
const maxReconnectAttempts = 10;
const baseReconnectDelay = 1000; // 1 second
function connect() {
const token = getAuthToken();
const ws = new WebSocket(`ws://localhost:8765/ws/sync?token=${token}`);
ws.onopen = () => {
console.log("Connected");
reconnectAttempts = 0; // Reset on successful connection
};
ws.onclose = (event) => {
if (reconnectAttempts < maxReconnectAttempts) {
const delay = baseReconnectDelay * Math.pow(2, reconnectAttempts);
console.log(`Reconnecting in ${delay}ms...`);
setTimeout(() => {
reconnectAttempts++;
connect();
}, delay);
} else {
console.error("Max reconnection attempts reached");
}
};
ws.onmessage = handleIncomingMessage;
return ws;
}
```
### Error Handling
Handle common errors:
```javascript
ws.onerror = (error) => {
console.error("WebSocket error:", error);
// Check specific error types
if (error.code === 1006) {
console.error("Abnormal closure - server may be down");
} else if (error.code === 1008) {
console.error("Policy violation - check authentication");
}
};
```
### Message Queueing
Queue messages while disconnected:
```javascript
let messageQueue = [];
let isConnected = false;
function queueMessage(message) {
if (isConnected) {
ws.send(JSON.stringify(message));
} else {
messageQueue.push(message);
}
}
ws.onopen = () => {
isConnected = true;
// Send queued messages
while (messageQueue.length > 0) {
const message = messageQueue.shift();
ws.send(JSON.stringify(message));
}
};
ws.onclose = () => {
isConnected = false;
};
```
---
## Examples
### Complete JavaScript Client
```javascript
class BookhoardWebSocket {
constructor(token) {
this.token = token;
this.ws = null;
this.reconnectAttempts = 0;
this.maxReconnectAttempts = 10;
this.messageHandlers = {};
}
connect() {
const url = `ws://localhost:8765/ws/sync?token=${this.token}`;
this.ws = new WebSocket(url);
this.ws.onopen = () => {
console.log("Connected to Bookhoard");
this.reconnectAttempts = 0;
};
this.ws.onmessage = (event) => {
const message = JSON.parse(event.data);
this.handleMessage(message);
};
this.ws.onerror = (error) => {
console.error("WebSocket error:", error);
};
this.ws.onclose = () => {
this.reconnect();
};
}
handleMessage(message) {
const handler = this.messageHandlers[message.type];
if (handler) {
handler(message);
}
}
on(messageType, callback) {
this.messageHandlers[messageType] = callback;
}
reconnect() {
if (this.reconnectAttempts < this.maxReconnectAttempts) {
const delay = 1000 * Math.pow(2, this.reconnectAttempts);
console.log(`Reconnecting in ${delay}ms...`);
setTimeout(() => {
this.reconnectAttempts++;
this.connect();
}, delay);
}
}
disconnect() {
if (this.ws) {
this.ws.close();
}
}
}
// Usage
const token = "your-jwt-token";
const client = new BookhoardWebSocket(token);
client.on("initial_state", (message) => {
console.log("Initial state received:", message.data);
updateUIWithInitialState(message.data);
});
client.on("progress_update", (message) => {
console.log("Progress updated:", message.data);
updateProgressBar(message.data.book_id, message.data.percentage);
});
client.on("annotation_update", (message) => {
console.log("Annotation updated:", message.data);
showNotification("New annotation from " + message.source_device.name);
});
client.on("conflict", (message) => {
console.log("Conflict detected:", message.data);
showConflictDialog(message.data);
});
client.connect();
```
---
## Troubleshooting
### Connection Fails
**Problem**: Can't connect to WebSocket
**Solutions**:
1. Check JWT token is valid
2. Verify server is running: `curl http://localhost:8765/health`
3. Check firewall settings
4. Verify WebSocket URL format
### Connection Drops Frequently
**Problem**: WebSocket disconnects unexpectedly
**Solutions**:
1. Check network stability
2. Verify server keepalive settings
3. Implement reconnection logic (see above)
4. Check server logs for errors
### No Messages Received
**Problem**: Connected but no messages
**Solutions**:
1. Check onmessage handler is registered
2. Verify initial_state message received
3. Test with manual progress update via API
4. Check browser console for errors
### Authentication Errors
**Problem**: 401 Unauthorized
**Solutions**:
1. Verify JWT token is not expired
2. Check token has correct claims (user_id)
3. For device tokens, verify Authorization header format
4. Regenerate token via `/api/auth/login`
### High Memory Usage
**Problem**: Client memory increases over time
**Solutions**:
1. Clean up old messages
2. Don't store entire message history
3. Use weak references for large data
4. Implement message pagination
---
## Performance Tips
1. **Debounce UI Updates**: Don't update DOM on every message
```javascript
const debouncedUpdate = debounce(updateUI, 100);
client.on("progress_update", (message) => {
debouncedUpdate(message);
});
```
2. **Use Web Workers** for heavy processing
3. **Limit Message History**: Keep only last N messages
4. **Virtual Scrolling**: For large lists of updates
5. **Connection Pooling**: Reuse connections across tabs
---
## Security Best Practices
1. **Use WSS in Production**: Always use secure WebSocket (wss://) in production
2. **Validate Tokens**: Always verify JWT tokens on server
3. **Rate Limiting**: Implement message rate limiting per connection
4. **Input Sanitization**: Sanitize all message data before displaying
5. **Origin Checks**: Validate WebSocket origin on server
---
## Support
For issues or questions:
- GitHub Issues: [Bookhoard Repository]
- Documentation: [Bookhoard Docs]
- API Reference: [Bookhoard API Docs]
---
**Last Updated**: 2026-02-01
**Version**: 1.0.0
**Protocol**: WebSocket (RFC 6455)
`source_device` is present on events that originate from a device
(progress and annotation updates), so clients can ignore their own echo.
## Message catalog
All ten types defined by `internal/sync/websocket.go`:
| Type | Emitted | `data` carries |
| --- | --- | --- |
| `initial_state` | Once, on connect | The connecting user's media items for sync |
| `progress_update` | On any device's progress save | `book_id`, `percentage` |
| `annotation_update` | On annotation changes (highlights, notes, bookmarks) | `book_id`, `annotation_type`, payload |
| `conflict` | When a sync conflict is detected | `book_id`, `notification_type`, `conflict_id` |
| `scan_progress` | During scan jobs | Job progress for the admin scan panel |
| `scan_complete` | When a scan job finishes | Job summary |
| `scan_error` | When a scan job fails | Job error |
| `scan_cancelled` | When a scan job is cancelled | Job identity |
| `collection_updated` | When books are added to / removed from one of *your* collections | `collection_id`, `action` (`books_added` / `book_removed` / `books_removed`), affected book ids |
| `heartbeat` | *Defined, not currently emitted* | — |
| `sync_complete` | *Defined, not currently emitted* | — |
(`collection_updated` is the one per-user-targeted event — it's emitted
with the connecting user's id directly. It's also currently the only
message type defined as a raw string in the collections handler rather
than in the sync package's constants block.)
## Keepalive
The server sends a WebSocket **ping frame every 30 seconds** and expects
the standard pong within a **90-second read deadline**. Browsers and
every mainstream WebSocket library answer pings automatically — there is
nothing to implement. A connection that stops answering is removed.
## Fan-out and backpressure — read this before building a client
- Sync and scan events fan out **instance-wide**: every connected client
receives progress, annotation, conflict, and scan events. The
exception is `collection_updated`, which is targeted to one user's
connections. (A per-user targeting helper exists and is used for
collections only — the sync/scan emitters do not use it.)
- Every connection has a bounded send queue. **If a client falls behind,
messages for that connection are skipped, not queued** (and the global
broadcast ring drops messages when saturated). Clients must therefore
treat the socket as a *hint* — on reconnect (or any gap), refetch real
state over REST. The fresh `initial_state` on reconnect is designed to
be that gap-healer.