docs: add comprehensive infrastructure enhancement plan
Add detailed implementation plan for leveraging underutilized job queue and WebSocket infrastructure. Key focus areas: **Core Design Principle:** - Job queue as concurrency control mechanism (not mutex blocking) - Non-blocking API responses for long-running operations - Expand job queue from 10% to 90% utilization **Phase 1 (2-3 hours): Core Concurrency Fixes** - Add watching atomic flag to MediaScanner (prevents duplicate WatchChanges) - Use job queue for folder configuration instead of blocking calls - Fix test isolation with system_settings snapshot/restore - Add job queue serialization tests **Phase 2 (6-8 hours): Job Queue Expansion** - Add 7 new job types: import, convert, thumbnails, reindex, backup, analytics, sync - Create JobsHandler with REST API endpoints - All operations support progress tracking via callbacks **Phase 3 (2-3 hours): WebSocket Scan Progress** - Real-time scan progress broadcasts to user's devices - Pass ConnectionManager to Worker for WebSocket integration - Add user ID to Job for targeted messaging **Phase 4 (2 hours): Caching & Monitoring** - Redis caching for frequently accessed data - Prometheus metrics for job queue performance **Phase 5 (4-6 hours): Job Queue Enhancements** - Priority queues for different job types - Job cancellation and retry logic - Rate limiting and backpressure handling Total estimated time: 16-22 hours for full implementation
This commit is contained in:
