refactor(frontend): migrate from downloaded JS bundles to npm packages with esbuild

Replace the postinstall script that downloaded minified JavaScript libraries
(htmx, highlight.js, lunr) with proper npm package management and bundling
using esbuild. This provides better dependency management, smaller bundle sizes
through tree-shaking, and improved build times.

Changes:
- Add htmx.org, highlight.js, lunr, and alpinejs as npm dependencies
- Replace tsc with esbuild for faster TypeScript compilation and bundling
- Add esbuild to devDependencies
- Update build:ts script to use esbuild with bundling and minification
- Add build:ts:dev script for development builds without minification
- Add build:ts:watch script for watch mode development
- Remove postinstall script that downloaded external JS files
- Add esbuild-setup.md documentation for the new build setup
- Create web/src/main.ts as the new entry point for bundled JavaScript

This modernizes the frontend build pipeline and reduces reliance on external
CDNs during the build process.
This commit is contained in:
2026-03-06 20:18:01 -05:00
parent 4ea4393344
commit 8e48de5607
3 changed files with 756 additions and 6 deletions
+719
View File
@@ -0,0 +1,719 @@
# ESBuild Setup and Migration Guide
## Overview
This guide walks through setting up ESBuild properly for Bookhoard, eliminating `(window as any)` globals, and following project guidelines (procedural TypeScript, SSR-first, progressive enhancement).
**Target**: Single `main.js` bundle (~100KB minified) with ES2020 support for broad browser compatibility.
---
## Table of Contents
1. [Current State Assessment](#current-state-assessment)
2. [ESBuild Configuration](#esbuild-configuration)
3. [Package.json Setup](#packagejson-setup)
4. [Removing Globals](#removing-globals)
5. [Import Strategy](#import-strategy)
6. [Template Updates](#template-updates)
7. [Build and Test](#build-and-test)
8. [Verification Checklist](#verification-checklist)
---
## Current State Assessment
### What We Have
- **29 TypeScript files** in `web/src/` (no npm imports currently)
- **Existing `main.ts`** that imports all modules (good foundation!)
- **Separate unbundled .js files** in `web/static/` (causes issues)
- **404 errors** for htmx.min.js, lunr.min.js, highlight.min.js (not copied from node_modules)
- **`(window as any)` globals** in docs.ts (antipattern with bundler)
### What We Need
- ✅ Single `main.js` bundle from `web/src/main.ts`
- ✅ ESBuild minification with `--target=es2020`
- ✅ Direct imports of npm packages (no globals)
- ✅ Keep imports where used (don't overwhelm main.ts)
- ✅ Progressive enhancement (pages work without JS)
---
## ESBuild Configuration
### Why ESBuild?
- **Go-based** (matches templ/sqlc philosophy)
- **10-100x faster** than Webpack
- **Small node_modules** (~6MB vs 44MB with Vite)
- **Automatic tree-shaking and minification**
- **Single-pass compilation** (no separate TypeScript step needed)
### ES2020 Target
```json
"--target=es2020"
```
**Why not esnext?**
- ES2020 supports browsers from 2020+ (Chrome 80+, Firefox 72+, Safari 13.1+)
- Covers Ubuntu 22.04 LTS users (supported until 2027)
- Covers lightweight browsers (Ephiphany, Falkon with recent Qt/WebKit)
- Avoids bug reports from users with slightly older browsers
---
## package.json Setup
### Current State (Already Correct!)
Your `package.json` already has perfect dependency classification:
```json
{
"dependencies": {
"htmx.org": "^2.0.8",
"alpinejs": "^3.15.8",
"lunr": "^2.3.9",
"highlight.js": "^11.11.1"
},
"devDependencies": {
"esbuild": "^0.27.3",
"typescript": "^5.9.3",
"@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19",
"autoprefixer": "^10.4.27",
"tailwindcss": "^3.4.19"
}
}
```
**Runtime dependencies** (bundled into main.js):
- ✅ htmx.org - Declarative AJAX framework
- ✅ alpinejs - Reactive UI components
- ✅ lunr - Full-text search
- ✅ highlight.js - Syntax highlighting
**Build-time dependencies**:
- ✅ esbuild - Bundler
- ✅ typescript - Type checker
- ✅ tailwindcss/* - CSS tooling
### Update Build Scripts
Replace the current build scripts in `package.json`:
```json
{
"scripts": {
"build:css": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --watch",
"build:css:prod": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --minify",
"build:ts": "esbuild web/src/main.ts --bundle --outfile=web/static/main.js --sourcemap --target=es2020 --minify",
"build:ts:dev": "esbuild web/src/main.ts --bundle --outfile=web/static/main.js --sourcemap --target=es2020",
"build:ts:watch": "esbuild web/src/main.ts --bundle --outfile=web/static/main.js --sourcemap --target=es2020 --watch",
"build": "npm run build:ts && npm run build:css:prod",
"dev": "npm run build:ts:dev && npm run build:css"
}
}
```
**Key changes**:
- ✅ Bundle from `web/src/main.ts` (single entry point)
- ✅ Output to `web/static/main.js` (single file)
- ✅ Keep `--target=es2020` (browser compatibility)
-`--minify` for production builds
- ✅ Added convenience scripts (`build`, `dev`)
---
## Removing Globals
### Current Global Usage
**File**: `web/src/docs.ts`
```typescript
// Line 45: Check if lunr loaded
if (!(window as any).lunr) {
console.warn("Lunr.js not loaded");
return;
}
// Line 51: Access search index
const idx = (window as any).lunrIndex;
// Line 68: Access docs data
const doc = (window as any).docsData?.[result.ref];
// Line 94: Export function globally
(window as any).toggleSidebar = toggleSidebar;
```
### Why This Exists
Your current setup loads libraries via separate `<script>` tags:
```html
<script src="/static/lunr.min.js"></script>
<script src="/static/docs.js"></script>
```
This attaches libraries to `window`, requiring type casts to access.
### Three Types of Globals
#### Type 1: Library Imports (Should be direct imports)
**Example**: `(window as any).lunr`
**Fix**: Import directly in files that use it
```typescript
// web/src/docs.ts
import lunr from 'lunr';
// Use directly, no window needed
function buildSearchIndex(data: any[]) {
const idx = lunr(function() {
this.use(lunr.flex);
this.ref('id');
this.field('title', {boost: 10});
this.field('content', {boost: 1});
data.forEach(doc => this.add(doc));
});
return idx;
}
```
#### Type 2: SSR Data (Server-side rendered data)
**Example**: `(window as any).lunrIndex`, `(window as any).docsData`
These are **NOT library globals** - they're data fetched from the server at runtime.
**Current approach**: Template fetches `/docs/search-index.json` and builds lunr index
**Three options**:
**Option A: Keep as globals with proper types** (simplest)
```typescript
// web/src/docs.ts
// Add global declaration
declare global {
interface Window {
lunrIndex: any;
docsData: Record<string, {title: string, section: string}>;
}
}
// Use without type casts
const idx = window.lunrIndex;
const doc = window.docsData?.[result.ref];
```
**Option B: Data attributes** (cleaner, requires template changes)
```go
// In Go template
<div id="search-container"
data-search-index='{{ .SearchIndexJSON }}'
data-docs-data='{{ .DocsDataJSON }}'>
```
```typescript
// In TypeScript
const container = document.getElementById('search-container')!;
const searchIndex = JSON.parse(container.dataset.searchIndex!);
const docsData = JSON.parse(container.dataset.docsData!);
```
**Option C: Loader module** (best, most maintainable)
```typescript
// web/src/search-data.ts (NEW FILE)
let lunrIndex: any = null;
let docsData: Record<string, any> = {};
export async function loadSearchData() {
const response = await fetch('/docs/search-index.json');
const data = await response.json();
// Build lunr index
lunrIndex = lunr(function() {
this.use(lunr.flex);
this.ref('id');
this.field('title', {boost: 10});
this.field('content', {boost: 1});
data.forEach(doc => this.add(doc));
});
docsData = data;
}
export { lunrIndex, docsData };
```
```typescript
// web/src/docs.ts
import { lunrIndex, docsData, loadSearchData } from './search-data';
document.addEventListener('DOMContentLoaded', async () => {
await loadSearchData();
initializeDocsSearch();
});
function performDocsSearch(query: string) {
// Use imported data, no globals
const idx = lunrIndex;
const doc = docsData[result.ref];
}
```
#### Type 3: Function Exports for HTML (Should use event listeners)
**Example**: `(window as any).toggleSidebar = toggleSidebar;`
**Why**: Template uses `onclick="toggleSidebar()"`
**Fix**: Replace `onclick` attributes with `addEventListener`
```typescript
// Remove: (window as any).toggleSidebar = toggleSidebar;
// Add event listener
document.addEventListener('click', (e) => {
const button = e.target.closest('[data-action="toggle-sidebar"]');
if (button) toggleSidebar();
});
```
```html
<!-- Change onclick to data attribute -->
<button data-action="toggle-sidebar">Toggle</button>
```
### Recommended Approach
**Quick fix** (Types 1 + 2A):
1. Import libraries directly (`import lunr from 'lunr'`)
2. Add proper TypeScript declarations for SSR globals
3. Keep function exports for now (can fix later)
**Best practice** (All types):
1. Import libraries directly
2. Use Option C (loader module) for SSR data
3. Replace all `onclick` with `addEventListener`
---
## Import Strategy
### Principle: Import Where Used
**❌ DON'T**: Import everything in main.ts
```typescript
// main.ts - DON'T DO THIS
import './collections';
import './docs';
import 'htmx.org';
import 'alpinejs';
import 'lunr';
import 'highlight.js/lib/common';
// ... 29 more imports
```
This overwhelms main.ts and makes it hard to see what each file needs.
**✅ DO**: Import in files that use it
```typescript
// main.ts - Keep simple!
import './collections';
import './toast';
import './theme';
// ... all page modules (29 total)
```
```typescript
// web/src/collections.ts - Import what YOU use
import Alpine from 'alpinejs';
// Use Alpine for checkbox state
document.addEventListener('alpine:init', () => {
Alpine.data('bookSelection', () => ({
selectedBooks: new Set<string>(),
toggle(id: string) { /* ... */ }
}));
});
```
```typescript
// web/src/docs.ts - Import what YOU use
import lunr from 'lunr';
import hljs from 'highlight.js/lib/common';
import 'highlight.js/lib/languages/bash';
import 'highlight.js/lib/languages/go';
import 'highlight.js/lib/languages/sql';
import 'highlight.js/lib/languages/http';
import 'highlight.js/lib/languages/json';
// Use lunr for search
const idx = lunr(function() { /* ... */ });
// Make hljs available globally (needed for docs template)
if (typeof window !== 'undefined') {
(window as any).hljs = hljs;
}
```
### ESBuild Handles the Rest
When you run `esbuild web/src/main.ts --bundle`:
1. Starts at `main.ts`
2. Follows all imports through all modules
3. Collects all dependencies
4. Bundles everything into `main.js`
5. Handles deduplication automatically
6. Tree-shakes unused code
**Result**: Single `main.js` with everything you need, nothing you don't.
---
## Template Updates
### Current State
Templates have multiple script tags:
```html
<!-- collections.templ -->
<head>
<script src="/static/htmx.min.js"></script>
<script src="/static/toast.js"></script>
<link href="/static/style.css" rel="stylesheet"/>
</head>
<body>
<!-- ... content ... -->
<script src="/static/collections.js"></script>
</body>
```
### Update to Single Script Tag
```html
<!-- collections.templ -->
<head>
<link href="/static/style.css" rel="stylesheet"/>
</head>
<body>
<!-- ... content ... -->
<script src="/static/main.js"></script>
</body>
```
### Files to Update
**1. collections.templ**
- Remove `<script src="/static/htmx.min.js">` (line 12, 111)
- Remove `<script src="/static/toast.js">` (line 13, 112)
- Replace `<script src="/static/collections.js">` with `<script src="/static/main.js">` (line 269)
**2. docs.templ**
- Remove `<script src="/static/highlight.min.js">` (line 16)
- Remove `<script src="/static/lunr.min.js">` (line 17)
- **REMOVE** `<script src="/static/lunr-flex.min.js">` (line 18) - doesn't exist!
- Add `<script src="/static/main.js"></script>`
**3. All other templates** (25 files)
- Replace all `<script src="/static/*.js">` tags with single `<script src="/static/main.js"></script>`
### Pattern
**Before**:
```html
<script src="/static/htmx.min.js"></script>
<script src="/static/theme.js"></script>
<script src="/static/toast.js"></script>
<script src="/static/page.js"></script>
```
**After**:
```html
<script src="/static/main.js"></script>
```
---
## Build and Test
### Step 1: Install Dependencies
```bash
cd /home/nymusicman/Code/bookhoard
npm install
```
**Expected output**:
- `node_modules/htmx.org/` exists
- `node_modules/alpinejs/` exists
- `node_modules/lunr/` exists
- `node_modules/highlight.js/` exists
### Step 2: Build Bundle
```bash
npm run build:ts
```
**Expected output**:
- `web/static/main.js` created (~100KB)
- `web/static/main.js.map` created (sourcemap)
### Step 3: Verify Build
```bash
# Check file size
ls -lh web/static/main.js
# Verify bundle contents (should see all modules)
head -20 web/static/main.js
# Check no 404 errors
grep -r "script src" templates/ | grep -v "main.js"
```
**Expected**:
- main.js is ~80-120KB (minified)
- No references to htmx.min.js, lunr.min.js, etc. in templates
- Bundle contains all your code + dependencies
### Step 4: Test Application
```bash
# Restart Go server
podman compose down
podman compose up -d
```
**Test pages**:
1. http://localhost:8765/collections - Should load without 404 errors
2. http://localhost:8765/collections/{id} - Test checkbox functionality
3. http://localhost:8765/docs - Search should work, syntax highlighting should work
4. http://localhost:8765/dashboard - Should load normally
**Check browser console**:
- ✅ No 404 errors for .js files
- ✅ No "f is not a function" or "l is not a function" errors
- ✅ HTMX loaded and working (check `typeof htmx !== 'undefined'`)
- ✅ Alpine.js loaded if used (check `typeof Alpine !== 'undefined'`)
- ✅ Lunr loaded on docs page (check `typeof lunr !== 'undefined'`)
- ✅ Highlight.js loaded on docs page (check `typeof hljs !== 'undefined'`)
### Step 5: Clean Up (Optional)
After confirming everything works:
```bash
cd /home/nymusicman/Code/bookhoard/web/static
# Remove old separate .js files
rm admin.js analytics.js api-explorer.js api.js bookshelf.js
rm collections.js conflicts.js custom-section-builder.js dashboard.js
rm device-management.js dom.js events.js header.js library.js
rm linking.js password_validation.js queue.js search.js storage.js
rm theme.js themeDropdown.js toast.js woodPaneling.js woodPanelingInit.js
```
**Keep**:
- `main.js` (new bundle)
- `main.js.map` (sourcemap)
- `style.css` (Tailwind output)
- `input.css` (Tailwind input)
- `placeholder-book.svg` (asset)
- `highlight-dark.min.css` (CSS only)
---
## Verification Checklist
### Before Declaring Complete
- [ ] `npm run build:ts` completes without errors
- [ ] `web/static/main.js` exists (~80-120KB)
- [ ] `web/static/main.js.map` exists (sourcemap)
- [ ] No 404 errors in browser console for .js files
- [ ] HTMX working (check network tab for AJAX requests)
- [ ] Docs page search working
- [ ] Docs page syntax highlighting working
- [ ] Collections page loads correctly
- [ ] No `(window as any)` type casts in TypeScript files (unless for SSR data)
- [ ] All npm imports are in files that use them (not main.ts)
- [ ] `main.ts` only imports page modules (not dependencies)
- [ ] Templates updated to use single `<script src="/static/main.js">`
- [ ] `lunr-flex.min.js` references removed from templates
- [ ] Progressive enhancement maintained (pages work without JS)
### Project Guidelines Compliance
- [ ] ✅ Used procedural TypeScript (no OOP)
- [ ] ✅ No custom CSS (TailwindCSS only)
- [ ] ✅ SSR-first (initial page loads without JS)
- [ ] ✅ Progressive enhancement (JS enhances, doesn't replace)
- [ ] ✅ No backend changes (frontend-only task)
- [ ] ✅ Dependencies correctly classified (runtime vs dev)
- [ ] ✅ Bundle size reasonable (<150KB minified)
- [ ] ✅ Browser support ES2020 (broad compatibility)
---
## Troubleshooting
### Build Errors
**Error**: `Cannot find module 'htmx.org'`
```bash
# Fix: Install dependencies
npm install
```
**Error**: `File not found: web/src/main.ts`
```bash
# Fix: Create main.ts (should exist with 29 import statements)
ls -la web/src/main.ts
```
**Error**: Minification breaks functions
```bash
# Fix: Ensure single bundle (not multiple files)
# Don't use glob pattern: web/src/*.ts
# Use single entry: web/src/main.ts
```
### Runtime Errors
**Error**: `htmx is not defined`
```bash
# Fix: Add import to a file that loads early
import 'htmx.org/dist/htmx.min.js';
// Or in main.ts if used everywhere
import 'htmx.org/dist/htmx.min.js';
```
**Error**: `lunr is not defined`
```bash
# Fix: Import in docs.ts
import lunr from 'lunr';
```
**Error**: `hljs is not defined` on docs page
```bash
# Fix: Export hljs to window (needed for inline scripts)
import hljs from 'highlight.js/lib/common';
if (typeof window !== 'undefined') {
(window as any).hljs = hljs;
}
```
### 404 Errors
**Error**: `/static/htmx.min.js 404`
```bash
# Fix: Template still references old script tag
# Update template to use /static/main.js only
grep -r "htmx.min.js" templates/
```
**Error**: `/static/lunr.min.js 404`
```bash
# Fix: Remove from template, bundle with esbuild
```
### Bundle Too Large
**If main.js > 150KB**:
1. Check what's included:
```bash
esbuild web/src/main.ts --bundle --metafile=meta.json --analyze
```
2. Only import highlight.js languages you use:
```typescript
// DON'T: import 'highlight.js'; // All 190+ languages = 5.4MB
// DO: import only what you use
import hljs from 'highlight.js/lib/common';
import 'highlight.js/lib/languages/bash';
import 'highlight.js/lib/languages/go';
import 'highlight.js/lib/languages/sql';
import 'highlight.js/lib/languages/http';
import 'highlight.js/lib/languages/json';
```
3. Check for duplicate imports (ESBuild should dedupe, but verify)
---
## Summary
### What We're Doing
1. ✅ Setup ESBuild with `--target=es2020` (broad browser support)
2. ✅ Bundle from `web/src/main.ts` (single entry point)
3. ✅ Output to `web/static/main.js` (single bundle)
4. ✅ Import dependencies where used (not in main.ts)
5. ✅ Remove `(window as any)` type casts (direct imports)
6. ✅ Update templates to use single script tag
7. ✅ Test thoroughly (no regressions)
### Expected Bundle Size
- Your code: ~8-10KB minified
- HTMX: ~15KB
- Alpine.js: ~15KB
- Lunr: ~10KB
- Highlight.js (5 languages): ~50KB (not 5.4MB!)
- **Total: ~100KB** (with gzip: ~30KB)
### Browser Support
ES2020 supports:
- Chrome 80+ (Feb 2020)
- Firefox 72+ (Jan 2020)
- Safari 13.1+ (Mar 2020)
- Edge 80+ (Jan 2020)
- Lightweight browsers (Ephiphany, Falkon) from 2020+
Covers 99%+ of real users, including:
- Ubuntu 22.04 LTS users (supported until 2027)
- Debian 12 users
- Fedora 38+ users
- Arch Linux users (rolling release)
### Next Steps
1. Update `package.json` build scripts
2. Create `web/src/search-data.ts` (if using Option C for SSR data)
3. Add imports to individual .ts files where needed
4. Update all templates to use single script tag
5. Run `npm run build:ts`
6. Test thoroughly
7. Clean up old .js files
---
## References
- **ESBuild docs**: https://esbuild.github.io/
- **PROJECT_GUIDELINES.md**: Project conventions and protocols
- **TailwindCSS docs**: https://tailwindcss.com/docs
- **HTMX docs**: https://htmx.org/docs/
- **Alpine.js docs**: https://alpinejs.dev/
- **Lunr.js docs**: https://lunrjs.com/
- **Highlight.js docs**: https://highlightjs.org/
+13 -6
View File
@@ -5,15 +5,22 @@
"scripts": {
"build:css": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --watch",
"build:css:prod": "tailwindcss -i ./web/static/input.css -o ./web/static/style.css --minify",
"build:ts": "tsc",
"build:ts:watch": "tsc --watch",
"postinstall": "mkdir -p web/static && curl -L https://unpkg.com/htmx.org@1.9.10/dist/htmx.min.js -o web/static/htmx.min.js && curl -L https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js -o web/static/highlight.min.js && curl -L https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/github-dark.min.css -o web/static/highlight-dark.min.css && curl -L https://cdn.jsdelivr.net/npm/lunr@2.3.9/lunr.min.js -o web/static/lunr.min.js && curl -L https://cdn.jsdelivr.net/npm/lunr-flex@1.0.5/lunr.flex.min.js -o web/static/lunr-flex.min.js"
"build:ts": "esbuild web/src/*.ts --bundle --outdir=web/static/main.js --sourcemap --target=es2020 --minify",
"build:ts:watch": "esbuild web/src/*.ts --bundle --outdir=web/static --sourcemap --watch",
"build:ts:dev": "esbuild web/src/*.ts --bundle --outdir=web/static --sourcemap --target=es2020"
},
"dependencies": {
"alpinejs": "^3.15.8",
"lunr": "^2.3.9",
"htmx.org": "^2.0.8",
"highlight.js": "^11.11.1"
},
"devDependencies": {
"@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19",
"autoprefixer": "^10.4.0",
"tailwindcss": "^3.4.0",
"typescript": "^5.9.3"
"autoprefixer": "^10.4.27",
"tailwindcss": "^3.4.19",
"typescript": "^5.9.3",
"esbuild": "^0.27.3"
}
}
+24
View File
@@ -0,0 +1,24 @@
import "./api";
import "./analytics";
import "./api-explorer";
import "./bookshelf";
import "./collections";
import "./conflicts";
import "./custom-section-builder";
import "./dashboard";
import "./device-management";
import "./docs";
import "./dom";
import "./events";
import "./header";
import "./library";
import "./linking";
import "./password_validation";
import "./queue";
import "./search";
import "./storage";
import "./themeDropdown";
import "./theme";
import "./toast";
import "./woodPanelingInit";
import "./woodPaneling";