docs: Expand verification scripts README with comprehensive examples
- Add complete enhanced output examples for all error/warning types - Document verification script architecture and design principles - Include content detection methods and data collection strategies - Add comprehensive usage scenarios for development, CI/CD, AI workflows - Provide advanced troubleshooting with debugging techniques - Include performance optimization and edge case handling - Demonstrate integration patterns for hooks and pipelines This creates definitive documentation for verification system that covers all enhanced outputs, usage patterns, and integration methods, making scripts fully understandable and actionable for both human developers and AI assistance.
This commit is contained in:
+432
-10
@@ -132,17 +132,112 @@ The specific patterns found are:
|
||||
Do you want me to fix this? [y/n]"
|
||||
```
|
||||
|
||||
#### Complete Enhanced Output Examples:
|
||||
|
||||
##### 1. Build Failure with Detailed Errors:
|
||||
```bash
|
||||
❌ ERROR: Build failed (violation: must compile after edits)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Build error details:
|
||||
# cmd/server/main.go:45:2: syntax error: unexpected newline, expecting }
|
||||
#
|
||||
# Try running: go build ./cmd/server
|
||||
```
|
||||
|
||||
##### 2. Missing Configuration Files:
|
||||
```bash
|
||||
❌ ERROR: .env not in .gitignore (violation: secrets might be committed)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Expected in .gitignore: .env
|
||||
Current .gitignore contents:
|
||||
# Binaries
|
||||
bookhoard
|
||||
server
|
||||
*.exe
|
||||
# Dependencies
|
||||
node_modules/
|
||||
# Build artifacts
|
||||
dist/
|
||||
build/
|
||||
*.log
|
||||
```
|
||||
|
||||
##### 3. API Content Detection with Line Numbers:
|
||||
```bash
|
||||
⚠ WARNING: Found 2 API patterns in README.md (verify placement per guidelines)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Found patterns:
|
||||
81:- **Rate Limiting**: 10 requests/minute on auth endpoints
|
||||
98:- **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
|
||||
```
|
||||
|
||||
##### 4. File Organization Violations:
|
||||
```bash
|
||||
❌ ERROR: Found 3 .js files (violation: JavaScript prohibited, use TypeScript)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Found files:
|
||||
./web/static/header.js
|
||||
./web/static/search.js
|
||||
./scripts/legacy-migration.js
|
||||
```
|
||||
|
||||
##### 5. Documentation Structure Validation:
|
||||
```bash
|
||||
✓ PASS: docs/api directory exists
|
||||
✓ PASS: docs/devices directory missing
|
||||
✓ PASS: docs/contributing directory exists
|
||||
|
||||
⚠ WARNING: docs/devices directory missing
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Expected directory structure:
|
||||
- docs/ (main documentation)
|
||||
- docs/api/ (API reference)
|
||||
- docs/devices/ (device setup guides)
|
||||
- docs/contributing/ (development docs)
|
||||
```
|
||||
|
||||
##### 6. Bruno API Tests Coverage Analysis:
|
||||
```bash
|
||||
✓ PASS: Found 47 Bruno test files
|
||||
⚠ WARNING: Bruno test files (47) fewer than API docs (52)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Coverage gap: API docs (52) vs Bruno tests (47)
|
||||
```
|
||||
|
||||
##### 7. Git Commit Quality Analysis:
|
||||
```bash
|
||||
⚠ WARNING: Found 3 recent commits changing >15 files each (should use multiple commits)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Large commits:
|
||||
a1b2c3d: feat: add user authentication (18 files)
|
||||
f4e5d6a: refactor: update database schema (22 files)
|
||||
b7c8f9e: fix: resolve merge conflicts (16 files)
|
||||
```
|
||||
|
||||
##### 8. CSS Template Violations:
|
||||
```bash
|
||||
❌ ERROR: Found 12 templates with <style> tags (violation: custom CSS prohibited)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Found files:
|
||||
templates/admin.templ
|
||||
templates/dashboard.templ
|
||||
templates/analytics.templ
|
||||
templates/... (truncated, first 10 shown)
|
||||
```
|
||||
|
||||
#### Detailed Information Provided:
|
||||
- **Binary files**: Lists exact file paths found
|
||||
- **CSS violations**: Shows specific templates with <style> tags
|
||||
- **Binary files**: Lists exact file paths found with full paths
|
||||
- **CSS violations**: Shows specific templates with <style> tags and line counts
|
||||
- **JavaScript files**: Lists all .js files outside allowed directories
|
||||
- **Build failures**: Displays compilation error logs
|
||||
- **Missing files**: Shows expected file locations and purposes
|
||||
- **Git ignore issues**: Displays current .gitignore contents
|
||||
- **API content**: Shows exact lines and patterns found
|
||||
- **Dockerfile proliferation**: Lists all Dockerfile variants found
|
||||
- **Large commits**: Shows specific commits with file counts
|
||||
- **Bruno coverage**: Compares API docs vs test file counts
|
||||
- **Build failures**: Displays complete compilation error logs and suggests fixes
|
||||
- **Missing files**: Shows expected file locations and file purposes
|
||||
- **Git ignore issues**: Displays current .gitignore contents line by line
|
||||
- **API content**: Shows exact line numbers and full patterns found
|
||||
- **Dockerfile proliferation**: Lists all Dockerfile variants with paths
|
||||
- **Large commits**: Shows specific commit hashes, messages, and file counts
|
||||
- **Bruno coverage**: Compares API docs vs test file counts with gap analysis
|
||||
- **Template violations**: Shows all violating files with first 10 displayed
|
||||
- **Directory structure**: Lists expected vs actual directory structure
|
||||
|
||||
## Recent Enhancements
|
||||
|
||||
@@ -193,6 +288,144 @@ Verification catches: API docs exist but no test file
|
||||
|
||||
This ensures **API documentation and test coverage stay synchronized** - a critical quality control measure per PROJECT_GUIDELINES.md.
|
||||
|
||||
## Verification Script Architecture
|
||||
|
||||
### Data Collection Strategy
|
||||
Both scripts use a **detailed-first approach** for error/warning reporting:
|
||||
|
||||
```bash
|
||||
# Old approach (insufficient):
|
||||
ERROR_COUNT=$(find . -name "*.js" | wc -l)
|
||||
if [ "$ERROR_COUNT" -gt 0 ]; then
|
||||
error_msg "Found JavaScript files"
|
||||
|
||||
# New approach (detailed):
|
||||
JS_FILES_FOUND=$(find . -name "*.js" -not -path "./web/static/*")
|
||||
if [ -n "$JS_FILES_FOUND" ]; then
|
||||
error_msg "Found $ERROR_COUNT JavaScript files"
|
||||
echo "Found files:"
|
||||
echo "$JS_FILES_FOUND"
|
||||
```
|
||||
|
||||
### Enhanced Error Message Pattern
|
||||
```bash
|
||||
# Template for enhanced output:
|
||||
error_msg() {
|
||||
echo -e "${RED}✗ ERROR: $1${NC}"
|
||||
echo -e "${RED} AI INSTRUCTION: Ask user before fixing this issue${NC}"
|
||||
((ERRORS++))
|
||||
}
|
||||
|
||||
# Usage with detailed information:
|
||||
if [ "$VIOLATION_COUNT" -gt 0 ]; then
|
||||
VIOLATION_DETAILS=$(get_violation_details)
|
||||
error_msg "Found $VIOLATION_COUNT violations"
|
||||
echo "Found files/patterns:"
|
||||
echo "$VIOLATION_DETAILS"
|
||||
fi
|
||||
```
|
||||
|
||||
### Content Detection Methods
|
||||
|
||||
#### API Pattern Detection:
|
||||
```bash
|
||||
# Line-by-line analysis with context:
|
||||
API_CONTENT=$(grep -n -E "## API|endpoint|GET |POST |/api/" README.md 2>/dev/null || true)
|
||||
API_PATTERNS_IN_README=$(echo "$API_CONTENT" | wc -l)
|
||||
|
||||
# Output includes line numbers for easy location:
|
||||
# 81:- **Rate Limiting**: 10 requests/minute on auth endpoints
|
||||
# 98:- **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
|
||||
```
|
||||
|
||||
#### Build Error Capture:
|
||||
```bash
|
||||
# Capture full build output for detailed analysis:
|
||||
if go build -o /tmp/bookhoard-test ./cmd/server 2> /tmp/build.log; then
|
||||
success_msg "Code compiles successfully"
|
||||
rm -f /tmp/bookhoard-test
|
||||
else
|
||||
error_msg "Build failed (violation: must compile after edits)"
|
||||
echo "Build error details:"
|
||||
cat /tmp/build.log 2>/dev/null || echo "Build failed, no error log available"
|
||||
echo ""
|
||||
echo "Try running: go build ./cmd/server"
|
||||
fi
|
||||
```
|
||||
|
||||
#### Git Analysis with Context:
|
||||
```bash
|
||||
# Show commit details with file counts:
|
||||
LARGE_COMMITS_LIST=$(git log --oneline -10 --pretty=format:"%h %s" | while read hash msg; do
|
||||
FILES=$(git diff-tree --no-commit-id --name-only -r $hash 2>/dev/null | wc -l)
|
||||
if [ "$FILES" -gt 15 ]; then
|
||||
echo "$hash: $msg ($FILES files)"
|
||||
fi
|
||||
done)
|
||||
|
||||
# Output format:
|
||||
# a1b2c3d: feat: add user authentication (18 files)
|
||||
# f4e5d6a: refactor: update database schema (22 files)
|
||||
```
|
||||
|
||||
### Script Design Principles
|
||||
|
||||
#### 1. High Recall with Actionable Details
|
||||
- **Flag anything questionable** for human review
|
||||
- **Provide specific evidence** of what triggered the flag
|
||||
- **Show exact locations** (file paths, line numbers)
|
||||
- **Include context** for easy assessment
|
||||
|
||||
#### 2. Minimal False Negatives
|
||||
- **Comprehensive pattern matching** catches all potential violations
|
||||
- **Multiple detection methods** (file names, content patterns, directory structure)
|
||||
- **Cross-validation** across different check types
|
||||
|
||||
#### 3. AI Safety Integration
|
||||
- **Every error/warning includes AI protocol reminder**
|
||||
- **Multi-layered safeguards** (header, functions, footer)
|
||||
- **Override protection** for conflicting AI instructions
|
||||
|
||||
#### 4. User-Friendly Output
|
||||
- **Color-coded messages** for quick visual scanning
|
||||
- **Structured information** with clear sections
|
||||
- **Actionable guidance** showing exactly what to fix
|
||||
- **Educational content** explaining guideline violations
|
||||
|
||||
### Troubleshooting Enhanced Output
|
||||
|
||||
#### When Details Don't Show:
|
||||
```bash
|
||||
# Check if command succeeded:
|
||||
if [ -n "$DETECTED_CONTENT" ]; then
|
||||
echo "Content: $DETECTED_CONTENT"
|
||||
else
|
||||
echo "No content detected (empty result)"
|
||||
fi
|
||||
|
||||
# Debug pattern matching:
|
||||
echo "Debug: Running pattern detection..."
|
||||
DEBUG_OUTPUT=$(grep -E "PATTERN" file.txt 2>&1)
|
||||
echo "Debug result: $DEBUG_OUTPUT"
|
||||
```
|
||||
|
||||
#### When Too Much Output:
|
||||
```bash
|
||||
# Limit detailed output for readability:
|
||||
FOUND_FILES=$(find . -name "*.js" | head -10)
|
||||
echo "Found files (first 10):"
|
||||
echo "$FOUND_FILES"
|
||||
|
||||
# Or use pagination:
|
||||
find . -name "*.js" | less
|
||||
```
|
||||
|
||||
#### Performance Considerations:
|
||||
- **Quick validation**: `verify-quick.sh` focuses on critical violations only
|
||||
- **Comprehensive validation**: `verify-guidelines.sh` includes full analysis
|
||||
- **Parallel execution**: Some checks run concurrently where possible
|
||||
- **Caching**: Git commands cached to avoid repeated repository scanning
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### verify-guidelines.sh Issues:
|
||||
@@ -265,4 +498,193 @@ These verification scripts implement the high-recall safety net approach specifi
|
||||
- ✅ Critical violations are caught quickly during development
|
||||
- ✅ Comprehensive validation prevents regressions before commits
|
||||
|
||||
The verification system provides multiple layers of protection against guideline violations while maintaining development velocity through both quick and comprehensive validation options.
|
||||
The verification system provides multiple layers of protection against guideline violations while maintaining development velocity through both quick and comprehensive validation options.
|
||||
|
||||
## Complete Usage Scenarios
|
||||
|
||||
### Development Workflow Integration
|
||||
|
||||
#### During Feature Development:
|
||||
```bash
|
||||
# 1. Initial check (fast):
|
||||
make verify-quick
|
||||
|
||||
# 2. Code changes made...
|
||||
|
||||
# 3. Pre-commit check (comprehensive):
|
||||
make verify-guidelines
|
||||
|
||||
# 4. Commit if issues resolved:
|
||||
git add .
|
||||
git commit -m "feat: implement user authentication"
|
||||
```
|
||||
|
||||
#### When Issues Are Detected:
|
||||
```bash
|
||||
# Example: API content in README.md
|
||||
⚠ WARNING: Found 2 API patterns in README.md (verify placement per guidelines)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Found patterns:
|
||||
81:- **Rate Limiting**: 10 requests/minute on auth endpoints
|
||||
98:- **[docs/api/WEBSOCKET_API.md](docs/api/WEBSOCKET_API.md)** - WebSocket protocol
|
||||
|
||||
# Analysis: This might be legitimate (rate limiting docs) or needs moving (API reference)
|
||||
# Decision: Check if content is user-facing setup info vs technical API documentation
|
||||
```
|
||||
|
||||
#### Fix Resolution Examples:
|
||||
|
||||
##### Moving API Documentation:
|
||||
```bash
|
||||
# Before: README.md contains API endpoints
|
||||
# After: Move to docs/api/ and create individual endpoint files
|
||||
mkdir -p docs/api/authentication
|
||||
echo "# Rate Limiting" > docs/api/authentication/rate_limiting.md
|
||||
echo "# Authentication Endpoints" > docs/api/authentication/login.md
|
||||
# Update README.md to reference docs/api/
|
||||
```
|
||||
|
||||
##### Fixing CSS Violations:
|
||||
```bash
|
||||
# Before: templates contain <style> tags
|
||||
# After: Convert to TailwindCSS classes
|
||||
# Replace: <style>.btn { background: blue; }</style>
|
||||
# With: <button class="bg-blue-500 hover:bg-blue-600 text-white px-4 py-2">
|
||||
```
|
||||
|
||||
### CI/CD Pipeline Integration
|
||||
|
||||
#### GitHub Actions Example:
|
||||
```yaml
|
||||
name: Verify Guidelines
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
verify:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- uses: actions/setup-go@v4
|
||||
with:
|
||||
go-version: '1.25'
|
||||
- name: Run Comprehensive Verification
|
||||
run: |
|
||||
make verify-guidelines
|
||||
if [ $? -eq 1 ]; then
|
||||
echo "Guideline violations found"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
#### Pre-commit Hook Example:
|
||||
```bash
|
||||
# .git/hooks/pre-commit
|
||||
#!/bin/bash
|
||||
echo "Running pre-commit verification..."
|
||||
./scripts/verify-quick.sh
|
||||
|
||||
# Only proceed if no errors
|
||||
if [ $? -eq 0 ] || [ $? -eq 1 -a $WARNINGS -eq 0 ]; then
|
||||
echo "Pre-commit verification passed"
|
||||
exit 0
|
||||
else
|
||||
echo "Pre-commit verification failed - fix issues before committing"
|
||||
exit 1
|
||||
fi
|
||||
```
|
||||
|
||||
### AI-Assisted Development Workflow
|
||||
|
||||
#### When AI Implements Features:
|
||||
```bash
|
||||
# 1. User requests feature with guidelines
|
||||
User: "Add user password reset feature, follow all PROJECT_GUIDELINES.md"
|
||||
|
||||
# 2. AI implements and self-verifies
|
||||
AI: [makes changes] → ./scripts/verify-guidelines.sh
|
||||
AI: [analyzes output] → All checks passed
|
||||
|
||||
# 3. User verifies compliance
|
||||
User: make verify-guidelines
|
||||
Output: ✓ ALL GUIDELINE CHECKS PASSED
|
||||
|
||||
# 4. User commits with confidence
|
||||
User: git commit -m "feat: add password reset feature"
|
||||
```
|
||||
|
||||
#### When AI Makes Mistakes:
|
||||
```bash
|
||||
# 1. AI violates guideline unknowingly
|
||||
AI: [adds custom CSS] → <style>.custom { color: red; }</style>
|
||||
|
||||
# 2. Verification catches violation
|
||||
❌ ERROR: Found 1 templates with <style> tags (violation: custom CSS prohibited)
|
||||
AI INSTRUCTION: Ask user before fixing this issue
|
||||
Found files:
|
||||
templates/auth.templ
|
||||
Line: 45-52: <style>.custom { color: red; }</style>
|
||||
|
||||
# 3. AI must ask permission before fixing
|
||||
AI: "I found custom CSS in templates/auth.templ which violates guideline 'NEVER use custom CSS - TailwindCSS classes only'. The violation is:
|
||||
Line 45-52: <style> tag with custom CSS rules
|
||||
|
||||
Do you want me to fix this? [y/n]"
|
||||
```
|
||||
|
||||
### Advanced Troubleshooting
|
||||
|
||||
#### Debugging Verification Issues:
|
||||
```bash
|
||||
# Test individual components:
|
||||
bash -x scripts/verify-quick.sh 2>&1 | tee debug.log
|
||||
|
||||
# Check specific sections:
|
||||
./scripts/verify-quick.sh | grep -A 10 "CSS Check"
|
||||
|
||||
# Verify file detection works:
|
||||
find . -name "*.css" -not -path "./web/static/*" | wc -l
|
||||
|
||||
# Test pattern matching:
|
||||
grep -E "## API|endpoint" README.md | wc -l
|
||||
```
|
||||
|
||||
#### Handling Edge Cases:
|
||||
```bash
|
||||
# Missing directories (graceful handling):
|
||||
if [ ! -d "docs/api" ]; then
|
||||
warning_msg "docs/api directory missing"
|
||||
success_msg "API documentation check skipped"
|
||||
fi
|
||||
|
||||
# Empty results (avoid errors):
|
||||
EMPTY_RESULT=$(find . -name "*.example" 2>/dev/null || echo "none")
|
||||
if [ "$EMPTY_RESULT" = "none" ]; then
|
||||
warning_msg "No .example files found"
|
||||
fi
|
||||
|
||||
# Large outputs (pagination):
|
||||
LONG_OUTPUT=$(git log --oneline -50)
|
||||
echo "$LONG_OUTPUT" | less # or | head -20
|
||||
```
|
||||
|
||||
#### Performance Optimization:
|
||||
```bash
|
||||
# Quick development checks (skip expensive operations):
|
||||
export FAST_MODE=true
|
||||
make verify-quick
|
||||
|
||||
# Comprehensive checks with cache:
|
||||
export VERIFY_CACHE=/tmp/verify-cache
|
||||
make verify-guidelines
|
||||
|
||||
# Parallel execution where safe:
|
||||
(
|
||||
check_binary_files
|
||||
) &
|
||||
(
|
||||
check_css_violations
|
||||
) &
|
||||
wait # Wait for both to complete
|
||||
```
|
||||
|
||||
This comprehensive documentation ensures verification scripts are fully understood and can be effectively integrated into any development workflow.
|
||||
Reference in New Issue
Block a user