Adds a full per-chapter evaluation logging system that captures every
score, critique, and quality decision made during writing, then renders
a self-contained HTML report shareable with critics or prompt engineers.
New file — story/eval_logger.py:
- append_eval_entry(folder, entry): writes per-chapter eval data to
eval_log.json in the book folder (called from write_chapter() at
every return point).
- generate_html_report(folder, bp): reads eval_log.json and produces a
self-contained HTML file (no external deps) with:
• Summary cards (avg score, auto-accepted, rewrites, below-threshold)
• Score timeline bar chart (one bar per chapter, colour-coded)
• Score distribution histogram
• Chapter breakdown table with expand-on-click critique details
(attempt number, score, decision badge, full critique text)
• Critique pattern frequency table (keyword mining across all critiques)
• Auto-generated prompt tuning observations (systemic issues, POV
character weak spots, pacing type analysis, climax vs. early
chapter comparison)
story/writer.py:
- Imports time and eval_logger.
- Initialises _eval_entry dict (chapter metadata + polish flags + thresholds)
after all threshold variables are set.
- Records each evaluation attempt's score, critique (truncated to 700 chars),
and decision (auto_accepted / full_rewrite / refinement / accepted /
below_threshold / eval_error / refinement_failed) before every return.
web/routes/run.py:
- Imports story_eval_logger.
- New route GET /project/<run_id>/eval_report/<book_folder>: loads
eval_log.json, calls generate_html_report(), returns the HTML as a
downloadable attachment named eval_report_<title>.html.
Returns a user-friendly "not yet available" page if no log exists.
templates/run_details.html:
- Adds "Eval Report" (btn-outline-info) button next to "Check Consistency"
in each book's artifact section.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. editor.py — Fix rewrite_chapter_content to use model_writer (was model_logic).
Chapter rewrites now use the creative writing model, not the cheaper analysis model.
2. editor.py — evaluate_chapter_quality now uses keep_head=True so the evaluator
sees the chapter opening (engagement hook, sensory anchoring) as well as the
ending; long chapters no longer scored on tail only.
3. editor.py — Consistency analysis sampling upgraded to head+middle+tail (was
head+tail), giving the LLM a complete view of each chapter's events.
4. writer.py — max_attempts is now adaptive: climax/resolution chapters
(position >= 0.75) receive 3 refinement attempts; others keep 2.
5. writer.py — Polish-skip threshold tightened from 0.012 to 0.008 (1 filter
word per 125 words vs. 1 per 83 words), so more borderline drafts are cleaned.
6. style_persona.py — Persona validation sample increased from 200 to 400 words
for more reliable voice quality assessment.
Version bumped: 3.0 → 3.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Step 1 (Bible-Tracking Merge):
- Added merge_tracking_to_bible() to story/bible_tracker.py — merges character
tracking state and lore back into bible dict after each chapter, making
blueprint_initial.json the single persistent source of truth.
- Integrated in cli/engine.py after each chapter's update_tracking + update_lore_index
calls so the persisted bible is always up-to-date.
Step 2 (Character-Specific Voice Profiles):
- story/writer.py: write_chapter now checks bp['characters'] for a voice_profile on
the POV character before falling back to the prebuilt_persona cache.
- story/style_persona.py: refine_persona() accepts pov_character=None; when a POV
character with a voice_profile is supplied it refines that profile's bio instead of
the global author_details bio.
- cli/engine.py: refine_persona call now passes ch.get('pov_character') so per-chapter
persona refinement targets the correct voice.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
core/utils.py:
- estimate_tokens: improved heuristic 4 chars/token → 3.5 chars/token (more accurate)
- truncate_to_tokens: added keep_head=True mode for head+tail truncation (better
context retention for story summaries that need both opening and recent content)
- load_json: explicit exception handling (json.JSONDecodeError, OSError) with log
instead of silent returns; added utf-8 encoding with error replacement
- log_image_attempt: replaced bare except with (json.JSONDecodeError, OSError);
added utf-8 encoding to output write
- log_usage: replaced bare except with AttributeError for token count extraction
story/bible_tracker.py:
- merge_selected_changes: wrapped all int() key casts (char idx, book num, beat idx)
in try/except with meaningful log warning instead of crashing on malformed keys
- harvest_metadata: replaced bare except:pass with except Exception as e + log message
cli/engine.py:
- Persona validation: added warning when all 3 attempts fail and substandard persona
is accepted — flags elevated voice-drift risk for the run
- Lore index updates: throttled from every chapter to every 3 chapters; lore is
stable after the first few chapters (~10% token saving per book)
- Mid-gen consistency check: now samples first 2 + last 8 chapters instead of passing
full manuscript — caps token cost regardless of book length
story/writer.py:
- Two-pass polish: added local filter-word density check (no API call); skips the
Pro polish if density < 1 per 83 words — saves ~8K tokens on already-clean drafts
- Polish prompt: added prev_context_block for continuity — polished chapter now
maintains seamless flow from the previous chapter's ending
marketing/fonts.py:
- Separated requests.exceptions.Timeout with specific log message vs generic failure
- Added explicit log message when Roboto fallback also fails (returns None)
marketing/blurb.py:
- Added word count trim: blurbs > 220 words trimmed to last sentence within 220 words
- Changed bare except to except Exception as e with log message
- Added utf-8 encoding to file writes; logs final word count
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Major changes to marketing/cover.py:
- Split evaluate_image_quality() into two purpose-built functions:
* evaluate_cover_art(): 5-rubric scoring (visual impact, genre fit, composition,
quality, clean image) with auto-fail for visible text (score capped at 4) and
deductions for deformed anatomy
* evaluate_cover_layout(): 5-rubric scoring (legibility, typography, placement,
professional polish, genre signal) with auto-fail for illegible title (capped at 4)
- Added validate_art_prompt(): pre-validates the Imagen prompt before generation —
strips accidental text instructions, ensures focal point + rule-of-thirds + genre fit
- Added _build_visual_context(): extracts protagonist/antagonist descriptions and key
themes from tracking data into structured visual context for the art director prompt
- Score thresholds raised to match chapter pipeline: ART_PASSING=7, ART_AUTO_ACCEPT=8,
LAYOUT_PASSING=7 (was: art>=5 or >0, layout breaks only at ==10)
- Critique-driven art prompt refinement between attempts: full LLM rewrite of the
Imagen prompt using the evaluator's actionable feedback (not just keyword appending)
- Layout loop now breaks early at score>=7 (was: only at ==10, so never)
- Design prompt strengthened with explicit character/visual context and NO TEXT clause
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix: chapter quality evaluation now uses model_logic (free Pro) instead of model_writer (Flash).
The model that wrote the chapter was also scoring it, causing circular, lenient grading.
- Increase max_attempts in write_chapter from 2 to 3 for more refinement passes per chapter.
- Update auto model selection prompt (ai/setup.py) to prioritize quality over budget framing:
free/preview/exp models preferred by capability (Pro > Flash, 2.5 > 2.0 > 1.5), not just cost.
Writer role now allowed to use best free Flash/Pro preview — not restricted to basic Flash only.
- Bump version to 3.0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copies bible.json as bible_snapshot.json into the run folder before
generation begins, preserving the exact blueprint used for that run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Each chapter card now has a footer with Prev/Next chapter anchor links
- First chapter shows only Next; last chapter shows 'End of Book'
- Back to Top link on every chapter footer
- Added get_chapter_neighbours() helper in story/bible_tracker.py for
programmatic chapter sequence navigation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Guard checks at task start verify: run exists in DB, project folder exists,
bible.json exists and is parseable, and bible has at least one book
- Any failed check marks the run as 'failed' and returns early, preventing
jobs from writing to the wrong book or orphaned project directories
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Added tags VARCHAR(300) column to Run model
- Added startup ALTER TABLE migration in app.py
- New POST /run/<id>/set_tags route saves comma-separated tags
- Tag badges + collapsible edit form in run_details.html header area
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New POST /run/<id>/delete route removes run from DB and deletes run directory
- Only allows deletion of non-active runs (blocks running/queued)
- Delete Run button shown in run_details.html header for non-active runs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New GET /run/<id>/download_bible route serves project bible.json as attachment
- Download Bible button added to run_details.html header toolbar
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove ai_blueprint.md from git tracking (already gitignored)
- web/app.py: Unify startup reset — all non-terminal states (running,
queued, interrupted) are reset to 'failed' with per-job logging
- web/routes/project.py: Add active_runs list to view_project() context
- templates/project.html: Add Active Jobs card showing all running/queued
jobs with status badge, start time, progress bar, and View Details link;
Generate button and Stop buttons now driven by active_runs list
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- web/db.py: Add last_heartbeat column to Run model
- core/utils.py: Add set_heartbeat_callback() and send_heartbeat()
- web/tasks.py: Add _robust_update_run_status() with 5-retry exponential backoff;
add db_heartbeat_callback(); remove all bare except:pass on DB status updates;
set start_time + last_heartbeat when marking run as 'running'
- web/app.py: Add last_heartbeat column migration; add _stale_job_watcher()
background thread (checks every 5 min, 15-min heartbeat threshold, 2-hr start_time threshold)
- cli/engine.py: Add phase-level logging banners and try/except wrappers in
process_book(); add utils.send_heartbeat() after each chapter save;
add start/finish logging in run_generation()
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Backend (web/routes/run.py): Extended /run/<id>/status JSON response with
server_timestamp, db_log_count, and latest_log_timestamp so clients can
detect whether the DB is being written to independently of the log text.
- Frontend (templates/run_details.html):
• Added Live Status Panel above the System Log card, showing:
- Polling state badge (Initializing / Requesting / Waiting Ns / Error / Idle)
- Last Successful Update timestamp (HH:MM:SS, updated every successful poll)
- DB diagnostics (log count + latest log timestamp from server response)
- Last Error message displayed inline when a poll fails
- Force Refresh button to immediately trigger a new poll
• Refactored JS polling loop: countdown timer with clearCountdown/
startWaitCountdown helpers, forceRefresh() clears pending timers before
re-polling, explicit pollTimer/countdownInterval tracking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- web/tasks.py: db_log_callback now writes non-OperationalError exceptions to data/app.log for visibility
- web/tasks.py: generate_book_task restructured with try...finally to guarantee final status update — run can never be left in 'running' state if worker crashes
- templates/project.html: added .catch() to fetchLog() with console.error + polling resume on failure; added manual Refresh button to status bar
- templates/run_details.html: improved .catch() in updateLog() with descriptive message + 5s retry; added manual Refresh button to status bar
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- web/tasks.py: db_log_callback bare `except: break` replaced with
explicit `except Exception as _e: print(...)` so insertion failures
are visible in Docker logs. Also fixed datetime.utcnow() → .isoformat()
for clean string storage in SQLite.
Same fix applied to db_progress_callback.
- web/routes/run.py (run_status): added db.session.expire_all() to
force fresh reads; raw sqlite3 bypass query when ORM returns no rows;
file fallback wrapped in try/except with stdout error reporting;
secondary check for web_console.log inside the run directory;
utf-8 encoding on all file opens.
- ai_blueprint.md: bumped to v2.11, documented root causes and fixes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
docker-compose.yml:
- Add PYTHONIOENCODING=utf-8 env var (guarantees UTF-8 stdout in all
Python environments, including Docker slim images on ARM).
- Add logging driver section: json-file, max-size 10m, max-file 5.
Without this the json-file log on a Raspberry Pi SD card grows
unbounded and eventually kills the container or fills the disk.
web/requirements_web.txt:
- Pin huey==2.6.0 so a future pip upgrade cannot silently change the
Consumer() API and re-introduce the loglevel= TypeError that caused
all tasks to stay queued forever.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- web/app.py: Startup banner to docker logs (Python version, platform,
Huey version, DB paths). All print() calls now flush=True so Docker
captures them immediately. Emoji-free for robust stdout encoding.
Startup now detects orphaned queued runs (queue empty but DB queued)
and resets them to 'failed' so the UI does not stay stuck on reload.
Huey logging configured at INFO level so task pick-up/completion
appears in `docker logs`. Consumer skip reason logged explicitly.
- web/tasks.py: generate_book_task now emits [TASK run=N] lines to
stdout (docker logs) at pick-up, log-file creation, DB status update,
and on error (with full traceback) so failures are always visible.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root cause: Consumer(huey, workers=1, worker_type='thread', loglevel=20)
raised TypeError on every app start because Huey 2.6.0 does not accept
a `loglevel` keyword argument. The exception was silently caught and only
printed to stdout, so the consumer never ran and all tasks stayed 'queued'
forever — causing the 'Preparing environment / Waiting for logs' hang.
Fixes:
- web/app.py: Remove invalid `loglevel=20` from Consumer(); configure
Huey logging via logging.basicConfig(WARNING) instead. Add persistent
error logging to data/consumer_error.log for future diagnosis.
- core/config.py: Replace emoji print() calls with ASCII-safe equivalents
to prevent UnicodeEncodeError on Windows cp1252 terminals at import time.
- core/config.py: Update VERSION to 2.9 (was stale at 1.5.0).
- ai_blueprint.md: Bump to v2.10, document root cause and fixes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ai/setup.py: Added threading import; OAuth block now detects background/headless
threads and skips run_local_server to prevent indefinite blocking. Logs a clear
warning and falls back to ADC for Vertex AI. Token file only written when creds
are not None.
- web/tasks.py: All sqlite3.connect() calls now use timeout=30, check_same_thread=False.
OperationalError on the initial status update is caught and logged via utils.log.
generate_book_task now touches initial_log immediately so the UI polling endpoint
always finds an existing file even if the worker crashes on the next line.
- ai_blueprint.md: Bumped to v2.9; Section 12.D sub-items 1-3 marked ✅; item 13
added to summary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Added Section 12 to ai_blueprint.md covering:
- A: API timeout hangs (ai/models.py 180s, ai/setup.py 30s, removed cascading init call)
- B: Huey consumer never started under flask/gunicorn (module-level start + reloader guard)
- C: 'Create new book not showing anything' — 3 root causes fixed:
(4) Jinja2 UndefinedError on s.tropes|join in project_setup.html
(5) Silent redirect when model_logic=None now renders form with defaults
(6) planner.enrich() called with wrong bible structure in create_project_final
Bumped blueprint version from v2.7 → v2.8.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1. templates/project_setup.html: s.tropes|join and s.formatting_rules|join
raised Jinja2 UndefinedError when AI failed and fallback dict lacked those
keys → 500 blank page. Fixed with (s.tropes or [])|join(', ').
2. web/routes/project.py (project_setup_wizard): Removed silent redirect-to-
dashboard when model_logic is None. Now renders the setup form with a
complete default suggestions dict (all fields present, lists as []) plus a
clear warning flash so the user can fill it in manually.
3. web/routes/project.py (create_project_final): planner.enrich() was called
with the full bible dict — enrich() reads manual_instruction from the top
level (got 'A generic story' fallback) and wrote results into book_metadata
instead of the bible's books[0]. Fixed to build a proper per-book blueprint,
call enrich, and merge characters/plot_beats back into the correct locations.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Root causes of indefinite spinning during book create/generate:
1. ai/models.py — ResilientModel.generate_content() had no timeout: a
stalled Gemini API call would block the thread forever. Now injects
request_options={"timeout": 180} into every call. Also removed the
dangerous init_models(force=True) call inside the retry handler, which
was making a second network call during an existing API failure.
2. ai/setup.py — genai.list_models() calls in get_optimal_model(),
select_best_models(), and init_models() had no timeout. Added
request_options={"timeout": 30} to all three calls so model init
fails fast rather than hanging indefinitely.
3. web/app.py — Huey task consumer only started inside
`if __name__ == "__main__":`, meaning tasks queued via flask run,
gunicorn, or other WSGI runners were never executed (status stuck at
"queued" forever). Moved consumer start to module level with a
WERKZEUG_RUN_MAIN guard to prevent double-start under the reloader.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sections 1 (RAG for Lore/Locations) and 2 (Thread Tracking) still showed
⏳ despite being fully implemented under Sections 8 and 9 in v2.5.
Updated both to ✅ with accurate implementation notes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- story/planner.py: enrich() and plan_structure() now extract series_metadata
and inject a SERIES_CONTEXT block (Book X of Y in series Z, with position-aware
guidance) into prompts when is_series is true.
- story/writer.py: write_chapter() builds and injects the same SERIES_CONTEXT
into the chapter draft prompt; passes series_context to evaluate_chapter_quality().
- story/editor.py: evaluate_chapter_quality() accepts optional series_context
parameter and injects it into METADATA so arc pacing is evaluated relative to
the book's position in the series.
- ai_blueprint.md: Section 11 marked complete (v2.7), summary updated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- story/style_persona.py: Expanded default ai_isms list with 20+ modern AI phrases
(delved, mined, neon-lit, bustling, a wave of, etched in, etc.) and added
filter_words (wondered, seemed, appeared, watched, observed, sensed)
- story/editor.py: Stricter evaluate_chapter_quality rubric — added
DEEP_POV_ENFORCEMENT block with automatic fail conditions for filter word
density and summary mode; strengthened criterion 5 scoring thresholds
- story/writer.py: Added get_genre_instructions() helper with genre-specific
mandates for Thriller, Romance, Fantasy, Sci-Fi, Horror, Historical, and
General Fiction; added DEEP_POV_MANDATE block banning summary mode and
filter words; expanded AVOID AI-ISMS banned phrase list
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Convert form POST to async fetch() in system_status.html
- Spinner + disabled button while request is in-flight
- Bootstrap toast notification on success/error
- Auto-reload page 1.5s after successful refresh
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement Section 3 of the AI Context Optimization Blueprint: before each
chapter draft, model_logic expands sparse scene_beats into a structured
Director's Treatment covering staging, sensory anchors, emotional shifts,
and subtext per beat. This treatment is injected into the writer prompt,
giving the model a detailed scene blueprint to dramatize rather than infer,
reducing rewrite attempts and improving first-draft quality scores.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- writer.py: Dynamic character injection — only POV + beat-named characters
are sent to the writer prompt, eliminating token waste and hallucinations
from characters unrelated to the current scene.
- writer.py: Smart tail truncation — prev_content trimmed to last 1,000 tokens
(the actual chapter ending) instead of a blind 2,000-token head slice,
preserving the exact hand-off point for continuity.
- writer.py: Scene state injected into char_visuals — current_location,
time_of_day, and held_items now surfaced per relevant character in prompt.
- bible_tracker.py: update_tracking expanded to record current_location,
time_of_day, and held_items per character after each chapter.
- core/config.py: VERSION bumped 1.4.0 → 1.5.0.
- README.md: Story Generation section and tracking_characters.json schema
updated to document new context optimization features.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed `from huey.contrib.mini import MiniHuey` which caused
`ModuleNotFoundError: No module named 'gevent'` on startup. MiniHuey
was never used; the app correctly uses SqliteHuey via `web.tasks`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
web/app.py was hardcoded to port 7070, causing Docker port forwarding
(5000:5000) and the Dockerfile HEALTHCHECK to fail. Changed to port 5000
to match docker-compose.yml and Dockerfile configuration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
