Commit Graph

9 Commits

Author SHA1 Message Date
d2c65f010a feat: Improve revision pipeline quality — 6 targeted enhancements (v3.1)
1. editor.py — Fix rewrite_chapter_content to use model_writer (was model_logic).
   Chapter rewrites now use the creative writing model, not the cheaper analysis model.

2. editor.py — evaluate_chapter_quality now uses keep_head=True so the evaluator
   sees the chapter opening (engagement hook, sensory anchoring) as well as the
   ending; long chapters no longer scored on tail only.

3. editor.py — Consistency analysis sampling upgraded to head+middle+tail (was
   head+tail), giving the LLM a complete view of each chapter's events.

4. writer.py — max_attempts is now adaptive: climax/resolution chapters
   (position >= 0.75) receive 3 refinement attempts; others keep 2.

5. writer.py — Polish-skip threshold tightened from 0.012 to 0.008 (1 filter
   word per 125 words vs. 1 per 83 words), so more borderline drafts are cleaned.

6. style_persona.py — Persona validation sample increased from 200 to 400 words
   for more reliable voice quality assessment.

Version bumped: 3.0 → 3.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-24 07:51:31 -05:00
ff5093a5f9 fix: Pipeline hardening — error handling, token efficiency, and robustness
core/utils.py:
- estimate_tokens: improved heuristic 4 chars/token → 3.5 chars/token (more accurate)
- truncate_to_tokens: added keep_head=True mode for head+tail truncation (better
  context retention for story summaries that need both opening and recent content)
- load_json: explicit exception handling (json.JSONDecodeError, OSError) with log
  instead of silent returns; added utf-8 encoding with error replacement
- log_image_attempt: replaced bare except with (json.JSONDecodeError, OSError);
  added utf-8 encoding to output write
- log_usage: replaced bare except with AttributeError for token count extraction

story/bible_tracker.py:
- merge_selected_changes: wrapped all int() key casts (char idx, book num, beat idx)
  in try/except with meaningful log warning instead of crashing on malformed keys
- harvest_metadata: replaced bare except:pass with except Exception as e + log message

cli/engine.py:
- Persona validation: added warning when all 3 attempts fail and substandard persona
  is accepted — flags elevated voice-drift risk for the run
- Lore index updates: throttled from every chapter to every 3 chapters; lore is
  stable after the first few chapters (~10% token saving per book)
- Mid-gen consistency check: now samples first 2 + last 8 chapters instead of passing
  full manuscript — caps token cost regardless of book length

story/writer.py:
- Two-pass polish: added local filter-word density check (no API call); skips the
  Pro polish if density < 1 per 83 words — saves ~8K tokens on already-clean drafts
- Polish prompt: added prev_context_block for continuity — polished chapter now
  maintains seamless flow from the previous chapter's ending

marketing/fonts.py:
- Separated requests.exceptions.Timeout with specific log message vs generic failure
- Added explicit log message when Roboto fallback also fails (returns None)

marketing/blurb.py:
- Added word count trim: blurbs > 220 words trimmed to last sentence within 220 words
- Changed bare except to except Exception as e with log message
- Added utf-8 encoding to file writes; logs final word count

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 22:31:22 -05:00
6684ec2bf5 feat: Improve book quality — stronger evaluator, more refinement attempts, quality-first model selection
- Fix: chapter quality evaluation now uses model_logic (free Pro) instead of model_writer (Flash).
  The model that wrote the chapter was also scoring it, causing circular, lenient grading.
- Increase max_attempts in write_chapter from 2 to 3 for more refinement passes per chapter.
- Update auto model selection prompt (ai/setup.py) to prioritize quality over budget framing:
  free/preview/exp models preferred by capability (Pro > Flash, 2.5 > 2.0 > 1.5), not just cost.
  Writer role now allowed to use best free Flash/Pro preview — not restricted to basic Flash only.
- Bump version to 3.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 21:28:49 -05:00
51b98c9399 refactor: Migrate file-based data storage to database 2026-02-22 10:23:40 -05:00
81340a18ea Auto-commit: v2.14 — Stuck job robustness (heartbeat, retry, stale watcher, granular logging)
- web/db.py: Add last_heartbeat column to Run model
- core/utils.py: Add set_heartbeat_callback() and send_heartbeat()
- web/tasks.py: Add _robust_update_run_status() with 5-retry exponential backoff;
  add db_heartbeat_callback(); remove all bare except:pass on DB status updates;
  set start_time + last_heartbeat when marking run as 'running'
- web/app.py: Add last_heartbeat column migration; add _stale_job_watcher()
  background thread (checks every 5 min, 15-min heartbeat threshold, 2-hr start_time threshold)
- cli/engine.py: Add phase-level logging banners and try/except wrappers in
  process_book(); add utils.send_heartbeat() after each chapter save;
  add start/finish logging in run_generation()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 19:00:29 -05:00
a324355cdf Auto-commit: v2.10 — Fix Huey consumer never starting (loglevel= TypeError)
Root cause: Consumer(huey, workers=1, worker_type='thread', loglevel=20)
raised TypeError on every app start because Huey 2.6.0 does not accept
a `loglevel` keyword argument. The exception was silently caught and only
printed to stdout, so the consumer never ran and all tasks stayed 'queued'
forever — causing the 'Preparing environment / Waiting for logs' hang.

Fixes:
- web/app.py: Remove invalid `loglevel=20` from Consumer(); configure
  Huey logging via logging.basicConfig(WARNING) instead. Add persistent
  error logging to data/consumer_error.log for future diagnosis.
- core/config.py: Replace emoji print() calls with ASCII-safe equivalents
  to prevent UnicodeEncodeError on Windows cp1252 terminals at import time.
- core/config.py: Update VERSION to 2.9 (was stale at 1.5.0).
- ai_blueprint.md: Bump to v2.10, document root cause and fixes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 12:02:18 -05:00
f71a04c03c Blueprint v1.5.0: AI Context Optimization — Dynamic Characters & Scene State
- writer.py: Dynamic character injection — only POV + beat-named characters
  are sent to the writer prompt, eliminating token waste and hallucinations
  from characters unrelated to the current scene.
- writer.py: Smart tail truncation — prev_content trimmed to last 1,000 tokens
  (the actual chapter ending) instead of a blind 2,000-token head slice,
  preserving the exact hand-off point for continuity.
- writer.py: Scene state injected into char_visuals — current_location,
  time_of_day, and held_items now surfaced per relevant character in prompt.
- bible_tracker.py: update_tracking expanded to record current_location,
  time_of_day, and held_items per character after each chapter.
- core/config.py: VERSION bumped 1.4.0 → 1.5.0.
- README.md: Story Generation section and tracking_characters.json schema
  updated to document new context optimization features.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 00:01:47 -05:00
db70ad81f7 Blueprint v1.0.4: Implemented AI Context Optimization & Token Management
- core/utils.py: Added estimate_tokens(), truncate_to_tokens(), get_ai_cache(), set_ai_cache(), make_cache_key() utilities
- story/writer.py: Applied truncate_to_tokens() to prev_content (2000 tokens) and prev_sum (600 tokens) context injections
- story/editor.py: Applied truncate_to_tokens() to summary (1000t), last_chapter_text (800t), eval text (7500t), propagation contexts (2500t/3000t)
- web/routes/persona.py: Added MD5-keyed in-memory cache for persona analyze endpoint; truncated sample_text to 750 tokens
- ai/models.py: Added pre-dispatch payload size estimation with 30k-token warning threshold

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 23:30:39 -05:00
f7099cc3e4 v2.0.0: Modularize project into single-responsibility packages
Replaced monolithic modules/ package with a clean architecture:

- core/       config.py, utils.py
- ai/         models.py (ResilientModel), setup.py (init_models)
- story/      planner.py, writer.py, editor.py, style_persona.py, bible_tracker.py
- marketing/  cover.py, blurb.py, fonts.py, assets.py
- export/     exporter.py
- web/        app.py (Flask factory), db.py, helpers.py, tasks.py, routes/{auth,project,run,persona,admin}.py
- cli/        engine.py (run_generation), wizard.py (BookWizard)

Flask routes split into 5 Blueprints; all templates updated with blueprint-
prefixed url_for() calls. Dockerfile and docker-compose updated to use
web.app entry point and new package paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 22:20:53 -05:00