Commit Graph

14 Commits

Author SHA1 Message Date
dc39930da4 feat: Implement ai_blueprint.md Steps 1 & 2 — bible-tracking merge and character voice profiles
Step 1 (Bible-Tracking Merge):
- Added merge_tracking_to_bible() to story/bible_tracker.py — merges character
  tracking state and lore back into bible dict after each chapter, making
  blueprint_initial.json the single persistent source of truth.
- Integrated in cli/engine.py after each chapter's update_tracking + update_lore_index
  calls so the persisted bible is always up-to-date.

Step 2 (Character-Specific Voice Profiles):
- story/writer.py: write_chapter now checks bp['characters'] for a voice_profile on
  the POV character before falling back to the prebuilt_persona cache.
- story/style_persona.py: refine_persona() accepts pov_character=None; when a POV
  character with a voice_profile is supplied it refines that profile's bio instead of
  the global author_details bio.
- cli/engine.py: refine_persona call now passes ch.get('pov_character') so per-chapter
  persona refinement targets the correct voice.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 22:45:54 -05:00
ff5093a5f9 fix: Pipeline hardening — error handling, token efficiency, and robustness
core/utils.py:
- estimate_tokens: improved heuristic 4 chars/token → 3.5 chars/token (more accurate)
- truncate_to_tokens: added keep_head=True mode for head+tail truncation (better
  context retention for story summaries that need both opening and recent content)
- load_json: explicit exception handling (json.JSONDecodeError, OSError) with log
  instead of silent returns; added utf-8 encoding with error replacement
- log_image_attempt: replaced bare except with (json.JSONDecodeError, OSError);
  added utf-8 encoding to output write
- log_usage: replaced bare except with AttributeError for token count extraction

story/bible_tracker.py:
- merge_selected_changes: wrapped all int() key casts (char idx, book num, beat idx)
  in try/except with meaningful log warning instead of crashing on malformed keys
- harvest_metadata: replaced bare except:pass with except Exception as e + log message

cli/engine.py:
- Persona validation: added warning when all 3 attempts fail and substandard persona
  is accepted — flags elevated voice-drift risk for the run
- Lore index updates: throttled from every chapter to every 3 chapters; lore is
  stable after the first few chapters (~10% token saving per book)
- Mid-gen consistency check: now samples first 2 + last 8 chapters instead of passing
  full manuscript — caps token cost regardless of book length

story/writer.py:
- Two-pass polish: added local filter-word density check (no API call); skips the
  Pro polish if density < 1 per 83 words — saves ~8K tokens on already-clean drafts
- Polish prompt: added prev_context_block for continuity — polished chapter now
  maintains seamless flow from the previous chapter's ending

marketing/fonts.py:
- Separated requests.exceptions.Timeout with specific log message vs generic failure
- Added explicit log message when Roboto fallback also fails (returns None)

marketing/blurb.py:
- Added word count trim: blurbs > 220 words trimmed to last sentence within 220 words
- Changed bare except to except Exception as e with log message
- Added utf-8 encoding to file writes; logs final word count

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 22:31:22 -05:00
4f2449f79b feat: Implement ai_blueprint_v2.md — Exp 5, 6 & 7 (persona validation, mid-gen consistency, two-pass drafting)
Exp 6 — Iterative Persona Validation (story/style_persona.py + cli/engine.py):
- Added validate_persona(): generates ~200-word sample in persona voice, scores 1–10 via
  lightweight voice-quality prompt; accepts if ≥ 7/10
- cli/engine.py retries create_initial_persona() up to 3× until validation passes
- Expected: -20% Phase 3 voice-drift rewrites

Exp 5 — Mid-gen Consistency Snapshots (cli/engine.py):
- analyze_consistency() called every 10 chapters inside the writing loop
- Issues logged as ⚠️ warnings; non-blocking; score and summary emitted
- Expected: -30% post-generation continuity error rate

Exp 7 — Two-Pass Drafting (story/writer.py):
- After Flash rough draft, Pro model (model_logic) polishes prose against a strict
  checklist: filter words, deep POV, active voice, AI-isms, chapter hook
- max_attempts reduced 3 → 2 since polished prose needs fewer rewrite cycles
- Expected: +0.3 HQS with no increase in per-chapter cost

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 22:08:47 -05:00
2100ca2312 feat: Implement ai_blueprint.md action plan — architectural review & optimisations
Steps 1–7 of the ai_blueprint.md action plan executed:

DOCUMENTATION (Steps 1–3, 6–7):
- docs/current_state_analysis.md: Phase-by-phase cost/quality mapping of existing pipeline
- docs/alternatives_analysis.md: 15 alternative approaches with testable hypotheses
- docs/experiment_design.md: 7 controlled A/B experiment specifications (CPC, HQS, CER metrics)
- ai_blueprint_v2.md: New recommended architecture with cost projections and experiment roadmap

CODE IMPROVEMENTS (Step 4 — Experiments 1–4 implemented):
- story/writer.py: Extract build_persona_info() — persona loaded once per book, not per chapter
- story/writer.py: Adaptive scoring thresholds — SCORE_PASSING scales 6.5→7.5 by chapter position
- story/writer.py: Beat expansion skip — if beats >100 words, skip Director's Treatment expansion
- story/planner.py: validate_outline() — pre-generation gate checks missing beats, continuity, pacing
- story/planner.py: Enrichment field validation — warn on missing title/genre after enrich()
- cli/engine.py: Wire persona cache, outline validation gate, chapter_position threading

Expected savings: ~285K tokens per 30-chapter novel (~7% cost reduction)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 22:01:30 -05:00
6684ec2bf5 feat: Improve book quality — stronger evaluator, more refinement attempts, quality-first model selection
- Fix: chapter quality evaluation now uses model_logic (free Pro) instead of model_writer (Flash).
  The model that wrote the chapter was also scoring it, causing circular, lenient grading.
- Increase max_attempts in write_chapter from 2 to 3 for more refinement passes per chapter.
- Update auto model selection prompt (ai/setup.py) to prioritize quality over budget framing:
  free/preview/exp models preferred by capability (Pro > Flash, 2.5 > 2.0 > 1.5), not just cost.
  Writer role now allowed to use best free Flash/Pro preview — not restricted to basic Flash only.
- Bump version to 3.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-22 21:28:49 -05:00
c3724a6761 feat: Cost-aware Pro model selection — free Pro beats Flash, paid Pro loses to Flash 2026-02-22 10:38:57 -05:00
353dc859d2 feat: Optimize AI model usage for cost reduction 2026-02-22 10:23:47 -05:00
d75186cb29 Auto-commit: v2.7 Series Continuity & Book Number Awareness
- story/planner.py: enrich() and plan_structure() now extract series_metadata
  and inject a SERIES_CONTEXT block (Book X of Y in series Z, with position-aware
  guidance) into prompts when is_series is true.
- story/writer.py: write_chapter() builds and injects the same SERIES_CONTEXT
  into the chapter draft prompt; passes series_context to evaluate_chapter_quality().
- story/editor.py: evaluate_chapter_quality() accepts optional series_context
  parameter and injects it into METADATA so arc pacing is evaluated relative to
  the book's position in the series.
- ai_blueprint.md: Section 11 marked complete (v2.7), summary updated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 01:51:35 -05:00
83a6a4315b Blueprint v2.4-2.6: Style Rules UI, Lore RAG, Thread Tracking, Redo Book
v2.4 — Item 7: Refresh Style Guidelines
- web/routes/admin.py: Added /admin/refresh-style-guidelines route (AJAX-aware)
- templates/system_status.html: Added 'Refresh Style Rules' button with spinner

v2.5 — Item 8: Lore & Location RAG-Lite
- story/bible_tracker.py: Added update_lore_index() — extracts location/item
  descriptions from chapters into tracking_lore.json
- story/writer.py: Reads chapter locations/key_items, builds LORE_CONTEXT block
  injected into the prompt (graceful degradation if no tags)
- cli/engine.py: Loads tracking_lore.json on resume, calls update_lore_index
  after each chapter, saves tracking_lore.json

v2.5 — Item 9: Structured Story State (Thread Tracking)
- story/state.py (new): load_story_state, update_story_state (extracts
  active_threads, immediate_handoff, resolved_threads via model_logic),
  format_for_prompt (structured context replacing the prev_sum blob)
- cli/engine.py: Loads story_state.json on resume, uses format_for_prompt as
  summary_ctx for write_chapter, updates state after each chapter accepted

v2.6 — Item 10: Redo Book
- templates/consistency_report.html: Added 'Redo Book' form with instruction
  input and confirmation dialog
- web/routes/run.py: Added revise_book route — creates new Run, queues
  generate_book_task with user instruction as feedback

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 01:35:43 -05:00
b1bce1eb55 Blueprint v2.3: AI-isms filter, Deep POV mandate, genre-specific writing rules
- story/style_persona.py: Expanded default ai_isms list with 20+ modern AI phrases
  (delved, mined, neon-lit, bustling, a wave of, etched in, etc.) and added
  filter_words (wondered, seemed, appeared, watched, observed, sensed)
- story/editor.py: Stricter evaluate_chapter_quality rubric — added
  DEEP_POV_ENFORCEMENT block with automatic fail conditions for filter word
  density and summary mode; strengthened criterion 5 scoring thresholds
- story/writer.py: Added get_genre_instructions() helper with genre-specific
  mandates for Thriller, Romance, Fantasy, Sci-Fi, Horror, Historical, and
  General Fiction; added DEEP_POV_MANDATE block banning summary mode and
  filter words; expanded AVOID AI-ISMS banned phrase list

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 01:19:56 -05:00
0667c31413 Blueprint v2.0: Pre-Flight Beat Expansion (Director's Treatment)
Implement Section 3 of the AI Context Optimization Blueprint: before each
chapter draft, model_logic expands sparse scene_beats into a structured
Director's Treatment covering staging, sensory anchors, emotional shifts,
and subtext per beat. This treatment is injected into the writer prompt,
giving the model a detailed scene blueprint to dramatize rather than infer,
reducing rewrite attempts and improving first-draft quality scores.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 00:15:43 -05:00
f71a04c03c Blueprint v1.5.0: AI Context Optimization — Dynamic Characters & Scene State
- writer.py: Dynamic character injection — only POV + beat-named characters
  are sent to the writer prompt, eliminating token waste and hallucinations
  from characters unrelated to the current scene.
- writer.py: Smart tail truncation — prev_content trimmed to last 1,000 tokens
  (the actual chapter ending) instead of a blind 2,000-token head slice,
  preserving the exact hand-off point for continuity.
- writer.py: Scene state injected into char_visuals — current_location,
  time_of_day, and held_items now surfaced per relevant character in prompt.
- bible_tracker.py: update_tracking expanded to record current_location,
  time_of_day, and held_items per character after each chapter.
- core/config.py: VERSION bumped 1.4.0 → 1.5.0.
- README.md: Story Generation section and tracking_characters.json schema
  updated to document new context optimization features.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-21 00:01:47 -05:00
db70ad81f7 Blueprint v1.0.4: Implemented AI Context Optimization & Token Management
- core/utils.py: Added estimate_tokens(), truncate_to_tokens(), get_ai_cache(), set_ai_cache(), make_cache_key() utilities
- story/writer.py: Applied truncate_to_tokens() to prev_content (2000 tokens) and prev_sum (600 tokens) context injections
- story/editor.py: Applied truncate_to_tokens() to summary (1000t), last_chapter_text (800t), eval text (7500t), propagation contexts (2500t/3000t)
- web/routes/persona.py: Added MD5-keyed in-memory cache for persona analyze endpoint; truncated sample_text to 750 tokens
- ai/models.py: Added pre-dispatch payload size estimation with 30k-token warning threshold

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 23:30:39 -05:00
f7099cc3e4 v2.0.0: Modularize project into single-responsibility packages
Replaced monolithic modules/ package with a clean architecture:

- core/       config.py, utils.py
- ai/         models.py (ResilientModel), setup.py (init_models)
- story/      planner.py, writer.py, editor.py, style_persona.py, bible_tracker.py
- marketing/  cover.py, blurb.py, fonts.py, assets.py
- export/     exporter.py
- web/        app.py (Flask factory), db.py, helpers.py, tasks.py, routes/{auth,project,run,persona,admin}.py
- cli/        engine.py (run_generation), wizard.py (BookWizard)

Flask routes split into 5 Blueprints; all templates updated with blueprint-
prefixed url_for() calls. Dockerfile and docker-compose updated to use
web.app entry point and new package paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-02-20 22:20:53 -05:00