Auto-commit: Blueprint v2.8 — document all v2.8 infrastructure & UI bug fixes

Added Section 12 to ai_blueprint.md covering:
- A: API timeout hangs (ai/models.py 180s, ai/setup.py 30s, removed cascading init call)
- B: Huey consumer never started under flask/gunicorn (module-level start + reloader guard)
- C: 'Create new book not showing anything' — 3 root causes fixed:
    (4) Jinja2 UndefinedError on s.tropes|join in project_setup.html
    (5) Silent redirect when model_logic=None now renders form with defaults
    (6) planner.enrich() called with wrong bible structure in create_project_final

Bumped blueprint version from v2.7 → v2.8.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

ai_blueprint.md

@@ -1,4 +1,4 @@
-# AI Context Optimization Blueprint (v2.7)
+# AI Context Optimization Blueprint (v2.8)
 
 This blueprint outlines architectural improvements for how AI context is managed during the writing process. The goal is to provide the AI (Claude/Gemini) with **better, highly-targeted context upfront**, which will dramatically improve first-draft quality and reduce the reliance on expensive, time-consuming quality checks and rewrites (currently up to 5 attempts).
 
@@ -121,6 +121,32 @@ The system generates books for a series, but the prompts in `story/planner.py` (
 1. ✅ **Planner Prompts Update:** Modified `enrich()` and `plan_structure()` in `story/planner.py` to extract `bp.get('series_metadata', {})` and inject a `SERIES_CONTEXT` block — "This is Book X of Y in the Z series" with position-aware guidance (Book 1 = establish, middle books = escalate, final book = resolve) — into the prompt when `is_series` is true. *(Implemented v2.7)*
 2. ✅ **Writer Prompts Update:** `story/writer.py` `write_chapter()` builds and injects the same `SERIES_CONTEXT` block into the chapter writing prompt and passes it as `series_context` to `evaluate_chapter_quality()` in `story/editor.py`. `editor.py` `evaluate_chapter_quality()` now accepts an optional `series_context` parameter and injects it into the evaluation METADATA so the editor scores arcs relative to the book's position in the series. *(Implemented v2.7)*
 
+## 12. Infrastructure & UI Bug Fixes (v2.8)
+
+**Problems Found & Fixed:**
+
+### A. API Timeout Hangs (Spinning Logs)
+The Gemini SDK had no timeout configured on any network call, causing threads to hang indefinitely:
+- `ai/models.py` `generate_content()` had no timeout → runs spun forever on API errors.
+- `ai/setup.py` all three `genai.list_models()` calls had no timeout → model init could hang.
+- `ai/models.py` retry handler called `init_models(force=True)` — a second network call during an existing failure, cascading the hang.
+
+**Fixes Applied:**
+1. ✅ `ai/models.py`: Added `_GENERATION_TIMEOUT = 180` class variable; all `generate_content()` calls now merge `request_options={"timeout": 180}`. Removed `init_models(force=True)` from retry handler. *(Implemented v2.8)*
+2. ✅ `ai/setup.py`: Added `_LIST_MODELS_TIMEOUT = {"timeout": 30}` passed to all three `genai.list_models()` call sites (`get_optimal_model`, `select_best_models`, `init_models`). *(Implemented v2.8)*
+
+### B. Huey Consumer Never Started (Tasks Queued But Never Executed)
+`web/app.py` started the Huey background consumer inside `if __name__ == "__main__":`, which only runs when the script is executed directly. Under `flask run`, gunicorn, or any WSGI runner the block is never reached — tasks were queued in `queue.db` but never processed.
+
+3. ✅ `web/app.py`: Moved Huey consumer start to module level with a Werkzeug reloader guard (`WERKZEUG_RUN_MAIN`) and a `FLASK_TESTING` guard to prevent duplicate/test-time consumers. Consumer runs as a daemon thread. *(Implemented v2.8)*
+
+### C. "Create New Book" Showing Nothing
+Three bugs combined to produce a blank page or silent failure when creating a new project:
+
+4. ✅ `templates/project_setup.html`: `{{ s.tropes|join(', ') }}` and `{{ s.formatting_rules|join(', ') }}` raised Jinja2 `UndefinedError` when AI analysis failed and the fallback dict lacked those keys → 500 blank page. Fixed to `{{ (s.tropes or [])|join(', ') }}`. *(Implemented v2.8)*
+5. ✅ `web/routes/project.py` (`project_setup_wizard`): When `model_logic` was `None`, the route silently redirected to the dashboard with a flash the user missed. Now renders the setup form with a complete default suggestions dict (all fields populated, lists as `[]`) and a visible `"warning"` flash so the user can fill in details manually. *(Implemented v2.8)*
+6. ✅ `web/routes/project.py` (`create_project_final`): `planner.enrich()` was called with the full project bible dict. `enrich()` reads `bp.get('manual_instruction')` from the top level (got `'A generic story'` fallback — the real concept was in `bible['books'][0]['manual_instruction']`), and wrote enriched data into a new `book_metadata` key instead of the bible's `books[0]`. Fixed to build a proper per-book blueprint, call enrich, and merge `characters`, `plot_beats`, and `structure_prompt` back into the correct bible locations. *(Implemented v2.8)*
+
 ## Summary of Actionable Changes for Implementation Mode:
 1. ✅ Modify `writer.py` to filter `chars_for_writer` based on characters named in `beats`. *(Implemented in v1.5.0)*
 2. ✅ Modify `writer.py` `prev_content` logic to extract the *tail* of the chapter, not a blind slice. *(Implemented in v1.5.0 via `utils.truncate_to_tokens` tail logic)*
@@ -133,3 +159,4 @@ The system generates books for a series, but the prompts in `story/planner.py` (
 9. ✅ **(v2.5)** Structured Story State (Thread Tracking): new `story/state.py`, `story_state.json`, structured prompt context replacing raw summary blob in `engine.py`. *(Implemented v2.5)*
 10. ✅ **(v2.6)** "Redo Book" form in `consistency_report.html` + `revise_book` route in `run.py` that creates a new run with the instruction applied as bible feedback. *(Implemented v2.6)*
 11. ✅ **(v2.7)** Series Continuity Fix: `series_metadata` (is_series, series_title, book_number, total_books) injected as `SERIES_CONTEXT` into `story/planner.py` (`enrich`, `plan_structure`), `story/writer.py` (`write_chapter`), and `story/editor.py` (`evaluate_chapter_quality`) prompts with position-aware guidance per book number. *(Implemented v2.7)*
+12. ✅ **(v2.8)** Infrastructure & UI Bug Fixes: API timeouts (180s generation, 30s list_models) in `ai/models.py` + `ai/setup.py`; Huey consumer moved to module level with reloader guard in `web/app.py`; Jinja2 `UndefinedError` fix for `tropes`/`formatting_rules` in `project_setup.html`; `project_setup_wizard` now renders form instead of silent redirect when models fail; `create_project_final` `enrich()` call fixed to use correct per-book blueprint structure. *(Implemented v2.8)*

templates/project_setup.html

@@ -121,12 +121,12 @@
                     
                     <div class="mb-4">
                         <label class="form-label">Tropes (comma separated)</label>
-                        <input type="text" name="tropes" class="form-control" value="{{ s.tropes|join(', ') }}">
+                        <input type="text" name="tropes" class="form-control" value="{{ (s.tropes or [])|join(', ') }}">
                     </div>
-                    
+
                     <div class="mb-4">
                         <label class="form-label">Formatting Rules (comma separated)</label>
-                        <input type="text" name="formatting_rules" class="form-control" value="{{ s.formatting_rules|join(', ') }}">
+                        <input type="text" name="formatting_rules" class="form-control" value="{{ (s.formatting_rules or [])|join(', ') }}">
                     </div>
 
                     <div class="d-grid gap-2">

web/routes/project.py

@@ -30,10 +30,6 @@ def project_setup_wizard():
     try: ai_setup.init_models()
     except: pass
 
-    if not ai_models.model_logic:
-        flash("AI models not initialized.")
-        return redirect(url_for('project.index'))
-
     prompt = f"""
     ROLE: Publishing Analyst
     TASK: Suggest metadata for a story concept.
@@ -67,13 +63,46 @@ def project_setup_wizard():
     }}
     """
 
+    _default_suggestions = {
+        "title": concept[:60] if concept else "New Project",
+        "genre": "Fiction",
+        "target_audience": "",
+        "tone": "",
+        "length_category": "4",
+        "estimated_chapters": 20,
+        "estimated_word_count": "75,000",
+        "include_prologue": False,
+        "include_epilogue": False,
+        "tropes": [],
+        "pov_style": "",
+        "time_period": "Modern",
+        "spice": "",
+        "violence": "",
+        "is_series": False,
+        "series_title": "",
+        "narrative_tense": "",
+        "language_style": "",
+        "dialogue_style": "",
+        "page_orientation": "Portrait",
+        "formatting_rules": [],
+        "author_bio": ""
+    }
+
     suggestions = {}
-    try:
-        response = ai_models.model_logic.generate_content(prompt)
-        suggestions = json.loads(utils.clean_json(response.text))
-    except Exception as e:
-        flash(f"AI Analysis failed: {e}")
-        suggestions = {"title": "New Project", "genre": "Fiction"}
+    if not ai_models.model_logic:
+        flash("AI models not initialized — fill in the details manually.", "warning")
+        suggestions = _default_suggestions
+    else:
+        try:
+            response = ai_models.model_logic.generate_content(prompt)
+            suggestions = json.loads(utils.clean_json(response.text))
+            # Ensure list fields are always lists
+            for list_field in ("tropes", "formatting_rules"):
+                if not isinstance(suggestions.get(list_field), list):
+                    suggestions[list_field] = []
+        except Exception as e:
+            flash(f"AI Analysis failed — fill in the details manually. ({e})", "warning")
+            suggestions = _default_suggestions
 
     personas = {}
     if os.path.exists(config.PERSONAS_FILE):
@@ -201,8 +230,33 @@ def create_project_final():
 
     try:
         ai_setup.init_models()
-        bible = planner.enrich(bible, proj_path)
-    except: pass
+        # Build a per-book blueprint matching what enrich() expects
+        first_book = bible['books'][0] if bible.get('books') else {}
+        bp = {
+            'manual_instruction': first_book.get('manual_instruction', concept),
+            'book_metadata': {
+                'title': bible['project_metadata']['title'],
+                'genre': bible['project_metadata']['genre'],
+                'style': dict(bible['project_metadata'].get('style', {})),
+            },
+            'length_settings': dict(bible['project_metadata'].get('length_settings', {})),
+            'characters': [],
+            'plot_beats': [],
+        }
+        bp = planner.enrich(bp, proj_path)
+        # Merge enriched characters and plot_beats back into the bible
+        if bp.get('characters'):
+            bible['characters'] = bp['characters']
+        if bp.get('plot_beats') and bible.get('books'):
+            bible['books'][0]['plot_beats'] = bp['plot_beats']
+        # Merge enriched style fields back (structure_prompt, content_warnings)
+        bm = bp.get('book_metadata', {})
+        if bm.get('structure_prompt') and bible.get('books'):
+            bible['books'][0]['structure_prompt'] = bm['structure_prompt']
+        if bm.get('content_warnings'):
+            bible['project_metadata']['content_warnings'] = bm['content_warnings']
+    except Exception:
+        pass
 
     with open(os.path.join(proj_path, "bible.json"), 'w') as f:
         json.dump(bible, f, indent=2)