Lead pipeline: local SQLite store as source of truth

Form submissions now land in /data/scout73.db before anything leaves the
box. The Google Sheet and the ntfy push become mirrors whose per-record
outcome is recorded, so a failed sheet write is replayable instead of
surviving only as a push telling you to retype it.

The table is a generic record store keyed by 'kind', with workflow columns
and an audit trail, so RSVPs and other forms can land in the same place
later without a migration. Admin panel talks to /api/admin over HTTP and
never opens the DB file.

- app/store.py     schema, writes, reads, one-time leads.jsonl backfill
- app/admin_api.py token-gated API; fails closed when ADMIN_TOKEN is unset
- app/app.py       three hunks: imports, boot init, join_post
- compose          ADMIN_TOKEN passed through from the Portainer stack env
This commit is contained in:
Mike Wichers
2026-08-26 19:38:33 -04:00
parent 5331e38d62
commit be2d81b3ab
4 changed files with 506 additions and 0 deletions
+26
View File
@@ -4,6 +4,11 @@ from fastapi import FastAPI, Form
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.staticfiles import StaticFiles
# Internal record store + admin API. The store is the source of truth for
# anything a family submits; see store.py for why.
import store
import admin_api
# ---------------------------------------------------------------------------
# Pack & Troop 73 - greenlanescouts73.org
# Calendar lives in ONE file: /data/events.json (seeded from app/events.json on
@@ -29,6 +34,10 @@ SHEET_HEADERS = ["Timestamp", "Parent Name", "Email", "Phone",
app = FastAPI(title="Pack & Troop 73")
app.mount("/static", StaticFiles(directory="/app/static"), name="static")
# Creates the schema if absent and backfills the legacy leads.jsonl once.
store.init(backfill_jsonl=LEADS)
app.include_router(admin_api.router)
if not EVENTS_LIVE.exists() and EVENTS_SEED.exists():
try: shutil.copyfile(EVENTS_SEED, EVENTS_LIVE)
except Exception: pass
@@ -628,13 +637,30 @@ def join_post(parent_name: str = Form(...), email: str = Form(...), phone: str =
f.write(json.dumps(rec) + "\n")
except Exception:
pass
# Land it internally FIRST. Everything below this point is a mirror of a
# row that already exists locally, and each mirror's outcome is recorded
# against the record so a failure is replayable instead of just shouted.
record_id = None
try:
record_id = store.insert_record(
kind="join_lead", source=rec["source"], payload=rec,
unit=store.unit_of(rec["interested_in"]),
display_name=rec["parent_name"], email=rec["email"],
phone=rec["phone"], created_at=rec["ts"])
except Exception as e:
print("STORE WRITE FAILED for %s <%s>: %s" % (rec["parent_name"], rec["email"], e), flush=True)
sheet_error = None
try:
sheet_append(rec)
except Exception as e:
sheet_error = str(e)
print("SHEET WRITE FAILED for %s <%s>: %s" % (rec["parent_name"], rec["email"], e), flush=True)
if record_id:
store.set_mirror(record_id, "google_sheet", sheet_error is None, sheet_error)
delivered = pipeline_notify(rec, sheet_error)
if record_id:
store.set_mirror(record_id, "ntfy", bool(delivered),
None if delivered else "ntfy push not delivered")
if sheet_error and not delivered and NTFY_URL:
# last-ditch push on the fallback topic so a lead is never silently lost
try: