🧬 Commentarium

🎨 Live UI demo: farazjz.github.io/Commentarium — static design preview of the SPA (fake data, no backend).

🚧 v2.0 branch — "Research Studio UI" · This branch (v2.0) modernises the frontend: a dark/light single-page app served by the same FastAPI backend (Streamlit remains available as a legacy fallback). The current stable release of the v1.x line is tag v1.9.0 (development Phases 1–9 complete).

v2 progress: ✅ Settings & jobs API · ✅ SPA shell + live Dashboard · ✅ OCR API · ✅ Library view + upload with live ingest · ✅ Chat with sources & citations · ✅ Citations workflow + bibliography export · ✅ Podcast studio (async generation + waveform player) · ✅ OCR studio view · ✅ Settings (.env writer) + live Logs · ✅ Background launchers (run.bat / stop_all.bat) — feature complete, released as v2.0.0.

A local research assistant for your deep learning, thesis and scientific writing. Upload PDF, Word, TXT and Markdown documents, chat over them with the AI model of your choice, and get answers with page-level citations in APA 7 or Vancouver style. Scanned PDFs and images are handled by the OCR studio.

Built as a lean, thesis-focused alternative to open-notebook — privacy-first, runs entirely on your machine, using your own OpenAI-compatible API key — any online router (OpenRouter, Gemini, OpenAI…) or a fully local AI (LM Studio, Ollama, vLLM, LiteLLM…).

---

✨ Features

• 🔒 100% local & private — your files never leave your computer
• 🖥️ Research Studio SPA (v2) — a dark/light single-page app served directly by the FastAPI process (no build step, no node): live Dashboard, Library, Chat, Podcast studio, OCR, Citations, Settings and a Log viewer in one sidebar; command palette (Ctrl/Cmd + K, Enter runs, Esc closes); live Dashboard quick actions (Upload / Chat / Podcast / OCR / Settings / Log viewer / Exit & shut down)
• 📄 Upload PDF, Word (DOCX/DOC), TXT and Markdown files — see Supported documents below; images and scanned PDFs go through the OCR studio first
• 🤖 Bring your own AI — paste any OpenAI-compatible key from an online router or a local server (OpenRouter, LiteLLM, LM Studio, Ollama via its OpenAI bridge, vLLM…), pick any chat model and embedding backend (local sentence-transformers is free & private)
• 🗂️ Projects to organise research (e.g. per thesis chapter)
• 💬 Chat over your documents, with grounded, page-level citations
  • resolved APA 7 / Vancouver references shown with each answer
• 🎛️ Per-project model control — each project can override the chat model and temperature (falling back to global defaults), plus a global temperature setting
• 📝 Citations — auto-extract metadata (title/authors/year/journal/DOI via Crossref), verify it, then render APA 7 or Vancouver references
• 🏷️ Tags & notes per document — freeform tags (filterable) and personal notes on every paper
• 🧠 Multi-kind per-document summaries — Brief, Detailed, Key points, TL;DR, and a study Quiz, all cached so they don't burn tokens twice
• 🎙️ Podcast studio (NotebookLM-style) — select papers, and the app writes a two-host "deep dive" conversation and reads it aloud as an MP3. Choose the TTS: edge-tts (free, no key), VoiceStudio (locally running voice cloning/design server), or any OpenAI-compatible /v1/audio/speech server (Kokoro / Silero / Piper / vLLM — fully local & private), with per-host voice ids / languages and model/format controls (transcript-only fallback)
• 🔍 OCR to Markdown — stand-alone page that turns scanned PDFs and images into clean Markdown files with per-page structure, using the OCR backend of your choice (live preview + download)
• 🛡️ Embedding-dim guard — prevents silent corruption if you change embedding models; Projects page shows corpus-model identity and warns on mismatch
• 🪵 Log viewer — live, filterable logs (file + console + in-app) for debugging

📄 Supported documents

Where	Formats	What happens
Library (upload → ingest → chat/citations)	.pdf · .docx · .doc · .txt · .md / .markdown / .text	Parsed (PyMuPDF for PDFs, python-docx for Word, direct read for text/Markdown), chunked with page/section mapping, embedded into the local NumPy + SQLite vector store
OCR studio (→ Markdown files)	.pdf (incl. scanned) · .png · .jpg / .jpeg · .webp · .bmp · .tiff	Extracted page-by-page (smart text read or forced full-OCR rasterization) with your chosen OCR backend, saved as clean .md files in data/ocr_output/

Notes:

• Anything the OCR studio produces (.md) can be uploaded to the Library like any other document — that's the path to make scans/image-only papers searchable and chat-able.
• purely text-based PDFs parse without OCR; scanned ones automatically fall back to OCR when a backend is configured (see troubleshooting).
• .doc (legacy Word) is accepted and routed to the Word parser; for best fidelity convert to .docx.

🧱 Architecture

Animated overview — a document flows down through ingest, embedding, retrieval, answering and citation:

Plain-text version of the diagram

Your files (PDF/DOCX)
      │  upload
      ▼
┌─────────────────────┐
│  Ingest pipeline     │  parse (PyMuPDF / python-docx)
│  parse → chunk       │  smart chunking w/ page mapping
│  embed → store       │  local embeddings → NumPy/SQLite vector store
└─────────┬───────────┘
          ▼
┌─────────────────────┐
│  Retrieval (RAG)     │  cosine search → top-k chunks
│                      │  each chunk carries file + page(s)
└─────────┬───────────┘
          ▼
┌─────────────────────┐        ┌─────────────────────┐
│  Answer generation   │ ─────▶ │  Citation layer      │
│  (your AI gateway)   │        │  APA 7 / Vancouver   │
└─────────────────────┘        └─────────────────────┘
   answer + in-text markers → resolved to real Reference list

• Backend: FastAPI (also serves the v2 SPA; web/ — no build step)
• Frontend: vanilla-JS single-page app (v2, default) · Streamlit (v1 legacy fallback, still available)
• AI: your OpenAI-compatible chat gateway — online router (OpenRouter, Gemini…) or fully local (LM Studio, vLLM, Ollama proxy, LiteLLM…)
• Embeddings: sentence-transformers (local) or your gateway's /embeddings
• Vector store: custom lightweight NumPy + SQLite (no C-compiler needed)
• Metadata store: SQLite (projects, documents, citation metadata)
• Parsers: PyMuPDF, python-docx (with optional OCR of scanned pages)
• Logging: rotating file + console + in-app ring buffer

Why a custom vector store? On Windows + Python 3.12, ChromaDB needs chroma-hnswlib, which must be compiled from source (Requires Visual C++ Build Tools). At the single-user / 100–300 file scale, direct cosine search over a NumPy matrix is simpler, dependency-free and more than fast enough.

🚀 Getting started

1. Install dependencies

pip install -r requirements.txt

If sentence-transformers needs to download a model on first use, ensure you have internet access (the default local model BAAI/bge-small-en-v1.5 is fetched from Hugging Face once).

2. Configure your AI provider

Commentarium talks to any OpenAI-compatible chat API — an online router or a server running locally on your machine. Copy the template (you can also do this in the UI):

copy .env.example .env

Option A — online router (e.g. OpenRouter). Set a key in .env:

OPENROUTER_API_KEY=sk-or-...

Get one at https://openrouter.ai/keys (a single key gives you access to literally thousands of models — Claude, Gemini, Llama, etc.).

Option B — fully local / offline. No key needed. Point OPENROUTER_BASE_URL (and leave the key empty, or use your local server's key if it requires one) at the OpenAI-compatible endpoint of your local server — e.g. LM Studio (http://127.0.0.1:1234/v1), Ollama (http://localhost:11434/v1), vLLM, or a LiteLLM proxy. The Settings page in the UI can configure the same values.

Embeddings are separate and work the same way: local (free & private sentence-transformers, no network), any OpenAI-compatible /embeddings endpoint (online or local), or Cloudflare Workers AI.

3. Run the app

Double-click run.bat in the project folder (or run it from a terminal):

run.bat

It starts the FastAPI server quietly in the background (no console window, via pythonw) — the v2 SPA is served by that same process — waits for the API port to come online, then opens your browser at http://localhost:8010 automatically. Stop it later with stop_all.bat, or the "Exit & shut down" buttons in the app (sidebar, Dashboard hero, or Settings → danger zone). A root-level copy of both launchers sits one folder above; both work.

The port comes from .env (PORT, default 8010 on this branch; the legacy Streamlit fallback would use UI_PORT, default 8511).

Prefer visible consoles / manual control? Run the server yourself:

python -m uvicorn app.main:app --host 127.0.0.1 --port 8010 --reload

Then open http://localhost:8010 in your browser. The legacy Streamlit UI (v1 look) can still be served with:

streamlit run app/ui/main.py --server.port 8511 --server.address 127.0.0.1 --server.headless true

4. First steps in the UI

1. ⚙️ Settings → paste your AI/gateway key → Test connection → pick your chat model. Local setups: point the base URL at your local server instead of using a key. Set OCR backends (incl. PaddleOCR-VL / TeleOCR) and podcast voices here.
2. 📚 Library → upload your PDF/Word/text papers, add tags/notes, and generate summaries (Brief/Key points/Quiz…).
3. 💬 Chat → ask questions; answers come with page-level citations. Pick a project scope first — it filters threads and drives what the model sees.
4. 📝 Citations → verify document metadata; export APA 7 / Vancouver.
5. 🔍 OCR → upload PDFs or images, extract text via your chosen OCR backend, and export clean Markdown files.
6. 🎙 Podcast → select papers (optionally scoped to a project) and generate an audio deep-dive (MP3 waveform player).
7. 🪵 Logs → debug any errors live.

On the Dashboard, the hero quick-action buttons jump straight to the common tasks (Upload / Chat / Podcast / OCR / Settings / Log viewer / Exit).

---

📁 Project layout

app/
  main.py               # FastAPI entry + endpoints (/health, /projects, /documents, /chat, /citations, /podcasts, ...); serves the v2 SPA
  config.py             # .env settings, storage paths
  logging_setup.py      # rotating file + console + in-app log buffer
  models_openrouter.py  # OpenAI-compatible provider client (test connection, list models, chat, embeddings)
  db/
    vectorstore.py      # NumPy + SQLite vector store (dimension guard, model identity)
    metadata.py         # SQLite projects/documents/citation metadata + tags/notes/summaries/podcasts
    chat.py             # SQLite chat sessions + message history
  ingest/               # parsers (PDF/DOCX/TXT/MD + OCR), chunking, embeddings, pipeline
  rag/                  # retrieval (focused + distributed), prompts, generation, chat engine, summaries
  podcast/              # NotebookLM-style script generation + edge-tts audio
  shutdown.py           # helper to stop the local server (Exit button)
  citations/            # Crossref lookup, APA 7 / Vancouver formatters, resolution, service
  ui/                   # Streamlit pages (v1 legacy fallback: main/Home/Projects/Chat/Podcast/Citations/OCR/Settings/Logs)
web/                    # v2 Research Studio SPA (served by FastAPI at :8010)
  index.html            # app shell: sidebar rail, topbar, context panel, svg sprite
  css/styles.css        # shell + view styles (dark/light themes)
  js/                   # vanilla ES modules, no build step (app.js = shell; views/ = per-page modules)
scripts/                # dev/QA: Playwright browser E2E (test_spa_browser.py), rail geometry probe, etc.
run.bat / stop_all.bat  # launch / stop the v2 server in the background (also present one folder up)
_run_api.py             # pythonw entrypoint used by run.bat
data/                   # git-ignored: uploads, sqlite, vectors, logs, podcasts, ocr_output
docs/                   # README assets (animated pipeline diagram, light + dark)
.env                    # git-ignored: your secrets

🛠️ Development status (phases)

• ✅ Phase 1 — Foundation: project skeleton, config, logging system, FastAPI backend (/health, /settings, /logs), Streamlit UI shell, custom vector store with insert/search/delete + persistence.
• ✅ Phase 2 — Ingestion: PDF/DOCX parsers (PyMuPDF / python-docx), OCR of scanned pages (Tesseract / vision), smart chunking with page + section metadata, local & OpenAI-compatible embedding backends, SQLite metadata store, and full project/document management (create project, upload, ingest, re-index, delete) with a working Streamlit Projects & Files page.
• ✅ Phase 3 — Chat + RAG: query embedding + cosine retrieval (project filter), grounded answer generation with a strict "only from context + [SRC:doc|page] source tags" prompt, in-text source-tag parsing into structured citations, and SQLite-backed chat sessions/history with a working Streamlit Chat page.
• ✅ Phase 4 — Citations: Crossref metadata extraction (by DOI or title search with confidence ranking), user verification workflow, APA 7 + Vancouver (NLM) reference formatters, a resolution layer that turns the [SRC:doc|page] tags from Chat answers into real in-text citations + a bibliography, and a Streamlit Citations page (verify/edit metadata, preview the formatted reference).
• ✅ Phase 5 — Polish: embedding-dimension consistency guard (fail loudly on model switch + corpus-model mismatch banner), corpus model identity tracking, stale-artifact cleanup, .env.example completeness, README QA/troubleshooting, citations & prefill UI bugfixes.
• ✅ Phase 6 — Management + retrieval fix: full project/chat management (rename/edit/delete with cleanup), professional UI redesign (project cards, inline document actions with on-demand summaries), section-weighted retrieval (Reference chunks no longer crowd out real content), plus distributed retrieval (top-K per document) for overview questions — with auto-detect heuristics and a "Specific facts / Summarize papers" toggle in Chat.
• ✅ Phase 7 — Exit / shutdown: a "🛑 Exit" button in the sidebar (and a POST /shutdown API endpoint) that gracefully stops both the API and UI servers via psutil PID lookup on the listening ports.
• ✅ Phase 8 — Research workflow + podcast studio: multi-kind per-document summaries (Brief / Detailed / Key points / TL;DR / Quiz, cached), per-project model & temperature overrides, a global temperature setting, document tags (filterable) + freeform notes, and a Podcast studio that writes a multi-host "deep dive" script over selected papers and reads it aloud (MP3 download; transcript-only fallback), with TTS via edge-tts, a local VoiceStudio server (voice cloning/design, per-host languages en/fa, dropdown voice picking, connection test), or any OpenAI-compatible /v1/audio/speech endpoint. Episode list supports delete (incl. failed / in-progress episodes). OCR also gains the PaddleOCR-VL-1.6 and TeleOCR vision-language backends (local VLM server first, gateway fallback).
• ✅ Phase 9 — One-click launcher + OCR to Markdown: a single run.bat that starts the API (then the UI) hidden in the background, waits for the API port, and opens the browser (replaces run_all.bat / run_api.bat / run_ui.bat; ports come from .env), plus a stand-alone 🔍 OCR to Markdown UI page — upload PDFs/images, per-page extraction (smart text read or forced full-OCR rasterization with adjustable DPI), any configured OCR backend (with a per-run vision-model override), live rendered/raw Markdown preview, individual or zipped downloads, and a saved-file history in data/ocr_output/ with delete support.
• ✅ Phase 10 — Research Studio SPA (v2.0): a fast single-page app in web/ (vanilla ES modules, no build step) served by the FastAPI process — live Dashboard with quick actions, Library with live ingest, Chat with project scope + sources, Podcast studio (async generation, waveform player), OCR studio, Citations, Settings (.env writer) and live Logs, plus a command palette (Ctrl/Cmd + K); dedicated background launchers (run.bat / stop_all.bat, pythonw + hidden console) with a root-level shim. Port-aware shutdown (/shutdown honors .env PORT/UI_PORT and pythonw).

📝 Notes & troubleshooting

• OCR of scanned PDFs requires Tesseract installed (free, local). Set its path in Settings (C:/Program Files/Tesseract-OCR/tesseract.exe).
• Embedding model consistency: the corpus vectors are tied to the embedding model used at ingest time. If you switch models afterwards, ingestion and chat will fail loudly with a "dimension mismatch" error. Fix it by switching the model back in Settings, or Re-index every document with the new model (Projects page → Re-index). The Projects page shows which model produced the current corpus and warns on mismatch.
• Flaky internet: embedding/chat calls and Crossref lookups need connectivity to your gateway / upstream providers. Timeouts and "Bad Gateway" errors usually resolve on retry; check the Log Viewer for details.
• Garbled or missing text in a PDF usually means the pages are scans — install Tesseract and set OCR Backend to tesseract (Settings → OCR & RAG tuning), then Re-index the document. For state-of-the-art document parsing, choose paddleocr-vl or teleocr and point them at a local VLM server (or let them fall back to the gateway's vision model).
• Podcast audio options (Settings → OCR & RAG tuning → Podcast audio):
  • edge-tts requires pip install edge-tts (in requirements.txt) + internet.
  • api uses any OpenAI-compatible /v1/audio/speech endpoint — fully local & private (e.g. Kokoro, Silero, Piper, vLLM). Set the URL, model id, and the two voice ids your server knows (some servers expose /v1/audio/voices; the app validates yours against it). A different voice per host makes them distinct.
  • VoiceStudio uses a locally running VoiceStudio backend (default http://localhost:3900) — fully local voice cloning/design. In Settings, Load voices lists your cloned profiles / OpenAI aliases and you pick each host's voice. Each host's dialogue language (en/fa) is chosen per-episode on the Podcast page — it drives both the script language and the language sent to VoiceStudio synthesis. Keep the VoiceStudio app running while generating (mp3/opus/aac output needs VoiceStudio's bundled ffmpeg; wav/flac/pcm do not). Optionally set a Bearer API key when VoiceStudio runs on another machine. If the script model still writes English for a Persian (fa) episode, the app auto-translates the entire script to Persian before it reaches VoiceStudio, so the audio is guaranteed Persian.
  • Every episode has a 🗑 Delete button (including failed or still-generating ones) that removes the episode and its audio file.
  • If TTS fails or is disabled, the podcast is still generated and saved as a transcript.
• All logs are stored in data/logs/app.log (rotating). Every UI page shows recent errors in a collapsible panel for quick debugging.
• Check .env.example for every available setting.

🔒 Privacy

Everything runs locally. Your API key is stored only in your local .env / database and used only to call your chat gateway (OpenRouter or a fully local provider). With a fully local setup nothing ever leaves your machine. Your documents and embeddings never leave your machine.

🙏 Acknowledgements

Commentarium is built with, and relies on, several open-source projects. Respect their licenses when distributing or modifying this app:

• OpenRouter — optional AI gateway (API key; usage subject to their Terms).
• FastAPI (MIT) — backend web framework.
• Streamlit (Apache-2.0) — UI frontend.
• sentence-transformers (Apache-2.0) — local embedding models (e.g. BAAI/bge-small-en-v1.5, BAAI's own terms apply to the model weights).
• PyMuPDF (AGPL-3.0) — PDF parsing.
• python-docx (MIT) — DOCX parsing.
• Tesseract OCR (Apache-2.0) — scanned-page OCR (installed separately, local).
• PaddleOCR / PaddleOCR-VL (Apache-2.0, repo; the PaddleOCR-VL-1.6 model weights are under separate terms — review them before distributing) — the vision-language OCR backends.
• TeleOCR — the teleocr vision-language OCR backend (check its license before redistributing the model).
• Crossref API (public metadata API, used under their terms) — citation lookup.
• edge-tts (LGPL-3.0) — podcast TTS.
• VoiceStudio (AGPL-3.0) — optional local voice-cloning TTS server (separate app, runs on :3900).
• open-notebook — the project Commentarium was inspired by (thesis-focused, leaner alternative).
• huggingface_hub / Hugging Face models — model downloads; each model's license governs the weights.

If you reuse or extend Commentarium, keep this list current so downstream users know the licenses of the components bundled or required at runtime.