Shandu is a Python deep research agent: ask a question, and a team of LLM agents plans research loops, runs web searches, scrapes pages and documents, scores every source for credibility, and writes a long-form markdown report with numbered citations. It covers the same ground as hosted deep-research tools (OpenAI Deep Research, Perplexity, Gemini), but it is open source and runs on your own machine with any model LiteLLM supports: DeepSeek, OpenRouter, Anthropic, OpenAI, or a local endpoint. Built on the Blackgeorge agent framework, usable from a terminal CLI or a Gradio web GUI.
- Architecture deep dive:
ARCH.md - Example long-form output: see the
examplesdirectory. - DeepSeek Flash example:
examples/deepseek-flash.md
- Lead orchestrator plans iterative research loops (plan, search, synthesize, repeat).
- Parallel search subagents run web search and evidence extraction concurrently.
- Citation subagent builds the final reference ledger.
- SQLite-backed memory tracks run context across steps.
- Rich CLI control deck renders run metrics and timeline.
- Gradio GUI control room provides live telemetry, task views, and report download.
- Scraper pipeline normalizes URLs, strips boilerplate HTML, and favors main-content blocks.
Recommended for end users (no manual venv management):
pipx install shanduStandard pip install:
pip install shanduInstall latest from GitHub:
pipx install "git+https://github.com/jolovicdev/shandu.git@main"uv sync --dev
source .venv/bin/activate
cp .env.example .env
# edit .env with your provider/model settingsshandu configure now asks for:
Default model(example:deepseek/deepseek-v4-flash,openrouter/minimax/minimax-m2.5)API key env var name(example:DEEPSEEK_API_KEY,OPENROUTER_API_KEY,ANYSUPPORTED_API_KEY)API key value(hidden input)
Shandu saves these in user config storage and exports the configured env var at runtime for LiteLLM if it is not already set in your shell.
Examples:
# DeepSeek
shandu configure
# model: deepseek/deepseek-v4-flash
# env var name: DEEPSEEK_API_KEY
# key value: <your key>
# OpenRouter
shandu configure
# model: openrouter/minimax/minimax-m2.5
# env var name: OPENROUTER_API_KEY
# key value: <your key>You can still configure keys only through shell env vars if you prefer:
export OPENROUTER_API_KEY="your_real_key"If you prefer not to use interactive configuration, set env vars directly.
Provider/model:
SHANDU_MODEL(primary model selector, exampledeepseek/deepseek-v4-flash)OPENAI_MODEL_NAME(compatibility fallback ifSHANDU_MODELis not set)
Provider API key routing:
SHANDU_API_KEY_ENV(name of provider key env var, exampleOPENROUTER_API_KEY)SHANDU_API_KEY(actual key value that Shandu exports intoSHANDU_API_KEY_ENVat runtime if missing)
Direct LiteLLM-style provider key env vars (examples):
DEEPSEEK_API_KEYOPENROUTER_API_KEYANTHROPIC_API_KEYOPENAI_API_KEY- Any other provider key name LiteLLM supports, for example
ANYSUPPORTED_API_KEY
Generation/runtime controls:
SHANDU_TEMPERATURE(default0.2)SHANDU_MAX_TOKENS(default16384)SHANDU_STORAGE_DIR(default.blackgeorge)SHANDU_PROXY(optional proxy for scraping)
Precedence:
- If your provider key env var (for example
OPENROUTER_API_KEY) is already set in shell, Shandu uses it. - Otherwise, Shandu uses
SHANDU_API_KEY_ENV+SHANDU_API_KEYfrom config/env.
shandu run "Who is the current president of the United States?" \
--max-iterations 1 \
--parallelism 2 \
--max-results-per-query 2 \
--max-pages-per-task 2 \
--output report.md--parallelism controls the maximum number of subagent tasks that execute concurrently inside each iteration. If set to 2, the lead planner creates at least two independent tasks when possible, and the orchestrator runs up to two tasks at the same time.
During shandu run, progress events stream live in the terminal:
BOOTSTRAP/PLAN/SEARCH/SYNTHESIZE/CITE/REPORT/COMPLETE- Per-task search events (
Task <id> startedandTask <id> completed) with metrics - Iteration index and task IDs for long-running model calls
- Run summary includes model call count across lead/subagents/citation
- Metered calls/tokens/cost appear when provider exposes billing/usage metrics
shandu aisearch "latest state of open-source browser automation in 2026" \
--max-results 8 \
--max-pages 3 \
--detail-level high \
--output aisearch.mdaisearch is the quick mode: a single Perplexity-style answer built from live web
search, with source citations, when a full deep-research run is more than you need.
Citation behavior:
- Final reports enforce numeric citation markers (
[1],[2], ...). - Raw internal evidence IDs are removed from the rendered markdown.
- The final
## Referencessection is rendered from the citation ledger to keep numbering stable.
Other commands:
shandu infoshandu configureshandu guishandu aisearch <query>shandu inspect <run_id>shandu clean
Launch the visual control room:
shandu gui --host 127.0.0.1 --port 7860gradio ships with the default Shandu install, so shandu gui works out of the box.
GUI features:
- live run stage timeline (
BOOTSTRAPthroughCOMPLETE) - per-subagent task board (status, focus, last query, evidence)
- search/scrape trace stream (query start/finish, hit counts, URLs scraped, extraction/fallback signals)
- final report + citation ledger panels
- one-click markdown download button after run completion
- run cost display (
usd_spent) when provider exposes cost metrics - runtime configuration editing (model, provider env var name, key, iteration/parallelism/search limits)
from shandu import ResearchRequest, ShanduEngine
engine = ShanduEngine.from_config()
result = engine.run_sync(
ResearchRequest(
query="AI inference infrastructure 2026",
max_iterations=2,
parallelism=3,
)
)
print(result.report_markdown)
engine.close() # release the shared HTTP sessionuv run ruff check .
uv run pytest -q- Three-layer HTML extraction: trafilatura → readability-lxml → BS4.
- Document-format support: PDF, DOCX, XLSX, CSV, plaintext, markdown.
- Structured blocks preserve headings, tables, code, blockquotes, and list items.
- Per-domain rate limiting with exponential backoff; 3 retry attempts with jitter.
- Fetch-error detection: paywall, captcha, empty JS shell, blocked, login-required.
- Publication-date extraction from OpenGraph, JSON-LD, DC, prism, sailthru, parsely meta tags.
- In-flight deduplication prevents concurrent duplicate fetches of the same URL.
- Redirect-aware: pages tracked by requested URL so redirects don't cause false misses.
A research agent is only as good as what it cites. Shandu grades every page it reads instead of treating all search results as equal, so reports lean on primary sources and peer-reviewed work rather than SEO farms, undated blogs, and marketing pages.
- The per-page extractor classifies each source on the existing extraction call (primary, official, peer-reviewed, journalism, corporate, community, personal blog, advocacy/marketing, aggregator, social profile, or unknown) and records authorship, dating, and whether the page only summarizes work it does not contain.
- Each evidence record carries a
source_class, acredibility_scorederived from those signals, andquality_flagssuch asundated,no_author,promotional,snippet_only, orunassessed. - The synthesizer and reporter weight claims by credibility and name weak-only support explicitly; undated evidence is treated as stale for time-sensitive queries.
- The adaptive loop treats high-confidence but low-credibility evidence as weak, so weak-source corpora keep searching; run stats report source-class counts and the fraction of dated evidence.
- The citation ledger excludes evidence below the credibility bar (weak pages can still inform caveats in prose), merges URL variants of the same work (abstract/HTML/PDF) into one reference, and sanitizes boilerplate titles.
MIT license.