# Intercal Intercal is an open, provenance-backed temporal knowledge substrate for agents and LLM apps. It serves public docs, REST, generated OpenAPI, and MCP from https://intercal.jami.studio. Source exports are generated from 12 source-owned pages listed in docs/public/manifest.json. --- # Introduction Intercal is an open, provenance-backed temporal knowledge substrate for agents and LLM apps. It turns source documents into extracted claims, resolved entities, typed temporal relationships, and append-only bitemporal fact versions. The public product surface is same-origin on `https://intercal.jami.studio`: - Human routes under `/` for entities, topics, graph/timeline exploration, evidence search, deltas, claim verification, freshness, coverage, subscriptions, feedback, and operator readouts. - REST under `/api/v1/*`. - Generated OpenAPI at `/api/openapi.json`. - MCP Streamable HTTP at `/api/mcp`. - Documentation under `/docs`. - Agent-readable exports at `/llms.txt` and `/llms-full.txt`. - Crawl metadata at `/sitemap.xml`, `/robots.txt`, canonical page metadata, structured data, and a source-owned share image route for OpenGraph/Twitter previews. Intercal does not maintain a separate docs-only knowledge model. Public UI, REST, SDK, and MCP all read the same contracts and query layer. If a public page cannot back a statement with evidence, it must show an explicit unknown, unavailable, stale, thin, or coverage-gap state. The `/ai-history` page gives crawlers and agents a concise public explanation of Intercal's role without duplicating the docs export or claiming broader coverage than the live corpus gates prove. ## Current launch posture The live code proves the V1 query surface: `get_entity`, `get_sources`, `get_freshness`, `search_evidence`, `get_delta`, and `verify_claim`. The corpus has a bounded reviewed broad AI-history proof slice and quality gates; it is not a claim of continuous full-web saturation. Public coverage language should stay no broader than the last passing live corpus gate. ## Source of truth - TypeSpec in `packages/shared/typespec/main.tsp` owns the contract. - SQL migrations in `db/` own schema, constraints, indexes, and seeds. - `@intercal/core` owns read semantics shared by REST, SDK, MCP, and dashboard pages. - `services/shared` owns adapter ports and provider-swappable source/LLM/storage/queue boundaries. - Durable docs explain the live system; they do not replace it. --- # Concepts Intercal models knowledge as cited temporal state, not as free-form summaries. ## Source documents A source document is an immutable evidence unit from an adapter-backed source. Source rows carry license and redistribution posture. Source documents snapshot that posture at ingest time so later policy edits do not silently rewrite what may be exposed. ## Claims A claim is an atomic factual assertion extracted from source evidence. Claims carry confidence, status, contradiction state, valid-world time, and transaction time. Public claim output must include citation paths or an explicit unavailable state. ## Entities Entities are conservative resolutions of mentions into canonical things: organizations, products, events, concepts, legislation, technical artifacts, datasets, jurisdictions, people, roles, and sources. False merges are corruption, so resolution is intentionally conservative. ## Relationships Relationships are typed temporal edges between entities. They are derived from claims and evidence, not invented by the UI. Relationship status and valid-time windows make point-in-time reads possible. ## Fact versions Fact versions are append-only bitemporal records. They let Intercal answer both "what was true in the world at this date?" and "what had Intercal recorded by this date?" ## Digests Digests are delivery artifacts, not canonical facts. `get_delta` produces token-budgeted cited summaries over already-recorded evidence and graph state. The digest does not become the source of truth. --- # Quickstart Use Intercal from the public UI first, then choose REST, SDK, or MCP for agent/application access. ## 1. Open the public surface Go to `https://intercal.jami.studio`. Useful starting routes: - `/entity/ChatGPT` - `/topic/frontier%20LLMs` - `/delta?topic=frontier%20LLMs&since=2023-03-01T00:00:00Z` - `/verify?claim=GPT-4%20Turbo%20supports%20a%20128k%20context%20window` - `/coverage` ## 2. Fetch the generated contract ```powershell Invoke-WebRequest "https://intercal.jami.studio/api/openapi.json" -UseBasicParsing ``` The OpenAPI document is generated from TypeSpec. Do not copy schemas out of docs prose; use the generated contract. ## 3. Call REST ```powershell $base = "https://intercal.jami.studio/api" Invoke-WebRequest "$base/v1/freshness?topic_or_entity=MCP%20protocol" -UseBasicParsing ``` Anonymous reads are allowed under a tight per-IP limit. A valid API key raises the rate limit and is required for subscription management. ## 4. Use the SDK ```ts import { IntercalClient } from "@intercal/sdk"; const client = new IntercalClient({ baseUrl: "https://intercal.jami.studio/api" }); const delta = await client.getDelta({ topic: "frontier LLMs", since_date: "2023-03-01T00:00:00Z", token_budget: 300, }); ``` ## 5. Connect an MCP client Point an MCP Streamable HTTP client at: ```text https://intercal.jami.studio/api/mcp ``` The server exposes the same V1 query semantics as REST through generated JSON Schema tool inputs. --- # MCP Intercal exposes MCP over Streamable HTTP at: ```text https://intercal.jami.studio/api/mcp ``` The MCP server is stateless on the Vercel route. Each request builds a fresh server/transport over the shared query layer, so there is no per-session state to pin to a serverless instance. ## Tools The V1 tool list is owned by `packages/shared/src/index.ts` and uses generated JSON Schemas for inputs: - `get_delta` - `get_entity` - `search_evidence` - `verify_claim` - `get_sources` - `get_freshness` Every tool maps to the same query semantics as its REST counterpart. Structured failures use the same error taxonomy as REST in the MCP tool result. ## Authentication MCP uses an OAuth 2.1 resource-server posture when `MCP_OAUTH_ISSUER` is configured. When no Authorization Server is configured, the current public-read posture allows anonymous reads. Invalid tokens never fail open when auth is enabled. Protected Resource Metadata is served from the well-known routes only when MCP OAuth is configured: - `/.well-known/oauth-protected-resource` - `/.well-known/oauth-protected-resource/api/mcp` ## Local smoke ```powershell node scripts/dev/verify-mcp.mjs https://intercal.jami.studio/api/mcp ``` Run this against a deployed or local endpoint. It proves transport availability; it does not replace live corpus quality gates. --- # REST REST is mounted under: ```text https://intercal.jami.studio/api/v1 ``` The generated OpenAPI document is served from: ```text https://intercal.jami.studio/api/openapi.json ``` ## Contract source TypeSpec in `packages/shared/typespec/main.tsp` is the contract source. It generates OpenAPI 3.1, JSON Schema, TypeScript types, and the SDK type aliases. Docs link to the generated contract instead of duplicating schemas. ## Read endpoints - `GET /v1/entity` - `GET /v1/sources` - `GET /v1/freshness` - `GET /v1/evidence` - `GET /v1/delta` - `GET /v1/claims/verify` ## Feedback and subscriptions Feedback creates audited review records and does not mutate canonical graph state: - `POST /v1/feedback` Subscription management requires an API key: - `GET /v1/subscriptions` - `POST /v1/subscriptions` - `POST /v1/subscriptions/poll` - `POST /v1/subscriptions/dispatch` - `POST /v1/subscriptions/delete` ## Errors Errors use the generated `ApiError` shape with `code`, `message`, and optional `details`. Known codes include `invalid_request`, `unauthorized`, `forbidden`, `not_found`, `rate_limited`, `not_implemented`, and `internal_error`. --- # SDK The TypeScript SDK is `@intercal/sdk`. It is a thin client over the generated REST contract and does not declare domain shapes of its own. ## Install from this workspace Inside the monorepo, packages consume it through the pnpm workspace: ```json { "dependencies": { "@intercal/sdk": "workspace:*" } } ``` Published package installation is a release concern. Until a package is published, external consumers should use REST or vendor the workspace intentionally. ## Client ```ts import { IntercalClient } from "@intercal/sdk"; const client = new IntercalClient({ baseUrl: "https://intercal.jami.studio/api", maxRetries: 2, retryBackoffMs: 250, }); const verification = await client.verifyClaim({ claim_text: "GPT-4 Turbo supports a 128k context window", as_of_date: "2024-04-01T00:00:00Z", }); ``` ## API keys Pass a key only when you need raised limits or scoped surfaces: ```ts const client = new IntercalClient({ baseUrl: "https://intercal.jami.studio/api", apiKey: process.env.INTERCAL_API_KEY, }); ``` Do not commit raw keys. The server stores only key hashes and never returns a raw key after issuance. --- # Authentication Intercal is a public read substrate. Authentication raises limits and unlocks scoped surfaces; it is not required for ordinary read queries today. ## REST Anonymous reads are allowed for `/api/v1/*` under a tight per-IP rate limit. A valid bearer API key raises the rate limit and unlocks scoped operations such as subscription management. ```http Authorization: Bearer ical_sk_... ``` Bad credentials return `401`; they do not fall back to anonymous mode. API keys are generated by the operator CLI, stored as SHA-256 hashes, and shown only once at issuance. Rotation means issuing a new key, handing it off, then revoking the old key id. ## MCP MCP uses OAuth 2.1 resource-server validation when an Authorization Server is configured by environment. The resource server validates bearer JWT access tokens against issuer, audience, expiry, signature, algorithm allowlist, and required scopes. When `MCP_OAUTH_ISSUER` is unset, MCP keeps the public-read posture and does not advertise Protected Resource Metadata. ## Subscriptions Subscription create, list, poll, dispatch, and delete require an API key. Keys are passed to the SDK/REST call and must not be persisted in browser state or echoed in public output. --- # Examples These examples use the generated contract and the live V1 operation names. They are checked by `pnpm docs:check` for route and operation drift. ## REST: delta ```http GET /api/v1/delta?topic=frontier%20LLMs&since_date=2023-03-01T00:00:00Z&token_budget=300 Host: intercal.jami.studio Accept: application/json ``` ## REST: claim verification ```http GET /api/v1/claims/verify?claim_text=GPT-4%20Turbo%20supports%20a%20128k%20context%20window&as_of_date=2024-04-01T00:00:00Z Host: intercal.jami.studio Accept: application/json ``` ## SDK: same calls ```ts import { IntercalClient } from "@intercal/sdk"; const client = new IntercalClient({ baseUrl: "https://intercal.jami.studio/api" }); const delta = await client.getDelta({ topic: "frontier LLMs", since_date: "2023-03-01T00:00:00Z", token_budget: 300, }); const verification = await client.verifyClaim({ claim_text: "GPT-4 Turbo supports a 128k context window", as_of_date: "2024-04-01T00:00:00Z", }); ``` ## MCP: tool call payloads ```json { "name": "get_delta", "arguments": { "topic": "frontier LLMs", "since_date": "2023-03-01T00:00:00Z", "token_budget": 300 } } ``` ```json { "name": "verify_claim", "arguments": { "claim_text": "GPT-4 Turbo supports a 128k context window", "as_of_date": "2024-04-01T00:00:00Z" } } ``` ## OpenAPI Fetch the generated contract instead of copying schemas from this page: ```powershell Invoke-WebRequest "https://intercal.jami.studio/api/openapi.json" -UseBasicParsing ``` --- # Source Policy Source policy controls what Intercal may fetch, store, and expose. ## Source-row posture Every source row carries license and redistribution posture: - `license_spdx` - `license_notes` - `redistribution_allowed` - `summary_allowed` - `citation_only` - `rate_limit_requests_per_minute` The policy booleans are snapshotted onto each `source_documents` row at ingest time. That snapshot travels with the immutable evidence unit and controls exposure. ## Before store Citation-only sources do not persist full body text. Sources without redistribution approval do not archive raw bytes to object storage. Intercal can still retain source metadata and cited derived facts where policy allows. ## Before expose Public responses may show citation metadata and policy-allowed derived snippets. They must not show raw source bodies. Restricted body text is neither emitted as a snippet nor searchable as an existence oracle. ## Fetch safety Built-in source adapters validate configured endpoints through the shared SSRF guard before fetch. The guard allows only `http` and `https`, rejects private/metadata/link-local/reserved destinations, pins connections to validated IPs, revalidates redirects, and enforces timeouts and body-size caps. ## Public page rule Dashboard pages consume the served shape from the query layer. They do not reimplement source policy and do not invent fallback source text when a citation body is unavailable. --- # Provenance Every publicly served fact must trace to evidence. The evidence chain is: ```text source row -> source document -> claim evidence -> claim -> entity / relationship / fact version -> public response ``` ## What public routes can show Public pages and APIs may show: - structured claims returned by the query layer; - citations with `sourceDocumentId`, URL, and published date; - source-document metadata returned by `get_sources`; - derived snippets only when source policy permits; - explicit unknown, stale, thin, unavailable, or coverage-gap states. They must not show raw source bodies. ## Bitemporal behavior Point-in-time reads use both valid-world time and transaction time. For example, `verify_claim` with `as_of_date` evaluates the claim against evidence available for that date and returns `unverified` rather than inventing support when the substrate has no matching cited evidence. ## Feedback Feedback creates review records for operators. It does not mutate canonical claims, entities, relationships, or fact versions. ## Contradictions Contradictions are explicit graph state. `verify_claim` uses recorded contradiction state and deterministic matching over cited evidence; it does not ask an LLM to decide unsupported facts from scratch. --- # Corpus And Backfill Coverage Intercal's launch target corpus is AI history from January 2023 onward: model releases, model cards, lab announcements, research papers, standards, SDK/framework releases, benchmarks, regulation, runtime infrastructure, and selected MediaWiki revisions. Coverage remains limited to reviewed sources and the last passing live corpus gate. ## Current proof status The current live code has: - historical adapters behind `SourcePort`; - reviewed first-proof and broad-corpus seed/catalog scripts; - quality gates for seeded proof, live first proof, and live full proof; - query proofs for entity point-in-time reads, freshness, deltas, claim verification, evidence search, contradictions, and source-policy restricted-body behavior. The broad live proof is a bounded reviewed slice. It is enough to prove machinery and public query paths. It is not continuous full-web saturation and should not be described that way. ## Quality gates Run gates against a migrated database: ```powershell node scripts/dev/verify-corpus-quality-gates.mjs seeded-proof node scripts/dev/verify-corpus-quality-gates.mjs live-first-proof node scripts/dev/verify-corpus-quality-gates.mjs live-full ``` Seeded mode rolls back probe rows. Live modes require reviewed source rows and backfilled evidence in the target database. ## Backfill posture Historical backfill uses the same worker path as scheduled ingestion. Operators bound it by source class, date range, source count, document count, and dry-run/apply mode. It must create provenance-bearing source documents, claims, evidence links, and fact versions. It must not insert shortcut facts that bypass the provenance chain. ## Public coverage language Public coverage claims must stay no broader than the last passing live gate. A passing seeded proof proves the verifier and query machinery only. --- # Operations Transparency Intercal is designed to run on swappable provider-backed ports while exposing honest operational state. ## Hosted topology The accepted hosted topology is: - Vercel for dashboard, REST, OpenAPI, and MCP on one domain. - A PostgreSQL + pgvector canonical store; the production provider is not selected yet and Neon is excluded for the launch corpus. - GitHub Actions for routine scheduled pipeline runs. - Cloud Run Jobs for heavy or on-demand pipeline execution. - Cloudflare R2 as the accepted object-storage target behind the S3 storage adapter. - Upstash Redis behind queue/cache and rate-limit ports. Storage, queue/cache, LLM, embeddings, and database provider swaps should be configuration and adapter changes, not schema rewrites. Front-door compute swaps are separate release proofs: the current launch uses Vercel-specific route mounts and trusted client-IP header assumptions that must be revalidated before moving production traffic to Cloudflare Workers or Pages. A prior operator record reports an R2 bucket metadata check for the `intercal` bucket in `ENAM` with Standard storage. It is historical metadata evidence, not a fresh source-document object write/read smoke through the S3 adapter; the production-readiness plan requires it to be revalidated before a live backfill. Missing provider telemetry is reported as unavailable, not guessed. ## Budgets Resource budget controls live in `docs/operations/resource-budget.md` and env-driven knobs. Backfill and scheduled ingestion should use bounded source counts, document counts, LLM request budgets, and local embeddings where possible. ## Observability Operator health reads come from SQL-owned views and append-only usage tables. Missing provider telemetry is reported as unavailable, not as zero. Do not insert guessed provider usage. ## Secret lanes App runtime secrets live only in `.env`, Vercel env, GitHub Actions secrets, Cloud Run Secret Manager, and provider secret stores. Operator credentials such as Vercel, Cloudflare, the selected database provider, and GCloud control-plane tokens are not product runtime auth. ## Smoke checks ```powershell Invoke-WebRequest "https://intercal.jami.studio/api/openapi.json" -UseBasicParsing Invoke-WebRequest "https://intercal.jami.studio/api/v1/freshness?topic_or_entity=MCP%20protocol" -UseBasicParsing node scripts/dev/verify-mcp.mjs "https://intercal.jami.studio/api/mcp" ``` Run corpus quality gates separately when validating coverage claims.