Drupal Copilot
RAG · tool calling
A working prototype of an LLM tool-calling agent that answers natural-language questions about a Drupal site's content and configuration, backed by a crawler that builds a searchable SQLite knowledge graph.
How it works
The problem
Anyone who works with a large Drupal site ends up asking the same questions over and over: what pages exist about a topic, where a specific component is used, what the current menu or theme configuration is. Answering that means knowing exactly where to look in the Drupal admin. I built Drupal Copilot to answer those questions in plain language instead: a crawler turns a Drupal site's JSON:API into a queryable knowledge graph, and a tool-calling agent sits on top of it so a question gets answered by looking something up, not by guessing.
Architecture
- It's a monorepo with two pieces: a Next.js chat app at the root, and a Knowledge Builder package (Express, SQLite, Drizzle) that owns crawling and storage. The chat app never talks to Drupal directly. Every answer comes from the knowledge graph the crawler built.
- A chat message goes through an API route into an agent loop that calls tools against the Knowledge Builder's REST API, which reads from a SQLite database the crawler populated ahead of time.
- The crawler talks to Drupal's JSON:API, not the database. It discovers the site's actual content types from the API's own index instead of assuming a fixed list, so it adapts to whatever content model a given site exposes.
- The knowledge graph is a real graph: an entities table and a relationships table with typed edges (component, media, taxonomy, menu, reference), not just a flat search index. A separate FTS5 virtual table stays in sync for full-text search.
- The Knowledge Builder exposes two storage implementations behind the same interface: a SQLite-backed one for real use and an in-memory one for tests.
Design decisions
Self-discovering content types
I built the crawler to read the JSON:API root's own links and classify every endpoint it finds by naming pattern instead of hardcoding the content types I happened to be building against. It's the one decision in this codebase that would let the same crawler point at a different Drupal site's content model without a code change.
A tool-calling agent with a hard step cap
The agent gets nine tools (search, get entity, find related entities, aggregate, page structure, site settings, and a couple of graph-inspection tools) and a 10-step cap. The first step forces a tool call, so the model has to look something up before it answers; later steps let it decide when it has enough. If it hits the cap, I show an explicit message instead of letting it fail silently.
A grounding heuristic for citations, not embeddings
Sources are capped at 3 per tool call by a structural score, then re-ranked at the end of the turn by whether the entity's title or path actually shows up as a substring in the generated answer, capped again to 8 for display. It's a cheap heuristic, not semantic matching. I chose it over adding an embeddings step because it's good enough for grounding citations without a second model call.
A CMS-provider interface, used for real
Crawling and relationship-resolving go through a provider interface with a single Drupal implementation today. The crawler and graph code depend on that interface, not the Drupal-specific class directly, so a second CMS would slot in at that seam instead of needing a rewrite.
Bulk-write mode for the crawl
A full crawl defers the FTS index sync until the end instead of updating it after every single entity write, then rebuilds it once. Syncing on every write was the obvious way to build it. It just doesn't scale to a full site crawl.
AI architecture
- The agent gets nine tools as OpenAI-style function schemas: search, get entity, find related entities, aggregate, page structure, site settings, and graph inspection. The model decides which to call and in what order.
- A regex-based classifier catches site-configuration questions before the model does and injects a pre-fetched settings snapshot straight into the system prompt, skipping the tool round-trip entirely for that class of question.
- Switching between OpenAI and OpenRouter is a config change, not a code change, since OpenRouter is OpenAI-API-compatible. It's provider switching through a base URL and headers, not a multi-implementation interface the way the CMS side is.
- Streaming is structured, not token-by-token. Every trace event, an LLM call starting, a tool call resolving, the answer forming, is encoded as its own line as it happens, so the client sees the agent's reasoning steps live instead of just watching text appear.
- Trace data lives in memory for the life of one request. Nothing about how the agent reasoned is persisted after the response finishes.
Security
- Drupal auth supports either a static API token or a full OAuth2 client-credentials flow with a cached token, refreshed shortly before it actually expires.
- The Knowledge Builder API is locked to localhost by CORS, and that's the only access control it has. There's no per-request auth on top of it.
- The chat app itself has no login or session layer. It's built to be run by one person against their own Knowledge Builder instance, not served to multiple untrusted users.
- Auth on the app's own API, an origin allowlist for the embed widget, and inbound rate limiting are the non-negotiable list before I'd point this at a site I don't fully control.
Constraints & tradeoffs
Concurrent writes during a crawl
The SQLite driver I'm using is synchronous under the hood and my ORM wraps it asynchronously, so concurrent writes during a crawl could race. I added a write queue that serializes them instead of trying to make SQLite handle concurrency it isn't built for.
No auth on the API or the chat app
The Knowledge Builder's API and the chat app itself have no authentication. The only real boundary is CORS locked to localhost, which is a fine boundary for a single-operator local tool and not one for a multi-tenant product. If this needs to serve more than one person, that's the first thing I'd add, not an afterthought.
The embed widget has no origin restriction
The embed script drops a chat widget into any page that includes it, iframed against the app's own embed route. There's no sandbox attribute, no postMessage origin check, and no allowlist on that route. Any site can embed it today. I know exactly what's missing here. I haven't built it yet.
The agent loop itself is untested
Test coverage is real and concentrated in the Knowledge Builder's pure logic: relationship building, pagination, discovery, normalization, the query engine. The agent loop and the LLM service, the most stateful and complex part of the system, have no tests yet.
Rate limits are handled outbound, not inbound
The crawler retries against Drupal with backoff and a configurable rate-limit delay, and it specifically detects the case where an expired session returns an HTML login page instead of JSON, surfacing that as a clear hint instead of a raw parse error. None of that protection exists on the app's own API surface yet.
Deployment
- This runs locally today. The chat app and the Knowledge Builder API run as two separate processes (npm run dev and a second dev command), with SQLite as a single file on disk.
- There's no Dockerfile, no CI, and no hosting config anywhere in the repo. Getting the crawl-to-answer pipeline right has been the goal so far, not deploying it anywhere beyond my own machine.
Current state
Drupal Copilot is Phase 1 of a two-phase plan. It can answer questions about content, structure, and site configuration, and I've written down exactly what it can't do yet rather than leaving that implicit. It's a local, single-operator tool right now, built in a focused implementation push rather than iterated over a long commit history, so I treat it as a working prototype that proves the architecture end to end, not a system that's carried real production usage yet.