Outpost (Jarvis)
Re-architected Jarvis from a hardcoded specialist-agent prototype into a dynamic Supervisor/Anonymous-Worker production topology across 300+ capabilities, human-in-the-loop write safety, and model-metered AI billing.
Problem
Outpost is a comprehensive business operations platform spanning six critical operational domains: Accounting, Inventory, CRM, Team Management, Field Operations, and Billing. Jarvis is the platform's natural-language conversational AI layer, designed to let operators query system state, execute business actions, and orchestrate cross-department workflows via text.
The original prototype was structured around hardcoded specialist agents. Each business domain was assigned a dedicated, monolithic agent persona with statically attached tools. As the platform expanded to over 300 operations, this design broke down under production constraints:
- Context Window Bloat & Tool Confusion: Declaring hundreds of tools in static agent definitions consumed excessive context tokens and caused the LLM to misidentify tools with similar semantic descriptions across different domains.
- Fragile Cross-Domain Execution: An operator request like "Check inventory for SKU-402, create an invoice for Customer A, and assign Driver B" required daisy-chaining multiple specialist agents. State handoffs between agents were brittle and lacked centralized error recovery.
- Uncontrolled Mutation Risk: Write operations (creating records, modifying financial ledger entries) were executed without transactional idempotency or pre-execution parameter verification, risking accidental or duplicate database writes.
- Unmetered Token Overhead: Token consumption was unmetered at the operation level, making it impossible to enforce granular credit limits or bill customers accurately by model tier.
Architectural Decisions
Supervisor and Anonymous-Worker Dynamic Dispatch
We eliminated the monolithic specialist-agent silos in favor of a dynamic Supervisor / Anonymous-Worker topology orchestrated in LangGraph (JS).
In this architecture, the Supervisor acts as a lightweight triage engine that never executes domain mutations directly. Instead, it inspects the conversation intent, evaluates the operator's runtime RBAC permissions, and dynamically spins up isolated, anonymous worker nodes with precisely filtered toolsets (least-privilege capability filtering). Workers execute their specific sub-tasks in isolated memory contexts and return structured execution receipts to the Supervisor, keeping prompt context windows lean and eliminating cross-domain tool hallucinations.
Human-in-the-Loop Write Safety & Entity Resolution
Allowing an LLM to mutate production databases requires absolute write safety. We instituted a strict architectural invariant: exactly one write operation per execution step.
Every mutating action is fingerprinted from the operation, its exact arguments, and the conversation turn — a retried or duplicated request returns the original result instead of executing again. Raw database IDs (such as cust_9481 or inv_0291) never reach the model or the operator. We built an Entity Resolution Engine that intercepts planned write arguments, resolves them to human-readable entity labels instead of raw IDs, and renders structured, interactive approval cards in the UI before any write transaction commits.
Multi-Tier Company Roster Cache
Jarvis requires accurate knowledge of the company roster, operational departments, and employee roles on every turn to route queries accurately. Querying MongoDB on every token generation step was unacceptable for latency.
We engineered a multi-tier roster cache using Mongoose and Redis: debounced cache invalidation on org mutations, lazy background TTL revalidation, and per-turn in-memory memoization. This feeds denormalized organizational context directly into prompt cache-prefixes, reducing token processing latency while ensuring zero stale authorization reads.
Parallel Read-Chain Executor & Scratchpad Planner
Analytical user queries frequently require reading data from multiple independent endpoints (e.g., comparing historical sales volume against current warehouse stock). Executing these reads sequentially inflated time-to-first-token (TTFT).
We implemented a parallel read-chain executor backed by a scratchpad planning engine. Read operations execute concurrently with pause/resume retry semantics, streaming aggregated context over Server-Sent Events (SSE) directly to the client interface.
AI Billing and Token Metering Engine
To support tiered SaaS pricing, we built a dedicated AI billing service integrating Stripe API webhooks with real-time credit-pool metering. The system meters prompt, completion, and cache tokens per execution turn, pricing each call in fractions of a cent against a per-organization budget, applying model-specific multipliers across tiers (Google Gemini Pro vs. Flash), and gating capabilities cleanly when a budget is exhausted rather than degrading silently.
Capability Definitions Derived Directly from Platform Routes
Rather than hand-authoring and separately maintaining a set of AI tool definitions, capability schemas are generated by reading the platform's own route and validation definitions directly. When a business operation changes on the platform, the agent's available capabilities update with it automatically — there's no second definition of an operation that can silently drift out of sync with the real one.
Key Challenges
The most demanding challenge was balancing execution autonomy with deterministic reliability. Multi-turn agent chains can accumulate compounding errors if intermediate tool outputs are ambiguous. By enforcing strict Pydantic/Zod schemas on all tool return payloads and restricting workers to read-only loops until explicit confirmation was granted, we eliminated cascading failure states.
Additionally, managing SSE connection stability across long-running proactive briefing pipelines (aggregating updates from 6 external business providers) required custom reconnection and message-replay mechanisms to prevent state loss during mobile network handoffs.
Outcomes
Jarvis is now the primary conversational interface for Outpost operators, handling real-time operations across inventory, financial summaries, and team dispatch.
- Zero Unintended Mutations: The Entity Resolution and write-approval pipeline has maintained a 100% precision record on verified mutations in production.
- Substantial Latency Reduction: Parallel read execution and prompt prefix caching reduced multi-source query latency by over 40%.
- Extensible Architecture: Adding new business capabilities no longer requires re-prompting or retraining agent personas; new tools are registered in the capability registry and dynamically discovered by the Supervisor.