# Arclyx

> An AI agent platform for teams, live at arclyx.ai. Core engineer on evals, observability, the agent runtime, and billing.

2026 → now · core engineer · 320+ merged PRs · evals · observability · agent runtime · agents evals observability rag fastapi

Link: https://arclyx.ai

Arclyx is the AI agent platform Arc10 builds and runs. A team describes a job in one sentence; Arclyx builds the agent, picks its tools, lets them try it on a dry run, then runs it on chat, on a schedule, from a webhook, or in Slack. Agents answer from the company's own documents with citations back to the source. It runs on FastAPI, Next.js, Postgres, and Qdrant, and it is live at [arclyx.ai](https://arclyx.ai). It launched as AuraMind and was renamed Arclyx in October 2026.

It is a team build: I am one of four engineers, with over 320 merged pull requests authored since March 2026. Teammates built the code sandbox, tenant isolation, the MCP client, and the original routing layer. What follows is the part I built.

## Evals that gate every agent change

An agent change ships only after it passes the eval harness. The first version ran suites through a batch runner with baselines that must be able to fail, so a passing score means something; a grader that cannot take down a batch, so one bad case never voids a run; and repeated runs that classify flaky behaviour before release instead of after. In October 2026 I rebuilt it: LLM-judge checks with a human review queue for the verdicts that need one, multi-turn scenarios that test a whole conversation, cost caps on every run, and run-to-run comparison so a regression shows up as a diff.

## Observability: every run accounted for

Each run gets a detail page, analytics, and an LLM-written story of what the agent did, with Sentry behind it and trace content visible only to its owner. Every model request, including side calls like chat titles, summaries, and the model picker, is recorded from one transport with its purpose, whether it finished, failed, or was cut off. Estimates are corrected to what OpenRouter actually charged, and that ledger feeds Stripe usage billing, which passes model costs through at the provider's reported price.

## The agent loop

The runtime is a tool-calling loop with sub-agent delegation, short-lived helpers, token and wall-clock budgets, and run cancel. Agents own their triggers: schedules and webhooks. The new-agent flow takes a plain-language description through describe, clarify, draft, test, and create, streamed, with a spend bound on every call.

## Auto mode

Auto mode routes each request by capability: an LLM decider picks the model, with retry and failover across OpenRouter and AkashML.

## What was hard

Production agents fail in specific ways, and I hardened the runtime against each as it appeared: streamed tool-call deltas reassembled by fragment and slot, hallucinated and near-miss tool names, sub-agent fallbacks that poisoned the parent run, run deadlocks, tool calls cut off by the output cap, provider and connector rate limits, and MCP connector health and token refresh. Under load the problem moved to the database: requests held a Postgres session open across slow model and connector calls, so sessions are now released before every external call and file ingestion runs off the event loop.

## Security

Retrieved content is scanned for prompt injection before it reaches an agent. I ran a platform security review and fixed its auth, credential, and permission findings.

## The website

I also built [arclyx.ai](https://arclyx.ai): the product and feature pages, solutions pages by team, comparison pages, the security page, a dozen product posts, and the SEO plumbing behind them.

Back: https://jigarpatel.ca/#work