Skip to content

AGENTS.md

Latest commit

 

History

History
466 lines (440 loc) · 32.6 KB

AGENTS.md

File metadata and controls

466 lines (440 loc) · 32.6 KB
Raw

AGENTS.md

Onboarding guide for anyone working in this repository — human or AI coding agent. Read this first.

1. What this is

Cloud Agent runs coding agents (Claude Code, Codex, pi, OpenCode, Kilo) in sandboxed containers against a user's repository. When the owner asks, the agent pushes a branch and opens a pull request itself, as that owner, through the platform's GitHub relay.

The extraction from the HybridInference monorepo (HarvardMadSys/hybridInference) is complete: production cut over on 2026-08-07 and the gateway's own agent code was deleted on 2026-08-09. The repository extraction record records the split, its decisions, and deferred work. The extraction plan in the HybridInference repo (docs/agents/plans/2026-08-03-cloud-agent-repo-split.md) is historical; new work tracks in this repository's issues and pull requests.

2. Repo map

apps/
├── backend/
│   ├── cloud_agent/           control plane (FastAPI): API, auth adapter,
│   │                          scheduler, MCP and GitHub relays, storage
│   ├── cloud_agent_host/      host agent: runner, sandbox, runtimes, sandbox manager
│   ├── cloud_agent_shared/    vocabulary both images ship: capability strings,
│   │   │                      protocol versions, pins, sandbox network, shared errors,
│   │   │                      and the operation-profiling collector both sides run
│   │   └── git/               workspace branch names and the source-archive builders
│   ├── migrations/            Alembic migrations (agent-owned database only)
│   └── openapi.json           the committed snapshot of this service's OpenAPI
└── frontend/                  Next.js user UI + admin dashboard
deploy/{control-plane,host,sandbox}/
tests/{unit,servers,integration,external}/  # plus fixtures/ and support/
docs/{developer,user,agents}/  maintained guides, user notes, agent records

Not every module is worth a line here; these are the ones a newcomer needs to find early. The control plane's database schema is one baseline revision plus additive ones (apps/backend/migrations/versions/0001_schema_baseline.py and on), not a long chain — the extraction record's § Schema baseline rewrite explains why, and docs/developer/operations.md § Rebuilding the agent database is the runbook for a database stamped with the chain that predates it.

Module What it owns
apps/backend/cloud_agent/app.py The four listeners: the main API, the MCP-only one, the GitHub relay and the model relay
apps/backend/cloud_agent/api/ One router module per surface — jobs, hosts, admin, session, MCP relay
apps/backend/cloud_agent/schemas/ Domain-organized wire models, explicitly exported by cloud_agent.schemas
apps/backend/cloud_agent/application/ Flows that run without a request: job admission and creation, native Fork materialization, residency credentials
apps/backend/cloud_agent/bootstrap/ Internal assembly: the resources the listeners are built from, and the lifespan that starts and stops them
apps/backend/cloud_agent/workers/ The background loops — attempt reaping, queue and goal recovery, sandbox deletion, notifications and maintenance
apps/backend/cloud_agent/ports/, apps/backend/cloud_agent/storage/ Narrow capability contracts and Postgres repositories: claims, fences, residency, pull-request records
apps/backend/cloud_agent/usage/ (runtime_usage.py, turn_report.py, admin_usage.py, sums.py) What agents reported about their model calls, per job and summed for the Usage pages; no cost or quota (usage)
apps/backend/cloud_agent/accounts/agent_access.py The admission decision (agent.use, agent.admin) every protected request reads
apps/backend/cloud_agent/deployment/identity_namespace.py Binds an empty database to the account system once, and refuses a Gateway-era one (--check reads it without writing)
apps/backend/cloud_agent/accounts/, application/owner_termination.py The account system: sign-in, sessions, the waiting list, the account directory, retired-configuration refusal, and stopping a revoked account's work (guide)
apps/backend/cloud_agent/mcp/, api/mcp_connections.py, api/mcp_relay.py Per-user connections, protocol hosting, authorization and the MCP relay
apps/backend/cloud_agent/mcp/runtime.py Process-local MCP host, fenced session maintenance, and shutdown
apps/backend/cloud_agent/source_control/ (github_relay_policy.py, source_control.py), api/github_relay.py The GitHub relay and its policy, and the GitHub/GitLab credentials behind them
apps/backend/cloud_agent/api/model_relay.py, api/admin_operator_models.py, domain/operator_models.py, domain/model_usage.py The model relay, the models a deployment provides, their prices and each owner's daily budget on them
apps/backend/cloud_agent/schemas/admin_observability.py The cross-user admin reading surface and its audit
apps/backend/cloud_agent_host/service/enrollment.py Host registration and the persisted host credential
apps/backend/cloud_agent_host/service/host_service.py, turn/turn_child.py The host service that claims, heartbeats and drains on SIGTERM, and the child process that executes one turn (runner.py is their command line)
apps/backend/cloud_agent_host/claimed_job.py, control_plane_client.py Claim payloads, lease errors, and attempt-scoped worker HTTP calls
apps/backend/cloud_agent_host/container/chat_sandbox.py, sandbox.py Retained group lifecycle and the Docker/Kata boundary
apps/backend/cloud_agent_host/network/chat_sandbox_network.py Per-group bridge and relay lifecycle, guest network proof, and exact network release
apps/backend/cloud_agent_host/container/chat_sandbox_models.py Retained-container specs, inspection/status values, and identity vocabulary
apps/backend/cloud_agent_shared/egress.py, apps/backend/cloud_agent_host/network/network_guard.py The one sandbox network: its configuration and host rules, and the helper that installs and verifies them
apps/backend/cloud_agent_host/runtimes/, cloud_agent_shared/runtime_catalog.py, shared_protocol.py The five Agent CLI adapters, shared discovery facts, and version pins

Code structure maps module responsibilities and dependency direction, including the shared profiler and frontend API/UI modules. Neither implementation package imports the other; shared facts live in cloud_agent_shared. Lifecycle wiring must not import HTTP routers, and callers outside storage must not import repository implementations.

3. Quality gates

make setup-dev   # uv venv + uv sync
make check       # format + lint + type-check + test — must be green before every PR

make test runs everything except the external and dbtest markers. Database-backed tests are opt-in. When a change touches storage, SQL, a migration, or a database test, run make test-db-local (needs Docker). It uses this machine's shared test PostgreSQL, gives the checkout fresh databases of its own on every run, and fails if any database test was skipped; ARGS="<paths> -x" narrows it. Never run make test-db-local-down: it discards every worktree's databases and fails their runs. make test-db runs the tier against a PostgreSQL you provide (see ENVIRONMENT.md).

For frontend changes, use Node.js 22 and run make ci-frontend. It installs the locked dependencies with npm ci, then checks lint, formatting, types, tests, and build guards. make setup-dev installs backend dependencies only.

The developer documentation site is published in English and Simplified Chinese. When a change edits a page under docs/developer/ or a root guide the site includes (ENVIRONMENT.md, CONTRIBUTING.md, SECURITY.md, deploy/local/README.md, deploy/release/README.md), run make docs-translate, translate the entries it adds or marks fuzzy, and run make docs-verify. On the stable pages (the home page, the developer guide, the code structure page, and the SECURITY.md and CONTRIBUTING.md the site includes), CI fails while a string has no catalog entry or a fuzzy one; on the other pages it warns. No page catches an entry left empty, which renders in English. Translating the docs has the workflow and terminology.

Complexity and size only ratchet down (#771). New code stays within the limits: backend functions within McCabe 15, 12 branches and 50 statements, backend modules within 1,500 lines, frontend functions within complexity 15 and frontend files within 800 lines. The backend limits cover apps/backend only, not tests, scripts, deploy/ or migrations; the frontend limits skip *.test.ts(x), __tests__/ and e2e/.

  • A backend function over a limit today carries a noqa naming #771. Never add one to new code. Once the function is back under, make check deletes the noqa (ruff's RUF100 fix).
  • Oversized backend modules are listed in tests/unit/test_file_size_budget.py and may not grow. Lower an entry when its module shrinks.
  • tests/unit/distribution_neutrality_allowlist.json lists, by reason, the tracked files that still name one particular deployment of this code, and how often. tests/unit/test_distribution_neutrality.py fails on a new mention, and on an entry above its file's count; after removing mentions, uv run python scripts/check_distribution_neutrality.py --write lowers the entries.
  • apps/frontend/eslint-ratchet.json lists the frontend files over a limit, each at its current value. ESLint applies a listed value to every function in that file, so it lets a new function there go past 15; keep new functions within 15 anyway. Lower an entry when its file improves.

If apps/frontend/node_modules is absent, install it before reporting checks as blocked. From apps/frontend, run npm ci, npm run type-check, and npm test; vitest and tsc come from the project's development dependencies.

4. Workflow

  • Branch off dev as <user>/<scope>/<name>. Pull requests target dev.
  • Keep each pull request focused on one feature, fix, or documentation change.
  • Put documentation where docs/README.md expects it. Maintained guides go in docs/developer/. New designs and implementation plans go in docs/agents/specs/; verification and review records (acceptance runs, reviews of a revision) go in docs/agents/reviews/. Name records YYYY-MM-DD-<topic>.md. Do not create other top-level folders under docs/: tools that default elsewhere, such as the superpowers plugin's docs/superpowers/, must write to these folders instead.
  • Never commit secrets. This repository will be made public; treat every commit as already public.
  • Hosts roll out before the control plane, and each side supports the previous release only. Managed compute hosts are the exception: the new control plane rolls them forward once it runs (Compute resources). Code kept for an older release is marked COMPAT(remove-after=YYYY-MM-DD): <what makes it removable> in a comment, and tests/unit/test_compat_markers.py fails once that date has passed: remove the code then, or move the date with the reason it must stay.

5. Migration rules — historical scope and permanent boundaries

The move-only restrictions in rules 1 and 2 expired when the split completed on 2026-08-09. They remain here for reading migration pull requests; cleanups, renames, and refactors are ordinary new work now. Rule 3 remains a quality requirement, and rules 4 and 5 remain permanent service boundaries.

  1. Move as-is. When a task says "move" a file, change imports and paths only. No renames, no splitting large files, no reformatting, no dead-code removal, no "while I'm here" fixes. A reviewer will diff your result against a pristine copy from the source repo and expect only import lines to differ. Cleanups happen after cutover, as their own pull requests.
  2. If it cannot move without a logic change, stop and report. Do not improvise a redesign inside a migration task.
  3. Never weaken a test to make it pass. A ported test that fails is a parity break — report it. Adapting fixtures and imports is fine; changing an assertion is not.
  4. No HybridInference secrets here. This service never sees the HybridInference Gateway's API_KEY_SECRET, JWT_SECRET_KEY or database. Its own tokens use its own secrets (AGENT_CONTROL_TOKEN_SECRET, AGENT_LOCAL_SESSION_SECRET, AGENT_SETTINGS_ENCRYPTION_KEY). A mail transport credential an operator chooses to share (SMTP) is configuration, not identity.
  5. No Gateway dependency. No Python import of serving.*, ever, and no Gateway sign-in, admission, grant or catalog: those were removed (decision). A configuration naming them is refused at startup.

6. Key concepts

  • Conversation / job / attempt / lease. A conversation (thread_id in code, a chat or task in the UI) is made of turns, and a job is one turn of it: one user message and the agent's work on it (turn_no; every turn after the first names its parent_job_id). An attempt is one execution of a turn on one host. Attempts carry a lease with a fencing generation — a superseded attempt's writes are rejected by the store, which is what makes retries safe and stops duplicate pull requests. Code structure has the full glossary.
  • Model connection. Every turn runs on a model connection: a provider Base URL, an API key and verified model ids. Usually it is one of its owner's own, with the owner's own key (FreeInference is one such provider, used with the owner's key). The claim and the residency of the owner's turn hand the key to the host; the member's CLI runs on it. A deployment may also provide connections: an administrator saves the endpoint and the deployment's key, and every admitted owner may run on it. That key stays in the control plane: the member's CLI holds a residency placeholder and reaches the provider through the model relay (design). A turn runs at Auto (the provider's default) unless its owner picks a reasoning level the provider's own model listing declares for that model and API and the agent's CLI can send (design). MCP servers are resolved per user by the control plane and reached through its MCP-only relay with a separate scoped credential. A chat member's CLI holds its residency's MCP credential, which the relay resolves on every call to the turn that residency is running, so follow-ups keep the live process. There is no per-task dollar budget and no billing — the provider keeps the key's quota and bill. The one local limit is on provided connections: each owner has a budget per UTC day on a provided connection's models, and the model relay charges each request the tokens its answer said it used at the prices the administrator set. That is an allowance, not a bill: nobody is invoiced, and the provider's own account of the key is what the deployment pays. The Usage pages show the tokens agents reported for calls attributable to a turn, labelled runtime_reported; they are not a provider bill and may undercount it. If a task seems to need a budget field, stop — the plan's C5 section records the decision.
  • Host. A machine in the pool running the host agent. It registers with a one-time enrollment token, heartbeats, and claims jobs it has capacity and capabilities for. Hosts are added and removed without redeploying the control plane.
  • Sandbox. A retained container and workspace per sandbox group. Each chat has its own native session and active CLI process; Fork adds a member to the same group and workspace. Native resume stays in that container. There is no transcript-replay or platform-checkpoint recovery path. See the implemented session design. The group has one network, the same for setup and for the agent, with nothing to select (design). Each group gets its own routable bridge, and before anything joins one the host installs this deployment's nftables table (cloud_agent_shared/egress.py renders it, cloud_agent_host/network/network_guard.py installs and verifies it):
    • public IPv4 on TCP 80/443, so curl, pip, npm and Git over HTTPS work;
    • the one named resolver on TCP/UDP 53;
    • the MCP relay's service port on the group's own bridge, which reaches the MCP-only listener for tools;
    • the model relay's service port on the group's own bridge, on a host configured with one, which reaches the model relay for provided models;
    • the exact public IPv4 address and port of each SSH server its owner registered, checked and enabled for one of the group's chats;
    • and nothing else: not the host, private or special-purpose addresses, other groups, other ports, UDP 443, or IPv6. No relay is a caller-selected forward proxy, and there is no proxy at all.

7. Security boundaries — do not relax without review

  • A sandbox group reaches public IPv4 on TCP 80/443, the one named resolver, its own MCP relay, the model relay where its host has one, and the exact public IPv4 address and port of each SSH server its owner registered, checked and enabled for one of the group's chats — and nothing else: none of the host's addresses, no private, link-local/metadata, CGNAT, multicast or reserved range, nothing in AGENT_EGRESS_PROTECTED_CIDRS, no other group, and no IPv6. Per-group bridges and this deployment's host nftables table enforce it on the actual packets; the table is installed and read back before any group joins its bridge, and a failed check stops the group rather than falling back to anything. There is no mode, flag or proxy that widens it, and adding one is a design change, not a configuration.
  • MCP tool calls are served only on the MCP-only listener; the public main API must not expose them. The MCP tunnel continues to target that listener.
  • Accepted owner-key exposure. A member sandbox holds its owner's provider key for the turn it runs, and code or instructions it encounters may cause that key to leave the sandbox (see owner model connections below). It is the owner's own key and quota; it grants no MCP or platform-management authority. See the decision and remaining boundaries.
  • Platform-held long-lived account and service credentials never enter the sandbox. Each member receives its own scoped inference and MCP credentials, which a sibling member cannot read: members have distinct UIDs, private provider homes and tmpfs, and no credential is passed as a process argument. That stops reading, not writing: siblings share one writable workspace, .git included, so code a member plants there — a Git hook or core.fsmonitor, a build or test script — runs with the credentials of whichever member executes it next. Model calls use the owner's own provider key and billing, or, on a provided model, the deployment's key, which only the model relay holds (provided models below). No credential GitHub accepts enters a sandbox. A member's GH_TOKEN is its residency's agh. placeholder, which only the GitHub relay accepts: while the residency serves a running turn it exchanges it for the owner's own GitHub user token, for the conversation's repository only, and refuses branch deletion, tags, GraphQL and repository administration. A member shares the owner's identity, not their token: commits are authored as the owner, and the owner's token, refresh token and the App's private key stay in the control plane. Nothing on the platform scans what goes through the relay; repository branch protection, GitHub's push protection and the repository's CI still apply (design).
  • Owner-provided environment credentials. An owner may deliberately inject personal variables and Secrets through a frozen environment binding. Repository code, dependencies and the agent can read them, send them through the existing public egress, or write them into files/model context. These are user-supplied credentials, not platform credentials. The platform encrypts stored values, keeps Secrets write-only, filters setup/runtime phases and masks known values in durable output. Nothing on the platform scans what an agent pushes; GitHub's push protection and the repository's CI are the only checks on a push. These controls do not prevent deliberate exfiltration. Same-owner Fork inherits the binding; the only cross-owner sharing is a team's environment, below. Reserved platform/host variable names and the existing fixed network and member credential boundaries still apply. An owner's OPENAI_*/ANTHROPIC_* variables supplied through an environment binding never replace an agent CLI's own inference credentials; a member-private BASH_ENV script gives them to the commands the CLI runs. See the environment design.
  • Team environments (2026-10-15). A team's owner and admins may save an environment for the team, and every active member may run their own tasks, in their own sandboxes, with it. Its Secrets are therefore readable by every member who runs it: the write-only editor hides them on screen, not between members, and the dialog says so. A member's personal globals join it only at run time, so its installation is the same for everyone. Its installation snapshots serve the whole team, and a member's task or test can queue a build from a commit that member checked out, so a member can shape what the others' sandboxes start from. Members of a team are one trust domain for its environments, as for a repository they can all push to. A binding records the member's access generation, and every read that hands its values to a container (each turn's claim, terminals, actions, Fork, preparation, tests) refuses once the member has left or the team is deleted; a value already read cannot be taken back, so rotate it. See the team design.
  • Owner SSH servers (2026-10-15). An owner may register their own SSH servers and enable them per chat. The platform generates one Ed25519 key per server, stores it encrypted, and writes it only into the enabled chat member's private tmpfs, with the host key the owner confirmed pinned. The agent, repository code and anything they run can read that key, use or exfiltrate it, and run anything the server account may run; an owner revokes it by deleting the server, which cuts live sessions and erases the stored key, and by removing the key line from authorized_keys. Keys are masked in the runner's durable output: a key reaches a member only when that member's own turn starts, from that turn's claim, and a push mid-turn only takes access away, so every key in a sandbox is one its running turn masks. The control plane dials a server only when its owner runs a check, under the same public-address rules, an hourly per-owner limit, and a 64 KB cap on what it reads back. See the SSH server design.
  • Owner model connections (2026-10-27). An owner may add their own inference endpoint (Base URL, API key, up to sixteen model ids) and run their own tasks on it; it is the only way anything runs, and they pay the provider. It is private to them: never shared with a team, never reachable through another owner's Fork, and every claim, residency, follow-up, waiting message, goal round and Fork re-reads it as theirs. The key is encrypted under its own domain, write-only, and handed to a host only in the claim and residency of its owner's turn on it; the member's CLI holds it in its environment and activation tmpfs, never in argv, and the runner masks it in durable output. Repository code, dependencies and the agent can read and exfiltrate it through the existing public egress, as with owner environment credentials. An encrypted scanning copy is still recorded when a key is bound to a residency and kept with the sandbox group record, even after connection deletion, but nothing reads it since the workspace snapshot scan was removed; execution paths never select it. It remains a real provider secret, and deletion here does not revoke it upstream. Login and Agent admission are the account system's. Nothing is saved unverified, and there is no manual declaration of APIs: creating a connection, and an edit that changes its key or Base URL or adds models, verifies the models it would list inside the saving request, on the owner's explicit action, before anything is written. Each model gets a streaming tool round trip on each of Chat Completions, Responses and Anthropic Messages (at most six provider requests and one of the owner's hourly checks, all reserved before any call, connection or not); the save also reads the provider's two model listings once for the reasoning levels it declares (no inference, no extra check). The save commits only if every model has an agent that can use it, storing the native outcomes (supported, unsupported or failed) per model; otherwise nothing is written, not even a new key, and each model's result is answered. An edit commits only while the row still has the revisions and model list and stored facts its verification read. Verification has a 240-second overall deadline; a timeout writes nothing. A model without current facts runs on nothing. A runtime runs on its own API when verified for that model (Chat Completions: OpenCode, Kilo, pi; Responses: Codex; Anthropic Messages: Claude Code). Otherwise only Codex and Claude Code may run on a model verified for Chat Completions, through a member-private translator: the CLI is exec'd behind a launcher that forks a loopback service as the same member UID, bound to that one upstream, key and model, answering only a platform-issued token scoped to the owner, member, connection revision, client protocol and model (masked by the runner like the key; a sibling's token is refused) and dying with the CLI. The CLI never holds the key on that path, and the service refuses, rather than drops, what Chat Completions cannot carry (an image a tool returned goes to the upstream as a following user image part). No other pair is translated, and nothing falls back to another connection, model or API at inference time; the composer offers, and every gate accepts, the same per-model paths. A claim and its residency must agree on both the credential revision and the plan. See the protocol compatibility design. A direct plan's CLI starts behind the same launcher, its loopback service in passthrough (2026-11-05): bound to the connection's one upstream origin, it forwards the CLI's own requests and credentials unchanged, holds no key and issues no token, and adds only a size record per model request on the CLI's stderr -- the system prompt's and tool schemas' characters, never their text -- which the host reports as a request_prefix lifecycle event. It is not a network proxy: the member's egress, and the network rules, are the same. A sibling that reaches its port can do nothing through it that the public egress does not already allow. A CLI pointed at its vendor's own host (Claude Code on api.anthropic.com) is not metered, so no CLI sees another host than it would; a meter that cannot start leaves its CLI pointed at the provider directly. See the request meter design. Listing a provider's model ids to fill in the form writes nothing and proves nothing; it uses the key just entered, or a saved key only at the exact Base URL it was saved with, and spends one hourly check. A verified new key or Base URL replaces every model's facts; that, disabling or deleting closes every process still running on the old state, while a rename, removing models or enabling makes no provider call and closes nothing. The control plane's verification dials only public IPv4 HTTPS under the MCP connection policy and the sandbox's protected ranges, pinned, with no redirects and no upstream body echoed. See the model connection design.
  • Provided models (2026-11-12). An administrator may save a provider's Base URL, the deployment's own key and up to eight model ids, and every admitted owner may run turns on it without any setup. The key is encrypted under the settings root key in its own domain, is write-only, and never leaves the control plane: it is in no claim, residency, host, sandbox, response, log or audit row. A turn's claim marks the connection relayed and carries no key; the member's CLI is launched at the connection's path on its host's model tunnel and holds its residency's amd. placeholder, which no provider accepts. Only the model relay accepts it, on a listener of its own that serves nothing else, and only while that residency is the chat's hot one, serving a running, leased, unstopped turn on an enrolled host, on that connection, at the revision its process started on. Stop, a superseded attempt, a rotated residency, a removed host, and a connection disabled, deleted or moved to another Base URL take effect on the next request, with nothing to revoke at the provider; a rotated key closes no process. The relay is not a proxy: the destination is the connection's Base URL, dialled under the rules of a connection check (public IPv4 HTTPS, pinned, no redirect, no proxy); it forwards only the inference and model-listing paths an agent CLI calls, matched whole, for a model the connection lists; and it refuses any credential that is not its own placeholder, so an owner's key never comes this way. Each owner has a daily budget on a provided connection's models together (a default on the connection, or the account's own amount; zero withholds it), and each model has prices and is open to every owner or to named accounts only. A request is charged when its answer ends: the tokens the answer said it used, at its model's prices, with the inference gateway's formula and rounding; or, when the answer named none or was cut short, an estimate of its input and of the output it had produced. A cost that could not be written is kept, counted against the budget, and written again. The budget is checked on every request and again when a turn is admitted and claimed; requests in flight reserve their estimate of it, and two caps limit how many are in flight. All three are kept per relay process. The cost is the relay's arithmetic on the administrator's prices, not the provider's bill: the provider's own budget on that key remains the limit on what the deployment spends. Accepted: an owner can relay their own running turn's requests to something outside the sandbox, within their own budget and under their own name; an owner can pass a budget by about what their requests in flight cost; a sibling member can cause a member's placeholder to be used, as with the other residency credentials; and the requests and answers of provided models pass through the control plane, which stores and logs none of their content. Nothing falls back between a provided connection and an owner's own. Saving and editing verify like an owner's connection, with the administrator's explicit action, and every change is in the admin audit trail without the key. See the design.
  • Public multi-tenant use requires Kata isolation, non-root, dropped capabilities, seccomp, and resource limits — plain Docker containers are for trusted internal repositories only.
  • The Docker socket is the host's authority. Only the runner and the workspace broker mount it, and :ro on that mount does not narrow the Docker API: either can start a privileged container, so a flaw in either is a host compromise. No other service may mount it; the deployment tests and CI's Compose check pin the holders.