borg

A supervised agent runtime for Rust.

cancels cleanly · isolates panics · resumes after a crash

[dependencies]
# Not on crates.io (the `borg` name there is an unrelated crate).
borg = { git = "https://github.com/joydle/borg.git", tag = "v1.0.0" }
async-trait = "0.1"

Run

A scripted run, played in the browser; no model is called. A child panics and is restarted, each loop iteration is checkpointed, the process crashes and resumes from its snapshot, and the deploy waits for a person.

span

Budget

0 / 40,000 tokens

Elapsed

0.00s

Checkpoints

0

Outcome

pending

  • running
  • ok
  • panicked
  • backoff
  • cancelled
  • approval, budget
  • checkpoint

Layers

Each layer is a trait; swapping a port leaves the layers above unchanged.

  1. Composition

    The tree your code declares.

    seq · par · branch · repeat · fallback
  2. Supervision

    Per-child cancel, panic catch, restarts, meltdown and breakers, inside execute and run_one.

    RestartPolicy · MeltdownPolicy · CircuitBreaker
  3. Guardrails

    Behavioural rules at turn boundaries.

    GuardrailEngine · GuardrailRule
  4. Checkpoint

    Resume at iteration boundaries.

    execute_resumable · CheckpointStore
  5. Agent

    One model call or deterministic step; a typed verdict.

    BorgAgent · Verdict
  6. Tools

    MCP-shaped schemas; serial or parallel dispatch.

    Tool · Registry<C> · ToolDispatch
  7. Ports

    Model client, budget, event sink, store.

    AgentLlmClient · Budget · EventEmitter · CheckpointStore

Composition

Ten node types. Leaves are agents; every other node supervises its children the same way.

  • sequencing
  • fan-out
  • control
  • recovery

Single

step(agent)

One agent, with its own restart policy and budget check.

Sequential

seq([..])

Steps in order; each reads the handoff of the last.

LoopUntil

repeat(n, until, ..)

Re-runs a body until a verdict says done, with an iteration cap.

Parallel

par · par_all

Fan-out under one supervisor; fail fast or collect every result.

Deferred 2.x

deferred(|h| ..)

Built when reached, from what earlier steps decided.

Branch

branch(..)

Routes on a verdict token or a handoff predicate.

Agent

agent_loop(..)

The tool-use loop, up to a step cap.

Heartbeat 2.x

heartbeat(..)

Emits a beat while a long step runs.

Fallback

fallback([..])

Tries a chain until one step succeeds.

Ladder 2.x

fallback_ladder

Fallback with a fall-through rule and self-selecting rungs.

Lifecycle

Runs that survive a crash, wait for a person, or last for days.

Checkpoints

execute_resumable(composition, ctx, store, run_id) saves snapshots to a CheckpointStore. After a crash the same call skips finished steps and continues. A snapshot whose composition fingerprint does not match the pipeline's shape is refused.

OnLoopIteration
Default. Saves after each LoopUntil iteration.
OnEveryStep
Saves after each step.
Manual
Leaves saving to you.

Resume points are Sequential steps and LoopUntil iterations; Parallel, Branch and Fallback re-run whole. InMemoryCheckpointStore ships; the trait is three methods, so Postgres, Redis or S3 fit behind it.

Pause and resume

An agent calls AgentContext::request_suspend and the run unwinds as BorgError::Suspended. A borg-loop actor returns HandleOutcome::Suspended; its snapshot is persisted. It resumes later with a Decision: approve, reject or edit.

  1. SuspendHandleOutcome::Suspended
  2. Persistsnapshot in the ActorStore
  3. Resumeresume_actor, then the Decision

borg_tools::approval::request_approval wraps the approve, reject or edit case. See borg-loop/examples/human_in_the_loop.rs.

Long-lived sessions

ActorAgent has a typed mailbox and checkpoints to an ActorStore. spawn_actor starts one; resume_actor restores it after a restart.

StreamingActorAgent, started with spawn_streaming_actor, streams each turn's events live. Dropping its TurnHandle mid-stream parks the actor with its snapshot intact.

Examples

use borg::prelude::*;

#[derive(BorgAgentDerive)]
#[borg(id = "reviewer", model = "anthropic/claude-haiku-5-5",
       max_tokens = 800, system = "Review code quality.")]
struct Reviewer;

impl Reviewer {
    fn user_prompt(&self, ctx: &AgentContext) -> Option<String> {
        ctx.handoff.get_as("code")
    }
}

let suggestions: Vec<Suggestion> =
    run_one_typed("review", Reviewer, ctx, handoff).await?;

A derive replaces the trait boilerplate; you write user_prompt. run_one_typed parses the verdict into your type.

Operability

Wired into one executor; each piece also works on its own.

Cancellation

A cooperative token tree; cancelling the root unwinds every nested call.

CancellationToken::child_token

Budgets

Admission before every dispatch; sub-budgets per composition; discounted cache reads.

Budget · sub_budget · with_cache_read_percent

Restarts

Exponential backoff with jitter, capped by attempts or wall-clock; Retry-After is honoured.

RestartPolicy::on_failure_within

Meltdown

Session-wide restart-storm detection; 20 restarts in 60 s by default.

MeltdownPolicy::new(max_restarts, window) · BorgError::Meltdown

Circuit breakers

Consecutive per provider and agent, or a shared sliding window; the half-open trial holds a lease.

BreakerPolicy::new(threshold, open_duration) · WindowBreaker<K>

Guardrails

Six built-in behavioural rules, or your own, checked at each loop boundary.

GuardrailEngine · GuardrailRule

Retry classification

One taxonomy for provider failures, including context length and credits.

FailureClass · ProviderFailure::classify_status

Fallback ladders

Your own fall-through rule; exhaustion keeps every rung's failure.

fallback_ladder · ladder::Ladder

Tracing

A span tree from session to tool, with tokens and time to first token.

borg_tracing::TracingEmitter

Context propagation

Task-locals and spans re-entered in every task borg spawns.

propagation::register_carrier · TaskLocal

Events

One emitter for lifecycle events; filter by kind, or record JSONL.

EventEmitter · with_lifecycle_events · FileEventRecorder

Parallel tool calls

Opt in per context; collect all or fail fast.

ConcurrentDispatcher · BatchFailureMode

Vision

Images travel with the prompt and tool schemas.

BorgAgent::images · ChatRequest::images

Terminal guard

A drop guard that runs the fallback if success never does.

TerminalGuard<T>

Subprocesses

Allowlisted programs in a workspace root, no inherited env, capped output; on Unix a cancel or timeout kills the process group.

SubprocessTool::builder

Testing

A scripted fake client and an eval harness for regression tests.

borg::testing::{eval, run_eval} · FakeLlmClient

Guarantees

Each has a regression test in the workspace.

  • Panics stay in their subtree

    A panicking child surfaces as BorgError::ChildPanicked under its own name; it never unwinds into its siblings.

  • Spans cross tokio::spawn

    Spawned work inherits the caller's span; no orphan trace roots.

    borg-loop/tests/tracing_propagation.rs
  • Dropping a turn parks the actor

    Dropping a TurnHandle mid-stream cancels the in-flight stream.

  • Registry claims are atomic

    Two callers racing for one parked actor cannot both win claim_parked.

  • Cancellation reaches the subprocess

    On cancel or timeout the process group is killed on Unix (the direct child on Windows), the log pumps are aborted, and the call returns Cancelled.

  • Nothing outlives its owner

    Actor handles cancel their actor on drop and are #[must_use]; dropping a running execution aborts its children.

  • Task-locals reach every branch

    A task-local set before par_all is visible in each branch.

    borg/tests/context_propagation.rs
  • Resume takes dependencies

    resume_actor takes a fresh and a restore closure.

Crates

One workspace, one version, one tag.

borgCompositions, supervisor, checkpointing, tools, streaming and the operability layer.
borg-macrosDerives for agents, tools and tool sets; #[borg_instrument]. Re-exported from borg.
borg-loopLong-lived actor sessions with a typed mailbox and checkpoint and resume.
borg-tracingA tracing emitter that turns borg events into one span hierarchy.
borg-mcpAny stdio MCP server as a tool dispatcher. Servers start from a cleared env (pass_env adds names); each call has a timeout (60 s by default), honours cancel, and caps results at 1 MiB.
borg-sandboxSubprocess execution as a tool: a required program allowlist and workspace root, no inherited env, a model that cannot set PATH, capped output. Not a sandbox: TokioBackend runs ordinary subprocesses.
borg-toolsFile operations over a workspace abstraction, and an approval helper.

Integration

A trait, with a default that ships, at every boundary.

Models
Implement AgentLlmClient once. The typed Provider covers OpenRouter, Anthropic, OpenAI, Bedrock, Google and Together, with Custom for the rest.
Stores
CheckpointStore and ActorStore are traits. CheckpointStore ships InMemoryCheckpointStore; ActorStore ships in-memory, LRU-caching and resilient primary-plus-fallback stores (the resilient store reconciles to the primary after an outage). Back either with Postgres, Redis or S3.
Telemetry
borg-tracing emits the span tree into tracing, so OTLP, Jaeger or JSON logs work through your subscriber. FileEventRecorder writes JSONL for replay.
Isolation
TokioBackend runs ordinary subprocesses; it is not a sandbox. Put real isolation behind SandboxBackend: gVisor, Firecracker, bubblewrap or nsjail, a hosted sandbox service, or agentjail (beta).
Pipelines
The dsl builders and the pipeline! macro make a composition a typed value, so resume can fingerprint its shape.
MCP
borg-mcp turns a stdio MCP server into tools the loop calls like native ones, from a cleared env, with a per-call timeout and a 1 MiB result cap.

Runs on tokio. MCP over stdio.