cancels cleanly · isolates panics · resumes after a crash
[dependencies]# Not on crates.io (the `borg` name there is an unrelated crate).borg={git="https://github.com/joydle/borg.git",tag="v1.0.0"}async-trait="0.1"
READMEGitHubv1.0.0 from GitHub (not on crates.io), Rust 1.88, tokio. MIT or Apache-2.0.
Run
A scripted run, played in the browser; no model is called. A child panics and is restarted, each loop iteration is checkpointed, the process crashes and resumes from its snapshot, and the deploy waits for a person.
span
Budget
0 / 40,000 tokens
Elapsed
0.00s
Checkpoints
0
Outcome
pending
running
ok
panicked
backoff
cancelled
approval, budget
checkpoint
Layers
Each layer is a trait; swapping a port leaves the layers above unchanged.
Composition
The tree your code declares.
seq · par · branch · repeat · fallback
Supervision
Per-child cancel, panic catch, restarts, meltdown and breakers, inside execute and run_one.
RestartPolicy · MeltdownPolicy · CircuitBreaker
Guardrails
Behavioural rules at turn boundaries.
GuardrailEngine · GuardrailRule
Checkpoint
Resume at iteration boundaries.
execute_resumable · CheckpointStore
Agent
One model call or deterministic step; a typed verdict.
Ten node types. Leaves are agents; every other node supervises its children the same way.
sequencing
fan-out
control
recovery
Single
step(agent)
One agent, with its own restart policy and budget check.
Sequential
seq([..])
Steps in order; each reads the handoff of the last.
LoopUntil
repeat(n, until, ..)
Re-runs a body until a verdict says done, with an iteration cap.
Parallel
par · par_all
Fan-out under one supervisor; fail fast or collect every result.
Deferred 2.x
deferred(|h| ..)
Built when reached, from what earlier steps decided.
Branch
branch(..)
Routes on a verdict token or a handoff predicate.
Agent
agent_loop(..)
The tool-use loop, up to a step cap.
Heartbeat 2.x
heartbeat(..)
Emits a beat while a long step runs.
Fallback
fallback([..])
Tries a chain until one step succeeds.
Ladder 2.x
fallback_ladder
Fallback with a fall-through rule and self-selecting rungs.
Lifecycle
Runs that survive a crash, wait for a person, or last for days.
Checkpoints
execute_resumable(composition, ctx, store, run_id) saves snapshots to a CheckpointStore. After a crash the same call skips finished steps and continues. A snapshot whose composition fingerprint does not match the pipeline's shape is refused.
OnLoopIteration
Default. Saves after each LoopUntil iteration.
OnEveryStep
Saves after each step.
Manual
Leaves saving to you.
Resume points are Sequential steps and LoopUntil iterations; Parallel, Branch and Fallback re-run whole. InMemoryCheckpointStore ships; the trait is three methods, so Postgres, Redis or S3 fit behind it.
Pause and resume
An agent calls AgentContext::request_suspend and the run unwinds as BorgError::Suspended. A borg-loop actor returns HandleOutcome::Suspended; its snapshot is persisted. It resumes later with a Decision: approve, reject or edit.
ActorAgent has a typed mailbox and checkpoints to an ActorStore. spawn_actor starts one; resume_actor restores it after a restart.
StreamingActorAgent, started with spawn_streaming_actor, streams each turn's events live. Dropping its TurnHandle mid-stream parks the actor with its snapshot intact.
Plan; write code, tests and docs in parallel; build and fix until PASS, at most eight times; deploy. A stop reaches every nested call.
#[derive(BorgToolDerive)]#[tool(name="read_file",description="Read a file from disk.",context=EditCtx,input_schema=serde_json::json!({"type":"object","properties":{"path":{"type":"string"}},"required":["path"],}))]structReadFile;implReadFile{asyncfnexecute(&self,input:serde_json::Value,ctx:&mutEditCtx)->Result<ToolCallResult,BorgError>{letpath=input["path"].as_str().unwrap_or_default();lettext=tokio::fs::read_to_string(path).await.map_err(|e|BorgError::Internal{message:e.to_string()})?;ToolCallResult::ok(text)}}letregistry=Registry::<EditCtx>::new().with(ReadFile).with(WriteFile).with(RunTests);letdispatcher:Arc<dynToolDispatch>=Arc::new(MutexDispatcher::new(registry,my_ctx));letcoder=agent_loop(MainCoder,dispatcher,/* max_steps */40);
Schemas reach the model in its provider's shape. Each tool call is supervised and draws on the same budget. MCP tools use the same path.
Rules watch model behaviour at turn boundaries. Verdicts merge by severity, Ok to Halt, and the executor enforces them.
useborg::{BorgError,StreamEvent};useborg_loop::*;usefutures::stream::BoxStream;#[async_trait]implStreamingActorAgentforChat{typeMessage=Msg;typeSnapshot=ChatSnap;asyncfnhandle_streaming(&mutself,msg:Msg,ctx:&ActorCtx)->Result<(BoxStream<'static,StreamEvent>,StreamingHandleOutcome),BorgError>{letevents=self.reply(msg,ctx).await?;// your model's event streamOk((events,StreamingHandleOutcome::Park))// wait for the next message}fncheckpoint(&self)->ChatSnap{ChatSnap{history:self.history.clone()}}fnrestore(s:ChatSnap)->Self{Chat::from_history(s.history)}}leth=spawn_streaming_actor("session-42",Chat::default(),ActorConfig::for_streaming(),store.clone(),None);matchh.run_session_loop(Msg::user("hi"),||Msg::Continue,|ev|{ifletStreamEvent::TextDelta(t)=ev{print!("{t}")}}).await{SessionOutcome::Parked=>{/* alive, waiting for the next user message */}SessionOutcome::Suspended{request}=>{/* ask a human, resume later */}_=>{/* terminal: spawn again for the next message */}}// After a restart: the same session, from its snapshot.leth=resume_streaming_actor("session-42",Chat::default,Chat::restore,ActorConfig::for_streaming(),store,None).await?;
A streaming actor streams each turn's events live and checkpoints to your store. Dropping a TurnHandle mid-stream parks it. Full example: borg-loop/examples/streaming_session.rs.
useborg::suspend::{Decision,SuspendKind,SuspendRequest};useborg_loop::*;// In the actor: pause for a human before deploying.Msg::Deploy{release}=>HandleOutcome::Suspended{request:SuspendRequest::new(format!("Deploy {release} to production?"),SuspendKind::Approval,),},// The runtime persists a snapshot and exits.letActorExit::Suspended{request}=handle.join().awaitelse{unreachable!()};// Later, when the human answers: restore, then deliver the decision.letresumed=resume_actor(session,fresh,Deployer::restore,ActorConfig::default(),store,None).await?;resumed.send(Msg::Decision(Decision::Approve)).await?;
Any stdio MCP server as a tool dispatcher. Servers start from a cleared env (pass_env adds names); each call has a timeout (60 s by default), honours cancel, and caps results at 1 MiB.
Subprocess execution as a tool: a required program allowlist and workspace root, no inherited env, a model that cannot set PATH, capped output. Not a sandbox: TokioBackend runs ordinary subprocesses.
File operations over a workspace abstraction, and an approval helper.
Integration
A trait, with a default that ships, at every boundary.
Models
Implement AgentLlmClient once. The typed Provider covers OpenRouter, Anthropic, OpenAI, Bedrock, Google and Together, with Custom for the rest.
Stores
CheckpointStore and ActorStore are traits. CheckpointStore ships InMemoryCheckpointStore; ActorStore ships in-memory, LRU-caching and resilient primary-plus-fallback stores (the resilient store reconciles to the primary after an outage). Back either with Postgres, Redis or S3.
Telemetry
borg-tracing emits the span tree into tracing, so OTLP, Jaeger or JSON logs work through your subscriber. FileEventRecorder writes JSONL for replay.
Isolation
TokioBackend runs ordinary subprocesses; it is not a sandbox. Put real isolation behind SandboxBackend: gVisor, Firecracker, bubblewrap or nsjail, a hosted sandbox service, or agentjail (beta).
Pipelines
The dsl builders and the pipeline! macro make a composition a typed value, so resume can fingerprint its shape.
MCP
borg-mcp turns a stdio MCP server into tools the loop calls like native ones, from a cleared env, with a per-call timeout and a 1 MiB result cap.