Concepts

The mental model behind fg-agents, in the order you'll meet it.

Defining an agent

An agent is a declarative AgentDefinition — no subclassing, no code. Minimally only name and model matter; model has no default and must be a "provider:model" string:

AgentDefinition(
    name="analyst",
    model="openai:gpt-4.1",             # or "anthropic:claude-sonnet-4-6", "ollama:qwen3:8b"
    fallback_models=["groq:llama-3.3-70b"],
    system_prompt="...",                 # identity + principles
    tools=["search", "manage_notes"],    # names registered in a ToolRegistry
    skills=["sql-analysis"],             # ids registered in a SkillsManager
    max_turns=50,
)

A bare model string (no provider prefix) defaults to ollama. Three provider code paths exist behind the same interface:

• Anthropic — native streaming; extended-thinking blocks are preserved (thinking + signature travel with tool_use); cache_control ephemeral markers on the system prompt and last tool.
• Google Gemini — non-streaming via google-genai, run through asyncio.to_thread, with full functionCall/functionResponse re-emission for multi-turn tool use.
• OpenAI-compatible — one adapter covers OpenAI plus ~20 providers via a base-URL registry (OPENAI_COMPATIBLE_PROVIDERS): openai, deepseek, mistral, xai, perplexity, cohere, groq, cerebras, fireworks, together, ollama, lmstudio, jan, llamacpp, localai, lemonade, jellybox, docker, vllm, azure, bedrock. AgentLLM(custom_providers=...) adds more.

fallback_models plus per-call retry with exponential backoff (in the engine's _call_llm_with_retry) give provider failover without any code on your side.

The ReAct loop

AgentEngine.run() is the heart. Each run:

1. Acquires a per-session asyncio.Lock — one run per session; concurrent calls queue.
2. Loads or creates the session; a cancelled session being resumed is lifted back to running.
3. Captures a cancellation baseline (a monotonic sequence number) so a leftover cancel from a previous run cannot kill a freshly-started run at turn 0.
4. Persists the user message and builds a WorkingMemory + ExecutionContext — identity (tenant/user) is taken only from the persisted session, never from request metadata.
5. Per turn: build the system prompt, build the message context via ContextManager, run before_llm_call middleware, call the LLM with retry + fallback, stream text / thinking / tool-call deltas, persist the assistant message (thinking blocks preserved).
6. No tool calls → standard ReAct termination, subject to require_explicit_completion and min_turns nudge logic (capped at 3 continue-nudges). Tool calls → execute each through before/after_tool_call middleware, persist results (failures are made unmistakable to the model), stream them.
7. Stop signals: the _task_complete / _session_complete working-memory flags set by the lifecycle tools, an external should_continue callback, or a context-health hard-stop.
8. Finalize → completed with final_output; any exception → failed + an error event.

Robustness details worth knowing: cancellation is cooperative via sequence numbers; the OpenAI streaming path flushes tool calls exactly once after stream drain (some providers emit multiple finish-reason chunks — without the dedup, tools would execute twice); an inline <think> splitter separates reasoning from output for models that emit it in-band; the API tool handler percent-encodes, pins hosts, and blocks redirects (SSRF hardening); the DB handler enforces read-only SQL backend-agnostically.

System prompt layering and prompt-cache stability

build_system_prompt assembles the prompt in fixed sections: identity + principles (from system_prompt, defaulting to ORCHESTRATOR_SYSTEM_PROMPT), mission/objective (capped at 2000 chars), skills (full templates for code-registered skills, an index for DB skills), knowledge index, tools index, notes, and plan.

With split_volatile=True the byte-stable sections (identity through tools) are separated from the per-turn notes/plan, which are delivered as a trailing non-persisted message. This keeps the prompt prefix byte-identical across turns, preserving provider prompt caches — Anthropic's ephemeral cache_control and the automatic prefix caching of Together/OpenAI.

The no-filesystem model

A laptop agent keeps its identity and memory in files. A web agent has no files — so its persistent state lives in the repository, exposed through built-in self-management tools and keyed by scoped_agent_key(tenant_id, agent_id), so two tenants running the same agent never share memory:

Application context flows in three ways:

• AgentDefinition.extra_context — static context baked into the definition.
• Per-run metadata — request-scoped values (connection_id, schema_hint, …) and callbacks (on_subagent_spawn / on_subagent_event / on_subagent_complete, should_continue).
• variables — values rendered into the prompt via Jinja2 / format templating.

Multi-agent orchestration

Orchestrator wraps the engine into a brain-and-workers pattern. It registers sub-agents, injects ORCHESTRATOR_SYSTEM_PROMPT for the brain (and TASK_AGENT_SYSTEM_PROMPT for task agents without their own prompt), and rewires the manage_agent tool from placeholder to real lifecycle actions:

• spawn — run one sub-agent task
• spawn_parallel — run several concurrently, capped by max_parallel_agents; each agent's output is capped at 60 KB with chart tokens preserved
• message — continue a conversation with an existing child session
• report / get_report — the only actions available to task agents themselves
• status, list — introspection

Cross-tenant access is blocked by same-tenant checks on every action.

SubAgentRunner does the actual spawning: it creates a linked child session (parent_session_id), runs an isolated ReAct loop, and applies a nudge timer (nudge_after_seconds) followed by a hard deadline (hard_deadline_seconds). Result capture is 3-tier — last assistant message → tool results → minimal summary — and if the model forgot to call session_complete, the runner synthesizes it. (The by recovering the full report from message history.)

External dispatch: if your on_subagent_spawn callback returns a dict, the framework treats the task as dispatched externally and does not run it inline — the seam for routing sub-agent work to a separate worker fleet without double execution.

Persistence and context management

All repositories are async implementations of BaseRepository — in-memory, SQLite, or PostgreSQL, chosen by create_repository(backend, ...). The engine, orchestrator, middleware, and built-in tools all speak only the interface.

ContextManager builds a message list that fits the model's window. It estimates tokens (~4 chars/token, per-model limits built in) and at 80% of the window applies two layers:

1. Truncate — shrink old tool arguments and large tool results.
2. Summarize — LLM auto-summarization; old messages are marked summarized and a summary message is injected.

Independently, ContextHealthConfig thresholds (compress 0.40 · compact 0.80 · warn 0.85 · hard-stop 0.90) drive per-turn assess_health actions when attached via agent_def.metadata["context_health"] — up to and including stopping the run before the context overflows.

Streaming

Everything the engine does is surfaced as typed StreamEvents — session/turn boundaries, text and thinking deltas, tool calls and results, sub-agent lifecycle, context compaction, errors. Each event serializes to SSE (event.to_sse()) or a dict. On the web side, sse_response() is a production SSE helper with heartbeats, disconnect detection, and Last-Event-ID replay — the piece that makes browser re-attach work.

Multi-tenant security model

Authentication is pluggable: an Authenticator is any Callable[[Request], Awaitable[AuthContext]], where AuthContext(tenant_id, user_id, scopes, is_admin) carries the verified identity. HeaderAuthenticator reads trusted-proxy headers (for deployment behind a gateway); the default UnauthenticatedAccess is a single shared admin tenant with a loud one-time warning — development only.

The enforcement rules:

• Sessions are stamped with tenant/user from the verified context, never the request body.
• Every route gates access through ownership checks where non-existence and non-ownership are indistinguishable — no session-id probing.
• Persistent agent memory is namespaced by scoped_agent_key(tenant_id, agent_id).
• Sub-agent actions verify same-tenant before touching a child session.
• Runtime tool/agent registration endpoints are admin_only by default (403 otherwise).

Middleware

Middleware hooks into every stage of the loop: before_llm_call, after_llm_call, before_tool_call (return None to block the call), after_tool_call, on_turn_complete, on_error. Subclass BaseMiddleware (a pass-through base) or implement the Middleware protocol.

Shipped implementations: AuditMiddleware (repository-backed audit trail), TokenTrackingMiddleware (usage + cost estimation), PermissionMiddleware (pattern-based rules with auto_approve / ask / deny / conditional levels — ask without an approval callback fails closed to deny), RateLimitMiddleware, LoopGuardMiddleware (repeated-call detection), and ContextCompressionMiddleware (progressive compression). The Orchestrator auto-adds audit + token tracking.

Scheduler

AgentScheduler is a DB-backed polling scheduler that runs agents on a cron expression (schedule_cron), at a moment (schedule_once), or in response to an application event (schedule_on_event + trigger_event). Schedules persist in the repository, so they survive restarts; pre_execute_hook / post_execute_hook are the host-app extension points. Requires the scheduler extra (croniter).

Integration seams

fg-agents imports no other fg-* package — integration with the rest of the platform is by injection point, not dependency: the authenticator (identity service), the vault_resolver on ToolRegistry (credential resolution for API/DB tools), db_url (the host's PostgreSQL), scheduler and sub-agent hooks (workflow dispatch), and the WORKFLOW tool type, which posts to an external workflow engine (/api/v1/workflows/{id}/execute).