--- name: pramana-integration description: Instrument Python AI agent code (OpenAI/Anthropic clients, tool calls, multi-agent handoffs) with Pramana so every LLM call, tool call, and inter-agent message is recorded for replay and evidence export. Use when the user asks to "add Pramana", "add tracing/observability with Pramana", "instrument this agent with Pramana", or "record this agent's runs". --- # Pramana integration Pramana records every non-deterministic decision an agent makes — LLM calls, tool calls, inter-agent messages — so a run can be replayed bit-for-bit later and its behavior proven with a signed evidence bundle. This skill instruments existing agent code with it. It does not rewrite the agent — it wraps what's already there. ## Before touching code Check for `PRAMANA_API_KEY` in the environment or a `.env` file. If it's missing, stop and tell the user: sign up at reliai.in, create an `engineer`-role API key from Settings → API keys, and set `PRAMANA_API_KEY` (or be ready to pass `api_key=` explicitly). Do not fabricate a key or proceed without one — every write is authenticated and a missing/wrong key fails loudly, not silently. Also get the tenant ID (shown on the same Settings page) — needed twice below. ## Step 1 — install ```bash pip install pramana-sdk ``` If the project uses `uv`/poetry/pipenv, add it the way that project's other dependencies are added, not a bare `pip install` into an unrelated environment. ## Step 2 — initialize once, near where the agent run actually starts Find where the agent's entrypoint is (a `main()`, a request handler, a CLI command — wherever one full "run" begins) and add this before any LLM/tool calls happen: ```python import uuid import pramana from pramana.sinks.http import HttpSink TENANT_ID = "" pramana.init( trace_id=str(uuid.uuid4()), # one fresh id per run — do not reuse across runs tenant_id=TENANT_ID, sink=HttpSink( "https://www.reliai.in/ingest/v1/events:batch", tenant_id=TENANT_ID, # api_key omitted here on purpose: HttpSink reads PRAMANA_API_KEY # from the environment by default. Pass api_key="..." explicitly # only if the project can't set env vars for some reason. ), ) ``` Multi-agent system (a supervisor + specialists, or several cooperating processes)? Call `pramana.init(...)` once **per agent**, all with the **same `trace_id`** but each with its own `agent_id=...` kwarg. Same trace_id is what groups them into one multi-agent view instead of N unrelated traces. ## Step 3 — instrument the LLM client Right after the OpenAI or Anthropic client is constructed: ```python pramana.instrument(client) # same call for either OpenAI or Anthropic — it detects which ``` Every `client.chat.completions.create(...)` (OpenAI) or `client.messages.create(...)` (Anthropic) call from then on is recorded automatically — model, prompt, response, latency, token counts. Nothing else about how the code calls the client changes. If the codebase wraps its own LLM calls behind a helper function, instrument the underlying vendor client inside that helper, not a mock or an abstraction layer Pramana can't see through. ## Step 4 — tool calls (optional, but do this for anything that isn't the LLM call itself) Wrap each tool invocation in **exactly one** helper function — the frame depth matters, so don't nest this inside another wrapper: ```python from pramana import interceptor from pramana_proto.v1.event_pb2 import TOOL_EXEC def tool_call(name, args, fn): """fn takes no arguments — close over them, or use a lambda.""" return interceptor.intercepted_call(fn, raw_input={name: args}, kind=TOOL_EXEC) result = tool_call("search_web", {"query": q}, lambda: real_search_function(q)) ``` Apply this pattern to each distinct tool the agent calls, at the actual call site in the code — not through one generic dispatcher that would blur every tool into the same recorded location. ## Step 5 — inter-agent messages (multi-agent systems only) Wherever one agent hands work to another (a queue publish, a direct function call, an HTTP request to another service): ```python from pramana import send_message, receive_message # In the sending agent, right where it currently ships the message: send_message(lambda envelope: real_transport.publish(envelope), payload, to_agent_id="worker") # In the receiving agent, right where it currently reads the message: payload = receive_message(lambda: real_transport.consume(), from_agent_id="planner") ``` `real_transport` is whatever the codebase already uses to move messages between agents — a queue client, an HTTP call, a shared object. Don't build a new transport; wrap the existing one. ## Step 6 — verify it worked Run the agent once, then either open `https://www.reliai.in` and confirm the trace appears, or run: ```bash uv run pramana doctor ``` (installed as the `pramana` CLI alongside the SDK) — it checks that ingest is reachable and the key authenticates, without needing to record a real trace first. ## Step 7 — replaying a recorded run (optional, but this is the point of recording) Once a run is recorded, replay it against the *same* code to confirm it reproduces, or against changed code to find out what a change would have done: ```bash export PRAMANA_API_KEY=... # the same engineer/admin key used to record pramana replay -- python your_agent.py ``` Nothing else is needed — the trace is read back over HTTPS with that key. Exits non-zero on any divergence, which is what makes it usable directly as a CI gate. `pramana model-diff -- ` is the same idea but makes real calls, for comparing a different model against the recording. Two things worth knowing before suggesting these to a user: - **An auditor-role key won't work.** Replay needs the recorded prompts and responses, and auditor keys never receive raw payloads by design. Use engineer or admin. - **Replay makes no outbound network calls at all.** Every LLM/tool call resolves from the recording, enforced at the socket level rather than by convention — so replaying a trace that contained a real payment or email cannot re-trigger it. This is why tool calls must be wrapped (step 4): an un-wrapped tool call is invisible to Pramana and would simply run for real during replay. ## Things to get right - **One `intercepted_call`/`tool_call` per real call site.** A shared wrapper used from many places in the code collapses them into one recorded location and breaks replay's ability to tell them apart — instrument at each actual call site, not through one dispatcher. - **Don't invent a `trace_id` scheme that reuses ids across runs.** Each run gets its own, generated fresh (`uuid.uuid4()` is fine). - **Never hardcode an API key into source.** Environment variable, or a secrets manager — the same standard the rest of the codebase already holds credentials to. - **This only sends recorded data to Pramana's ingest endpoint** — never a payload, prompt, or secret to anywhere else. If the agent handles genuinely sensitive data the user doesn't want captured at all, that's a real conversation to have with them before instrumenting, not something to silently work around.