Build a data-analyst agent that returns charts

An LLM agent that writes pandas code, runs it in a PandaStack code-interpreter sandbox, and hands back charts as objects — code execution with charts, no stdout parsing.

The hard part of a data-analyst agent isn't the prompt — it's the plumbing. The model writes pandas code, something has to run it safely, and a chart has to come back as an image the model can see, not as a base64 blob you scraped out of stdout. The wedge here is the code interpreter: a persistent kernel that captures rich results (charts as PNG, DataFrames as HTML) automatically, so your tool returns a chart object — ex.png — with no savefig, no temp file, no second download.

This recipe wires OpenAI tool-calling to a PandaStack code-interpreter sandbox. The agent loads a CSV once, queries it many times across calls (state persists), and the loop feeds each generated chart back to the model as an image.

Setup

export PANDASTACK_API_KEY=pds_...
export OPENAI_API_KEY=sk-...
pip install pandastack openai

The code-interpreter template bakes IPython, matplotlib, plotly, kaleido, and pandas. Rich output is captured by the kernel automatically — you don't install or configure anything inside the sandbox.

Load the data once

Create the sandbox and a persistent code context (a Jupyter-style kernel). Upload the CSV, then load it into a df that survives across every later run_code call. This is the state-persistence win: parse the file once, query it for the rest of the session.

from pandastack import Sandbox

sandbox = Sandbox.create(template="code-interpreter", ttl_seconds=3600)
ctx = sandbox.create_code_context()  # persistent kernel; state survives across calls

# Push a local CSV into the sandbox filesystem (or write bytes inline).
sandbox.filesystem.write("/workspace/sales.csv", open("./sales.csv").read())

# Load it ONCE. `df` now lives in the kernel for the whole session.
ctx.run_code("""
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv('/workspace/sales.csv', parse_dates=['date'])
""")

import { readFileSync } from "node:fs";
import { Sandbox } from "@pandastack/sdk";

const sandbox = await Sandbox.create({ template: "code-interpreter", ttlSeconds: 3600 });
const ctx = await sandbox.createCodeContext(); // persistent kernel

await sandbox.filesystem.write("/workspace/sales.csv", readFileSync("./sales.csv", "utf8"));

await ctx.runCode(`
import pandas as pd
import matplotlib.pyplot as plt
df = pd.read_csv('/workspace/sales.csv', parse_dates=['date'])
`);

The tool: run pandas in the sandbox

Declare one tool. The model writes Python that uses the already-loaded df and draws on plt. The execute function runs that code in the same code context — so the df from the previous step is still there — and returns two things: a text result for the model to reason over, and (if a chart was drawn) the captured PNG.

The chart comes back as ex.png directly off the Execution. No savefig. No temp file. No second download. The kernel intercepts plt.show() and captures the figure's rich repr as a base64 PNG the moment the cell runs.

import json
from openai import OpenAI

client = OpenAI()
MODEL = "gpt-5.1"  # any vision-capable model id

tools = [
    {
        "type": "function",
        "function": {
            "name": "run_pandas",
            "description": "Run pandas code against the loaded DataFrame `df` and optionally draw a matplotlib chart on `plt`.",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {
                        "type": "string",
                        "description": "Python using `df` (pandas) and `plt` (matplotlib). To return a chart, draw on plt and call plt.show().",
                    }
                },
                "required": ["code"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    }
]

def execute(code: str):
    """Run model-authored code in the persistent kernel; return (text, png_b64|None)."""
    ex = ctx.run_code(code, timeout_seconds=60)
    if ex.error:
        return f"Error:\n{ex.error}", None
    # ex.png is the captured chart — already base64, no save-to-disk involved.
    png_b64 = ex.png
    # A DataFrame's last-expression value is captured as an HTML table.
    text = ex.results[0].html if (ex.results and ex.results[0].html) else (ex.text or ex.stdout)
    return text, png_b64

const tools = [
  {
    type: "function",
    function: {
      name: "run_pandas",
      description: "Run pandas code against the loaded DataFrame `df` and optionally draw a matplotlib chart on `plt`.",
      parameters: {
        type: "object",
        properties: { code: { type: "string" } },
        required: ["code"],
        additionalProperties: false,
      },
      strict: true,
    },
  },
];

async function execute(code: string): Promise<[string, string | undefined]> {
  const ex = await ctx.runCode(code, { timeoutSeconds: 60 });
  if (ex.error) return [`Error:\n${ex.error}`, undefined];
  const text = ex.results?.[0]?.html ?? ex.text ?? ex.stdout;
  return [text, ex.png]; // ex.png is the captured chart, no savefig
}

Each Execution exposes typed results so you never parse stdout:

Field	What you get back
`ex.png`	first chart as a base64 PNG (the chart object — no `savefig`)
`ex.results[0].html`	a DataFrame rendered as an HTML `<table>`
`ex.results[0].chart`	the primary image (`png ?? jpeg ?? svg`)
`ex.text`	last-expression text repr
`ex.error`	formatted traceback if the cell raised

The tool-calling loop

Standard OpenAI Chat Completions loop, with one chart-specific subtlety. After each turn you append the assistant message object itself (it carries the tool_calls), then one tool message per tool call with the text result. Tool-role messages are text-only, so when a chart is produced you feed it back as a follow-up user message containing an image_url data-URL — that's how the model actually sees the chart.

messages = [
    {"role": "system", "content": "You are a data analyst. Use run_pandas to compute on `df` and to draw charts."},
    {"role": "user", "content": "Plot total revenue by month and tell me the peak month."},
]

while True:
    resp = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
    msg = resp.choices[0].message
    messages.append(msg)  # the assistant message object carries tool_calls

    if not msg.tool_calls:
        print(msg.content)  # final natural-language answer
        break

    pending_charts = []
    for tc in msg.tool_calls:
        args = json.loads(tc.function.arguments)  # arguments is a JSON string
        text, png_b64 = execute(args["code"])
        # One tool message per tool_call_id — required, must be a string.
        messages.append({"role": "tool", "tool_call_id": tc.id, "content": text})
        if png_b64:
            pending_charts.append(png_b64)

    # Charts can't ride in a tool message — send them as a follow-up user turn.
    for png_b64 in pending_charts:
        messages.append({
            "role": "user",
            "content": [
                {"type": "text", "text": "Here is the chart that was generated."},
                {"type": "image_url",
                 "image_url": {"url": f"data:image/png;base64,{png_b64}", "detail": "auto"}},
            ],
        })

The same chart bytes that go to the model can render straight in a notebook or UI — ex.png is the deliverable:

import base64
from IPython.display import Image

ex = ctx.run_code("df.groupby(df['date'].dt.month)['revenue'].sum().plot.bar(); plt.show()")
Image(data=base64.b64decode(ex.png))   # display inline, no file written

Because the kernel is persistent, the follow-up question ("now break the peak month down by region") reuses the loaded df with no re-parse — the agent just emits another run_pandas call against in-memory state.

Why this is fast

Every Sandbox.create is a snapshot restore of the baked code-interpreter template — sub-second, with IPython and pandas already warm. To explore several analyses from the same loaded dataset in parallel, fork the live sandbox: forks share the parent's memory pages copy-on-write, so ten branches off a 500 MiB session cost a few MiB each until they diverge. See snapshots and forks for the fork API.

Clean up when the conversation ends:

ctx.close()
sandbox.kill()

Gotchas

Tool messages are text-only. You cannot put an image_url part in a {"role": "tool"} message — send the chart in a follow-up user message, as above. This is the most common mistake in chart-returning agents.
tc.function.arguments is a JSON string, not a dict — always json.loads it.
One tool message per tool_call.id. If the model emits N tool calls in a turn, append N tool messages before the next request, or the API rejects it.
Pick a vision-capable model. The model has to read the chart you send back; use a current multimodal model id.

Next steps

Code interpreter — the pillar guide for code contexts, state persistence, and rich results
Run code from a LangGraph agent — the same execution wedge wired into a graph runtime
CrewAI code execution — the drop-in tool for CrewAI agents
Snapshots and forks — fork a loaded session to explore analyses in parallel