Real-Time Tool Call Updates in DSPy with Status Streaming

Agents are getting more capable — and slower. A simple chatbot responds in 2 seconds. An agent that searches the web, queries databases, and synthesizes results? That’s 15-30 seconds of dead air.

Users don’t mind waiting if they know something’s happening. A UI pattern has emerged: show intermediate status updates while the agent works. You’ve seen this in ChatGPT (“Searching the web…”), Perplexity, and Claude’s tool use. It turns an anxiety-inducing wait into something users can follow along with.

Turns out, this is surprisingly easy to do with DSPy.

What It Looks Like

A ReAct agent calling tools now shows:

🔍 Searching the web for: climate change...
✅ Result: Climate change refers to long-term shifts...
🌤️ Fetching weather for: Tokyo...
✅ Result: Weather in Tokyo: 80°F, Sunny
🤖 Thinking...

How It Works

DSPy’s streamify accepts a StatusMessageProvider that hooks into tool and LM lifecycle events:

import dspy

class MyStatusProvider(dspy.streaming.StatusMessageProvider):
    def tool_start_status_message(self, instance, inputs):
        return f"🔍 Calling {instance.name}..."
    
    def tool_end_status_message(self, outputs):
        return f"✅ Done: {str(outputs)[:50]}"
    
    def lm_start_status_message(self, instance, inputs):
        return "🤖 Thinking..."

Available hooks:

tool_start_status_message / tool_end_status_message
lm_start_status_message / lm_end_status_message
module_start_status_message / module_end_status_message

Wiring It Up

# Create your agent
react = dspy.ReAct("question -> answer", tools=[...])

# Wrap with streaming + status provider
streaming_agent = dspy.streamify(
    react,
    status_message_provider=MyStatusProvider(),
)

# Consume the stream
async for item in streaming_agent(question="..."):
    if isinstance(item, dspy.streaming.StatusMessage):
        print(item.message)  # Tool/LM status updates
    elif isinstance(item, dspy.Prediction):
        print(item.answer)  # Final result

Multi-Module Pipelines

For pipelines with multiple stages, use instance to identify which tool/module fired:

class PipelineStatusProvider(dspy.streaming.StatusMessageProvider):
    
    def tool_start_status_message(self, instance, inputs):
        tool_name = instance.name
        if tool_name == "search_web":
            return f"🔍 [Research] Searching: {inputs.get('query', '')[:40]}"
        elif tool_name == "calculate":
            return f"🧮 [Analysis] Computing: {inputs.get('expr', '')}"
        return f"⚙️ Running {tool_name}..."
    
    def module_start_status_message(self, instance, inputs):
        name = instance.__class__.__name__
        if name == "ResearchModule":
            return "📚 Starting research phase..."
        elif name == "AnalysisModule":
            return "📊 Starting analysis phase..."
        return None  # Return None to skip status for this module

Server-Sent Events

For web apps, wrap this in FastAPI and emit SSE:

@app.post("/v1/query")
async def query_stream(q: Query):
    async def generate():
        async for item in streaming_agent(question=q.text):
            if isinstance(item, dspy.streaming.StatusMessage):
                yield f"data: {json.dumps({'type': 'status', 'msg': item.message})}\n\n"
            elif isinstance(item, dspy.Prediction):
                yield f"data: {json.dumps({'type': 'result', 'answer': item.answer})}\n\n"
        yield "data: [DONE]\n\n"
    
    return StreamingResponse(generate(), media_type="text/event-stream")

Full Demo

I put together a self-contained script that spins up a FastAPI server with a ReAct agent, runs a demo query with status streaming, and cleans up:

GitHub Gist: dspy_streaming_demo.py

Setup:

    export OPENAI_API_KEY="sk-your-key"
    python3 -m venv .venv
    source .venv/bin/activate
    python3 -m pip install fastapi uvicorn httpx dspy

Run:

    python3 dspy_status_streaming.py
    # OR
    python3 dspy_status_streaming.py "Your custom question here"

Status streaming turns a black-box agent into something users can actually follow. Small addition, big UX win.

DSPy Status Streaming