Fix FastAPI WebSocket 1006 During LLM Streaming: Timeouts, Async Blocking, Keepalive

The Symptom: WebSocket Closed Before the LLM Finished

Your FastAPI WebSocket opens normally. The client sends a prompt. The model begins a long inference operation. Nothing comes back for a while. Then the browser reports:

WebSocket closed
code: 1006
reason: ""

Code 1006 is not a close code sent by FastAPI. RFC 6455 reserves 1006 for abnormal closure where a proper WebSocket Close frame was not received. An endpoint must not put 1006 into a Close control frame. (RFC Editor)

That means 1006 tells you the connection disappeared without a clean handshake. It does not tell you which layer disappeared first.

The Diagnostic Fork: Four Failure Classes

Before changing any timeout, determine which of these four failures you actually have:

Browser / Mobile / Voice Client
              |
              v
        CDN / Edge
              |
              v
     Load Balancer / WAF    <-- Class 3: idle timeout (often 60s)
              |
              v
       NGINX / Ingress      <-- Class 3: proxy_read_timeout (default 60s)
              |
              v
        Uvicorn / ASGI       <-- Class 2: ping/pong timeout (default 20s)
              |
              v
            FastAPI          <-- Class 1: event loop blocked
              |
              v
        LLM Provider          <-- Class 4: upstream hung or cancelled
              |
              v
   Model inference / reasoning

The fastest debugging signal is timing. Measure the gap between last successful traffic and disconnect:

Why the Common Fix Is Wrong

Search for this problem and you will find a confident explanation:

FastAPI automatically kills WebSockets when the LLM takes longer than 10 seconds.

That is wrong. FastAPI does not have a built-in 10-second LLM generation timeout. Current Uvicorn defaults are 20 seconds for --ws-ping-interval and 20 seconds for --ws-ping-timeout. (Uvicorn)

The common proposed fix is a JSON heartbeat daemon:

async def websocket_ping_daemon(websocket):
    while True:
        await asyncio.sleep(5)
        await websocket.send_json({"type": "ping"})

This has good instincts but three critical problems. First, {"type": "ping"} is an application data message, not a WebSocket protocol Ping control frame. Second, if the event loop is blocked by a synchronous call, the heartbeat cannot run either. Third, it does not address proxy or load-balancer idle timeouts that operate independently of application-level traffic.

Class 1: You Are Blocking the Event Loop

This is the failure that most closely matches the original theory. Consider two operations that both take 20 seconds:

# Genuinely async - event loop stays responsive
result = await async_llm_client.generate(prompt)

# Blocking - event loop frozen for 20 seconds
result = synchronous_llm_client.generate(prompt)

With a genuinely asynchronous operation, await gives control back to the event loop. During those 20 seconds, the event loop services other WebSockets, ping/pong handling, timers, and HTTP requests. A synchronous blocking call inside an async handler freezes everything. (Python documentation)

The same principle applies to time.sleep() versus asyncio.sleep(). The problem is not sleep. The problem is blocking the event loop.

Fix A: Use the async SDK

async for token in async_model.stream(prompt):
    await websocket.send_text(token)

Long latency is not automatically event-loop starvation. A model can take 30 seconds before producing output without freezing the event loop, provided the operation is implemented asynchronously.

Fix B: Wrap synchronous SDKs with asyncio.to_thread

result = await asyncio.to_thread(
    blocking_model_call,
    prompt,
)

Python documents asyncio.to_thread() specifically for synchronous I/O that would otherwise block the event loop. (Python documentation)

Fix C: Move CPU-bound work to a process pool

For local inference, embedding post-processing, or expensive parsing, asyncio.to_thread() is limited by CPython's GIL. Use a process pool or a dedicated compute service. Your WebSocket process should not quietly become your compute scheduler. For production voice and real-time AI patterns, see voice agent architecture at Kestrel Voice.

Class 2: Uvicorn WebSocket Ping Timeout

Your model request may be correctly asynchronous and the process responsive, but the WebSocket transport still fails ping/pong health checks. Server logs may show:

websockets.exceptions.ConnectionClosedError:
sent 1011 (internal error) keepalive ping timeout;
no close frame received

The websockets library documents this as a keepalive timeout. (websockets)

Parameter Breakdown

Do not confuse --timeout-keep-alive with WebSocket ping settings. They are separate timeout classes documented separately by Uvicorn. (Uvicorn settings)

Class 3: Proxy or Load-Balancer Idle Timeout

If disconnects happen consistently near 60 seconds, inspect infrastructure before rewriting the application. NGINX defaults proxy_read_timeout to 60 seconds. (Nginx) AWS Application Load Balancers default their idle timeout to 60 seconds. (AWS Documentation)

A WebSocket-aware NGINX configuration:

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

location /stream {
    proxy_pass http://fastapi_upstream;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;
    proxy_read_timeout 300s;
}

The websockets project notes that real HTTP infrastructure commonly closes connections after idle periods in the 30-120 second range. (websockets) When a disconnect occurs at a repeatable infrastructure-sized interval, inspect infrastructure first. For broader production architecture patterns, see production RAG architecture patterns.

Production Configuration: Async Streaming with Serialized Heartbeat

Once the event loop is healthy, add an application heartbeat for session liveness and proxy traffic. Serialize all outbound writes with a lock to prevent interleaved frames:

import asyncio
import time

from fastapi import FastAPI, WebSocket, WebSocketDisconnect

app = FastAPI()

async def send_json_safe(websocket, lock, payload):
    async with lock:
        await websocket.send_json(payload)

async def application_heartbeat(websocket, lock, interval=15.0):
    while True:
        await asyncio.sleep(interval)
        await send_json_safe(websocket, lock, {
            "type": "heartbeat",
            "timestamp": time.time(),
        })

@app.websocket("/stream")
async def stream(websocket: WebSocket):
    await websocket.accept()
    send_lock = asyncio.Lock()
    heartbeat_task = asyncio.create_task(
        application_heartbeat(websocket, send_lock)
    )
    try:
        while True:
            prompt = await websocket.receive_text()
            async for token in stream_model_async(prompt):
                await send_json_safe(websocket, send_lock, {
                    "type": "token", "payload": token,
                })
            await send_json_safe(websocket, send_lock, {
                "type": "complete"
            })
    except WebSocketDisconnect:
        pass
    finally:
        heartbeat_task.cancel()
        await asyncio.gather(heartbeat_task, return_exceptions=True)

For high-concurrency systems with multiple producers, use a bounded asyncio.Queue(maxsize=128) with a single writer task. This gives explicit backpressure instead of unbounded buffering. FastAPI's WebSocket API raises WebSocketDisconnect when a receive operation detects the closed connection. (FastAPI) Use that event to cancel upstream generation and log structured metadata.

What Not to Deploy

Do not increase timeouts to hide event-loop starvation

If the event loop is blocked for 45 seconds and you increase --ws-ping-timeout to 60, you have hidden the defect, not fixed it. A timeout should represent your operating policy, not camouflage.

Do not send JSON heartbeats and call them Ping frames

{"type": "ping"} is an application data message. Native WebSocket Ping/Pong uses protocol control frames handled by Uvicorn's transport layer. Application heartbeats solve a different problem: session liveness, UI status, and keeping intermediaries active.

Do not conflate session lifetime with generation lifetime

A WebSocket can live for hours. That does not mean one LLM request should. Add a separate generation deadline:

async with asyncio.timeout(90):
    async for token in stream_model_async(prompt):
        ...

This lets you fail one generation cleanly while keeping the user's session alive. For governance frameworks around production AI deadlines and evidence, see seven layers of AI compliance.

Diagnostic Decision Tree

Start: WebSocket closed with 1006
  |
  +-- Check server logs for "keepalive ping timeout"
  |     |
  |     +-- YES: Transport latency or event-loop lag
  |     |     +-- Is LLM call async? NO -> Fix: use async SDK or to_thread
  |     |     +-- Is LLM call async? YES -> Increase ws-ping-timeout
  |     |           (only after measuring event-loop lag)
  |
  |     +-- NO: Check disconnect timing pattern
  |           |
  |           +-- Consistent ~60s -> Check proxy_read_timeout,
  |           |                     ALB idle timeout
  |           |
  |           +-- Only during sync calls -> Event-loop blocking
  |           |
  |           +-- Random -> Network, client, process restart
  |           |
  |           +-- Under concurrency -> CPU saturation, backpressure

Verification Checklist

• Capture the exact client close code and timestamp
• Check server-side disconnect or error logs for ping timeout
• Search async handlers for time.sleep(), requests.get(), or sync SDK calls
• Measure event-loop lag with a monitoring task (see below)
• Verify Uvicorn --ws-ping-interval and --ws-ping-timeout values
• Confirm --timeout-keep-alive is not being changed for WebSocket issues
• Inspect NGINX proxy_read_timeout and ALB idle timeout
• Test slow generation deliberately: 5s, 15s, 30s, 60s, 120s with async calls
• Test under production-like concurrency
• Test client reconnection and process restarts
• Confirm generation cancellation fires on client disconnect

Event-loop lag monitor for production:

async def monitor_event_loop_lag(interval=0.5, warning_threshold=0.25):
    loop = asyncio.get_running_loop()
    expected = loop.time() + interval
    while True:
        await asyncio.sleep(interval)
        now = loop.time()
        lag = now - expected
        if lag >= warning_threshold:
            logger.warning("event_loop_lag_seconds=%.3f", lag)
        expected = now + interval

If a blocking call freezes the event loop, this task freezes too. When scheduling resumes, the delay becomes observable. In production, export this as a metric rather than relying on logs alone.

What to Monitor for a Production LLM Streaming Service

A WebSocket service can remain technically up while a significant portion of long-running sessions are unusable. API uptime of 99.9% does not capture that. For production AI operations and deployment patterns, see 12 production readiness checks for AI pilots.

Liveness Is Not Progress

A heartbeat proves the session is alive. It does not prove the model is making progress. A mature streaming system tracks three separate timers:

Transport liveness
     |
     +-- Is the WebSocket alive? (ping/pong, proxy idle)

Generation liveness
     |
     +-- Has upstream produced meaningful activity? (TTFT, inter-event gap)

Application deadline
     |
     +-- How long are we willing to keep this request alive? (product policy)

Keepalive can turn a failed request into a connection that remains successfully stuck forever. Separate the timers. Cancel or account for orphaned model generations when the client disappears. At enterprise scale, uncancelled generations become spend.

If you are deploying real-time LLM streaming and need an independent systems advisor to audit your WebSocket architecture, schedule a strategic evaluation.