Advanced: Behind a Reverse Proxy

Note: If you do not have a reverse proxy configured, you can skip this guide.

When deploying AiAgentNexus behind a reverse proxy, it is essential to configure your proxy to handle streaming output properly. This guide provides step-by-step instructions to enable streaming support, ensuring smooth operation.

This guide is intended for advanced deployments. A basic understanding of reverse proxies is recommended.

Nginx Configuration

If you’re using Nginx as your reverse proxy, include the following configuration within your server block:

server {
    listen 80;
    server_name your-domain.com;

    location / {
        # Proxy settings
        proxy_pass http://localhost:3000;  # Replace with your AiAgentNexus service address
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;

        # Enable chunked transfer encoding for streaming
        chunked_transfer_encoding on;
        proxy_buffering off;             # Disable buffering for streaming responses (SSE)
        proxy_cache off;                 # Disable caching
        tcp_nodelay on;                  # Reduce latency by disabling delayed ACKs
        tcp_nopush on;                   # Optimize TCP performance by disabling the Nagle algorithm

        # Extend timeouts for long-running connections
        keepalive_timeout 300;
        proxy_connect_timeout 300;
        proxy_read_timeout 300;
        proxy_send_timeout 300;
    }
}

Key Configuration Details:

  1. Disable Buffering: Disabling proxy buffering ensures real-time delivery of Server-Sent Events (SSE).

  2. Enable Chunked Transfer Encoding: Allows data to be sent in chunks, suitable for streaming responses.

  3. Adjust Timeouts: Extends timeout settings to prevent premature disconnections during long-running connections.

  4. Optimize TCP Settings:

    • tcp_nodelay on: Reduces latency by sending data immediately without delay.

    • tcp_nopush on: Enhances performance by combining small TCP packets where appropriate.

Troubleshooting

If streaming does not work as expected when using a reverse proxy, verify the following:

  • Buffering: Ensure that proxy_buffering is turned off in your configuration.

  • Timeouts: Confirm that the timeouts are sufficiently long to accommodate streaming requirements.

  • Caching: Ensure proxy_cache is disabled to prevent cached responses from interfering with streaming.

  • Reverse Proxy Placement: Verify that the reverse proxy is correctly forwarding requests to the AiAgentNexus service.

Additional Resources

For more detailed information, refer to the following guides:

  • Docker Deployment Guide

  • Kubernetes Deployment Guide

  • Installation Guide

If you encounter any persistent issues, feel free to reach out to our community support channels.

PreviousAnalyticsNext.ENV