> ## Documentation Index > Fetch the complete documentation index at: https://docs.strata.io/llms.txt > Use this file to discover all available pages before exploring further. # Scale for Production By the end of this guide, you will have a horizontally scaled Maverics deployment with multiple Orchestrator instances, sticky sessions, local session storage, and mode-appropriate Redis caching. A single Orchestrator instance can handle a significant amount of traffic -- the Orchestrator is small, lightweight, and efficient. But production workloads often require multiple instances for high availability (if one instance goes down, the others continue serving traffic) and for handling peak loads that exceed a single instance's capacity. Scaling the Orchestrator horizontally requires sticky sessions (cookie-based session affinity) on your load balancer so that each user's requests consistently reach the same Orchestrator instance, plus Redis caching when your mode requires cross-instance data sharing. **Console terminology:** In the Maverics Console, Orchestrator instances and configuration delivery are managed through **Deployments**. When working directly with YAML, configuration is managed as files delivered via the `-config` flag or `MAVERICS_CONFIG` environment variable. ## Prerequisites * **A running Orchestrator deployment** -- If you have not deployed to production yet, follow the [Deploy to Production guide](/guides/operations/deploy) first. * **A load balancer that supports cookie-based session affinity (sticky sessions)** -- Any HTTP load balancer that supports health check-based routing and cookie-based affinity (NGINX Plus, HAProxy, AWS ALB, F5 BIG-IP, GCP Load Balancer, etc.). * **A Redis instance** -- Required for OIDC Provider mode; recommended for SAML Provider mode; not needed for other modes. See the [Redis Cache reference](/reference/orchestrator/caches/redis) for setup. ## Scale Your Deployment Sessions are stored locally on each Orchestrator instance. Sticky sessions (session affinity) ensure that each user's requests consistently reach the same instance, so the Orchestrator can find the user's session on every request. Configure your load balancer to use **cookie-based affinity** targeting the `maverics_session` cookie (or your custom `session.cookie.name` value). Cookie-based affinity is more reliable than IP-based affinity because it works correctly when multiple users share the same IP address (e.g., behind a corporate NAT) or when a user's IP changes mid-session (e.g., switching networks on a mobile device). **Console UI documentation is coming soon.** This section will walk you through configuring this component using the Maverics Console's visual interface, including step-by-step screenshots and field descriptions.

Configure local session storage on each Orchestrator instance. The load balancer handles routing -- the Orchestrator configuration is the same as a single-node deployment: ```yaml maverics.yaml theme={null} session: cookie: name: "maverics_session" domain: ".example.com" lifetime: maxTimeout: "12h" idleTimeout: "30m" store: type: "local" local: capacity: 50000 ``` See [Sessions Reference](/reference/orchestrator/sessions) for all session configuration fields. If an Orchestrator node goes down, sessions on that node are lost. Users are transparently re-authenticated via their upstream IdP session -- they experience a brief redirect but are not prompted to log in again. Redis is a **cache** for connector tokens and provider data -- not a session store. In multi-node deployments, Redis ensures that provider-specific data (authorization codes, SAML request data, connector tokens) is accessible from any Orchestrator instance, even though each user's requests are routed to the same instance via sticky sessions. The need for Redis depends on which Orchestrator mode you are using: * **OIDC Provider:** Redis cache is **required** for multi-node deployments. The OIDC Provider stores authorization codes, token state, and provider data in the cache. Without Redis, an authorization code issued by one instance cannot be redeemed by another -- and even with sticky sessions, the token endpoint request may arrive from the application server rather than the user's browser, bypassing session affinity. * **SAML Provider:** Redis cache is **recommended**. The SAML Provider caches SAML request data (AuthnRequest and LogoutRequest form parameters) so that after a user authenticates upstream, any Orchestrator instance can restore the original request context and generate the SAML response. Without Redis, if the authentication callback arrives at a different instance than the one that received the original AuthnRequest, the SAML response flow will fail because the request data is only in local memory. * **HTTP Proxy:** Redis cache is **not needed**. Each instance manages its own connector tokens independently. * **LDAP Provider:** A shared cache is **not typically needed** for standalone LDAP Provider deployments -- the LDAP Provider uses per-connection bind state. However, in **facade (sandwich) deployments** where HTTP Proxy and LDAP Provider run as separate Orchestrator instances, a shared cache **is required** so that one-time credentials generated by the proxy can be validated by the LDAP Provider. When both modes run on a single Orchestrator, the local cache is sufficient. See [LDAP Provider -- Pairing with HTTP Proxy](/reference/modes/ldap-provider#pairing-with-http-proxy) and the [Caches reference](/reference/orchestrator/caches). * **AI Identity Gateway:** A cache is **not typically needed**. AI Identity Gateway uses token-based state tied to the upstream OAuth authorization server. **Console UI documentation is coming soon.** This section will walk you through configuring this component using the Maverics Console's visual interface, including step-by-step screenshots and field descriptions.

Define a Redis cache and reference it from your provider configuration: ```yaml maverics.yaml theme={null} caches: - name: "my-redis" type: "redis" redis: addresses: - "redis.example.com:6379" password: tls: "redis-tls" # Reference the cache from your provider: oidcProvider: cache: "my-redis" # ... other oidcProvider config samlProvider: cache: "my-redis" # ... other samlProvider config ``` See [Redis Cache Reference](/reference/orchestrator/caches/redis) for all Redis cache fields including encryption and hashing options. A load balancer distributes incoming requests across your Orchestrator instances. Configure it to use health check-based routing so that traffic only goes to healthy instances, and cookie-based sticky sessions so that each user's requests reach the same instance. Key load balancer configuration: * **Health check endpoint** -- Point the load balancer's health check at the Orchestrator's `/status` endpoint. Remove instances from the pool when they return an unhealthy status. * **Sticky sessions** -- Configure cookie-based session affinity using the `maverics_session` cookie (or your custom `session.cookie.name` value). This ensures each user's requests consistently reach the same Orchestrator instance. * **Connection draining** -- When an instance is removed from the pool (during a rolling update or failure), allow existing connections to complete before terminating the instance. This prevents request failures during deployments. * **TLS termination** -- Decide whether TLS terminates at the load balancer or at the Orchestrator. Terminating at the load balancer is simpler to manage when running multiple instances. Load balancers are external to the Orchestrator -- they are configured in your infrastructure layer, not in `maverics.yaml`. Select your load balancer below for configuration examples: ```nginx theme={null} upstream maverics { server 10.0.1.10:9443; server 10.0.1.11:9443; server 10.0.1.12:9443; # Use the Orchestrator's session cookie for affinity. # If you customized session.cookie.name, use that value instead. sticky cookie maverics_session; } server { listen 443 ssl; server_name app.example.com; location / { proxy_pass https://maverics; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; } location /status { proxy_pass https://maverics/status; } } ``` ```haproxy theme={null} frontend maverics_fe bind *:443 ssl crt /etc/haproxy/certs/app.pem default_backend maverics_be backend maverics_be cookie maverics_session insert indirect nocache server orch1 10.0.1.10:9443 ssl check cookie orch1 server orch2 10.0.1.11:9443 ssl check cookie orch2 server orch3 10.0.1.12:9443 ssl check cookie orch3 option httpchk GET /status ``` HAProxy's `cookie` directive in the backend inserts a cookie to track which server handles each session. The `insert indirect nocache` options tell HAProxy to manage the affinity cookie transparently. Enable application cookie-based stickiness on the ALB target group: ```bash theme={null} aws elbv2 modify-target-group-attributes \ --target-group-arn arn:aws:elasticloadbalancing:REGION:ACCOUNT:targetgroup/maverics/XXXXX \ --attributes \ Key=stickiness.enabled,Value=true \ Key=stickiness.type,Value=app_cookie \ Key=stickiness.app_cookie.cookie_name,Value=maverics_session \ Key=stickiness.app_cookie.duration_seconds,Value=86400 ``` In the AWS Console: navigate to **EC2 > Target Groups > your target group > Attributes > Stickiness**, enable stickiness, select **Application-based cookie**, and set the cookie name to `maverics_session`. Use an iRule or a persistence profile with cookie-based persistence targeting the `maverics_session` cookie. Consult your load balancer's documentation for cookie-based affinity configuration. Enable cookie-based affinity on the backend settings. Azure Application Gateway uses its own affinity cookie by default, but can be configured for application-managed affinity. Consult the Azure documentation for cookie-based affinity configuration. Use a backend service with `GENERATED_COOKIE` or `HTTP_COOKIE` session affinity and specify the `maverics_session` cookie name. Consult the GCP documentation for cookie-based affinity configuration. Use the sticky sessions configuration in the load balancer service definition with a cookie name of `maverics_session`. Consult the Traefik documentation for sticky session configuration. Always use **cookie-based affinity** rather than IP-based affinity. Cookie-based affinity works correctly when users share IPs (corporate NAT) or change networks mid-session (mobile devices). Each user's requests will consistently reach the same Orchestrator instance based on their session cookie. With multiple instances behind a load balancer and sticky sessions configured, verify that your deployment is truly highly available by testing failover. ```bash theme={null} # Verify all instances are healthy via the load balancer curl -s https://your-load-balancer:9443/status | jq . # Authenticate through the load balancer and verify your session works # Make several requests to confirm sticky sessions are routing correctly ``` Test a failover scenario: 1. **Authenticate through the load balancer** -- Establish a session 2. **Stop one Orchestrator instance** -- Simulate a failure 3. **Make another request** -- If the stopped instance was handling your session, the load balancer routes you to a healthy instance. The Orchestrator redirects you to your upstream IdP, which recognizes your existing IdP session and completes authentication silently -- you experience a brief redirect but are not prompted to log in again. 4. **Restart the stopped instance** -- It should rejoin the pool automatically once the health check passes **Success!** Your Orchestrator deployment is horizontally scaled with high availability. Each instance stores sessions locally, the load balancer uses cookie-based sticky sessions to route users consistently, and failover triggers transparent re-authentication via the upstream IdP. ## Troubleshooting If users are being asked to re-authenticate unexpectedly, sticky sessions may be misconfigured: * Verify the load balancer has cookie-based session affinity configured on the session cookie (`maverics_session` by default, or your custom `session.cookie.name` value). * Verify all instances use `store.type: "local"`. * Check load balancer logs to confirm requests from the same user consistently reach the same instance. * If using IP-based affinity instead of cookie-based, switch to cookie-based affinity. IP-based affinity breaks when multiple users share the same IP (corporate NAT) or when a user's IP changes mid-session. If a new or restarted Orchestrator instance is not receiving traffic, check that: * The instance's health endpoint is returning a healthy status * The load balancer's health check is configured to poll the correct port and path * The health check interval and threshold are set appropriately -- some load balancers require multiple consecutive healthy responses before adding an instance to the pool * Network rules allow the load balancer to reach the instance on both the application port and the health check port ## Related Pages Back to the Operations guides hub Scale the two-tier AI Identity Gateway architecture with MCP-specific load balancing and token exchange tuning Set up your production deployment before scaling horizontally Session store types and session configuration Redis cache configuration for connector tokens and provider data