Production checklist
Go through this list before going live and again after every larger change to your setup.
Infrastructure
- Your server is in the same region as your endpoints, ideally in the same data center, and you have measured the round-trip time. See Latency & benchmarking.
- The clock is synchronized with PTP or NTP, and you monitor its offset.
- You have your own RPC endpoint for state reads and confirmations, and a second one as a fallback.
- The server has CPU headroom during busy periods. Hot threads are not competing with other workloads.
Credentials and security
- The API key is stored in a secret manager or an environment variable. It is not in the repository and not in the logs.
- The IP allowlist matches the public IP addresses of all your servers, including standby ones.
- You know how to rotate the key quickly. See Authentication.
- The API key and your wallet keys are stored separately. Hot wallets hold only the balance they need.
Stream consumers (Shreds, Private Orderflow)
- Keepalive is configured: gRPC keepalive pings or WebSocket ping/pong.
- The client reconnects automatically with exponential backoff and jitter, and resubscribes after every reconnect.
- The receive loop only puts messages in a queue. Decoding and evaluation run on separate workers.
- The queue is bounded, and its length is monitored.
- Duplicates are handled by transaction signature.
- Private Orderflow over gRPC: your servers are reachable around the clock and keep up with the stream (our relayer drops messages a slow server cannot take). They accept only our relayer's identity key, keep their accounts and programs of interest up to date and send heartbeats. See gRPC push.
- The number of open connections stays within your limit, also during deployments, when the old and the new version may run at the same time. See Limits.
Transaction sending (Private TPU Send)
- Every transaction uses a fresh blockhash, and you keep its
lastValidBlockHeight. - The compute unit limit matches what the transaction actually uses, and the priority fee follows network conditions.
- Transactions are simulated through your RPC wherever your latency budget allows.
- A client-side rate limiter keeps you slightly below your agreed TPS.
- Unconfirmed transactions are resent about once per slot until they are confirmed or their blockhash expires.
- Outcomes are confirmed through your RPC, and you track the landing rate.
- One long-lived HTTPS connection is reused for all requests.
- A fallback sending path exists for outages, and you have tested switching to it.
Monitoring and alerting
Track at least these metrics:
| Metric | Alert when |
|---|---|
| Messages per second, Shreds | No messages for more than a few seconds while the network is active |
| Messages per second, Private Orderflow | No messages during the leader slots of validators in your stake pool. Quieter periods between those slots are normal. See Coverage and stake pool |
| Queue length | It keeps growing |
| Reconnects per hour and close codes | Reconnects become frequent, or any 1008 or authentication error appears |
| Private TPU Send errors by code | Any 401 or 403, or a regular share of 429 or 503 |
| Landing rate and slots to land | A clear drop compared with your baseline |
| Round-trip time to the endpoint | A clear rise compared with your baseline |
Operations
- Logs use UTC timestamps and include the product, region, error codes and close codes. See Errors.
- You follow the Announcements topic in our Telegram group, where maintenance windows are published.
- You have tested recovery by deliberately dropping connections and restarting the consumer.
- Everyone on call knows how to reach our support (@ZoriDev) and what information to include. See Troubleshooting.
Need help or access? Contact our support on Telegram.
Contact