Infrastructure
Exchange Co-Location in the Cloud Era: AWS Local Zones, Tokyo/Singapore Strategy for Asian Crypto Venues
Practical infrastructure decisions for crypto trading latency: venue geography, AWS Local Zones vs Direct Connect, Tokyo-Singapore peering, and latency tables from Dubai to each major venue.
ZeroCopy’s infrastructure has a constraint that shapes almost every network and compute decision: I’m based in Dubai, and the major crypto venues are in Tokyo, Singapore, and US East. Designing a trading system that achieves sub-10ms round-trip to Asian venues from a Dubai origin requires understanding the physical geography of internet routing, where the exchange matching engines actually live, and when cloud infrastructure is good enough versus when you need dedicated lines.
This post covers the practical geography of crypto venue infrastructure, the math for when Direct Connect becomes worth it, and the specific latency numbers I’ve measured from Dubai to each major venue.
Where the Matching Engines Actually Live
Most crypto exchange documentation is intentionally vague about server location for security and competitive reasons. Here’s what’s publicly known or can be inferred from latency measurements:
Binance:
- USDT-M Futures matching engine: AWS us-east-1 (Northern Virginia)
- USDC-M: likely same region
- Binance’s co-location program (via DRW’s Cumberland) offers servers in us-east-1
- WebSocket endpoints route to CDN edge nodes, but orders ultimately touch us-east-1
Bybit:
- Primary matching infrastructure: Tokyo, Japan (Equinix TY11)
- Secondary/backup: Singapore (Equinix SG3)
- WebSocket endpoints: Akamai CDN, routes to closest edge
OKX:
- Primary: Singapore (Equinix SG3)
- Co-location available via Equinix SG3 partnership
- Direct WebSocket to matching:
ws.okx.comresolves to Singapore IP ranges
Deribit:
- Amsterdam, Netherlands (Equinix AM1/AM2)
- The only major crypto options exchange with explicit Amsterdam location
Kraken:
- Primarily US-based (AWS us-east-1 and us-west-2)
- European futures: London (Equinix LD4)
HTX (Huobi):
- Singapore primary
- Tokyo secondary
This geography determines everything about your latency budget. If you’re co-located in Tokyo and trading Bybit, you have a structural advantage over someone in Singapore. If you’re trading Binance from Singapore, you have 170ms RTT to us-east-1 regardless of how much you spend on local networking.
The Tokyo-Singapore Latency Reality
Tokyo and Singapore are the two major Asian crypto trading hubs. The physical distance is approximately 5,300 km, which gives a speed-of-light lower bound of ~18ms for a direct fiber path.
Actual measured latency between Equinix TY11 (Tokyo) and Equinix SG3 (Singapore):
Provider RTT (measured 2024)
--------- -------------------
NTT 21-23ms
PCCW 22-25ms
Equinix IX 19-21ms (using Equinix's own peering fabric)
Telia 23-28ms
SoftBank 20-22ms
The Equinix Internet Exchange (ECX) achieves the lowest latency because traffic stays within the Equinix physical network - no public internet routing. If you co-locate at both TY11 and SG3, connect via ECX rather than routing over the public internet.
For a strategy that trades both Bybit (Tokyo) and OKX (Singapore) simultaneously, with a risk engine that needs to see both books before placing orders, the round-trip latency between venues (not to Dubai) is your constraint. At 20ms TY11→SG3, you can synchronize state between venues every 20ms at best.
AWS Local Zones: When They Help, When They Don’t
AWS Local Zones are AWS infrastructure extensions physically located in major metro areas, closer to end users. The relevant ones for Asian crypto trading:
AWS Local Zone Distance to Venue
------------------- -----------------
ap-northeast-1-osaka-1 ~500 km to Tokyo (Osaka to Tokyo by fiber)
ap-southeast-1-bkk-1 ~2,200 km to Singapore (Bangkok to Singapore)
The honest assessment: AWS Local Zones don’t help much for crypto trading latency to exchange matching engines. The matching engines are in specific datacenters (Equinix TY11, Equinix SG3), not in AWS. An AWS Local Zone in Osaka reduces your latency to AWS infrastructure in Osaka, not to Bybit’s matching engine in Tokyo.
The use case where Local Zones do help: latency-sensitive applications where your users are in a metro area and the normal AWS region is too far. For crypto trading, where you’re communicating with specific exchange endpoints, you want to co-locate in the same datacenter as the exchange, not in the closest AWS zone.
The relevant AWS infrastructure for crypto trading:
Region Venue benefit
----------- -----------------
us-east-1 Binance (matching engine in same region)
ap-northeast-1 Best AWS option for Bybit/Tokyo (still 10-15ms to Equinix TY11)
ap-southeast-1 Best AWS option for OKX/Singapore (still 3-8ms to Equinix SG3)
eu-west-1 Best AWS option for Deribit/Amsterdam (still 5-10ms to Equinix AM)
For pure cloud deployments with no co-location:
- Binance trading: us-east-1
- Bybit trading: ap-northeast-1
- OKX/Deribit trading: ap-southeast-1 or eu-west-1 respectively
Direct Connect Math: When It’s Worth $500/Month
AWS Direct Connect provides a dedicated 1 Gbps or 10 Gbps private connection from your datacenter to AWS. Cost: approximately $200-500/month for 1 Gbps depending on location.
The latency benefit comes from two sources:
- Elimination of public internet jitter (±2-5ms on congested paths vs ±0.1ms on dedicated)
- Potentially shorter physical path (Direct Connect PoPs often have more direct fiber routes than public internet peering)
The math for whether Direct Connect pays:
Annual Direct Connect cost: ~$4,800/year (1 Gbps at $400/month)
Break-even calculation:
- If the 2ms latency improvement enables capturing N additional edge cases per year
- Each captured edge = P(capture) × edge_value
- For break-even: N × P × value = $4,800
Concrete example:
- Binance latency from Singapore: 170ms via public internet, 168ms via Direct Connect
- 2ms improvement on 170ms RTT: ~1.2% latency reduction
- At $1M/day notional in strategies sensitive to latency: < $100/day marginal value
→ Direct Connect to us-east-1 from Singapore does NOT pay for itself
- But: Direct Connect from Singapore to ap-southeast-1 (OKX is in Singapore)
- Latency: 3ms via Direct Connect, 5-8ms via public internet
- On strategies sensitive to 2-5ms: depends entirely on alpha
The short answer: Direct Connect is worth it when you’re in the same metro as an AWS region that is itself close to your exchange venue (Singapore → ap-southeast-1 → OKX), and your strategy alpha degrades meaningfully with 2-5ms additional latency. It’s not worth it for cross-ocean paths (Dubai → us-east-1 → Binance) where you’re already at 170ms and the jitter improvement is a rounding error.
Latency Table: Dubai to Each Major Venue
These are measured round-trip times from a Dubai datacenter (AE-Dubai-1, commercial datacenter near Jebel Ali) using ping-equivalent UDP probes to exchange WebSocket endpoints:
Venue (WebSocket endpoint) Avg RTT P99 RTT Geographic route
---------------------------- ------- ------- ----------------
Binance USDT-M Futures 174ms 198ms Dubai → Europe → US East
fstream.binance.com
Bybit Linear 98ms 118ms Dubai → India → SEA → Tokyo
stream.bybit.com
OKX Swap 92ms 112ms Dubai → India → Singapore
ws.okx.com
Deribit 68ms 82ms Dubai → Europe → Amsterdam
www.deribit.com
Kraken Futures 170ms 195ms Dubai → Europe → US East
futures.kraken.com
HTX Global 94ms 115ms Dubai → Singapore
api.huobi.pro
These are WebSocket CDN edge latencies, not matching engine latencies. The CDN adds variable overhead but is representative of order submission round-trip time for most strategies.
With VPN/dedicated line optimization:
- Dubai → Bybit (Tokyo): 92-98ms (vs 98ms above) - small improvement via NTT direct routing
- Dubai → Deribit (Amsterdam): 60-68ms - larger improvement, direct fiber to Amsterdam
- Dubai → Binance (us-east-1): 168-174ms - minimal improvement, route is already optimized
The Co-Location Decision Framework
Given these numbers, here’s how to think about whether to co-locate:
Co-locate if:
- Your strategy’s edge degrades more than 5% with 10ms additional latency
- You’re making more than 10,000 orders/day and are competing with co-located firms
- Your venue is in Tokyo or Singapore where co-location is available at Equinix TY11/SG3
Use cloud if:
- Your strategy’s holding period is > 30 seconds (latency matters less)
- You’re trading Binance from anywhere except us-east-1 (Binance’s matching engine is in AWS; cloud-to-cloud within the same region is nearly equivalent to co-location)
- Your capital allocation doesn’t justify the operational overhead of physical co-location
Hybrid approach (what I run):
- Cloud (AWS ap-southeast-1) for OKX and HTX connectivity
- Cloud (AWS ap-northeast-1) for Bybit connectivity
- No physical co-location - the 5-10ms premium over physical co-location is not alpha-meaningful for my strategies’ holding periods
The key constraint: crypto exchanges don’t offer the same co-location infrastructure as CME or Euronext. CME co-location in Aurora, IL gives you a rack in the same room as the matching engine - 5µs latency. Crypto venue co-location means being in the same Equinix datacenter as the exchange’s internet uplink, typically 1-3ms latency. The absolute values are different, and for crypto, that 1-3ms matters less than for equities HFT.
Infrastructure Configuration for Dubai→Asia Routing
For cloud deployments in the Dubai→Asia routing context, the key optimization is BGP path selection. Public internet routing from Dubai to Singapore often goes Dubai → London → Singapore (the legacy submarine cable route), adding 40ms compared to the Dubai → India → Singapore path.
Force the shorter path with a cloud provider that has direct Dubai→SEA routing:
# Terraform: AWS VPC with optimal routing for Dubai-to-Singapore
resource "aws_vpc" "trading" {
provider = aws.ap_southeast_1 # Singapore
cidr_block = "10.0.0.0/16"
}
# Direct Connect (if warranted by alpha)
resource "aws_dx_connection" "dubai_to_singapore" {
name = "dubai-to-sg-1gbps"
bandwidth = "1Gbps"
location = "SGDF2" # Equinix SG DXC Direct Connect location
}
For Bybit specifically, measure latency via ap-northeast-1 (Tokyo region) as baseline, then compare against a Singapore-based server routing through Equinix ECX to Tokyo. The ECX path from Singapore to Tokyo is often faster than routing AWS ap-northeast-1 traffic through the public internet, depending on AWS’s BGP path choices.
How This Breaks in Production
1. Binance order submission from Singapore - 170ms baseline is unavoidable Symptom: Competitors using co-located servers at Equinix NY4 (next to Binance’s AWS us-east-1 presence) consistently fill before you. Your strategy is profitable in backtests but loses to speed in live trading. Root cause: 170ms RTT from Singapore to us-east-1 means your orders arrive 170ms after they’re generated. A competitor at NY4 has ~1ms RTT. At any moment of directional price movement, they win. Fix: For strategies that compete on speed against Binance co-located firms, you need to either co-locate in us-east-1 (or nearby), or shift to strategies with holding periods where 170ms is irrelevant. Acknowledge this constraint in strategy design.
2. Public internet jitter spike causes spurious OKX fills Symptom: Occasional orders fill at prices significantly worse than mid. P99 of fill price vs mid shows fat tails. Root cause: Public internet jitter from Dubai to Singapore spiked to 150ms on a specific path, causing an order to arrive at OKX 100ms later than expected. Market moved in the interim; your limit order was now a taker order against the new best price. Fix: Implement a maximum acceptable latency check - if the RTT to OKX exceeds a threshold (e.g., 200ms), pause order placement until latency recovers. Monitor via ICMP echo to the exchange WebSocket IP before each order.
3. AWS ap-northeast-1 routing changes increase Tokyo latency Symptom: Bybit latency increases by 8ms overnight with no infrastructure changes. Root cause: AWS updated its BGP peering with a Tokyo ISP. The new path to Equinix TY11 routes through a different PoP with more hops. Fix: Monitor latency continuously with a sidecar process that pings exchange endpoints every 30 seconds. Alert when P50 increases by >5ms from baseline. AWS routing changes are rare but real.
4. Bybit WebSocket CDN edge doesn’t match matching engine location
Symptom: Your Dubai server sees 98ms to stream.bybit.com, but your actual order-to-fill time is 150ms.
Root cause: Bybit’s CDN resolves to a nearby edge node (98ms), but the WebSocket traffic then traverses Bybit’s internal network from the CDN edge to the matching engine in Tokyo. The CDN hop adds processing overhead and a second network segment.
Fix: Use Bybit’s direct API endpoints (not CDN-fronted) if available for institutional users. Measure order-to-fill RTT (time from order send to fill ExecutionReport received) rather than ICMP ping, which only measures the CDN edge.
5. Co-location network misconfiguration - LACP bond failure Symptom: After a weekend maintenance window at the co-location facility, trading latency doubles. No connectivity loss, just slower. Root cause: The LACP (Link Aggregation Control Protocol) bond between your server and the TOR switch failed, falling back to a single 1G link instead of the 4×1G bond. All traffic now flows through one link, creating queue buildup during burst. Fix: Monitor link utilization on the aggregated bond. Alert at >50% on the bond (which means >200% on a single link if the bond fails). Implement automatic failover detection.
For the TCP-level optimizations that matter once you have the right geographic position, see TCP Tuning for Trading. For building the smart order routing that decides which of these venues to use for each order, see Building a Smart Order Router for Fragmented Crypto Liquidity. For the AWS infrastructure patterns for trading workloads, see AWS for Trading: Instances and Networking.
Continue Reading
Enjoyed this?
Get one deep infrastructure insight per week.
Free forever. Unsubscribe anytime.
You're in. Check your inbox.