mirror of
https://github.com/tailscale/tailscale.git
synced 2026-11-20 08:41:19 -08:00
kernel/linux/DESIGN.md: record the 24-peer, MTU 1280 subnet router results
With 24 clients at MTU 1280, tswg forwards 40 Gbps through a c7i.48xlarge (5.6x wireguard-go, 3x kernel WireGuard), using about 22 of its 192 CPUs, and is then limited by EC2's packet rate allowance for the instance. Updates #426 Signed-off-by: Brad Fitzpatrick <[email protected]> Change-Id: Ifdb78f7d85d182a89a42c2f0f9f77bd14344fb59
This commit is contained in:
1 parent
95c00517eb
commit
003fef170a
1 file changed
+13
-2
+13
-2
@@ -369,8 +369,19 @@ At 50 Gbps (2026-11-07): router and backend c7i.48xlarge (192 vCPUs,
|
||||
c7i sizes), and with 8 peers, flows collide on queues (4 clients on 3
|
||||
queues: the two sharing got half each). More peers spread better; a
|
||||
larger MTU needs 7x fewer packets. The router's CPUs weren't the limit.
|
||||
* For 100 Gbps, the next step: c6in instances (up to 200 Gbps), which
|
||||
the IAM policy doesn't allow yet.
|
||||
* With more peers at MTU 1280 (24 clients: the 8 above plus 16
|
||||
c7i.2xlarge, 8 flows each, 192 flows), traffic spreads over all the
|
||||
queues: tswg 39996 Mbps (4.49 CPU-s/GB, about 22 of the 192 CPUs busy),
|
||||
wireguard-go 7107 (5.6x less), kernel WireGuard 13463 (78% of 192
|
||||
CPUs). There the limit is EC2's packet rate allowance for the
|
||||
instance (ENA's `pps_allowance_exceeded` rises during tswg's runs): at
|
||||
MTU 1280 everything the router handles is small (tunnel packets in,
|
||||
1280-byte TCP segments out to the backend, ACKs both ways), about 12M
|
||||
packets per second at 40 Gbps. (The plain baseline isn't comparable
|
||||
here: without a tunnel, the clients' TCP uses the VPC's 9001-byte MTU.)
|
||||
* For more: instances with higher packet rate allowances, such as c6in
|
||||
(network optimized, up to 200 Gbps), which the IAM policy doesn't allow
|
||||
yet.
|
||||
|
||||
Profile of the tswg router at 4 CPUs (CPU-bound), before the
|
||||
optimizations: ChaCha20-Poly1305 about 23% (of which 5.5% was a scalar
|
||||
|
||||
Reference in new issue
Block a user