Files
tailscale/kernel/linux/src
Brad Fitzpatrick 51ec695feb kernel/linux/src: restore TCP backpressure, and fix tx and queue latency
RTT under a TCP load had grown to about 11ms at p99 (wireguard-go: 3.7).
Instrumenting the queues found three causes:

Backpressure. Upstream encrypts in place, so a packet stays charged to
its socket (for TCP, counted by its small queues, TSQ) until the NIC has
sent it, and TCP is held to the NIC's pace. Our encrypted packets are
new (and earlier ones were orphaned), so TCP could fill the NIC's qdisc
as fast as we encrypt: on a slow uplink with a FIFO qdisc, that would be
massive bufferbloat. Now each encrypted packet takes over its packets'
socket ownership and charge (tswg_move_owner), for destructors that
just uncharge truesize (sock_wfree, and TCP's), settled to its real
size (segments' truesize is about twice their length). TSQ then allowed
TCP only about 1ms in flight, which cost 15-20% of throughput, so
sockets sending over tswg get sk_pacing_shift 9 (about 2ms), as Wi-Fi
drivers set 8 for their aggregation.

The tx worker. Upstream pins a peer's tx worker to one CPU by peer ID.
When that's the CPU taking the NIC's interrupts, softirq work starves
it, and encrypted packets waited up to 20ms to be sent. Queue it on the
CPU that finished encrypting instead (also saving an IPI).

Queue length. The peer queue's limit counted entries, but a GRO entry
is up to 64 packets; count packets, with a limit of 2048.

Medians of interleaved runs in the lean harness, tswg vs. wireguard-go:
TCP 1 stream 10.1 vs. 9.7 Gbps, 4 streams 12.3 vs. 10.8; RTT under load
p50 1.7 vs. 1.7ms. p99 under load is still about 10ms on this testbed:
at full load, the receiving VM's workers sometimes don't run for tens of
milliseconds. That's next to investigate.

Updates #426

Signed-off-by: Brad Fitzpatrick <[email protected]>
Change-Id: Ief80aac9a1e443727c89b990c86c7c8f71690954
2026-11-07 14:32:55 +00:00
..