mirror of
https://github.com/tailscale/tailscale.git
synced 2026-11-20 08:41:19 -08:00
For a 50,000-node tailnet, every peer operation must be O(1), and a peer that rotates its key must keep its traffic. Measured in a lab VM with up to 50,000 peers (tailscale.com's TestLeanScale), three things were off, plus key rotation: Deleting a device removed its peers one at a time, each waiting for an RCU grace period (netif_napi_del) and three workqueue flushes: 6ms a peer, 5 minutes for 50,000, on every tailscaled restart. Now all of a device's peers wait together (peer_remove_dead_batch): 50,000 in 0.1-0.2s. Each peer's route cache was per CPU, 40 bytes on each: 5KB a peer on a 128-CPU host, 250MB for 50,000 peers, most of which never send. It's now allocated on a peer's first send. A peer is 2.1KB, whatever the CPUs. The pubkey and index hash tables had 2^11 and 2^13 buckets, chains of 25 at 50,000 peers; now 2^16 and 2^17 (1.5MB a device). And TSWGPEER_A_NEW_PUBLIC_KEY renames a peer: in one step, under device_update_lock, it moves to its new key in the pubkey table, with its static-static and cookie keys recomputed and its sessions cleared, keeping its allowed IPs, endpoint and queues. Removing it and adding it anew would drop its traffic in between. At 50,000 peers, adding, removing, renaming and editing one take 26-83 microseconds, as at 500. Updates #426 Signed-off-by: Brad Fitzpatrick <[email protected]> Change-Id: I2abc5369ce8321267b39df9d9dc137ef7dc040a7