ClientPrepareNewFlowForMagicIP folded resolving a Magic IP and counting the
flow into one call, and the datapath dropped its own IsValid check on the
resolved Transit IP in favor of trusting the new ok return. That left the
validity requirement stated only in the Conn25Datapath doc comment, and an
implementation that violates it panics the packet path two different ways.
A zero Transit IP is neither Is4 nor Is6, so it slips past both of
checksum.UpdateDstAddr's family-mismatch panics and reaches As4, which
panics in turn; a valid Transit IP in the other family hits one of those
family-mismatch panics outright. Check for both, releasing the counted
reference since no flow is installed, so the packet passes through
unmodified as before.
Updates https://github.com/tailscale/corp/issues/44273
Change-Id: Ib0c0eaa92bc4f0470299da08d48c865e1d101849
Signed-off-by: Mazdak Nasab <[email protected]>
A Magic IP gets its address assignment when the DNS response is rewritten,
but the Transit IP it maps to only becomes usable later: handleAddressAssignment
registers it with the connector over PeerAPI, and only then does
addTransitIPForConnector put it in byConnKey, where extraWireGuardAllowedIPs
picks it up and hands it to WireGuard as an allowed IP for that peer.
Until then, transitIPForMagicIP would happily return the Transit IP, the
datapath would DNAT to it, and WireGuard would drop the packet because no
peer claims the address. The client application saw a black hole and hung.
That window stays open indefinitely when the registration never lands: a
failed PeerAPI request, a full assignment queue, or a reset that clears
byConnKey.
transitIPForMagicIP now checks that the Transit IP is registered under its
connector's node key, and returns the new ErrTransitIPNotRoutable if it
isn't. The datapath treats that like an unknown Magic IP and injects an
ICMP host unreachable, so the application fails fast instead of hanging.
Fixes https://github.com/tailscale/corp/issues/50405
Change-Id: Id232e27063f27278e65279d9b1adad5d67d3b6ef
Signed-off-by: Mazdak Nasab <[email protected]>
The datapath resolved a Magic IP to its Transit IP and then, in a second
lock acquisition, counted the new flow against the assignment holding
that Transit IP. A concurrent reserveAddresses could expire the
assignment in between and hand the Transit IP to an unrelated
destination, so the flow was installed DNATing to an address another
destination now owned, counted against that stranger's assignment.
Merge the two steps into ClientFlowCreatedForMagicIP, which resolves and
counts under one c.mu acquisition.
Fixes https://github.com/tailscale/corp/issues/44273
Change-Id: Iefae902267fd06a5d2c965a3570b6977921be76f
Signed-off-by: Mazdak Nasab <[email protected]>
The cleanup returned by flowCreated decremented activeFlowCount on every
call, and its documented contract pushed the burden of calling it exactly
once onto the datapath. A caller that lost track and ran a cleanup twice
would undercount the flows still forwarding over the assignment, letting
popExpired reclaim both of its addresses back to the pools mid-connection,
or drive the count to -1, which can never again satisfy popExpired's
zero-flow test and so pins the assignment in the expiry heap and leaks
both addresses for the life of the process.
The cleanup now releases its hold at most once, guarded by a bool read and
written under c.mu alongside the decrement it protects. Repeat calls are
no-ops. The Conn25Datapath docs no longer ask callers to guarantee this.
No such double-call is reachable through FlowTable today: removeFlowLocked
deletes both cache entries, so NewFlow's second lookup misses, and the
capacity eviction takes lru.Back(), never the flow just pushed to the
front. This removes the class rather than a live bug.
Updates https://github.com/tailscale/corp/issues/50760
Change-Id: I1888f36eb8fdec9b219ccfd2d290dc4eb19c55a0
Signed-off-by: Mazdak Nasab <[email protected]>
flowRemoved identified an address assignment by transit IP alone, but a
transit IP does not identify an assignment for the lifetime of a flow.
An assignment can be dropped while a flow created against it is still in
the client flow table, and the pools then hand the same transit IP back
out to an unrelated assignment. The stale removal was charged to that
stranger, dropping its activeFlowCount while its own flows were still
forwarding, so popExpired reclaimed both of its addresses back to the
pools mid-connection. If the stranger had no flows of its own the count
went to -1, which can never again satisfy popExpired's zero-flow test,
pinning it in the expiry heap and leaking both addresses for the life of
the process.
flowCreated now returns a function, which the datapath calls on removal
to clean up the flow.
Fixes https://github.com/tailscale/corp/issues/50760
Change-Id: I3693c487b1031737763b5e49f8f8400c72848e8e
Signed-off-by: Mazdak Nasab <[email protected]>
extension.Shutdown closed client.addrsCh, but the hooks that feed that
channel are still live at that point. LocalBackend.shutdown shuts the
extension host down before closing the engine, and nothing unregisters
what installHooks registered on the DNS manager and the tun device, so a
DNS response arriving during the shutdown window still reaches
mapDNSResponse -> rewriteDNSResponse -> reserveAddresses ->
enqueueAddressAssignment, and sends on the closed channel. That's a
panic, not a dropped assignment.
Every send already happens under c.mu, so the client can shut its own
queue down: client.shutdown takes the lock, sets a closed flag and
drains the queue, and enqueueAddressAssignment refuses with
errClientShutdown from then on.
A late DNS response now fails the reservation instead of queueing it for
a sendLoop that is gone. reserveAddresses rolls back, returning the
magic and transit IPs to their pools, and the query servfails rather
than being answered with a magic IP that no connector will ever hear
about.
Fixes https://github.com/tailscale/corp/issues/51235
Change-Id: I712ba8493e77b8176e133decc6097efb52b1b4ab
Signed-off-by: Mazdak Nasab <[email protected]>
Give the client a generation counter, bump it in reset, and stamp each
assignment with it as it is queued. insertTransitConnMapping rejects
assignments from an older generation; because that check happens under
c.mu it is exclusive with reset, which is what actually closes the
window. handleAddressAssignment also checks before sending, so an
assignment invalidated while it was still queued doesn't reach a
connector at all.
Fixes https://github.com/tailscale/corp/issues/51125
Change-Id: Ib524324b6e8a196fdcb5b4cd307854f61c7a0f04
Signed-off-by: Mazdak Nasab <[email protected]>
client.reset clears the IP pools, assignments and byConnKey, but leaves
addrsCh alone. The channel is buffered, so assignments queued before the
reset are still delivered by sendLoop afterwards: each one POSTs to a
connector from the profile we just left, registering a transit IP this
client no longer knows about, and then fails in addTransitIPForConnector
because the assignment is gone from c.assignments.
Drain addrsCh at the end of reset. The drain is non-blocking, so holding
c.mu can't deadlock against reserveAddresses, which enqueues under the
same lock, or against sendLoop, which receives without it.
Fixes https://github.com/tailscale/tailscale/issues/22185
Change-Id: I579131bc54e82d2c797b4741ccaf6cbe7abfdc4a
Signed-off-by: Mazdak Nasab <[email protected]>
reserveAddresses inserts the assignment into c.assignments before
enqueueing it, and leaves it there when the enqueue fails. The connector
never learns about such a mapping, and nothing tell it later.
Add addrAssignments.remove and use it to undo the insert and return both
addresses to their pools when enqueueAddressAssignment fails. The caller
still gets a SERVFAIL, but the next DNS query starts from a clean slate.
Fixes https://github.com/tailscale/corp/issues/50917
Change-Id: I5a387bdcf5a5e87e2dca44867e88ed6149678729
Signed-off-by: Mazdak Nasab <[email protected]>
Similar to b6fc02d for private destinations, a destination inside one of
the configured magic or transit IP pools is never somewhere an app can be
reached: the magic range is installed as a route towards Tailscale on
every configured node, and the transit range belongs to conn25's own
peer-to-connector mappings. Forwarding to one routes back into the
mapping layer instead of out to an app.
Updates https://github.com/tailscale/corp/issues/50118
Updates https://github.com/tailscale/corp/issues/46169
Change-Id: I8116543d4b7a02e1f4c3c6b882ee3ddec1ac4f97
Signed-off-by: Mazdak Nasab <[email protected]>
Similar to 7304dd4 for host-scoped destinations, conn25 needs to
block private destinations: a connector forwards to app destinations,
and the packet filter only ever sees the transit IP, never the
destination, so a private destination would let a client reach into the
connector's own network.
Updates https://github.com/tailscale/corp/issues/50118
Updates https://github.com/tailscale/corp/issues/46169
Change-Id: I700b0b5a270723f5a77907e35c52d729f88c44b6
Signed-off-by: Mazdak Nasab <[email protected]>
Addresses handed out by ippool.next() are marked in use, and the only path
back to the pool is the popExpired loop in reserveAddresses, which can
only see an addrs once it's in the client's assignments. So a reservation
that fails after taking an address leaks it from the pool
for the lifetime of the process.
The leak is the result of interaction between the magic and transit pools.
First an address from magic pool is picked. Then another address from
the transit pool is picked. If an error happens during picking an address
from transit pool, the address from the magic pool needs to be returned,
otherwise that address is leaked, as the address in not in the client
assignments and won't be released.
Fixes https://github.com/tailscale/tailscale/pull/22019
Change-Id: I3d98282762576cc57c135761859902a1d0a63c31
Signed-off-by: Mazdak Nasab <[email protected]>
Additionals and authorities are two optional sections
in a DNS response. Conn25 treats DNS responses differently,
based on the response type. Adding test coverage for these
optional sections to assert the expected behavior.
Updates #cleanup
Change-Id: I778af5b368f53f6209fe2934aa98d9009c7fa395
Signed-off-by: Mazdak Nasab <[email protected]>
Lock mutex in lookupTransitIPsByConnKey right
before critical section, rather than in the caller.
Updates #cleanup
Change-Id: I8ee8950dce056e1945d4356d82f0bffbc6688624
Signed-off-by: Mazdak Nasab <[email protected]>
Write through email related DNS responses (MX and TXT),
and voip ones (SRV and NAPTR) as we try to not break them.
Fixes https://github.com/tailscale/corp/issues/39982
Change-Id: I920c4e4401f5a20bcdcad092e94fce8ffa0f12b5
Signed-off-by: Mazdak Nasab <[email protected]>
Lock shared resources right before access
for clarity and preventing bugs.
Updates #cleanup
Change-Id: I1f905a3e4ec8b9ede4e8846e84e7bc50d78c67f0
Signed-off-by: Mazdak Nasab <[email protected]>