The varz handler got its memstats_* metrics from the expvar package's
"memstats" func, which calls runtime.ReadMemStats and so stopped the
world on every Prometheus scrape. Keep the names but compute them from
runtime/metrics, and never call that func, even from
WritePrometheusExpvar.
While there, export the /tailscale/ metrics from our Go fork (stack
size histogram, stack copy counters, timer zombie counts and lifetime
histogram), which nothing could see before, plus a few upstream ones
with no MemStats equivalent: scheduling latency and GC pause
histograms, live heap, GC and total CPU seconds, thread count, and
mutex wait time. The last replaces derper's hand-rolled version.
Everything read is cheap and a scrape allocates nothing after the
first. Names use a go_runtime_ namespace rather than go_ so they can't
collide with the Prometheus Go client's collector in promvarz binaries.
The runtime's 162-bucket time histograms are reduced to one bucket per
factor of four from 256ns to 1s.
Updates #21300
Updates tailscale/go#189
Updates golang/go#75935
Signed-off-by: Brad Fitzpatrick <[email protected]>
Change-Id: I7e3c9a41f2b85d6e0c4a9b1d3f8e7c2a5b6d4e19
To decide whether we can bump GOAMD64 for our production binaries, we
first need to know what x86-64 microarchitecture levels the fleet's
CPUs actually support. Add two gauges to the default metric set on
linux/amd64: goamd64_capable, the maximum GOAMD64 level the host CPU
supports per /proc/cpuinfo, and goamd64_compiled, the level the
running binary was built with per its build info. Both are computed
once at init and report 0 on error.
The per-level feature lists match the checks in Go's
runtime/asm_amd64.s.
Updates tailscale/corp#47099
Signed-off-by: Brad Fitzpatrick <[email protected]>
Change-Id: I3b4e5431ae1501f0b7d584c12d9af805e9ab6125
Export the machine's boot time (the btime line from Linux's
/proc/stat) as node_boot_time_seconds, named to match what
Prometheus's node exporter uses for the same value. Combined with
process_start_unix_time, this can be used to distinguish process
restarts from whole node restarts.
The value is parsed once per process lifetime, not per scrape, and
the metric is only published when a value is available, so non-Linux
systems don't export a bogus zero.
Updates tailscale/corp#44743
Signed-off-by: Brad Fitzpatrick <[email protected]>
Change-Id: I5f53186b97bb1482bd1a5387c0910b0ae26544ff
Avery found a bunch of tests that fail with -count=2.
Updates tailscale/corp#40176 (tracks making our CI detect them)
Change-Id: Ie3e4398070dd92e4fe0146badddf1254749cca20
Signed-off-by: Brad Fitzpatrick <[email protected]>
Co-authored-by: Avery Pennarun <[email protected]>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <[email protected]>
Currently the expvar exporter attempts to write expvar.String, which
breaks the Prometheus metric page.
Updates tailscale/corp#36552
Signed-off-by: Anton Tolchanov <[email protected]>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <[email protected]>
We were missing this metric, but it can be important for some workloads.
Varz memstats output allocation cost reduced from 30 allocs per
invocation to 1 alloc per invocation.
Updates tailscale/corp#28033
Signed-off-by: James Tucker <[email protected]>
Co-authored-by: Brad Fitzpatrick <[email protected]>
This commit adds a new usermetric package and wires
up metrics across the tailscale client.
Updates tailscale/corp#22075
Co-authored-by: Anton Tolchanov <[email protected]>
Signed-off-by: Kristoffer Dalby <[email protected]>
Some of our labels contain UTF-8 and get mojibaked in the browser
right now.
Updates tailscale/corp#18687
Change-Id: I6069cffd6cc8813df415f06bb308bc2fc3ab65c4
Signed-off-by: Brad Fitzpatrick <[email protected]>
From a problem we hit with how badger registers expvars; it broke
trunkd's exported metrics.
Updates tailscale/corp#1297
Change-Id: I42e1552e25f734c6f521b6e993d57a82849464b2
Signed-off-by: Brad Fitzpatrick <[email protected]>
This splits Prometheus metric handlers exposed by tsweb into two
modules:
- `varz.Handler` exposes Prometheus metrics generated by our expvar
converter;
- `promvarz.Handler` combines our expvar-converted metrics and native
Prometheus metrics.
By default, tsweb will use the promvarz handler, however users can keep
using only the expvar converter. Specifically, `tailscaled` now uses
`varz.Handler` explicitly, which avoids a dependency on the
(heavyweight) Prometheus client.
Updates https://github.com/tailscale/corp/issues/10205
Signed-off-by: Anton Tolchanov <[email protected]>