663 Commits
Author SHA1 Message Date
Ethan af4421348f Use camera timing for independent One Euro filters (#268) 2026-10-27 19:04:05 -07:00
Hash eb85ce1932 Added BrowPinch mapping (#266)
* Mapping of EyeBrowUp to Eye Wide
* Added brow Pinch values
2026-10-27 19:04:05 -07:00
Hash 15c226094d Mapping of EyeBrowUp to Eye Wide (#261)
Mapping of EyeBrowUp to Eye Wide
2026-10-27 19:04:04 -07:00
dan2wik d84de1cd1a IpCameraCapture: resolve .local via in-process mDNS when the OS resolver can't
The app runs inside the Steam Linux Runtime (pressure-vessel) sandbox, whose minimal nsswitch has no nss-mdns/Avahi, so getaddrinfo returns 'Name or service not known' for .local hosts even though the host shell resolves them. Add a one-shot multicast-DNS A-record query (RFC 6762) as a fallback when the OS resolver fails for a .local name; it only needs multicast UDP, which the sandbox has. Direct IPs and normal DNS are unaffected (OS resolver still tried first).
2026-10-27 19:04:04 -07:00
dan2wik a0d731a920 ActivationService: comment out boot-time OpenVR autostart to avoid OpenComposite crash 2026-10-27 19:04:04 -07:00
dan2wik f3782d2fb0 IpCameraCapture: resolve mDNS .local unfiltered; surface real failure reason
- Resolve DNS unfiltered (AF_UNSPEC) like browsers/curl, then order results IPv4-first instead of querying IPv4-only; the family-filtered lookup failed to resolve .local mDNS names on many systems (direct IP worked because it skips resolution)
- Log the resolved address(es) for the host
- Flatten the exception chain into the failure log so the underlying cause is visible even when the log sink doesn't render exceptions
- Add a 'connecting...' debug line; drop the HttpClient overall timeout (stream is open-ended, ConnectTimeout bounds the open)
2026-10-27 19:04:03 -07:00
dan2wik 37d2d2c72f LocalSettingsService: atomic writes + corruption-safe load to stop config wipes
- Write via temp file + atomic rename, keeping the prior contents as .bak; a kill/crash mid-write can no longer truncate the live settings file
- On load, recover from .bak when the main file is corrupt; set an unrecoverable file aside as .bad instead of silently resetting to empty; refuse to overwrite a file that couldn't be read at all
- SaveSetting skips unchanged values to stop the redundant write churn (VM init re-saves, per-retry camera-state saves)
- App.axaml.cs: final atomic flush right before the Windows force-kill so the last state lands and no debounced write is left mid-flight
2026-10-27 19:04:03 -07:00
dan2wik 15451f51ad IpCameraCapture: IPv4-preferring connect, forceful teardown, error logging
- Prefer IPv4 (A-record) connect with a 4s ConnectTimeout so a .local name opens as fast/deterministically as a raw IP, instead of stalling on an IPv6 link-local past the frame window
- StopCapture disposes the HttpClient (not just cancels) to abort a stuck socket read and close it, freeing the single-connection camera's slot so failover/retry can connect; non-blocking
- Log HTTP status, non-MJPEG content-type, connection failures, and first-frame delivery so silent 'no data' failures become diagnosable
2026-10-27 19:04:02 -07:00
dan2wik dd33a67008 Drive titlebar version from informational version; allow CI override
- MainWindow: read AssemblyInformationalVersion (keeps -prerelease suffix), strip +<git-sha>
- Baballonia.csproj: add Version default, disable source-revision suffix
- build.yml: pass -p:Version from the workflow version-number input when set
2026-10-27 19:04:02 -07:00
dan2wik 9393875494 Fix streaming-camera regression; tighten capture URL routing
- OpenCvCapture: scope IsOpened() fail-fast to local devices so network streams aren't aborted before the read loop pumps frames
- IpCameraCaptureFactory: accept https MJPEG streams, not just http
- SingleCameraSourceFactory: prefix scheme-less hosts (e.g. openiristracker.local) with http:// so bare endpoints reach a backend
- OpenCVCaptureFactory: claim only an explicit streaming-scheme allowlist instead of any absolute URI
2026-10-27 19:04:02 -07:00
dan2wik 6e9d9b83de DefaultInferenceRunner: throw when output size matches no known layout 2026-10-27 19:04:01 -07:00
dan2wik 33715d69bb DefaultInferenceRunner: demote model-expr init log to Debug 2026-10-27 19:04:01 -07:00
dan2wik 72467f2659 OneEuroFilter: copy spans via CopyTo in zero-dt branch 2026-10-27 19:04:01 -07:00
dan2wik 63e828890e OneEuroFilter: use MathF.Tau instead of local TwoPi const 2026-10-27 19:04:00 -07:00
dan2wik 9e1626cc0d Prefer the VFT backend for identified Vive Facial Trackers
Identify the tracker during enumeration by USB VID 0BB4/PID 0321 or the
"HTC Boot" name, and force the VFT capture backend over the generic
"Normal Camera"/"V4L2 Camera" ones, which otherwise open it un-activated
and mis-decode the image.
2026-10-27 19:04:00 -07:00
dan2wik 219a29ff2c Run the calibration overlay invariant so it boots under Steam Runtime 4
BabbleCalibration is a separate .NET (Godot Mono) binary with its own runtime and
no app-local ICU, so under steamrt4 (no libicui18n) its globalization init
FailFast/SIGABRTs at launch, which blocked calibration and training. Launch the
overlay child process with DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1.
2026-10-27 19:03:59 -07:00
dan2wik f731165051 Renaming linux workaround key to avoid stale translations 2026-10-27 19:03:59 -07:00
dan2wik 2c34a57ca3 Updated and improved linux workaround instructions. 2026-10-27 19:03:59 -07:00
dan2wik 7fd9cd79eb Flag flawed/unused SingleCameraSource branches in EyePipelineManager
Left/right start paths build source but never assign it to VideoSource, and are unreachable: TryStart{Left,Right}IfNotRunning no-op on a SingleCameraSource.
2026-10-27 19:03:58 -07:00
dan2wik ee9bf83e13 Mount the BabbleTrainer symlink target into the steamrt4 build container 2026-10-27 19:03:58 -07:00
dan2wik 9513d4e41b Fix SteamVR autostart toggle off path and crash
- Apply user toggles reliably instead of swallowing them during load
- Use a short-lived OpenVR session so toggling after SteamVR closes can't crash
2026-10-27 19:03:58 -07:00
dan2wik 932ca3ea92 Fix build warnings and the broken SteamVR autostart toggle
- Clear NU1510/CA2017/CS0105/CS0649/MSB3568 warnings
- Wire up OpenVRService so the autostart toggle works
2026-10-27 19:03:57 -07:00
dan2wik 7b2d62b876 Bump Tmds.DBus.Protocol to 0.21.3 to fix GHSA-xrw6-gwf8-vvr9 2026-10-27 19:03:57 -07:00
dan2wik 80339ebe96 Exit cleanly on the Linux DBus teardown crash at shutdown
At exit, Avalonia's Linux DBus integration (tray / platform) disconnects on a
background thread and marshals a notification to the already-stopped UI dispatcher,
surfacing as an unhandled TaskCanceledException that aborted the process.

- Add global AppDomain.UnhandledException + TaskScheduler.UnobservedTaskException
  handlers (there were none).
- During our shutdown, a benign cancellation now Environment.Exit(0)s instead of
  aborting; all other unhandled/unobserved exceptions are logged, not swallowed.
- Complements 76516a9 (Windows exit path); this covers the Linux path.
2026-10-27 19:03:57 -07:00
dan2wik 757f07dca3 Fix Windows exit hang from wedged camera capture thread
- App: bounded host dispose then TerminateProcess on Windows exit
- LocalSettings: ForceSave writes synchronously before terminate
- DebounceFunction: add Cancel()
- VFT backend: only probe devices present in enumeration
2026-10-27 19:03:56 -07:00
dan2wik 5c9944a631 Make OSC send/receive self-healing across consumer restarts
VRChat / the VRCFT module / DFR can start, stop, or restart at any time relative to
us. Previously a failed connect or bind left the path permanently dead and a send
error was swallowed with no recovery, so any disconnect needed an app restart.

Send (OscSendService):
- Lazily (re)establish the connected UDP socket with backoff, so the consumer can
  come up before or after us and output starts flowing on its own.
- Stop trusting Socket.Connected as UDP liveness; a send failure no longer tears the
  path down. Transient errors are rate-limited; the socket is rebuilt only when
  disposed. On Windows suppress ICMP UDP-connreset so output resumes silently when
  the consumer returns.
- Snapshot/lock the socket so a re-target can't race an in-flight send.

Receive (OscRecvService):
- The receive loop owns binding and retries with backoff, so a temporarily-held
  in-port (companion app / restart) recovers automatically instead of staying dead.
- Re-targeting closes the socket to wake the in-flight receive; socket access is
  serialized; a malformed datagram is isolated so it can't blackout the loop.

VrcftModuleSendService:
- TryParse the stored address with loopback fallback (Parse ran during DI and a bad
  stored value crashed host startup); also re-apply on DestinationAddress changes.
2026-10-27 19:03:56 -07:00
dan2wik f71d5e18ab Fix DFR/native eye OSC crash that stopped all eye cameras
ProcessNativeVrcEyeTracking read /leftEyePitch, /leftEyeYaw, /rightEyePitch,
/rightEyeYaw — keys the eye runner never produces (it emits /leftEyeX, /leftEyeY,
/leftEyeLid, ...). The throwing OrderedFloatMap indexer raised KeyNotFoundException
on the first eye frame whenever Use-DFR or VRChat-native eye tracking was enabled;
the exception propagated into ProcessingLoopService.EyeWorker, whose catch calls
StopAllCameras, permanently killing eye tracking until restart.

- Read the produced /leftEyeX//leftEyeY//leftEyeLid keys via non-throwing TryGetValue
- Wrap the native/DFR fan-out so an OSC build error can't reach the worker catch
2026-10-27 19:03:55 -07:00
dan2wik 3ecbb860cc Re-enumerate cameras on a friendly-name miss
The device map was cached with ??= and only ever populated on the first start attempt. If that happened with the camera absent (booted with none connected, or clicked start early), the friendly name never resolved to its /dev/video path, so CanConnect rejected every backend and the camera was unopenable until an app restart. Refresh the map once on a lookup miss so a later hot-plug is picked up.
2026-10-27 19:03:55 -07:00
dan2wik 5493fa9dd0 Send raw, unfiltered eye data to the DFR/native OSC output
The OneEuroFilter ran upstream of the OSC fan-out, so enabling it also smoothed the DFR (port 9020) and VRChat-native eye streams, adding latency where native eye tracking wants the rawest data. Keep VRCFT filtered; feed the native paths the pre-filter values.

- EyeProcessingPipeline: expose RawEyeResult, the un-smoothed result, geometry-corrected like the returned map but skipping the filter.
- ProcessingLoopService: carry the raw map on the Expressions event.
- ParameterSenderService: DFR and native eye tracking read the raw map; VRCFT keeps the filtered map.
2026-10-27 19:03:55 -07:00
dan2wik de7d596f0b Prevent eye/mouth sharing one camera; fix capture wedging
Block the mouth and an eye using the same device (split left+right stays
shared); autostart eyes first so stale configs heal. OpenCvCapture: back
off on failed reads and make StopCapture non-blocking.
2026-10-27 19:03:54 -07:00
dan2wik db0c3c34df Show Linux CI jobs as steamrt4-x64 / steamrt4-arm 2026-10-27 19:03:54 -07:00
dan2wik a5031b83e9 Install patchelf in the steamrt4 build container for the arm64 SkiaSharp fixup
The arm64 PatchSkiaSharp target runs `patchelf --add-needed` on libSkiaSharp.so,
but the steamrt4 SDK container has no patchelf (the old ubuntu-arm runner did).
Add it to the CI build prerequisites and the local build image.
2026-10-27 19:03:54 -07:00
dan2wik 18f4924971 Run under Steam Runtime 4 and harden firmware/serial handling
Steam Linux Runtime 4 deployment:
- Bundle app-local ICU so the app boots under steamrt4 (no system libicui18n)
- Use the GStreamer-free OpenCV mini-runtime (4.11.0.2) on Linux
- Access V4L2 through libc instead of the unversioned libv4l2 soname (LibV4L2 + VFT)
- Build Linux CI artifacts inside the steamrt4 SDK container
- Exclude the local CUDA trainer bundle (_internal) and fix the MSB3024 copy error
- Fall back across capture backends without an unhandled exception aborting startup
- Add steamrt4 build/run scripts; gitignore .vscode and the trainer _internal bundle

Firmware / serial:
- Surface live espflash progress and a clear failure state instead of always "Done!"
- Stop only serial camera feeds on Refresh (never on tab navigation); release all
  firmware serial handles when leaving the tab
- Skip version-unsupported requests (SetPausedRequest on v2) and drop dead-port sessions
- Fix the SerialCameraCapture dispose crash on device unplug
2026-10-27 19:03:53 -07:00
dan2wik a12f9cab3e Re-init OneEuroFilter when the model output shape changes
A model hot-swap (training/calibration) replaces the inference service but
reuses the filter, whose per-element state was sized to the first frame. A
model with a different output count then overran those arrays
(IndexOutOfRangeException). Detect a shape change and re-initialize.
2026-10-27 19:03:53 -07:00
dan2wik 6df5eb66e7 Fall back to the next capture backend when one can't deliver frames
Preferring the V4L2 backend only fixes fresh setups; users who already
have "Normal Camera" persisted still get the OpenCV/GStreamer backend,
which can't open /dev/video* without the unbundled v4l2src plugin.

CreateStart now tries every compatible backend in preference order (the
requested one first, then the rest) until one actually produces a frame,
so a dead GStreamer open automatically falls back to LibV4L2. Also drops
the ArgumentNullException-on-no-provider path in favour of a logged null,
and waits on the frame signal instead of busy-polling.
2026-10-27 19:03:53 -07:00
dan2wik 8a5d8c4e50 Load libudev by versioned soname so enumeration works under Steam runtime
The udev P/Invokes named "libudev.so", the dev-package symlink. Minimal
runtimes like the Steam Linux Runtime (sniper) ship only the versioned
"libudev.so.1", so the load failed there, camera enumeration returned
nothing, friendly names never mapped to /dev/videoN, and starting a camera
threw "No provider ... not found".

Register a DllImportResolver that maps "libudev.so" to whichever soname
exists (libudev.so.1, then .so, then .so.0), so enumeration works under
sniper, scout, and bare distros without the -dev package.
2026-10-27 19:03:52 -07:00
dan2wik 5620a102b0 Prefer V4L2 backend on Linux and fail fast when GStreamer can't open
The bundled mini OpenCV runtime captures only through GStreamer, which
needs the v4l2src plugin (gst-plugins-good) to read /dev/video*. That
plugin isn't bundled, so on systems without it the OpenCV "Normal Camera"
backend silently fails. Which backend got used was decided by arbitrary
Directory.GetFiles order, so /dev/video* devices could default to the
broken GStreamer path instead of the dependency-free LibV4L2 one.

- DesktopConnector: deterministically prefer "V4L2 Camera" on Linux.
- OpenCvCapture: when the device fails to open, log an actionable message
  (install gst-plugins-good or use V4L2 Camera) and bail instead of
  starting a dead capture loop.
- SingleCameraSource.Start: return the backend's real start result so a
  failed open fails fast rather than waiting out the 13s frame timeout.
2026-10-27 19:03:52 -07:00
dan2wik 0c79d94a4e Fix Linux capture failing to open /dev/videoN paths
The mini OpenCV runtime has no V4L2 backend, so opening a "/dev/videoN"
path via the string ctor falls back to GStreamer's file source and tries
to read the char device as a media file ("unable to start pipeline").
Resolve the path to an index and open via FromCamera so GStreamer builds
a proper v4l2src pipeline instead.
2026-10-27 19:03:52 -07:00
dan2wik 17a33653d7 Show per-camera debug stats for up to 3 cameras
- Replace the single eye-camera card with a dynamic Cameras list: one card
  per active capture, with its own delivered fps, 60s drop count, negotiated
  fps, resolution and format
- Handle both a single split eye feed (Eye + Mouth) and two independent
  left/right eye feeds (Eye left/right + Mouth)
- Publish the active capture sources from the workers; the sampler reads each
  one's counters live, keyed per camera
2026-10-27 19:03:51 -07:00
dan2wik a7247cff18 Accurate Debug fps metrics + dropped-frame counter
- Sample every rate against a monotonic Stopwatch clock and read the camera
  frame count live, then EWMA-smooth, so delivered fps stops jittering
- Count genuinely lost frames at the capture (overwritten before the pipeline
  acquired them) and show the total over a rolling 60s window
- Tidy the Debug page header spacing
2026-10-27 19:03:51 -07:00
dan2wik fe3d9b2c6e Add render-thread fps to Performance metrics and tidy labels
- Add a 'UI Render thread' row showing Avalonia's compositor frame rate,
  counted via a per-frame RequestCompositionUpdate hook in the view.
- Rename 'UI thread' to 'UI data thread' and the header to 'Performance
  metrics'; drop the (responsiveness) and (NNN% = fully saturated) notes.
- Relabel Face -> Mouth (inference rows, pipeline header, worker thread).
- Rename 'CPU hotspots (per thread)' to 'Thread CPU Usage' and
  '% of one core' to '% x 1 core'.
2026-10-27 19:03:50 -07:00
dan2wik fe32cbbb0b Fix camera freeze from Mat reuse in capture loop
The update loop reused one Mat that SetRawMat hands to the consumer, so
the next Read overwrote the buffer the consumer was still using and the
feed stalled. Allocate a fresh Mat per frame and dispose it on a miss.
2026-10-27 19:03:50 -07:00
dan2wik 935e8622f7 Revert "Fix Windows capture for cameras whose default DShow pin is Y800"
This reverts commit 5a2a7b10ce.
2026-10-27 19:03:50 -07:00
Rames The Generic 610367715b Add checkpoints for Linux Trainer 2026-10-27 19:03:49 -07:00
Rames The Generic 7e4344d4a2 Fix Stabilize Eyes not enabling 2026-10-27 19:03:49 -07:00
dan2wik 181f27cc14 Fix Windows capture for cameras whose default DShow pin is Y800
OpenCV's DSHOW backend can't stream Y800 (raw greyscale) - which some
trackers expose as their default pin - and in fact can't open such
devices at all. Result was an endless "No data received" with the camera
never producing frames on Windows (it works on Linux via GStreamer/MJPEG).

- Add DirectShowModeSelector: enumerate a device's DirectShow modes and
  pick the best non-Y800 mode (prefer MJPG, then larger frame, then higher
  fps). Generic - no per-camera assumptions.
- OpenCvCapture: apply the selected mode as size -> fps -> FourCC (last),
  since the frame rate is what drives the backend onto the MJPG pin when a
  camera exposes the same resolution as both raw and compressed. Fall back
  to Media Foundation (MSMF) when DSHOW yields no frames; additive, so
  cameras already working on DSHOW are unaffected.
- Hand off a fresh Mat per frame in the capture loop. Reusing a single Mat
  raced the consumer (concurrent write / use-after-dispose) and crashed
  natively (0xC0000005) once frames actually started flowing.
- Log opened backend, negotiated format and a frame probe for diagnostics;
  sleep when idle to avoid busy-spinning a CPU core.
2026-10-27 19:03:48 -07:00
dan2wik bd0fbc87d0 Expression pipeline optimization + diagnostics
- ONNX: cap IntraOpNumThreads=2 to eliminate intra-op thread fan-out CPU.
  Eye-only CPU inference on a Ryzen 9 9900X: ~225% -> ~67% of one core
- Add always-on ThreadProfiler + Debug/Performance page: per-thread CPU,
  pipeline stage timings, throughput counters
- Settings: add "Split Eye Video Swap" (default unswapped) and opt-in
  "Show Debug menu"; move Advanced options to the bottom of the list
- Bundle in-progress expr-dev capture/pipeline work (V4L2 capture, video
  sources, OSC recv/send, parameter sender, home/calibration view models)
2026-10-27 19:03:48 -07:00
Rames The Generic 9a487f438a Add Alpha warning banner 2026-10-27 19:03:47 -07:00
Rames The Generic fd39e03713 version number ig 2026-10-27 19:03:47 -07:00
Rames The Generic 36019ded88 Revert sequence times 2026-10-27 19:03:47 -07:00