- Posted on
- 2026-07-29T23:18:00Z
- Tags
- Content warnings
- None
Table of Contents
§ Remaining before publishing (draft: false)
- Take video(s) of working tracking; both VRChat-side and application(s) side.
- Show libsnout + oscavmgr + VrcAdvert working in screenshots.
- Get proof-readings.
§ My background with VR
I've been playing Beat Saber for years and years (since ~2020...? on the Rift CV1?). You can see some of my playing here and my BeatLeader. This was most of what I have done with my headsets (I eventually upgraded to the Quest 3 because I enjoyed Beat Saber so much), other than some time spent playing SUPERHOT VR and Half-Life Alyx.
§ VRChat
During the pandemic, although briefly, a partner had me come onto VRChat. I didn't really get it, nor spend any time with it past that initial meeting in some apartment-like setting. Fast-forward several years, a friend reintroduced me to it and this time, it stuck a lot better. I spent a good period of time world-hopping initially, eventually made friends and found places to be! I must really thank Cozy Calm (by Sam) for this.
These experiences made me a lot more interested in avatar creation and VR technology.
To my VRChat friends: hi! you're cute, you're based.
§ Full body tracking
§ Pre-VRChat attempts at FBT
A long while back (must have been 2021) I had obtained an XBOX 360 Kinect for using for body tracking with Beat Saber via K2VR with owoTrack. This was not a good experience (no fault of the K2VR people); I did not have enough space for the required distance and I can't say the Kinect is a great tracking solution period. Around this time I had heard of SlimeVR for the first time as a result of the crowdfunding campaign, but I was not willing to part with my money for it at the time.
§ Earlier attempts with joycons
§ SlimeVR joycon wrangler
Due to circumstances, I had two sets of joycons (these circumstances being having a prior Switch 1 to my current, that I had modchipped but later broke in an attempt to replace a bezel) and I noticed that slimevr-wrangler existed. I noticed a Linux-specific issue with MAC address duplication and created a PR to fix it. Despite these changes and also having tested the WIP quaternions PR later, the accuracy of tracking with this was absolutely attrocious; drift was instant and rotation would end up going the wrong way in comparison to reality. I also used owoTrack during this for the waist tracker.
§ everything-imu
To see what it would look like in the "best case" I had decided to try everything-imu, a project I absolutely cannot recommend given its either heavily AI-assisted or vibecoded status. This slightly improved things in comparison to the aformentioned, but it was still woefully inadequate. While trying it, I noticed that on my system the UI could not even correctly connect to the server itself, so that was fun.
§ SlimeVR Smol Slimes
§ Prints
- Ibis Trackers - I don't like Bris particularly much because of their heavy use of AI, but Ibis Trackers as prints seem to predate this.
- Raye's Smolslime case edits - Had some tracker trays I needed for a particular width strap!
§ Results
After repeatedly doing cost calculations and staring at my 3D printer longingly, I realised I could probably make my own Slime Smols, regardless of the fact that Butterfly trackers are now pre-orderable. I decided to build them around the SlimeVR Mumo Breakout Modules (ICM-45686) because they were the best IMUs according to the IMU comparison. I used the typical knock-off nice!nanos (superminis, whatever) that are NRF52840s in a pro micro format. The pinecil, as always, came in clutch. I used the aliexpress straps that were not recommended, but hey, they were cheap!
I used the soldering fixture for the Ibis Trackers, it went relatively well other than pin mistakes that I made that required reworking what I had done. The first tracker took 11 hours, the remaining each took 30 minutes (and it could've been less than 30 mins for the remainder had I not had to do some rework from doing all of them a certain way initially).
With ~120mAh cells, I get around 30hrs of battery life on these trackers. They weigh very, very little.
§ Mouth & eye tracking
§ Project Babble & EyeTrackVR (with Baballonia + VRCFaceTracking.Avalonia)
Do not confuse Project Babble with Project Babel!
For this project, I opted for the Seeed Studio XIAO Sense ESP32-S3; it's tiny and I think the Project Babble DIY trackers are far smaller than the official boards for them. I bought 3 initially and later bought a fourth to replace one that I immediately broke; the camera hat board connector sheared off backwards when I was initially removing one of them to put on an antenna.
§ Part 1 - mouth!
§ Prints
- Clip-on Quest 3 Project Babble Mount - Would not recommend! Had trouble with this staying on properly in PLA or PETG.
- Seeed Studio XIAO ESP32S3 Project Babble Mount Universal - I hotglued the "Index Sticky mount" on and this worked wonderfully and still does.
- Xiao ESP32s3 Case for Quest 2 - EyeTrackVr - I hotglued these to the sides of my Quest 3, seems to work nicely.
§ Results
Initially, I printed in PLA (because I didn't want to open a new reel of PETG and I don't have a drybox yet) for testing. My first attempt with the mouth tracker was over WiFi with Baballonia and VRCFaceTracking.Avalonia. It did, in fact work, though initially it was off-center and I had to figure out how to do the calibrations.
Around this time, I also removed the IR filters on all of the 120/130° FOV cameras I had obtained (with two pairs of pliers to twist the lenses off and a pair of tweezers to shatter the filter).
§ Part 2 - eye tracker!
The print for this I got from someone as a 3MF + blend set of files on the EyeTrackVR discord as I use VROptician prescription lenses with my Quest 3. I would otherwise link them here. My apologies!
A few days into this, Baballonia got the expression beta release! How very exciting!
§ Part 3 - reprints in PETG for the mouth tracker, slime smol strap trays and Quest 3 ESP and IR power board housings
At this point I was ready to finalize things, given that the mouth tracker was a little janky by virtue of PLA issues. I also think, had I committed early enough to using UVC for the cameras I would've had less issue with antenna wire routing causing issues with putting together the mouth tracker.
§ All done!
By the end, I moved to using the cameras via UVC with the USB hub instead of using WiFi and treating them as IP cameras. I already use the Quest 3 via ADB-forwarded WiVRn over a Syntech Link Cable, 16ft with Charging, but having to have a second long cable is pretty rough. I accept it for now but I would like to get the whole setup down to one and a lot cleaner in future.
§ libsnout + oscavmgr + VrcAdvert
Thank goodness for Rust developers! I have been able to avoid the C# Avalonia programs due to the work of those mentioned below. libsnout even allows for proper GPU accelerated training on Linux, something that Baballonia natively does not (outside of Windows). I was also able to make an adjusted version of the Nix flake libsnout has to allow for NixOS libtorch training!
I can't express enough how much I appreciate and respect Darksecond for making libsnout. (It also seems libsnout was moved to codeberg! Chad shit.)
The same can be said for galister and oscavmgr, VrcAdvert and WayVR.
Over time, libsnout has gotten better and better for my use case, including being able to use eye models trained by Baballonia including expression beta things (because with their own training, it seemed a little rough for me on certain parameters, mostly blink did not seem to work!). I can't complain.
§ Conclusion
Although a little janky, I've been able to make a Linux-only VRChat experience that works well. I have all the tracking I could want, although it requires adjustment from time to time and could do with cleanup efforts. I'm relatively happy with the results!
Something I would like to see is modded Beat Saber having SlimeVR that works because right now it is dependent upon SteamVR (which I do not use) set to a compatibility mode to expose it as fake Vive trackers since that is what the Custom Avatars mod requires. I like being able to dance in my Beat Saber and it'd be nice to be able to stream that working without having to use Windows.
§ Remaining things to do
- Clean up the wire mess! Shorter cables, better routing.
- Figure out how to get the setup to only require one cable, two long cables is rough!