emulate/docs/joycon-esp32-bridge.md

Joy-Con ESP32 Bridge — Design Notes

Design sketch for replacing the 8BitDo USB Wireless Adapter 2 with a DIY ESP32-based Joy-Con bridge. Not yet implemented — this doc captures the plan so it's ready to pick up later.

Why

The current "working" path uses an 8BitDo USB Wireless Adapter 2 in PS-Classic mode, plugged into the Samsung Fold 6 via an OTG cable. It works, but has three annoying problems:

1. It wedges periodically. The adapter stops sending HID reports every 60–120 seconds on this phone. Unplug + replug clears it. Likely root cause is Samsung's aggressive USB selective-suspend / OTG power management kicking the idle device off the bus. See the "dongle rabbit hole" notes below for everything we tried.
2. PS-Classic mode is fragile. We picked it because it's the only mode where Samsung's kernel usbhid actually binds the device as a real gamepad. Direct Joy-Con Bluetooth, XInput mode, DInput mode, and macOS/DS4 mode all fail for various reasons specific to Samsung's kernel + this adapter's firmware.
3. It's a cable dangling off a foldable phone. Not great ergonomically.

Doing our own Joy-Con-to-gamepad translation on an ESP32 lets us control both ends of the conversation: we pair the Joy-Con to hardware we own, we do the init handshake correctly, and we present the result to the phone as either a standard BLE HID gamepad (no cable) or a USB-CDC serial stream (cable, but we control the firmware so the selective-suspend problem goes away).

Why the Joy-Con is non-trivial

Joy-Cons are Bluetooth Classic HID devices. They are NOT BLE. This means any bridge MCU must have BT Classic on its radio.

Once paired, they default to input report mode 0x3F, which only emits a small subset of buttons as a dumb HID "joystick". To get the full button-mask + analog sticks + IMU + battery, you have to send a sequence of subcommands:

1. Read SPI flash calibration at 0x6020 (stick factory calibration) and 0x8010 (stick user calibration)
2. set_player_lights (subcommand 0x30)
3. enable_imu (subcommand 0x40) if you want gyro/accel
4. set_input_report_mode to 0x30 (subcommand 0x03) — this is the important one; switches to the 12-byte input report that contains everything you actually want

Once the Joy-Con is in 0x30 mode, every input report is a fixed layout:

byte  purpose
  0   0x30 (report id)
  1   timer (rolling counter)
  2   battery + connection info
  3   buttons right (Y, X, B, A, SR, SL, R, ZR)
  4   buttons shared (-, +, R-stick, L-stick, home, capture)
  5   buttons left (down, up, right, left, SR, SL, L, ZL)
  6-8 left stick (12-bit X + 12-bit Y, packed)
  9-11 right stick (same)
  12  vibrator ack
  13+ IMU samples (3 frames x 12 bytes) if IMU enabled

The full protocol is documented here:

• https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering
• https://github.com/DanielOgorchock/joycond (Linux userspace init daemon)
• Linux mainline drivers/hid/hid-nintendo.c (kernel driver that does the same init sequence in C)

Either joycond or hid-nintendo is a fine starting point for porting to C on ESP32.

Hardware

Joy-Cons use Bluetooth Classic (BR/EDR), not BLE. This is the single most important constraint on the hardware. Most modern MCU boards that have "Bluetooth" actually only have BLE, which won't work.

Only these chips have BT Classic:

Chip	BT Classic	BLE	Native USB	Fits?
ESP32 (D0WDQ6 / WROOM-32 / WROVER-32)	✅	✅	❌	✅ best default
Raspberry Pi Pico W (CYW43439)	✅	✅	✅	✅ also works
ESP32-S3	❌	✅	✅	❌ BLE only (common mistake)
ESP32-S2	❌	❌	✅	❌ no BT at all
ESP32-C3	❌	✅	❌	❌ BLE only
ESP32-C6	❌	✅	❌	❌ BLE + 802.15.4 only
ESP32-H2	❌	✅	❌	❌ BLE + 802.15.4 only

⚠️ Do not buy ESP32-S3 for this project even though it looks like the obvious upgrade over the original ESP32. Espressif confirmed in their own datasheet that ESP32-S3 only supports Bluetooth 5 LE — no BR/EDR — and you cannot connect a Joy-Con to it over Bluetooth. The Bluepad32 FAQ makes this explicit: "controllers like Switch, Wii, DualSense, DualShock, etc. only talk BR/EDR... you cannot use an ESP32-S3 as a Bluetooth HID host to connect to Joy-Con controllers."

Recommended: original ESP32-WROOM-32 dev board — any of the Amazon "DOIT DEVKIT V1", "HiLetgo", "ELEGOO", "DIYmall" variants. They're all the same chip, usually with a CP2102 or CH340 USB-to-serial chip on board so you get programming + serial communication over a single USB port. $8–12 Canadian. BT Classic is built into the ESP32-D0WDQ6 chip. No native USB-OTG, so in Path B the chip talks to the phone over the on-board USB-serial bridge.

Also good: Raspberry Pi Pico W — the CYW43439 WiFi/BT combo chip on the Pico W supports BT Classic, and the RP2040 has native USB. If you prefer C SDK + CMake to Arduino, Pico W is a nicer dev experience, and the Bluepad32 library supports it as a first-class target alongside the original ESP32.

ESP32-WROVER-32 is the same silicon as WROOM-32 plus an extra PSRAM chip. Works identically for this project but costs a bit more for PSRAM we don't need. Fine if that's what you already have.

Two architectures

Path A — fully wireless (BT Classic host + BLE peripheral)

[Joy-Con]  --BT Classic HID-->  [ESP32-S3]  --BLE HID-->  [Phone]

The ESP32 runs two radio profiles at once on its dual-mode stack:

• BT Classic HID host (ESP-IDF esp_hidh API) — pairs with the Joy-Con, runs the init subcommand sequence, subscribes to 0x30 input reports
• BLE HID device (ESP-IDF esp_hids API, or the ESP32-BLE-Gamepad Arduino library) — advertises itself to the phone as a standard BLE gamepad with a generic HID descriptor

The Android side needs no code changes. Samsung's kernel binds BLE HID gamepads natively (BLE HID is a completely different code path from the USB HID mess we've been fighting), the existing InputDevice/ dispatchKeyEvent path picks up button events, and our ControllerManager routes them into the emulator the same way it does for the 8BitDo today.

Pros

• No cable. Charge the ESP32 off a tiny LiPo or a power brick, use it as a wireless dongle sitting next to the phone.
• Nothing to change in the app.
• Once it works, it works the same for every game on every emulator we already support.

Cons

• Running BT Classic host + BLE peripheral simultaneously on the same radio is non-trivial. ESP-IDF supports dual mode but the profiles have to share a single controller — expect to spend time in menuconfig and the Bluetooth controller's coexistence settings.
• Bigger firmware, more things that can go wrong during bring-up.

Path B — wired hybrid (BT Classic host + USB-CDC serial)

[Joy-Con]  --BT Classic HID-->  [ESP32-S3]  --USB CDC serial-->  [Phone]

Same Joy-Con side. Different phone side: instead of re-broadcasting over BLE, the ESP32 streams button state as short binary frames over USB CDC. The phone reads them via usb-serial-for-android or directly via the UsbManager + bulkTransfer APIs we already use.

Pros

• Simpler firmware — no dual-mode radio, no BLE HID descriptor, no BLE pairing dance. Get it working in a weekend.
• We own the wire protocol, so it's trivially debuggable. Dump the serial bytes in Serial Monitor and you see exactly what the Joy-Con is sending.
• Selective-suspend goes away because we control the firmware on both sides. Send a 60-Hz heartbeat frame even when no buttons are held and the USB bus stays active. The ESP32 can also be externally powered, so the phone's bus-power policy becomes irrelevant.
• Direct fit for the existing ControllerManager.dispatchButton plumbing.

Cons

• Still a cable between the ESP32 and the phone (we're swapping the 8BitDo OTG cable for an ESP32 OTG cable — ergonomically the same).
• Requires a small addition to the app to read USB serial and parse the wire protocol.

Recommended order of attack: build Path B first. The hard part (Joy-Con BT Classic host + init sequence + 0x30 report parsing) is identical in both paths, so Path B gets you to a working gamepad fastest and validates the Joy-Con side. Once Path B is rock-solid, swap the output stage for BLE HID and you have Path A for free (modulo the dual-mode coexistence tuning).

Wire protocol sketch (Path B)

Keep it tiny, keep it fixed-size, keep it self-synchronizing.

byte 0   0xA5   sync / frame header
byte 1   seq    rolling sequence counter, wraps at 255
byte 2   btns_lo (A, B, X, Y, L1, R1, L2, R2)
byte 3   btns_hi (Start, Select, L3, R3, Home, Capture, reserved, reserved)
byte 4   dpad   packed: 4 bits for hat direction (0-7 clockwise from N, 8=none),
                4 bits for Joy-Con-specific buttons (SR/SL etc)
byte 5   lx     signed int8, -127..127 (already deadzoned + calibrated on ESP32)
byte 6   ly     signed int8
byte 7   rx     signed int8
byte 8   ry     signed int8
byte 9   flags  bit 0: battery low, bit 1: charging, bit 2: imu_valid, bits 3-7: reserved
byte 10  crc8   CRC-8 over bytes 0..9 (or just XOR checksum if we're lazy)

11 bytes per frame, 60 Hz = 660 bytes/sec. USB CDC at 115200+ baud handles this with orders of magnitude to spare. Even ESP32's default 921600 baud is fine for 240 Hz polling.

If later we want motion / gyro for games that use it, we append another ~12 bytes and bump a version byte in the header. Forward-compatible.

App-side changes (Path B)

All contained in ControllerManager.kt and a new helper file:

1. New dependency on usb-serial-for-android (tiny, ~50 KB; supports CH340, CP210x, FTDI, and CDC-ACM, which covers every ESP32 dev board).
2. Add ESP32-S3 native USB VID/PID (0x303A / 0x1001) and CP2102 (0x10C4 / 0xEA60) to targetVidPids in ControllerManager.
3. In onUsbDeviceAttached, after the existing HID-class short-circuit, check whether the device matches one of the ESP32 VID/PIDs. If so, open it as a USB serial port instead of going through the HID path, start a reader thread, and parse incoming [0xA5, seq, btns_lo, btns_hi, dpad, lx, ly, rx, ry, flags, crc] frames.
4. For each frame, diff against the previous frame and call dispatchButton(...) for any changed button bits, plus onAnalogEvent(...) for stick updates. Existing plumbing carries it into the emulator core.
5. Heartbeat / liveness: track the frame sequence counter, and if we don't see a frame for 500 ms, log a warning and mark the controller as disconnected so the test screen reflects it.

Approximate LOC: ~150 lines in ControllerManager + 50 lines for the wire protocol parser. No changes needed anywhere else in the app.

Firmware-side notes

Bluepad32 changes the math

There's a project called Bluepad32 that already implements BT Classic HID host for original ESP32 and Pico W, with explicit first-class Joy-Con support alongside DualShock 3/4/5, Switch Pro Controller, Wii Remote, Xbox, and generic HID gamepads. It does the init subcommand dance, SPI-flash calibration reads, and 0x30 input report parsing for you. MIT licensed.

This means the Joy-Con side of this project essentially does not need to be written — it's library-level #include. Our firmware job shrinks to:

1. Initialize Bluepad32 and register a gamepad callback.
2. Format the callback's button/stick state into our wire protocol.
3. Send it out the output stage (USB serial for Path B, BLE HID for Path A).

The Joy-Con init quirks, reconnect handling, stick calibration, and BT pairing UX are already solved.

Starting points

• Bluepad32 — the main library. Has ESP-IDF and Arduino examples. The controllers/ demo example pairs gamepads and prints button state to serial; adapting it to our wire format is ~50 lines.
• For Path A (BLE output): glue the Bluepad32 input side to ESP32-BLE-Gamepad — working BLE HID gamepad profile in ~30 lines of Arduino. Caveat: BLE peripheral coexisting with BT Classic host on one chip is tricky.
• For Path B (USB serial): just Serial.write() the frame from the Bluepad32 callback. Trivial.
• If you'd rather not use Bluepad32 and write the protocol yourself (for learning or licensing reasons), reference joycond (C++, Apache 2.0) or the Linux kernel drivers/hid/hid-nintendo.c (C, GPL).

Prior art

• https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering — full protocol docs
• https://github.com/DanielOgorchock/joycond — Linux userspace init daemon
• https://github.com/pipe01/joycon-rs — Rust Joy-Con protocol library
• Various ESP32-based Nintendo Switch Pro Controller emulators also exist — they go the other direction (pretend to be a Pro Controller talking to a Switch console) but share the same protocol knowledge.

Estimated effort (with Bluepad32)

• Path B (serial out): a weekend to get working, another weekend to harden against reconnect/pair-loss edge cases. Total ~10–20 hobby hours.
• Path A (BLE out): same Path B starting point plus 1–2 weeks of wrestling with BT Classic + BLE dual-mode coexistence on one chip. Bluepad32 itself only targets single-mode BT Classic, so you'd be extending it or running the BLE side in parallel via raw ESP-IDF calls. Harder to estimate.

The dongle rabbit hole (what we already tried and why it didn't work)

For anyone picking this up later, here's the short history so you don't re-run the same experiments:

• Direct Joy-Con over Bluetooth to Samsung Fold 6. Pairs fine, kernel creates two evdev nodes (main + IMU), IMU streams MSC_TIMESTAMP so the link is alive, but the main node has only BTN_TL/BTN_TL2 in its key capability set (no face buttons, no dpad, no stick) and even those fire zero events. Samsung's hid-nintendo driver parks the node waiting for a userspace init handshake that nothing on the system provides, and shell can't chmod /sys/bus/hid/devices/.../ to kick it.
• 8BitDo adapter, XInput mode. Enumerates as an Xbox 360 wired controller (vendor-spec class 0xFF / subclass 0x5D). Android has no XInput driver in its kernel. Adapter endlessly cycles modes because it can't complete the Xbox 360 handshake with the phone.
• 8BitDo adapter, DInput mode. Enumerates as a clean HID-class device, but Samsung's kernel usbhid refuses to bind it for unknown reasons (verified by uninstalling the app entirely and replugging — no /dev/input/event* is ever created). Not fixable from userspace.
• 8BitDo adapter, macOS mode. Enumerates as a Sony DualShock 4 (054C:05C4). Same failure as DInput — Samsung's kernel doesn't bind it. Despite Android having native DS4 support.
• 8BitDo adapter, PS-Classic mode. Enumerates as a "Sony Interactive Entertainment Controller" (054C:0CDA). Samsung's kernel actually binds this one as a real gamepad. This is what we ship today. Downside: the adapter wedges every 60–120 seconds and needs an unplug+replug to recover, presumably from OTG selective-suspend.

Things that partially work or haven't been fully explored:

• Plugging the adapter into a powered USB hub between the phone and the adapter helps somewhat with the wedging but does not fully fix it.
• Disabling Samsung battery optimization for the app does not fix the wedging.
• Holding FLAG_KEEP_SCREEN_ON on the game window helps (the system stays out of the deep-sleep regime that aggravates selective-suspend) but again doesn't fully fix it.

All three workarounds stack with the ESP32 bridge idea if we go Path B — but Path A (BLE) makes them all irrelevant because there's no USB bus to suspend in the first place.

Open questions to resolve during bring-up

1. Does the Samsung Fold 6 kernel actually bind ESP32-S3 native USB-CDC as a /dev/bus/usb/ device visible to our app, or does it do something weird to USB-CDC too? Should be fine — CDC-ACM is boring and well-supported — but verify before writing firmware.
2. What's the latency floor from Joy-Con button press to emulator core? Joy-Con → BT Classic (~4-8 ms) + ESP32 parse (<1 ms) + USB serial (<1 ms) + app dispatch (<1 ms) should come in under 15 ms, comfortably below the ~16 ms/frame budget at 60 Hz. Worth measuring once hardware exists.
3. Left+Right Joy-Con combined as one "Pro Controller" style pad, or single-Joy-Con sideways mode? Single-Joy-Con is simpler for NES/SNES and matches the NES Joy-Con we have. Pairing two Joy-Cons to the same ESP32 is a separate BT Classic multi-device problem that we can punt on.
4. Rumble — do we care? Joy-Con rumble is HD Rumble, which is a nightmare even by Joy-Con standards. Probably punt; the emulated consoles we support didn't have rumble anyway.