emulate/docs/joycon-esp32-bridge.md

# Joy-Con ESP32 Bridge — Design Notes

Design sketch for replacing the 8BitDo USB Wireless Adapter 2 with a DIY
ESP32-based Joy-Con bridge. Not yet implemented — this doc captures the
plan so it's ready to pick up later.

## Why

The current "working" path uses an 8BitDo USB Wireless Adapter 2 in PS-Classic
mode, plugged into the Samsung Fold 6 via an OTG cable. It works, but has three
annoying problems:

1. **It wedges periodically.** The adapter stops sending HID reports every
   60–120 seconds on this phone. Unplug + replug clears it. Likely root cause
   is Samsung's aggressive USB selective-suspend / OTG power management
   kicking the idle device off the bus. See the "dongle rabbit hole" notes
   below for everything we tried.
2. **PS-Classic mode is fragile.** We picked it because it's the only mode
   where Samsung's kernel `usbhid` actually binds the device as a real
   gamepad. Direct Joy-Con Bluetooth, XInput mode, DInput mode, and macOS/DS4
   mode all fail for various reasons specific to Samsung's kernel + this
   adapter's firmware.
3. **It's a cable dangling off a foldable phone.** Not great ergonomically.

Doing our own Joy-Con-to-gamepad translation on an ESP32 lets us control both
ends of the conversation: we pair the Joy-Con to hardware we own, we do the
init handshake correctly, and we present the result to the phone as either a
standard BLE HID gamepad (no cable) or a USB-CDC serial stream (cable, but we
control the firmware so the selective-suspend problem goes away).

## Why the Joy-Con is non-trivial

Joy-Cons are Bluetooth **Classic** HID devices. They are NOT BLE. This means
any bridge MCU must have BT Classic on its radio.

Once paired, they default to input report mode `0x3F`, which only emits a
small subset of buttons as a dumb HID "joystick". To get the full
button-mask + analog sticks + IMU + battery, you have to send a sequence of
subcommands:

1. Read SPI flash calibration at `0x6020` (stick factory calibration) and
   `0x8010` (stick user calibration)
2. `set_player_lights` (subcommand `0x30`)
3. `enable_imu` (subcommand `0x40`) if you want gyro/accel
4. **`set_input_report_mode` to `0x30`** (subcommand `0x03`) — this is the
   important one; switches to the 12-byte input report that contains
   everything you actually want

Once the Joy-Con is in `0x30` mode, every input report is a fixed layout:

```
byte  purpose
  0   0x30 (report id)
  1   timer (rolling counter)
  2   battery + connection info
  3   buttons right (Y, X, B, A, SR, SL, R, ZR)
  4   buttons shared (-, +, R-stick, L-stick, home, capture)
  5   buttons left (down, up, right, left, SR, SL, L, ZL)
  6-8 left stick (12-bit X + 12-bit Y, packed)
  9-11 right stick (same)
  12  vibrator ack
  13+ IMU samples (3 frames x 12 bytes) if IMU enabled
```

The full protocol is documented here:
- https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering
- https://github.com/DanielOgorchock/joycond  (Linux userspace init daemon)
- Linux mainline `drivers/hid/hid-nintendo.c`  (kernel driver that does the
  same init sequence in C)

Either joycond or hid-nintendo is a fine starting point for porting to C on
ESP32.

## Hardware

Joy-Cons use **Bluetooth Classic (BR/EDR)**, not BLE. This is the single
most important constraint on the hardware. Most modern MCU boards that
have "Bluetooth" actually only have BLE, which won't work.

Only these chips have BT Classic:

| Chip                       | BT Classic | BLE | Native USB | Fits? |
|----------------------------|:----------:|:---:|:----------:|:-----:|
| **ESP32** (D0WDQ6 / WROOM-32 / WROVER-32) | ✅ | ✅ | ❌ | ✅ **best default** |
| **Raspberry Pi Pico W** (CYW43439)        | ✅ | ✅ | ✅ | ✅ also works |
| ESP32-S3                   | ❌ | ✅ | ✅ | ❌ BLE only (common mistake) |
| ESP32-S2                   | ❌ | ❌ | ✅ | ❌ no BT at all |
| ESP32-C3                   | ❌ | ✅ | ❌ | ❌ BLE only |
| ESP32-C6                   | ❌ | ✅ | ❌ | ❌ BLE + 802.15.4 only |
| ESP32-H2                   | ❌ | ✅ | ❌ | ❌ BLE + 802.15.4 only |

**⚠️ Do not buy ESP32-S3 for this project** even though it looks like the
obvious upgrade over the original ESP32. Espressif confirmed in their own
datasheet that ESP32-S3 only supports Bluetooth 5 LE — no BR/EDR — and
you cannot connect a Joy-Con to it over Bluetooth. The Bluepad32 FAQ
makes this explicit: *"controllers like Switch, Wii, DualSense, DualShock,
etc. only talk BR/EDR... you cannot use an ESP32-S3 as a Bluetooth HID
host to connect to Joy-Con controllers."*

**Recommended: original ESP32-WROOM-32 dev board** — any of the Amazon
"DOIT DEVKIT V1", "HiLetgo", "ELEGOO", "DIYmall" variants. They're all
the same chip, usually with a CP2102 or CH340 USB-to-serial chip on
board so you get programming + serial communication over a single USB
port. $8–12 Canadian. BT Classic is built into the ESP32-D0WDQ6 chip.
No native USB-OTG, so in Path B the chip talks to the phone over the
on-board USB-serial bridge.

**Also good: Raspberry Pi Pico W** — the CYW43439 WiFi/BT combo chip
on the Pico W supports BT Classic, and the RP2040 has native USB. If
you prefer C SDK + CMake to Arduino, Pico W is a nicer dev experience,
and the Bluepad32 library supports it as a first-class target alongside
the original ESP32.

**ESP32-WROVER-32** is the same silicon as WROOM-32 plus an extra PSRAM
chip. Works identically for this project but costs a bit more for PSRAM
we don't need. Fine if that's what you already have.

## Two architectures

### Path A — fully wireless (BT Classic host + BLE peripheral)

```
[Joy-Con]  --BT Classic HID-->  [ESP32-S3]  --BLE HID-->  [Phone]
```

The ESP32 runs two radio profiles at once on its dual-mode stack:

- **BT Classic HID host** (ESP-IDF `esp_hidh` API) — pairs with the Joy-Con,
  runs the init subcommand sequence, subscribes to `0x30` input reports
- **BLE HID device** (ESP-IDF `esp_hids` API, or the `ESP32-BLE-Gamepad`
  Arduino library) — advertises itself to the phone as a standard BLE
  gamepad with a generic HID descriptor

The Android side needs **no code changes**. Samsung's kernel binds BLE HID
gamepads natively (BLE HID is a completely different code path from the
USB HID mess we've been fighting), the existing `InputDevice`/
`dispatchKeyEvent` path picks up button events, and our
`ControllerManager` routes them into the emulator the same way it does
for the 8BitDo today.

**Pros**
- No cable. Charge the ESP32 off a tiny LiPo or a power brick, use it as a
  wireless dongle sitting next to the phone.
- Nothing to change in the app.
- Once it works, it works the same for every game on every emulator we
  already support.

**Cons**
- Running BT Classic host + BLE peripheral simultaneously on the same radio
  is non-trivial. ESP-IDF supports dual mode but the profiles have to share
  a single controller — expect to spend time in `menuconfig` and the
  Bluetooth controller's coexistence settings.
- Bigger firmware, more things that can go wrong during bring-up.

### Path B — wired hybrid (BT Classic host + USB-CDC serial)

```
[Joy-Con]  --BT Classic HID-->  [ESP32-S3]  --USB CDC serial-->  [Phone]
```

Same Joy-Con side. Different phone side: instead of re-broadcasting over
BLE, the ESP32 streams button state as short binary frames over USB CDC.
The phone reads them via `usb-serial-for-android` or directly via the
`UsbManager` + `bulkTransfer` APIs we already use.

**Pros**
- Simpler firmware — no dual-mode radio, no BLE HID descriptor, no BLE
  pairing dance. Get it working in a weekend.
- We own the wire protocol, so it's trivially debuggable. Dump the serial
  bytes in Serial Monitor and you see exactly what the Joy-Con is sending.
- **Selective-suspend goes away** because we control the firmware on
  both sides. Send a 60-Hz heartbeat frame even when no buttons are held
  and the USB bus stays active. The ESP32 can also be externally
  powered, so the phone's bus-power policy becomes irrelevant.
- Direct fit for the existing `ControllerManager.dispatchButton` plumbing.

**Cons**
- Still a cable between the ESP32 and the phone (we're swapping the 8BitDo
  OTG cable for an ESP32 OTG cable — ergonomically the same).
- Requires a small addition to the app to read USB serial and parse the
  wire protocol.

Recommended order of attack: **build Path B first**. The hard part (Joy-Con
BT Classic host + init sequence + `0x30` report parsing) is identical in
both paths, so Path B gets you to a working gamepad fastest and validates
the Joy-Con side. Once Path B is rock-solid, swap the output stage for BLE
HID and you have Path A for free (modulo the dual-mode coexistence tuning).

## Wire protocol sketch (Path B)

Keep it tiny, keep it fixed-size, keep it self-synchronizing.

```
byte 0   0xA5   sync / frame header
byte 1   seq    rolling sequence counter, wraps at 255
byte 2   btns_lo (A, B, X, Y, L1, R1, L2, R2)
byte 3   btns_hi (Start, Select, L3, R3, Home, Capture, reserved, reserved)
byte 4   dpad   packed: 4 bits for hat direction (0-7 clockwise from N, 8=none),
                4 bits for Joy-Con-specific buttons (SR/SL etc)
byte 5   lx     signed int8, -127..127 (already deadzoned + calibrated on ESP32)
byte 6   ly     signed int8
byte 7   rx     signed int8
byte 8   ry     signed int8
byte 9   flags  bit 0: battery low, bit 1: charging, bit 2: imu_valid, bits 3-7: reserved
byte 10  crc8   CRC-8 over bytes 0..9 (or just XOR checksum if we're lazy)
```

11 bytes per frame, 60 Hz = 660 bytes/sec. USB CDC at 115200+ baud handles
this with orders of magnitude to spare. Even ESP32's default 921600 baud is
fine for 240 Hz polling.

If later we want motion / gyro for games that use it, we append another ~12
bytes and bump a version byte in the header. Forward-compatible.

## App-side changes (Path B)

All contained in `ControllerManager.kt` and a new helper file:

1. **New dependency** on [usb-serial-for-android](https://github.com/mik3y/usb-serial-for-android)
   (tiny, ~50 KB; supports CH340, CP210x, FTDI, and CDC-ACM, which covers
   every ESP32 dev board).
2. **Add ESP32-S3 native USB VID/PID** (`0x303A / 0x1001`) and CP2102
   (`0x10C4 / 0xEA60`) to `targetVidPids` in `ControllerManager`.
3. **In `onUsbDeviceAttached`**, after the existing HID-class short-circuit,
   check whether the device matches one of the ESP32 VID/PIDs. If so, open
   it as a USB serial port instead of going through the HID path, start a
   reader thread, and parse incoming `[0xA5, seq, btns_lo, btns_hi, dpad,
   lx, ly, rx, ry, flags, crc]` frames.
4. **For each frame**, diff against the previous frame and call
   `dispatchButton(...)` for any changed button bits, plus `onAnalogEvent(...)`
   for stick updates. Existing plumbing carries it into the emulator core.
5. **Heartbeat / liveness**: track the frame sequence counter, and if we
   don't see a frame for 500 ms, log a warning and mark the controller as
   disconnected so the test screen reflects it.

Approximate LOC: ~150 lines in ControllerManager + 50 lines for the wire
protocol parser. No changes needed anywhere else in the app.

## Firmware-side notes

### Bluepad32 changes the math

There's a project called **[Bluepad32](https://bluepad32.readthedocs.io/)**
that already implements BT Classic HID host for original ESP32 and Pico W,
with **explicit first-class Joy-Con support** alongside DualShock 3/4/5,
Switch Pro Controller, Wii Remote, Xbox, and generic HID gamepads. It does
the init subcommand dance, SPI-flash calibration reads, and 0x30 input
report parsing for you. MIT licensed.

This means the Joy-Con side of this project essentially does not need to
be written — it's library-level `#include`. Our firmware job shrinks to:

1. Initialize Bluepad32 and register a gamepad callback.
2. Format the callback's button/stick state into our wire protocol.
3. Send it out the output stage (USB serial for Path B, BLE HID for Path A).

The Joy-Con init quirks, reconnect handling, stick calibration, and BT
pairing UX are already solved.

### Starting points

- **[Bluepad32](https://github.com/ricardoquesada/bluepad32)** — the main
  library. Has ESP-IDF and Arduino examples. The `controllers/` demo example
  pairs gamepads and prints button state to serial; adapting it to our
  wire format is ~50 lines.
- For **Path A** (BLE output): glue the Bluepad32 input side to
  [`ESP32-BLE-Gamepad`](https://github.com/lemmingDev/ESP32-BLE-Gamepad) —
  working BLE HID gamepad profile in ~30 lines of Arduino. Caveat: BLE
  peripheral coexisting with BT Classic host on one chip is tricky.
- For **Path B** (USB serial): just `Serial.write()` the frame from the
  Bluepad32 callback. Trivial.
- If you'd rather not use Bluepad32 and write the protocol yourself (for
  learning or licensing reasons), reference joycond (C++, Apache 2.0) or
  the Linux kernel `drivers/hid/hid-nintendo.c` (C, GPL).

### Prior art

- https://github.com/dekuNukem/Nintendo_Switch_Reverse_Engineering — full
  protocol docs
- https://github.com/DanielOgorchock/joycond — Linux userspace init daemon
- https://github.com/pipe01/joycon-rs — Rust Joy-Con protocol library
- Various ESP32-based Nintendo Switch Pro Controller *emulators* also
  exist — they go the other direction (pretend to be a Pro Controller
  talking to a Switch console) but share the same protocol knowledge.

### Estimated effort (with Bluepad32)

- **Path B (serial out)**: a weekend to get working, another weekend to
  harden against reconnect/pair-loss edge cases. Total ~10–20 hobby hours.
- **Path A (BLE out)**: same Path B starting point plus 1–2 weeks of
  wrestling with BT Classic + BLE dual-mode coexistence on one chip.
  Bluepad32 itself only targets single-mode BT Classic, so you'd be
  extending it or running the BLE side in parallel via raw ESP-IDF calls.
  Harder to estimate.

## The dongle rabbit hole (what we already tried and why it didn't work)

For anyone picking this up later, here's the short history so you don't
re-run the same experiments:

- **Direct Joy-Con over Bluetooth to Samsung Fold 6.** Pairs fine, kernel
  creates two evdev nodes (main + IMU), IMU streams MSC_TIMESTAMP so the
  link is alive, but the main node has only `BTN_TL`/`BTN_TL2` in its key
  capability set (no face buttons, no dpad, no stick) and even those fire
  zero events. Samsung's `hid-nintendo` driver parks the node waiting for a
  userspace init handshake that nothing on the system provides, and shell
  can't `chmod` `/sys/bus/hid/devices/.../` to kick it.
- **8BitDo adapter, XInput mode.** Enumerates as an Xbox 360 wired
  controller (vendor-spec class 0xFF / subclass 0x5D). Android has no
  XInput driver in its kernel. Adapter endlessly cycles modes because it
  can't complete the Xbox 360 handshake with the phone.
- **8BitDo adapter, DInput mode.** Enumerates as a clean HID-class device,
  but Samsung's kernel `usbhid` refuses to bind it for unknown reasons
  (verified by uninstalling the app entirely and replugging — no
  `/dev/input/event*` is ever created). Not fixable from userspace.
- **8BitDo adapter, macOS mode.** Enumerates as a Sony DualShock 4
  (`054C:05C4`). Same failure as DInput — Samsung's kernel doesn't bind
  it. Despite Android having native DS4 support.
- **8BitDo adapter, PS-Classic mode.** Enumerates as a "Sony Interactive
  Entertainment Controller" (`054C:0CDA`). Samsung's kernel actually binds
  this one as a real gamepad. **This is what we ship today.** Downside: the
  adapter wedges every 60–120 seconds and needs an unplug+replug to
  recover, presumably from OTG selective-suspend.

Things that partially work or haven't been fully explored:
- Plugging the adapter into a powered USB hub between the phone and the
  adapter helps somewhat with the wedging but does not fully fix it.
- Disabling Samsung battery optimization for the app does not fix the
  wedging.
- Holding `FLAG_KEEP_SCREEN_ON` on the game window helps (the system
  stays out of the deep-sleep regime that aggravates selective-suspend)
  but again doesn't fully fix it.

All three workarounds stack with the ESP32 bridge idea if we go Path B —
but Path A (BLE) makes them all irrelevant because there's no USB bus to
suspend in the first place.

## Open questions to resolve during bring-up

1. Does the Samsung Fold 6 kernel actually bind ESP32-S3 native USB-CDC
   as a `/dev/bus/usb/` device visible to our app, or does it do something
   weird to USB-CDC too? Should be fine — CDC-ACM is boring and
   well-supported — but verify before writing firmware.
2. What's the latency floor from Joy-Con button press to emulator core?
   Joy-Con → BT Classic (~4-8 ms) + ESP32 parse (<1 ms) + USB serial
   (<1 ms) + app dispatch (<1 ms) should come in under 15 ms, comfortably
   below the ~16 ms/frame budget at 60 Hz. Worth measuring once hardware
   exists.
3. Left+Right Joy-Con combined as one "Pro Controller" style pad, or
   single-Joy-Con sideways mode? Single-Joy-Con is simpler for NES/SNES
   and matches the NES Joy-Con we have. Pairing two Joy-Cons to the same
   ESP32 is a separate BT Classic multi-device problem that we can punt
   on.
4. Rumble — do we care? Joy-Con rumble is HD Rumble, which is a nightmare
   even by Joy-Con standards. Probably punt; the emulated consoles we
   support didn't have rumble anyway.