mirror of
https://github.com/Droid-Deck/DroidDeck.git
synced 2026-11-07 10:46:06 -10:00
Kurt Himebauch
22a675a664
## Problem and intent Desktop sessions need configurable mouse, keyboard and application actions from a controller. On dual-screen devices, a lower-screen trackpad may already handle the pointer, leaving the physical controls available for other assignments. This PR gives physical controllers and local touch controls a shared profile/action model instead of separate configuration paths. This follows the centralization direction discussed in [#1202](https://github.com/Droid-Deck/DroidDeck/pull/1202#issuecomment-6299647180) and addresses the controller-input portion of #860 and #1356. Those discussions are context, **not prior maintainer approval** of this implementation. I would appreciate architectural feedback and guidance on splitting or coordinating this work if that would make review easier. ## Profiles and configuration - Controller stays within the existing Setup tabs. Game and Desktop each have a default-profile selector, followed by one profile list and device settings. - Named profiles reference three independently reusable parts: Physical controller, Main screen and Second screen. Layout geometry and action bindings are stored separately. Profiles can share physical controls while assigning different touch actions. - The session menu selects a complete profile, or expands to mix the three parts. A mixed selection can be saved as a new profile; Game and Desktop retain separate selections. - Both screens use the same full-size editor, with add, move, resize, snapping, undo and shared properties. The main-screen editor has movable, collapsible tools and is transparent above a running session. When editing the secondary display, its entire area is the canvas and the main display hosts tools/properties. - The old fixed gamepad, fixed Deck controls and separate Screen keys editor are replaced with editable presets and the shared renderer. Existing positions, bindings, shapes, symbols, imported images and toggle behavior migrate; legacy Screen keys become additional profiles without replacing the user's selected profile. Migration retains a backup before removing old preferences. - Saved layouts have thumbnails and screen targeting. Actions include compatible native output/remapping, keys/combinations, mouse buttons, pointer movement, vertical/horizontal scrolling, directional keys, existing menu/keyboard commands, and disabled. Buttons can use symbols, shapes and toggle behavior. ## Architecture and latency ```mermaid flowchart LR Library[Profiles + reusable layouts and bindings] --> Router[InputRouter] Physical[Physical controller adapters] --> Router Touch[Main + secondary InputSurfaceView] --> Sources[Owned pad / Deck sources] Sources --> Router BLE[Steam Controller BLE adapter] --> Router Router --> Native[Existing controller publication] Native --> Steam[Steam Input / guest controller] Router --> Held[Shared key and mouse ownership] Held --> Guest[Existing guest input outputs] Router --> UI[Existing menu / keyboard hosts] ``` Bindings resolve when the profile changes, outside event dispatch. Native forwarding has no additional frame/timer queue; identity routes reuse existing state/reports. Only synthesized continuous stick pointer/scroll motion uses a frame callback. One source releasing a key/button does not release another source's hold. Menu, profile, suspension and removal transitions release old outputs and gate held controls until neutral. Steam continues to own game-specific mappings and action sets. Steam Mobile Touch and VR retain their specialized paths. Raw device adapters remain responsible for transport/calibration and must join routing before publishing consumed controls; this does not claim that every adapter has been unified. Historical Thor measurements on the native Game path found kernel-event-to-guest-observer upper-bound medians of **3.061 ms baseline versus 2.999–3.403 ms candidate**, with a subsequent installed-APK check at **3.039 ms / 300 matched edges**. These are relative software-path measurements, not button-to-photon latency. Some tails increased, simultaneous BLE contention remains unmeasured, and **this UI revision has not had a new latency measurement**. The previous cursor fix changed visual update intervals from roughly 33.3 ms to 16.7 ms without changing sensitivity or adding smoothing. [Full design, migration and boundaries](https://github.com/Delete228/DroidDeck/blob/codex/unified-controller-input/docs/development/unified-input-proposal.md). ## Validation Updated through upstream 542bbf89c; current PR head08526f828. The latest CodeRabbit review generated no actionable comments. - **1,017 tests passed in 133 suites after the latest upstream merge**, with no failures, errors or skips: all input/UI tests, global reset, component updates, PiP policy, runtime auto-update and Proton menu auto-selection. The pre-merge review-fix run passed **967 input/UI/global-reset tests**. The subsequent second-screen refresh fix passed **35 focused surface/presentation tests**, including three new state-preservation regressions. - Full local run before the final review follow-ups: **3,598 tests, 3,573 passed, 25 failed**. The 25 are unchanged upstream macOS limitations: log-export tests requiring SecureDirectoryStream (24) and a case-sensitive WoW add-on test (1). The global-reset regression also passes. Earlier disk-exhausted attempts were discarded; the completed run bounded temporary resources with one worker and JVM recycling, without excluding tests or changing repository test configuration. - APK build, runtime/session assets, native dependencies, alignment and v1/v2/v3 signatures passed. Installed over the separate Input Test app on **AYN Thor** with its environment retained. Local packaging omits Rust store engines; the [full Build APK workflow for08526f828passed](https://github.com/Droid-Deck/DroidDeck/actions/runs/42295403480), including Android/JVM, Python/native, Rust store-engine and Xbox checks. - Review follow-ups keep migration IDs stable, preserve unreadable supported-version libraries before recovery, retain forward-version protection, release keyboard owners on focus loss, handle unavailable presentation displays, and allow controller confirmation/cancellation after key capture. Deck router cleanup checks ownership; BLE teardown synchronously blocks late routing/publication, and shared Triton slot ownership protects a replacement controller from stale instances. Legacy Select-to-Steam migrates to Desktop without replacing saved bindings; mouse backend transitions are serialized with publication. Profile refresh now updates custom controls in place, preserving terminal/dashboard state, the field keyboard and WoW chat. No physical BLE-controller test or new latency measurement was performed. - The latest08526f828APK is installed on Thor. Profile preferences stayed byte-for-byte identical and the retained Steam environment reached READY at 1280×720 / 60 Hz. The earlier capture-dialog fix was also exercised on Thor. APK SHA-256: `ed6ef888648468b4ebeb94e8f1ae74a195bb1f0a3f27e4fb3646d5462a4beb6b`. - Upstream integration preserves the new WoW chat panel, platform-pointer and PiP fixes. The fast-double-tap stick-click rule and its six regressions were adapted to the shared renderer; WoW field selection also preserves Ctrl held by a profile. - Latest Thor UI pass: existing Setup navigation, profile/layout previews, action persistence, centered element creation, main-screen editing over Steam, secondary canvas with primary tools/properties, and X returning from the docked menu to controls. - Across the documented device sessions, the user tested native physical buttons/sticks/triggers, Deck pads/grips, desktop pointer/clicks/scrolling/arrows, Android keyboard with physical input, Ctrl+F and Ctrl+A/X/V/S, simultaneous physical/touch holds, menu release and Home/resume. These historical checks are distinguished from the latest UI-only pass. [Revision-specific results, APK hashes and limitations](https://github.com/Delete228/DroidDeck/blob/codex/unified-controller-input/docs/development/unified-input-validation.md). ## Coordination and remaining work - #1202 (jmarti326): shortcut capture/device identity and the shared-assignment discussion informed this work. - #1305 (kingokksa): shared PC-key definitions were adapted from that PR; this does not absorb its entire feature set. - #1414, #581, #1226 and #1150 remain relevant for external-device input, calibration, keyboard routing and DualSense integration. The DualSense coordination discussion in this PR describes adapter boundaries and duplicate-stream suppression. Import/export, automatic per-app selection, macros, multi-button activation chords and automatic text-field keyboard detection remain out of scope. A single binding emitting a key combination is implemented. The configuration UI is now implemented and exercised on Thor. I am happy to continue UX/accessibility/localization refinements here before merge, or in follow-up PRs, according to maintainer feedback. The main review questions remain the shared model, source ownership/lifecycle contract, migration policy and integration with other input work. ## UI walkthrough on AYN Thor Actual captures from the installed Input Test build. In paired images, the main display is above the second display. The test application name and local build badge are not production branding changes. <details> <summary>Controller settings: separate defaults for Desktop and Game, followed by the profile library.</summary> <img width="960" alt="Controller settings: separate defaults for Desktop and Game, followed by the profile library." src="https://github.com/user-attachments/assets/a4c156d6-9ded-43df-b0f8-e5f34028806d" /> </details> <details> <summary>Physical controller: bindings grouped by control type, with each current action visible.</summary> <img width="960" alt="Physical controller: bindings grouped by control type, with each current action visible." src="https://github.com/user-attachments/assets/a8a840e3-19ac-473f-9f55-b06ae9c637b5" /> </details> <details> <summary>Reusable layouts: bounded previews and separate screen targets.</summary> <img width="960" alt="Reusable layouts: bounded previews and separate screen targets." src="https://github.com/user-attachments/assets/cf0cf7d1-42f3-46cd-b50b-f01e1fa8ed41" /> </details> <details> <summary>Main-screen editor over the live Steam UI, with the movable tools expanded. The background is Steam artwork, not a running game.</summary> <img width="960" alt="Main-screen editor over the live Steam UI, with the movable tools expanded. The background is Steam artwork, not a running game." src="https://github.com/user-attachments/assets/0781f7c8-fd10-47e5-8a23-817f945a6289" /> </details> <details> <summary>Dual-screen editor: tools and selection on the main display, full-size placement canvas on the second display.</summary> <img width="720" alt="Dual-screen editor: tools and selection on the main display, full-size placement canvas on the second display." src="https://github.com/user-attachments/assets/f13d4ef6-be6d-4d95-9d42-e047e0a63b8a" /> </details> <details> <summary>The selected lower-screen trackpad stays visible while its properties open on the main display.</summary> <img width="720" alt="The selected lower-screen trackpad stays visible while its properties open on the main display." src="https://github.com/user-attachments/assets/26baf597-f037-419e-9601-e48f086c0d4f" /> </details> <details> <summary>Session menu: expand Profile to select the physical, main-screen and second-screen parts independently.</summary> <img width="720" alt="Session menu: expand Profile to select the physical, main-screen and second-screen parts independently." src="https://github.com/user-attachments/assets/27ac4d9c-68b3-4257-8353-23104e2343de" /> </details>