templisandClaude Opus 5
e65f9db3ae
Mirrors the solver's Windows-only voyage-import.ahk on KDE Wayland: - poe-clip-bridge.py: clipboard watcher that keeps only PoE item text, serves it on loopback and can trigger a sweep from the browser - voyage-sweep.py: hovers every cell of the in-game chart grid, Ctrl+C's it and hands over the whole batch in one go - userscript: feeds the batch into the solver page without focusing it Includes notes on the four Wayland pointer pitfalls this ran into. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
108 lines
3.9 KiB
Markdown
108 lines
3.9 KiB
Markdown
# Notes on driving the pointer under KDE Wayland
|
||
|
||
Everything here was measured on KDE Plasma 6 / `kwin_wayland`, multi-monitor,
|
||
game running as an XWayland client through Proton. If you are automating input
|
||
on Wayland, these four are what will bite you.
|
||
|
||
## 1. `ydotool mousemove --absolute` is useless on multi-monitor
|
||
|
||
Absolute positioning gets clamped into whatever output the pointer currently
|
||
sits on, so the mapping is neither linear nor monotonic:
|
||
|
||
| requested | landed on |
|
||
|---|---|
|
||
| `0,0` | `302,1081` |
|
||
| `1000,500` | `1855,747` |
|
||
| `5000,2500` | `1418,1411` |
|
||
| `6000,3000` | `301,1080` |
|
||
|
||
`xdotool mousemove` (XWarpPointer) does not work either — KWin ignores warp
|
||
requests from clients that do not own the pointer focus.
|
||
|
||
**Use relative motion.**
|
||
|
||
## 2. Injected relative motion is not applied 1:1
|
||
|
||
On one of the test monitors every injected delta came out multiplied by exactly
|
||
2.00 — not acceleration, a constant factor:
|
||
|
||
```
|
||
sent=-25,-96 moved=-50,-192 ratio=2.00,2.00
|
||
sent=+24,+99 moved=+48,+198 ratio=2.00,2.00
|
||
```
|
||
|
||
A naive "move by the remaining error" loop therefore oscillates forever around
|
||
the target, roughly ±100 px, and never converges.
|
||
|
||
**Fix:** learn the factor at runtime instead of hardcoding it. Send
|
||
`error / gain`, measure what actually happened, update `gain` with an
|
||
exponential moving average. Converges in a handful of iterations regardless of
|
||
scale factor or acceleration profile — see `move_to()` in `bin/voyage-sweep.py`.
|
||
|
||
Large single deltas additionally get distorted by pointer acceleration, so cap
|
||
each step (150 px works well).
|
||
|
||
## 3. `xdotool getmouselocation` freezes outside XWayland surfaces
|
||
|
||
X only learns the pointer position while the pointer is over an XWayland
|
||
surface. Anywhere else it keeps returning a stale value — for hours, with no
|
||
error:
|
||
|
||
```
|
||
KWin says: 5102,2532
|
||
xdotool says: 4300,2662 # stale, and stays that way
|
||
```
|
||
|
||
That silently breaks any control loop that starts outside the game window.
|
||
|
||
**Fix:** two-phase. Coarse approach with KWin's own cursor position, which is
|
||
always correct, then fine control with `xdotool` once the pointer is inside the
|
||
(XWayland) game window, where it is live and fast.
|
||
|
||
Reading KWin's cursor position without a compositor plugin:
|
||
|
||
```bash
|
||
echo 'print("PROBE " + workspace.cursorPos.x + "," + workspace.cursorPos.y);' > /tmp/probe.js
|
||
id=$(qdbus6 org.kde.KWin /Scripting org.kde.kwin.Scripting.loadScript /tmp/probe.js probe)
|
||
qdbus6 org.kde.KWin /Scripting/Script$id org.kde.kwin.Script.run
|
||
journalctl _COMM=kwin_wayland -n 5 -o cat | grep PROBE
|
||
qdbus6 org.kde.KWin /Scripting org.kde.kwin.Scripting.unloadScript probe
|
||
```
|
||
|
||
Roughly 300 ms per reading — fine for a coarse approach, too slow for a loop.
|
||
|
||
## 4. `wl-copy --clear` is asynchronous
|
||
|
||
Clearing the clipboard and then polling for "something appeared" is a race: the
|
||
poll still reads the *previous* owner's data. In a sweep this looks like every
|
||
cell copying the same item — 48 identical charts, all of them the first one.
|
||
|
||
**Fix:** write a unique sentinel token into the clipboard, then wait for it to be
|
||
replaced:
|
||
|
||
```python
|
||
token = f"sweep-{row}-{col}-{time.monotonic_ns()}"
|
||
subprocess.run(["wl-copy"], input=token, text=True)
|
||
send_ctrl_c()
|
||
# ... poll until clipboard != token
|
||
```
|
||
|
||
## Bonus: do not home the pointer into a screen corner
|
||
|
||
The obvious bootstrap for "get to a known position" — send a huge negative delta
|
||
and let it clamp in the corner — triggers KDE's hot corners. Ours opened the
|
||
application overview mid-sweep. Approach a known target directly instead.
|
||
|
||
## Bonus: measure the grid from a screenshot, do not eyeball it
|
||
|
||
Hand-calibrating cell corners is imprecise. Take a window screenshot, run a
|
||
brightness projection per axis to get the cell pitch, then draw the computed
|
||
centres back onto the screenshot and look at it:
|
||
|
||
```bash
|
||
spectacle -b -n -a -o window.png
|
||
```
|
||
|
||
That is how the 66.8 × 66.6 px pitch in this repo was derived — and how a wrong
|
||
row count (which silently changes the computed pitch) was caught.
|