voyage-bulk-autoimporter/docs/wayland-pointer-notes.md

Notes on driving the pointer under KDE Wayland

Everything here was measured on KDE Plasma 6 / kwin_wayland, multi-monitor, game running as an XWayland client through Proton. If you are automating input on Wayland, these four are what will bite you.

1. ydotool mousemove --absolute is useless on multi-monitor

Absolute positioning gets clamped into whatever output the pointer currently sits on, so the mapping is neither linear nor monotonic:

requested	landed on
0,0	302,1081
1000,500	1855,747
5000,2500	1418,1411
6000,3000	301,1080

xdotool mousemove (XWarpPointer) does not work either — KWin ignores warp requests from clients that do not own the pointer focus.

Use relative motion.

2. Injected relative motion is not applied 1:1

On one of the test monitors every injected delta came out multiplied by exactly 2.00 — not acceleration, a constant factor:

sent=-25,-96   moved=-50,-192   ratio=2.00,2.00
sent=+24,+99   moved=+48,+198   ratio=2.00,2.00

A naive "move by the remaining error" loop therefore oscillates forever around the target, roughly ±100 px, and never converges.

Fix: learn the factor at runtime instead of hardcoding it. Send error / gain, measure what actually happened, update gain with an exponential moving average. Converges in a handful of iterations regardless of scale factor or acceleration profile — see move_to() in bin/voyage-sweep.py.

Large single deltas additionally get distorted by pointer acceleration, so cap each step (150 px works well).

3. xdotool getmouselocation freezes outside XWayland surfaces

X only learns the pointer position while the pointer is over an XWayland surface. Anywhere else it keeps returning a stale value — for hours, with no error:

KWin says:    5102,2532
xdotool says: 4300,2662     # stale, and stays that way

That silently breaks any control loop that starts outside the game window.

Fix: two-phase. Coarse approach with KWin's own cursor position, which is always correct, then fine control with xdotool once the pointer is inside the (XWayland) game window, where it is live and fast.

Reading KWin's cursor position without a compositor plugin:

echo 'print("PROBE " + workspace.cursorPos.x + "," + workspace.cursorPos.y);' > /tmp/probe.js
id=$(qdbus6 org.kde.KWin /Scripting org.kde.kwin.Scripting.loadScript /tmp/probe.js probe)
qdbus6 org.kde.KWin /Scripting/Script$id org.kde.kwin.Script.run
journalctl _COMM=kwin_wayland -n 5 -o cat | grep PROBE
qdbus6 org.kde.KWin /Scripting org.kde.kwin.Scripting.unloadScript probe

Roughly 300 ms per reading — fine for a coarse approach, too slow for a loop.

4. wl-copy --clear is asynchronous

Clearing the clipboard and then polling for "something appeared" is a race: the poll still reads the previous owner's data. In a sweep this looks like every cell copying the same item — 48 identical charts, all of them the first one.

Fix: write a unique sentinel token into the clipboard, then wait for it to be replaced:

token = f"sweep-{row}-{col}-{time.monotonic_ns()}"
subprocess.run(["wl-copy"], input=token, text=True)
send_ctrl_c()
# ... poll until clipboard != token

Bonus: do not home the pointer into a screen corner

The obvious bootstrap for "get to a known position" — send a huge negative delta and let it clamp in the corner — triggers KDE's hot corners. Ours opened the application overview mid-sweep. Approach a known target directly instead.

Bonus: measure the grid from a screenshot, do not eyeball it

Hand-calibrating cell corners is imprecise. Take a window screenshot, run a brightness projection per axis to get the cell pitch, then draw the computed centres back onto the screenshot and look at it:

spectacle -b -n -a -o window.png

That is how the 66.8 × 66.6 px pitch in this repo was derived — and how a wrong row count (which silently changes the computed pitch) was caught.