Random issue with unexpected events generation

[larixer]

From time to time I see that tripletap and other clicking events occur when I'm not tapping touchpad at all, just moving my finger over surface, the tapping events are generated very often, Ubuntu goes crazy and opens Unity menu, switches between window (Alt+Tab behavior), and clicks. sudo evtest shows tripletap events despite me tapping at all. I'm using Linux kernel 4.8.6 and latest version of this driver.

I have not yet fully figured out what triggers this issue, but I'm 90% sure that it might be due to incomplete report error. Short before this issue started to happen in dmesg I saw:
[31545.072847] i2c_hid i2c-FTE1001:00: i2c_hid_get_input: incomplete report (30/512)
This weird behavior lasted for around a minute and then everything returned back to normal. I'm not sure 100% that incomplete report error triggers this issue. I will keep an eye on it and report back.

[redmcg]

Note that when you see the "incomplete report" message - this takes place within the i2c-hid module and the data is not passed on to the driver.

It could be the Touchpad is sending garbage or it could be trying to report a change in state (and how we should therefore interpret the following messages). Without documentation it's difficult to know.

I think the easiest way to confirm what the Touchpad is sending is by turning on the 'debug' parameter within i2c-hid. This will dump out everything that it is being passed to the driver. In fact it could be worth adding additional debug to the i2c-hid module so we can see all the data it receives; whether it's passed to the driver or not.

[larixer]

Yes, I will try to catch this issue with i2c-hid debug on for the start. It occurs rarely, once in two days approximately.

[ain101]

I am maybe able to reproduce this.
When i put an usb cable over the touchpad and position a finger just right the touchpad goes nuts.
(seems like a good source of fast randomness)
I am not seeing any incomplete report.
Windows behaves exactly the same.
Keep in mind that my laptop is a bit hardware modded, but i don't think it matters. (Asus ROG G501VW-FI074T)
@Vlasenko do you wear a ring or a watch.

My KDE desktop is now full of unwanted widgets. Also executed things from the bash history (like reboot).

[larixer]

@ain101 I wear a ring... Sometimes I also see another brain blowing issue - that cursor is moving and clicking all by itself with NO debug messages in dmesg when i2c-hid debugging is on! lol, when I start to move cursor with my finger I see the i2c-hid debug messages...

[ain101]

Cursor moving without debug messages might be a buffer problem. To much events to display, or the kernel is processing old events trying to catch up. The events are very fast after all.

Back to the usb cord problem:
Maybe the touchpad can be calibrated better. More insight into the firmware would be nice.
Maybe this could also be worked around an abstraction layer higher than this driver? I don't know how palm rejection works. It does fine in kubuntu.
Would be nice if we could perform better than the windows driver.

[redmcg]

Windows behaves exactly the same

@Vlasenko Have you seen this problem in Windows too? Or Linux exclusively?

Would be nice if we could perform better than the windows driver.

@ain101 It already does. It's in a Linux environment 😉

[larixer]

@redmcg The first thing I did when I bought my laptop was registration on Asus website inside Windows 10. It took me 10 mins. Ten very painful minutes with Windows, which drove me crazy by proposing to install anti-virus protection, to update web browser, to restart computer, etc, etc. After that I have removed all Windows 10 altogether, installed Linux and told myself that it was a sleep dream nightmare.

[redmcg]

@Vlasenko OK - that's fair enough. Sounds exactly like the Window's experience I know

[larixer]

@redmcg I was able to reproduce the issue. I have recorded several seconds of dmesg output and evtest events, attached to this comment. During this time I didn't do any clicks or taps on TouchPad only moves, but still tap events were triggered.

logs.zip

[redmcg]

OK cool. This provides a bit of insight.

I ran:

awk ' /Event:/ { print strftime(PROCINFO["strftime"], $3, 1) " -- " $0 } ' < ev.log

to convert the ev.log timestamp to a format that matched the dm.log - but they didn't quite match up. They look to be off by about 6 minutes - but that doesn't really matter as we can see from ev.log that the mouse clicks are originating from the device (I'm ruling out noise on the bus as the messages are all correctly formatted).

To make each event within dm.log easier to reference - I numbered each one with:

awk ' { print NR ") " $0 } ' < dm.log

So the first odd record is record 92:

92) [Wed Mar  1 04:38:13 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 94 01 37 ff ff 86 fa 30 ff ff 26 76 3d ff ff 00 00 00 00 00 00 00 00 00 00 00

The fourth byte has a value of 0x38. This indicates that three fingers are on the Touchpad. They are all registered at maximum pressure too (indicated by the 0xff x0ff bytes). This would trigger palm detection if it were on. That may be why I don't see this behaviour (as I have palm detection on).

What makes this look strongly like the Touchpad sending garbage is the fact that the contacts are registered for only one event. If it were a legitimate contact - that would be quite the ninja move.

It does continue to correctly track the original contacts movement though.

We can list all events that registered multiple contacts with the following grep:

$ awk ' { print NR ") " $0 } ' < dm.log | grep '1e 00 5d [^0]'
92) [Wed Mar  1 04:38:13 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 94 01 37 ff ff 86 fa 30 ff ff 26 76 3d ff ff 00 00 00 00 00 00 00 00 00 00 00
131) [Wed Mar  1 04:38:14 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 73 20 a6 30 7f 26 7f 12 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
202) [Wed Mar  1 04:38:14 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 63 d3 ff 30 7f 16 00 06 10 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
214) [Wed Mar  1 04:38:14 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 74 74 86 20 7f 75 e9 c1 10 35 25 7d ee ff ff 00 00 00 00 00 00 00 00 00 00 00
231) [Wed Mar  1 04:38:15 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 84 fb 20 30 7f 25 7b ee ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
277) [Wed Mar  1 04:38:15 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 74 b6 2b 30 7f 16 e1 0c 10 3e 46 0e 09 10 3e 00 00 00 00 00 00 00 00 00 00 00
348) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 83 d4 f2 30 7f 26 80 17 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
379) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 85 68 8d ff ff 26 03 03 10 4d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
381) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 85 ae 94 30 7f 96 0e cd 10 2b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
382) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 85 ca 93 30 7f 96 0e cd 10 2b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
385) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 95 2b 99 ff ff 25 7b f3 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
396) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 95 a7 45 ff ff 25 7d ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
409) [Wed Mar  1 04:38:17 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 84 b1 2b 30 7f 26 7c 0d ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
443) [Wed Mar  1 04:38:18 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 95 9e 4b 40 7f 76 7a 26 10 4b 26 7c 21 ff ff 00 00 00 00 00 00 00 00 00 00 00
526) [Wed Mar  1 04:38:18 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 64 ca 1e 30 7f 16 fc 28 10 54 76 6d 07 10 35 00 00 00 00 00 00 00 00 00 00 00
557) [Wed Mar  1 04:38:19 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 78 84 5e 1a ff ff a5 01 e8 ff ff 25 7d e6 ff ff 75 04 eb ff ff 00 00 00 00 00 00
701) [Wed Mar  1 04:38:20 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 91 3a 6f 30 7f 56 6b 12 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
767) [Wed Mar  1 04:38:20 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 93 18 b4 30 7f 96 fe 1d 10 4c 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
916) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 84 ea 8d 20 7f 96 e4 03 10 3a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
929) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 94 cc 17 30 7f 96 f9 28 30 62 26 79 2a ff ff 00 00 00 00 00 00 00 00 00 00 00
964) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 84 fe e3 ff ff 26 7e 12 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
973) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 95 b2 2f ff ff 26 f4 1f 20 57 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
985) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 a4 31 b2 ff ff 25 81 fc ff ff 85 f6 fb ff ff 00 00 00 00 00 00 00 00 00 00 00
1012) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 38 85 9d 6b ff ff 86 f9 c2 ff ff 16 f4 18 10 3e 00 00 00 00 00 00 00 00 00 00 00
1014) [Wed Mar  1 04:38:22 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 85 dd ab ff ff 25 7c f4 ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1075) [Wed Mar  1 04:38:23 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 95 4a 0a ff ff 36 f7 3f 10 59 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1104) [Wed Mar  1 04:38:23 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 78 93 43 58 30 7f a5 01 f6 10 3c 75 7c fb 10 3b 25 7d fb ff ff 00 00 00 00 00 00
1123) [Wed Mar  1 04:38:23 2017] i2c_hid i2c-FTE1001:00: input: 1e 00 5d 18 94 6f 11 20 7f a6 78 03 10 2d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

So we can see that with the exception of events 381 and 382 - all of the multiple contact events exist in isolation (i.e. they exist for no longer than the single event).

And for 17 of these 28 events (60%), palm detection would be triggered (i.e. at least one contact has the 0xff 0xff bytes). And with the exception of events 929 and 973 - it looks like the rest are registered with a width value (ABS_MT_TOUCH_MAJOR) of 1 and a pressure value (ABS_MT_PRESSURE) less than 0x60. The vast majority of your legitimate contacts has a pressure value of 0x7f.

So that's what we can see - the next question is how to fix it.

Given the problem appears to be with the physical device - the best place to fix it would obviously be in the device. However - I don't expect to see ASUS issuing a recall any time soon.

So that leaves software. Software will need to be able to determine between what is a legitimate contact and what is noise from the device. The two software options are:

1. The Kernel (i.e. the driver); or
2. User-space (i.e. with the xorg drivers)

I'm leaning strongly towards user-space for two reasons:

1. If we are to target the isolated events - we will need to keep track of what was happening in the previous events. The driver doesn't really keep this state. It just passes on what the device reports. User space does keep this information as it needs to know how long a contact was present so it can determine the difference between a click and a drag
2. If we are to target the pressure value - I would want the "sensitivity" to be configurable. Too strict and we'll eliminate legitimate movements, too lax and we'll let too much noise through. Given user space is usually less restrictive in terms of security - I'd prefer to put that solution here.

The synaptic driver already offers some settings that might help. The man synaptics page has a lot of information. For example - under the section Tapping it states:

       A  tap event happens when the finger is touched and released in a time interval shorter than MaxTapTime, and the touch and release coordinates are less than MaxTapMove units apart.  A "touch"
       event happens when the Z value goes above FingerHigh, and an "untouch" event happens when the Z value goes below FingerLow.

So you could try setting FingerHigh to 100. This will eliminate the events where ABS_MT_PRESSURE was less than 100. Although this would mean your lighter contacts would be ignored - so you may need to experiment with this value.

Unfortunately there doesn't appear to be a MinTapTime (or something similar) - so we can't eliminate those isolated (one-off) events.

I also mentioned I have palm detect on:

    PalmDetect              = 1
    PalmMinWidth            = 5
    PalmMinZ                = 100

This would eliminate those 17 events where the bytes corresponding to ABS_MT_TOUCH_MAJOR and ABS_MT_PRESSURE are set to 0x7f.

Have a play and see if you can get some satisfactory results. I do think the values could be different for each user - as there are probably manufacturing variances. It could even be dependant on the model of laptop (assuming its local interference causing the phantom contacts).

[larixer]

@redmcg Cool! Awesome insight! I think it would be great to post this information on README page for driver users, so that they would be able to configure their userspace in an optimal way.

[redmcg]

@Vlasenko I just updated the README page with a link to my comment above. My comment above is aimed at the Synaptics driver - but some of the information may still be helpful for users of a different X Input Touchpad Driver.

Out of curiosity - have you been able to stop this from occurring?

[larixer]

@redmcg No I haven't been able to stop it. But I think the cause of this behavior is the small amount of water or dirt, this touchpad feels very unreliable when this happens compared to other touchpads.

[larixer]

I'm closing this for now, as it seems the problem is due to unusual touchpad sensivity to traces of water or dirt.