Thursday, May 5, 2016

The difference between uinput and evdev

A recurring question I encounter is the question whether uinput or evdev should be the approach do implement some feature the user cares about. This question is unfortunately wrongly framed as uinput and evdev have no real overlap and work independent of each other. This post outlines what the differences are. Note that "evdev" here refers to the kernel API, not to the X.Org evdev driver.

First, the easy flowchart: do you have to create a new virtual device that has a set of specific capabilities? Use uinput. Do you have to read and handle events from an existing device? Use evdev. Do you have to create a device and read events from that device? You (probably) need two processes, one doing the uinput bit, one doing the evdev bit.

Ok, let's talk about the difference between evdev and uinput. evdev is the default input API that all kernel input device nodes provide. Each device provides one or more /dev/input/eventN nodes that a process can interact with. This usually means checking a few capability bits ("does this device have a left mouse button?") and reading events from the device. The events themselves are in the form of struct input_event, defined in linux/input.h and consist of a event type (relative, absolute, key, ...) and an event code specific to the type (x axis, left button, etc.). See linux/input-event-codes.h for a list or linux/input.h in older kernels.Specific to evdev is that events are serialised - framed by events of type EV_SYN and code SYN_REPORT. Anything before a SYN_REPORT should be considered one logical hardware event. For example, if you receive an x and y movement within the same SYN_REPORT frame, the device has moved diagonally.

Any event coming from the physical hardware goes into the kernel's input subsystem and is converted to an evdev event that is then available on the event node. That's pretty much it for evdev. It's a fairly simple API but it does have some quirks that are not immediately obvious so I recommend using libevdev whenever you actually need to communicate with a kernel device directly.

uinput is something completely different. uinput is an kernel device driver that provides the /dev/uinput node. A process can open this node, write a bunch of custom commands to it and the kernel then creates a virtual input device. That device, like all others, presents an /dev/input/eventN node. Any event written to the /dev/uinput node will re-appear in that /dev/input/eventN node and a device created through uinput looks just pretty much like a physical device to a process. You can detect uinput-created virtual devices, but usually a process doesn't need to care so all the common userspace (libinput, Xorg) doesn't bother. The evemu tool is one of the most commonly used applications using uinput.

Now, there is one thing that may cause confusion: first, to set up a uinput device you'll have to use the familiar evdev type/code combinations (followed-by a couple of uinput-specific ioctls). Events written to uinput also use the struct input_event form, so looking at uinput code one can easily mistake it for evdev code. Nevertheless, the two serve a completely different purpose. As with evdev, I recommend using libevdev to initalise uinput devices. libevdev has a couple of uinput-related functions that make life easier.

Below is a basic illustration of how things work together. The physical devices send their events through the event nodes and libinput is a process that reads those events. evemu talks to the uinput module and creates a virtual device which then too sends events through its event node - for libinput to read.

Monday, April 18, 2016

libinput and graphics tablet pad support

When we released graphics tablet support in libinput earlier this year, only tablet tools were supported. So while you could use the pen normally, the buttons, rings and strips on the physical tablet itself (the "pad") weren't detected by libinput and did not work. I have now merged the patches for pad support into libinput.

The reason for the delay was simple: we wanted to get it right [1]. Pads have a couple of properties that tools don't have and we always considered pads to be different to pens and initially focused on a more generic interface (the "buttonset" interface) to accommodate for those. After some coding, we have now arrived at a tablet pad-specific interface instead. This post is a high-level overview of the new tablet pad interface and how we intend it do be used.

The basic sign that a pad is present is when a device has the tablet pad capability. Unlike tools, pads don't have proximity events, they are always considered in proximity and it is up to the compositor to handle the focus accordingly. In most cases, this means tying it to the keyboard focus. Usually a pad is available as soon as a tablet is plugged in, but note that the Wacom ExpressKey Remote (EKR) is a separate, wireless device and may be connected after the physical pad. It is up to the compositor to link the EKR with the correct tablet (if there is more than one).

Pads have three sources of events: buttons, rings and strips. Rings and strips are touch-sensitive surfaces and provide absolute values - rings in degrees, strips in normalized [0.0, 1.0] coordinates. Similar to pointer axis sources we provide a source notification. If that source is "finger", then we send a terminating out-of-range event so that the caller can trigger things like kinetic scrolling.

Buttons on a pad are ... different. libinput usually re-uses the Linux kernel's include/input.h event codes for buttons and keys. But for the pad we decided to use plain sequential button numbering, starting at index 0. So rather than a semantic code like BTN_LEFT, you'd simply get a button 0 event. The reasoning behind this is a caveat in the kernel evdev API: event codes have semantic meaning (e.g. BTN_LEFT) but buttons on a tablet pad don't those meanings. There are some generic event ranges (e.g. BTN_0 through to BTN_9) and the Wacom tablets use those but once you have more than 10 buttons you leak into other ranges. The ranges are simply too narrow so we end up with seemingly different buttons even though all buttons are effectively the same. libinput's pad support undoes that split and combines the buttons into a simple sequential range and leaves any semantic mapping of buttons to the caller. Together with libwacom which describes the location of the buttons a caller can get a relatively good idea of how the layout looks like.

Mode switching is a commonly expected feature on tablet. One button is designated as mode switch button and toggles all other buttons between the available modes. On the Intuos Pro series tablets, that button is usually the button inside the ring. Button mapping and thus mode switching is however a feature we leave up to the caller, if you're working on a compositor you will have to implemented mode switching there.

Other than that, pad support is relatively simple and straightforward and should not cause any big troubles.

[1] or at least less wrong than in the past
[2] They're actually linux/input-event-codes.h in recent kernels

Thursday, April 7, 2016

Why libinput doesn't have a lot of config options

Most days, at least one of the bugs I deal with requests something along the lines of "just add $FOO as a config option". In this post, I'll explain why this is usually a bad solution. First, read http://www.islinuxaboutchoice.com/ and keep those arguments in mind. Generally, there are two groups of configuration options - hardware options and user options. Hardware options are those that deal with specific quirks needed on some hardware, but not on other hardware. User options are those that deal with user preferences such as tapping or two-finger vs. edge scrolling.

In the old synaptics driver, we added options whenever something new came up and we tried to make those options generic. This was a big mistake. The driver now has over 70 configuration options resulting in a test matrix with a googolplex of combinations. In other words, it's completely untestable. To make a device work users often have to find the right combination of options from somewhere, write out a root-owned config file and then hope this works. Why do we still think this is acceptable? Even worse: some options are very specific to hardware but still spread in user forum examples like an STD during spring break.

In libinput, we're having none of that. When hardware doesn't work we expect a user to file a bug, we get it fixed upstream for the specific model and thus automatically fix it for all users of that device. We're leaning heavily on udev's hwdb which we have extended to correct devices when the firmware announces wrong information. This has the advantage that there is only one authoritative source of quirks a device needs to work. And we can update this as time goes by without having to worry about stale configuration options. One good example here is the custom acceleration profile that Lenovo X230 touchpads have in libinput. All in all, there is little pushback for the lack of hardware-specific configuration options and most users are fine with it once they accept the initial waiting period to get the patch into their distribution.

User-specific options are more contentious. In our opinion, some features should be configurable and others should not. Where to draw that line is of course quite undefined. For example, tapping on or off was one of the first configuration options available and that was never a cause for arguments either way (except whether the default should be on or off). Other options are more contentious. Clickpad software buttons are always on the bottom edge and their size is hardcoded (synaptics allowed almost free positioning of those buttons). Other features such as changing a two-finger tap to some other button event is not supported at all in libinput. This effectively comes down to cost. You see, whenever you write "it's just 5 lines of code to make this an option", what I think is "once the patch is reviewed and applied, I'll spend two days to write test cases and documentation. I'll need to handle any bug reports related to this, and I'm expected to make sure this option works indefinitely. Any addition of another feature may conflict with this option, so I need to make sure the right combination is possible and test cases are written." So your work ends after writing a 5 line patch, my work as maintainer merely starts. And unless it pays off long-term, the effort is not worth it. Some features make that cut, others don't if they are too much of a niche feature.

All this is of course nothing new and every software project needs to make these decisions. Input isn't even a special case here, it pales in comparison with e.g. the decisions UI designers need to make. However, in FOSS we have a tendency to think that because something is possible, it should be done. Legally, you have freedom to do almost anything with the software, so you can maintain a local fork of libinput with that extra feature applied. If that isn't acceptable, why would it be acceptable to merge the patch and expect others to shoulder the costs?

Wednesday, April 6, 2016

libinput now has a touchpad software middle button

I just pushed a patch to libinput master to enable a middle button on the clickpad software buttons. Until now, our stance was that clickpads only get a left and right software button, split at the 50% mark. The reasoning is simple: touchpads only have markings for left and right buttons (if any!) and the middle button's extents are not easily discoverable if there is no visual or haptic feedback. A middle button event could however be triggered through middle button emulation, i.e. by clicking the touchpad with a finger on the left and right software button area (see the instructions here).

This is nice in theory but, as usual, reality gets in the way. Most interactions with the middle button are quick and short-lived, i.e. clicking the button once to paste. This interaction is what many touchpads are spectacularly bad at. For middle button emulation to be handled correctly, both fingers must be registered before the physical button press. The scanout rate on a touchpad is often too low and on touchpads with extremely light resistance like the T440 it's common to register the physical click before we know that there's a second finger on the touchpad. But even on a T450 and an X220 with much higher clickpad resistance I barely managed to get above 7 out of 10 correctly registered middle button events. That is simply not good enough.

So the patch I just pushed out to master enables a middle software button between the left and the right button. The exact width of the button scales with the touchpad but it's usually around 20-25mm and it's centered on the touchpad so despite the lack of visual or haptic guides it should be reliable to hit. The new behaviour is hard-coded and for now middle button emulation continues to work on touchpads. In the future, I expect I will remove middle button emulation on touchpads or at least disable it by default.

The middle button will be available in libinput 1.3.

Monday, March 14, 2016

why libinput does not have a configuration storage system

A question that pops up with some regularity is whether libinput has a global configuration storage system, and, subsequently, why it doesn't have one. Comparisons are drawn to the X-specific tool xinput that allows to trigger all configuration options (see below though).

As maintainer of libinput, I can state that libinput will not get a configuration storage system. That job is squarely in the realm of the caller (i.e. usually the compositor and/or the desktop environment). Here are a few reasons:

First, you'd get conflicts with the caller. You need to prioritise which configuration has precendence to decide what to do when libinput and the caller disagree on a configuration item. You (the user) also have to figure why a configuration doesn't work when it's clearly enabled in one of those systems. Ok, so you can work around this by putting in a warning somewhere, provided that you make sure that the warning shows up in the right place and users know where to look. And to know when to send the warning, because again, that requires libinput to know which config has priority.

This is amplified by the lack of an autoritative support system. Speaking from X experience, the number of posts on, say, the ubuntu forums that advocate setting configuration options that haven't existed for years is quite sad. Or users advocate config snippets that set everything but the feature they claim it enables. That gets copy-pasted, etc.

Some configuration options can be incompatible with each other. If you have conflicting configuration systems it gets harder because each configuration system cannot make certain options an either/or anymore. We don't have any of these options just yet, but they may come in the future.

Over time, supported features will change. A setting may be exposed in the magic libinput config system and, a few months later, it is now also exposed by, say, GNOME. All the documentation that points to the libinput configuration is now obsolete because GNOME overrides it. Unless you're running version Foo.Bar, or maybe the patched version from $DISTRIBUTION. It gets messy quickly.

How fine-grained do you need the configuration? libinput's config API applies separately for each device. For most desktop environments it is sufficient to have configuration per device type (touchpad vs mouse vs tablet, etc.). But some exceptions may apply and a newer generic touchpad configuration may need to override device-specific older configuration. Because the newer config is designed around the new libinput version, but the older config is something you copied off a forum post from 2 years ago.

Oh, implicit in the request for a configuration system in libinput is usually also: please write and maintain that system for free, and fix any bugs in a timely manner. And I can't shake the feeling that a large number of these requests are only triggered by "my desktop doesn't expose this setting, if we get it in libinput I can force it into any desktop".

Ironically enough, the tool that is usually used as an example for how it could work is xinput. The only reason xinput works is because the xf86-input-libinput driver exposes properties and maps changes in those to the libinput API calls. i.e. everything is handled by libinput's caller, not libinput itself. Just as it should be.

Lest I be accused of only shooting down ideas here is a constructive idea: write a library or DBus-daemon that exposes some configuration storage and makes it easy to query. Then convince the various desktop environments to use that system instead their existing solutions, and bang, you have a generic configuration storage system for libinput.

Tuesday, March 1, 2016

libinput and graphics tablet support

Last week's libinput 1.2 release included the new graphics tablet support. This work, originally started as Lyude's 2014 GSoC project enables the use of drawing tablets through libinput (think Wacom tablets, not iPad/Android tablet).

Wacom tablets provide three input types: pen-like tools, buttons and touch rings/strips on the tablet itself, and touch. libinput's tablet support work focuses on the tool support only, pad buttons are still work in progress. Touch is already supported, either through the touchpad interfaces for external tablets (e.g. Intuos) or touchscreen interfaces for direct-touch tablets (e.g. Cintiq). So the below only talks about how to get events from tools, the pad events will be covered in a separate post when it's ready.

How do things work in the xf86-input-wacom driver, the current standard for tablet support in Linux? The driver checks the kernel device node for capabilities and creates multiple X devices, usually pen, eraser, cursor, and pad. When a pen comes into proximity the driver sends events through the pen device, etc. The pen device has all possible axes available but some (e.g. rotation) won't be used unless you're actually using an Wacom Art Pen. Unless specifically configured, all pens send through the same device, all erasers send through the same device, etc.

The libinput tablet API is a notable departure from this approach. In libinput, each tool is a separate entity generating events. A client doesn't wait for events from the tablet, it waits for events from a tool. The tablet itself is little more than an idle device. This has a couple of effects:

  • A struct libinput_tablet_tool is created on-the-fly as a tool comes into proximity and its this struct that events are tied to.
  • This means we default to per-tool handling. Two pens will always be separate and never multiplexed through one device. [1]
  • The tool can be uniquely identified [1]. It's easy to track a tool across two tablets even though this is quite a niche case.
  • A client can query the tool struct for capabilities, but not the tablet. Hence you cannot know what capabilities are available until a tool is in proximity.
  • The tool-based approach theoretically enables us to have multiple tools in proximity simultaneously, though no current hardware supports this.
Now, the advantages for the professional use-case where artists have multiple tools and/or multiple tablets is quite obvious. But it also changes some things for the average user with a single tool, specifically: the data provided by libinput is now precise and reflects the tool you're using. No more fake rotation axis on your standard pen. But you cannot query that information until the pen has been in proximity at least once. This is a change that clients will have to eventually deal with.

I just pushed the tablet support for the xf86-input-libinput driver and this driver reflects the new approach that libinput takes. When a tablet is detected, we create one device that has no axes and serves as the parent device. Once the first tool comes into proximity, a new device is created with the exact capabilities that the tool provides and the serial number in the name:

$> xinput list
⎡ Virtual core pointer                     id=2 [master pointer  (3)]
[...]
⎜   ↳ Wacom Intuos5 touch M Pen Pen (0x99800b93) id=21 [slave  pointer  (2)]
⎣ Virtual core keyboard                    id=3 [master keyboard (2)]
[...]
    ↳ Wacom Intuos5 touch M Pen                id=11 [slave  keyboard (3)]
Device 11 is effectively mute (in the future this may become the Pad device, not sure yet). Device 21 appeared once I moved the tool into proximity. That tool has all the properties for configuration as they appear on the device. So far this works, but it means static configuration becomes more difficult. If you are still using xinit scripts or other scripts that only run once on login and not once per new device then the options may not apply correctly.

[1] provided the hardware can uniquely identify the pens

Monday, January 25, 2016

libinput and semi-mt touchpads

libinput 1.1.5 has a change in how we deal with semi-mt touchpads, in particular: interpretation of touch points will cease and we will rely on the single touch position and the BTN_TOOL_* flags instead to detect multi-finger interaction. For most of you this will have little effect, even if you have a semi-mt touchpad. As a reminder: semi-mt touchpads are those that can detect the bounding box of two-finger interactions but cannot identify which finger is which. This provides some ambiguity, a pair of touch points at x1/y1 and x2/y2 could be a physical pair of touches at x1/y2 and x2/y1. More importantly, we found issues with semi-mt touchpads that go beyond the ambiguity and reduce the usability of the touchpoints.

Some devices have an extremely low resolution when two-fingers are down (see Bug 91135), the data is little better than garbage. We have had 2-finger scrolling disabled on these touchpads since before libinput 1.0. More recently, Bug 93583 showed that some semi-mt touchpads do not assign the finger positions for some fingers, especially when three fingers are down. This results in touches defaulting to position 0/0 which triggers palm detection or results in scroll jumps, neither of which are helpful. Other semi-mt touchpads assign a straightforward 0/0 as position data and don't update until several events later (see Red Hat Bug 1295073). libinput is not particularly suited to handle this, and even if it did, the touchpad's reaction to a three-finger tap would be noticeably delayed.

In light of these problems, and since these affect all three big semi-mt touchpad manufacturers we decided to drop back and handle semi-mt touchpads as single-finger touchpads with extra finger capability. This means we track only one touchpoint but detect two- and three-finger interactions. Two-finger scrolling is still possible and so is two- and three-finger tapping or the clickfinger behaviour. What isn't possible anymore are pinch gestures and some of the built-in palm detection is deactivated. As mentioned above, this is unlikely to affect you too much, but if you're wondering why gestures don't work on your semi-mt device: the data is garbage.