device filtering support

[Xiaofan Chen]

On Sat, Feb 4, 2012 at 11:04 PM, Segher Boessenkool
<segher at kernel.crashing.org> wrote:
> Hi guys,
>
> It seems there are a few separate issues here.  Let me try to
> summarise and untangle it a bit, and propose a few things.
>
> First, there is "enumeration".  In the context of USB, this is
> often understood to mean walking the bus, sending lots of packets.
> We do not want that, that is the kernel's job, not ours.  The only
> thing libusbx should do is ask the OS "what devices are there?",
> and get some list of identifying information for those devices from
> that.  This might or might not contain some descriptors; it is hard
> to imagine an OS that does not at least give us VID/PID for the
> devices, for example.

VID/PID is not the issue here. Right now libusb tries to get the
descriptor and other properties of all device, which can be an
expensive process and that is why some users complaint about
the slow process of getting the device list.

Just an example here under Mac OS X.
http://libusb.6.n5.nabble.com/OpenUSB-LibUSB-on-Mac-OS-X-tp5444418p5457996.html

Take note of all those [darwin_cache_device_descriptor] call. Once
you hit the device like the suspended Bluetooth controller, you get
the following. Is this necessary if my program has nothing to do
with the Bluetooth controller?

No matter what we argue whether this is a really serious issue
(like causing other program to fail) or not, I think it is reasonable to
classify it as a performance issue.

libusb:info [process_new_device] found device with address 5 port = 2
parent = 0x7fec82c0a398 at 0x7fec84903200
libusb:info [process_new_device] allocating new device for location 0xfa113000
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe000404f. sleeping for 30 ms before trying again
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe00002ed. sleeping for 30 ms before trying again
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe000404f. sleeping for 30 ms before trying again
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe00002ed. sleeping for 30 ms before trying again
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe000404f. sleeping for 30 ms before trying again
libusb:debug [darwin_cache_device_descriptor] kernel responded with
code: 0xe00002ed. sleeping for 30 ms before trying again
libusb:warning [darwin_cache_device_descriptor] could not retrieve
device descriptor 05ac:8281: device not responding. skipping device
libusb:debug [libusb_unref_device] destroy device 0.0

> With hotplug (and hot unplug), all we need to immediately handle
> from the OS is again such a device handle ("minimal identifying
> information").
>
> I understand the libusb core wants to keep all kinds of other info
> cached; we should change that to not ask for that information until
> it is actually needed.  And most of that information should be
> cached, not asked for every time!

I think that is the fundamental "fix" or "optimization" we need.

> If there is anything in our API that requires us to ask the OS for
> lots more (expensive) information than what is actually used, then
> we need to change things there (add a new cheaper API).  But I've
> seen no evidence for that yet luckily.

Does the above serve as an evidence?

> Then, filtering.  There are two aspects to filtering: it can reduce
> the amount of communication between libusbx and the OS (if the OS's
> interfaces allow for that); and it can let the user of the library
> say up front what devices it cares for (so we don't bother it with
> uninteresting devices).

That is the main thing I am concerned. Filtering on VID/PID or
filtering on GUID (under Windows) can help reduce the traffic
on the bus.

> This can potentially reduce the boilerplate code somewhat, too; it
> isn't so great that every program using libusbx has to do the whole
> dance with libusb_get_device_list() etc. (or use
> libusb_open_device_with_vid_pid() which is completely inadequate
> for anything but toy programs).

Correct. It is not great at all for that every program using libusbx
has to do the whole dance with libusb_get_device_list() which is
not a cheap process.

> We could add a get_device_list_with_filter() thing; that reduces
> the OS communication, but not the boilerplate.  We could do an
> open_nth_device_with_filter() thing, but that is not ideal if you
> actually want to filter on more things.  It also doesn't help
> listing the devices.
>
> I propose we do a libusb_init_with_filter() instead; what it should
> do is completely ignore (as far as the user of the library is
> concerned, and as much as possible internally as well) all devices
> that the filter doesn't match.

I think this is a reasonable compromise.

> I don't have a strong opinion what a filter should look like, except
> it should be *simple*, so the filters people actually use can map
> to the filters the OS let's you filter by in its API.  I would avoid
> "deny" filters unless they are really really useful, since a) again,
> OSes do not give you those, and b) they are expensive (a list of
> allow/deny requires you to walk the whole list always, instead of
> using a hash or whatever).  So maybe just a list of accepted VID/PID,
> and classcode perhaps.  I dunno, let's experiment, and/or investigate
> what real programs want to use.

I like your proposal and I think it is a reasonable compromise.
Those who like to use the traditional way of libusb-1.0 can
call the normal libusb_init() and libusb_get_device_list(). Those
who do not want that behavior can call libusb_init_with_filter()
and libusb_get_device_list_with_filter().

In this case, I think the filtering will not break libusb (Pete's
words).

The filter can then be used for hotplug as well since in that
case we only care about hotplug event of interested device.


-- 
Xiaofan