Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

View: New views
14 Messages — Rating Filter:   Alert me  

Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by ChMeessen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,

I'm a computer scientist and beginner with USB programming on linux.
Our problem is as following.

We have a card developped by electronicians in our lab with an FPGA and
a QuickUSB controller.
It is currently used with a linux computer but has a very low throughput
~3MB/s. We need at least 10MB/s and preferably 30MB/s.

As stated in the QuickUSB doc, the driver provided by quickUSB for linux
doesn't support asynchronous bulk transfer.

Reading the following thread of this mailing list
http://www.nabble.com/Write-Linux-USB-Driver-td14913265.html

I saw that libusb did not support asynchronous transfer and that quick
USB has no interrupt link. Unfortunately I can't change the controller
now. Maybe in future card versions.

I assume that the linux driver provided with the QuickUSB is just a thin
layer on top of libusb and reading the latest libusb specification it is
advertised to support isochronous and asynchronous transfer. This would
be a great news.

On windows we achieved 30MB/s throughput, but the controller didn't stop
sending data. It kept sending whatever data was on the controller's data
write port.

So the question is if there is any code example and tutorial I could
look at to learn how to get the best throughput out of this QuickUSB
controller. Is this at least feasible ?

I assume the linux side is kind of standard way to do things for which I
should easily find some example code. One of the questions I'm asking my
self is if the interrupt channel is required to get the maximum
throughput ?

--
Ch. Meessen

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by Peter Stuge-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Christophe,

Christophe Meessen wrote:
> We have a card developped by electronicians in our lab with an FPGA and
> a QuickUSB controller.

That product is really overpriced! Wow. :(


> It is currently used with a linux computer but has a very low
> throughput ~3MB/s. We need at least 10MB/s and preferably 30MB/s.
>
> As stated in the QuickUSB doc, the driver provided by quickUSB for
> linux doesn't support asynchronous bulk transfer.

You have to dump all your technical information on the list. Then we
would actually be able to provide help, or ask you for some testing.

Right now, you haven't specified the problem very clearly so noone
knows exactly what you need to do.


> One of the questions I'm asking my self is if the interrupt channel
> is required to get the maximum throughput ?

No. USB interrupt pipes will not give very high throughput. If you
need reliable transfers you should be using bulk. This is specified
by the device descriptors, so can not be chosen by the software
developer.


Feedback to your department is that USB absolutely must be a
crossover project with both hardware and software groups working
TOGETHER - or the end result will likely be useless hardware and
certainly lack of understanding among students. Time wasted.


Please read the USB specification to learn more about the USB
software stack/model. (The hardware guys have already done this,
and you should of course have been doing that together.)

USB is completely host centric. The device must follow many rules and
specifications to work correctly. This means that software developers
actually create some requirements for the hardware developers. This
isn't really the norm, so it is important to have both groups working
together on some parts.

Please send lsusb -v output (the descriptors) for your device.


Finally, when writing your driver using libusb, make sure that you
have many transfer requests pending at all times, that is the only
way you will get the throughput you want. (The request queue travels
all the way down to the wire, so if you make sure there are pending
requests, the wire utilization will be maximal.)

There is also a Cypress FX2 community for people working with those
USB chips in Linux.

Note that you will quite likely have to throw away whatever software
quickusb provided you with, unless they are actually using libusb as
outlined above. You should look into what their software does, and
maybe you can actually get some support from them - I think that
should be included in the high price for their product.


//Peter

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by Marko Cebokli :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Cristophe,

I think your problem is mainly in the system latencies.

I play with the same EZ-USB cypress chip:

http://lea.hamradio.si/~s57uuu/uuusb/index.htm
(see also the software section linked from there)

and I think we have the same problem - it is not hard to get a lot of average
bandwidth (>20 MB/s), but the problem is when you do not have much
device-side buffering, like only the cypress internal buffers, that you
experience data loss.

With libusb 0.1 I can get cca 2..10 MB/s of quasi reliable transfers (1% lost
packets), depending on the machine. (that is with a "relatime" kernel and
scheduling)

Currently, I am experimenting with libusb 1.0. I found synchronous or
asynchronous mode with a single transfer to be quite similar to libusb 0.1.

Throwing out more than one transfer at a time does help.
I foun the difference between one and two issued asynchronous transfers quite
pronounced, but using more than two does not seem to bring very much. I hope
to tidy up the code soon and put it at the above website soon...

Marko Cebokli



On Thursday 08 October 2009 10:15:10 Christophe Meessen wrote:

> Hello,
>
> I'm a computer scientist and beginner with USB programming on linux.
> Our problem is as following.
>
> We have a card developped by electronicians in our lab with an FPGA and
> a QuickUSB controller.
> It is currently used with a linux computer but has a very low throughput
> ~3MB/s. We need at least 10MB/s and preferably 30MB/s.
>
> As stated in the QuickUSB doc, the driver provided by quickUSB for linux
> doesn't support asynchronous bulk transfer.
>
> Reading the following thread of this mailing list
> http://www.nabble.com/Write-Linux-USB-Driver-td14913265.html
>
> I saw that libusb did not support asynchronous transfer and that quick
> USB has no interrupt link. Unfortunately I can't change the controller
> now. Maybe in future card versions.
>
> I assume that the linux driver provided with the QuickUSB is just a thin
> layer on top of libusb and reading the latest libusb specification it is
> advertised to support isochronous and asynchronous transfer. This would
> be a great news.
>
> On windows we achieved 30MB/s throughput, but the controller didn't stop
> sending data. It kept sending whatever data was on the controller's data
> write port.
>
> So the question is if there is any code example and tutorial I could
> look at to learn how to get the best throughput out of this QuickUSB
> controller. Is this at least feasible ?
>
> I assume the linux side is kind of standard way to do things for which I
> should easily find some example code. One of the questions I'm asking my
> self is if the interrupt channel is required to get the maximum
> throughput ?
>
> --
> Ch. Meessen
>
> ---------------------------------------------------------------------------
>--- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Libusb-devel mailing list
> Libusb-devel@...
> https://lists.sourceforge.net/lists/listinfo/libusb-devel



------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2(linux) ?

by Michael Plante :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

>> and I think we have the same problem - it is not hard to get a lot of
average
>> bandwidth (>20 MB/s), but the problem is when you do not have much
>> device-side buffering, like only the cypress internal buffers, that you
>> experience data loss.

If this is the problem, I imagine it's easy to fix, since he's already using
an FPGA.  If the data is generated on-demand, you have dedicated logic to do
it.  If it's a stream of data back to the host from some bursty source,
throw a FIFO into the FPGA.  But is it the problem in this case?

Michael



------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Parent Message unknown Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by Tim Roberts :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

You wrote:
>
>We have a card developped by electronicians in our lab with an FPGA and
>a QuickUSB controller.
>It is currently used with a linux computer but has a very low throughput
>~3MB/s. We need at least 10MB/s and preferably 30MB/s.
>
>As stated in the QuickUSB doc, the driver provided by quickUSB for linux
>doesn't support asynchronous bulk transfer.

To achieve that, you must use asynchronous transfers.

>I saw that libusb did not support asynchronous transfer and that quick
>USB has no interrupt link. Unfortunately I can't change the controller
>now. Maybe in future card versions.

Libusb 0.x does not support asynchronous transfers.  Libusb 1.x certainly does.  Interrupt vs bulk makes no difference.  And the term is "pipe" or "endpoint", not "link".

>I assume that the linux driver provided with the QuickUSB is just a thin
>layer on top of libusb and reading the latest libusb specification it is
>advertised to support isochronous and asynchronous transfer. This would
>be a great news.

"Isochronous" and "asynchronous" are not mutually exclusive.  Isochronous pipe requests can be submitted asynchronously, just as bulk and interrupt pipe requests can.

>So the question is if there is any code example and tutorial I could
>look at to learn how to get the best throughput out of this QuickUSB
>controller. Is this at least feasible ?

All you need to know are the endpoint numbers, and those are specified in the firmware your engineers burned into the firmware.

>One of the questions I'm asking my
>self is if the interrupt channel is required to get the maximum
>throughput ?

On the contrary, to get the maximum throughput, you must use a bulk pipe.  An interrupt pipe can do no more than 24 MB/s.
--
Tim Roberts, timr@...
Providenza & Boekelheide, Inc.

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by ChMeessen :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,

thank you very much for all your answers. First a bit more information
on the device behind the usb. It is an X ray imaging prototype that we
are developing in our lab. It has to download 560x960 16/32bit pixels
images fast (~1MB/2MB data). It also has to upload some configurations,
but performance is less critical. The current data throughput we get on
Linux with the quick usb driver is about 3 MB/s and is too low.

The colleague who implemented it is from the hardware side. I'm from the
sofware side and I'm trying to figure out how I can get close to 30MB/s
data transfer, which seams a reasonable goal according to the USB 2.0
specifications.

I have to deal with the existing card and USB device. Looking at the FX2
data-sheet I have the impression it is a good controller. The choice was
driven by the rational to have a plug and play USB controller which is
true ... to some extend.

The current code (3MB/s) uses the bulk synchronous transfer mode. The
firmware was chosen for fastest data transfer rate and measured with the
oscilloscope to be 480Mb/s. The oscilloscope, also showed huge waiting
time between data sending which sounds logic with synchronous data
transfer and a driver in user space.

Now that libusb 1.0 support asynchronous transfer, I'm planning to give
it a try. But I have some probably noob questions to ask.

1. Marko Cebokli reports 1% loss of packets. I was told USB bulk
transfer is reliable and robust which means that data is transmitted
uncorrupted and without loss to endpoints. If the USB controller detects
corrupted packets it takes care to retransmit them. Is that true ?
There is enough buffer space in our fpga card, so buffer overflow should
not occur. Bus speed is much faster than USB speed.

2. Did anyone achieve to get 30MB/s throughput using libusb 1.0 or is
this too ambitious ?  Should I better consider writing a kernel driver
which, I was told, would have much less latency ?

3. Is libusb 1.0 still depending on usbfs kernel driver ? If yes, is
their a constrain on its version ? I'm using Ubuntu with the linux
kernel 2.6.28-15 on which libusb 0.1-4 is installed.

Here is what I see in the message log file when I plug the controller:

Oct  9 11:37:50 xxx kernel: [330902.700012] usb 2-6: new high speed USB
device using ehci_hcd and address 11
Oct  9 11:37:50 xxx kernel: [330902.833697] usb 2-6: configuration #1
chosen from 1 choice

Here is the output of lsusb -v

Bus 002 Device 011: ID 0fbb:0001 Bitwise Systems, Inc.
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x0fbb Bitwise Systems, Inc.
  idProduct          0x0001
  bcdDevice            2.11
  iManufacturer           1 Bitwise Systems
  iProduct                2 QuickUSB QUSB2 Module v2.11.10 (Simple I/O)
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x86  EP 6 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

The product Id indicates simpleIO which is the firmware currently
installed. This one allows to reach 480Mb/s sending speed. I see two
endpoints. Apparently one for input and the other for outpu, each one
configured with 512 byte (words?) buffer. In the DX2 datasheet it is
stated that there should be four 512 buffers for each end point. What
does the 1x512 mean ? Does it mean there is only one 512 buffer ?

4. I downloaded libusb 1.0.3 I would like to give it a try. Is there
anything I have to do before being trying to use it ? For instance do
something with udev so it somehow attaches usbfs driver to it ?

5. I wrote a piece of code compiled with libusb 0.1(pre-installed on
Ubuntu) that locates the  QuickUSB device and get its description
informations. Apparently it can't get the product identification string
unless it is run with root privilege (sudo). Same for lsusb. Is this the
normal behavior ?

Ch. Meessen










------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by Alan Stern :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Fri, 9 Oct 2009, Christophe Meessen wrote:

> 1. Marko Cebokli reports 1% loss of packets. I was told USB bulk
> transfer is reliable and robust which means that data is transmitted
> uncorrupted and without loss to endpoints. If the USB controller detects
> corrupted packets it takes care to retransmit them. Is that true ?

Yes.  Marko was referring to the possibility that the buffer space on
the device might be too small, so that packets overflow the buffer and
are lost before they can be sent to the host (IN) or before the device
can act on them (OUT).

> There is enough buffer space in our fpga card, so buffer overflow should
> not occur. Bus speed is much faster than USB speed.
>
> 2. Did anyone achieve to get 30MB/s throughput using libusb 1.0 or is
> this too ambitious ?  Should I better consider writing a kernel driver
> which, I was told, would have much less latency ?

Bus latency for kernel drivers is no smaller than for properly-written
asynchronous userspace drivers.

I don't know what speeds people have attained using libusb-1.0.  In the
end, it is probably limited more by the device than by the host.

> 3. Is libusb 1.0 still depending on usbfs kernel driver ?

Yes.

> If yes, is
> their a constrain on its version ? I'm using Ubuntu with the linux
> kernel 2.6.28-15 on which libusb 0.1-4 is installed.

Libusb-1.0 certainly will work with your kernel.  While there are
constraints, you shouldn't have to worry about them.  I imagine libusb
would work with any kernel later than 2.6.9, and probably with earlier
kernels too.

> Here is the output of lsusb -v
...

>       Endpoint Descriptor:
>         bLength                 7
>         bDescriptorType         5
>         bEndpointAddress     0x02  EP 2 OUT
>         bmAttributes            2
>           Transfer Type            Bulk
>           Synch Type               None
>           Usage Type               Data
>         wMaxPacketSize     0x0200  1x 512 bytes
>         bInterval               0
>       Endpoint Descriptor:
>         bLength                 7
>         bDescriptorType         5
>         bEndpointAddress     0x86  EP 6 IN
>         bmAttributes            2
>           Transfer Type            Bulk
>           Synch Type               None
>           Usage Type               Data
>         wMaxPacketSize     0x0200  1x 512 bytes
>         bInterval               0
...

> The product Id indicates simpleIO which is the firmware currently
> installed. This one allows to reach 480Mb/s sending speed. I see two
> endpoints. Apparently one for input and the other for outpu, each one
> configured with 512 byte (words?) buffer.

Bytes.  And the number is not a buffer size; it is the maximum packet
size.

> In the DX2 datasheet it is
> stated that there should be four 512 buffers for each end point. What
> does the 1x512 mean ? Does it mean there is only one 512 buffer ?

No.  The "1x" means that the endpoint is not capable of
"high-bandwidth" operation.  This is expected, because
"high-bandwidth" applies only to periodic endpoints (interrupt and
isochronous).

In full-speed or low-speed devices, interrupt and isochronous transfers
are limited to one transaction per frame.  In high-speed devices,
non-"high-bandwidth" interrupt and isochronous endpoints are likewise
limited to one transaction per microframe (that's what the "1x" refers
to).  However "high-bandwidth" interrupt and isochronous endpoints are
allowed up to three transactions per microframe; that's where the name
"high-bandwidth" comes from.

By contrast, high-speed bulk endpoints are allowed as many as 13
transactions per microframe.  Hence they can enjoy even more bandwidth
than the so-called "high-bandwidth" endpoints.

> 4. I downloaded libusb 1.0.3 I would like to give it a try. Is there
> anything I have to do before being trying to use it ? For instance do
> something with udev so it somehow attaches usbfs driver to it ?

You don't need to do anything.

> 5. I wrote a piece of code compiled with libusb 0.1(pre-installed on
> Ubuntu) that locates the  QuickUSB device and get its description
> informations. Apparently it can't get the product identification string
> unless it is run with root privilege (sudo). Same for lsusb. Is this the
> normal behavior ?

This depends on the permissions for the device file:
/dev/bus/usb/BBB/DDD.  If write access is restricted to root then you
will need root privilege to do anything nontrivial with the device.

The device file's permissions are generally determined by a udev rule.  
If you like, you can add a rule to give your QuickUSB device more
relaxed permissions.

Alan Stern


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by Ray S :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

At 01:28 PM 10/9/2009 +0200, you wrote:
The colleague who implemented it is from the hardware side. I'm from the
sofware side and I'm trying to figure out how I can get close to 30MB/s
data transfer, which seams a reasonable goal according to the USB 2.0
specifications.

Hello,
Look at Marko Cebokli's pages:
http://lea.hamradio.si/~s57uuu/uuusb/index.htm#fifo
where he implements 30MB/s
http://lea.hamradio.si/~s57uuu/uuusb/uuusb_software.htm#bandwidth
the main issue he reports finding is the occasional long wait for bulk_read. Since the internal buffer is max of 4k, an external FIFO would be needed for high speed unless you run RTOS.
I've implemented his simple_prg_rd.c in Python/libusb, if you're interested in Python.

I also have an FX2 (Sermod-100 board) and am implementing a high frame rate camera with his suggestions, and
with an external FIFO like the
 http://www.semiconductorstore.com/cart/pc/viewPrd.asp?idproduct=10565#details

Also see this FPGA project with FX2
http://www.myhdl.org/doku.php/users:cfelton:projects:usbp
and
http://www.acquiredevices.com/usbsoftware.html
if you have not looked at their code yet...

More bookmarks:
http://lea.hamradio.si/~s57uuu/uuusb/uuusb_software.htm
http://github.com/mulicheng/fx2lib/tree/master
http://allmybrain.com/tag/fx2/
http://volodya-project.sourceforge.net/fx2_programmer.php
http://www.fpgaz.com/usbp/
http://www.triplespark.net/elec/periph/USB-FX2/software/

Ray Schumacher
R&D Manager - Cognitive Vision
8580 Production Ave. Ste. B, San Diego, CA 92121
rays@CognitiveVision.com
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel


Re: Asynchronous bulk transfer with QuickUSB FX2 (linux) ?

by azdes :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Using libusb 1.0, on Linux, with asyncrhonous bulk transfers on a PLX
USB chip, we were able to comfortably sustain between 25MBytes and
28MBytes per second. This was even without any ping ponging of buffers
(and writing out to a file on disk on the user side). You just have to
make sure your firmware side has the IN data ready to go and can turn
around quickly. And for that operation, you may want to keep other USB
devices off that particular bus.

Our PC side was running java through jni to a C++ dll that interacts
with libusb.

Good luck.


-----Original Message-----
From: Christophe Meessen [mailto:meessen@...]
Sent: Friday, October 09, 2009 4:28 AM
Cc: Libusb List
Subject: Re: [Libusb-devel] Asynchronous bulk transfer with QuickUSB FX2
(linux) ?

Hello,

thank you very much for all your answers. First a bit more information
on the device behind the usb. It is an X ray imaging prototype that we
are developing in our lab. It has to download 560x960 16/32bit pixels
images fast (~1MB/2MB data). It also has to upload some configurations,
but performance is less critical. The current data throughput we get on
Linux with the quick usb driver is about 3 MB/s and is too low.

The colleague who implemented it is from the hardware side. I'm from the
sofware side and I'm trying to figure out how I can get close to 30MB/s
data transfer, which seams a reasonable goal according to the USB 2.0
specifications.

I have to deal with the existing card and USB device. Looking at the FX2
data-sheet I have the impression it is a good controller. The choice was
driven by the rational to have a plug and play USB controller which is
true ... to some extend.

The current code (3MB/s) uses the bulk synchronous transfer mode. The
firmware was chosen for fastest data transfer rate and measured with the
oscilloscope to be 480Mb/s. The oscilloscope, also showed huge waiting
time between data sending which sounds logic with synchronous data
transfer and a driver in user space.

Now that libusb 1.0 support asynchronous transfer, I'm planning to give
it a try. But I have some probably noob questions to ask.

1. Marko Cebokli reports 1% loss of packets. I was told USB bulk
transfer is reliable and robust which means that data is transmitted
uncorrupted and without loss to endpoints. If the USB controller detects
corrupted packets it takes care to retransmit them. Is that true ?
There is enough buffer space in our fpga card, so buffer overflow should
not occur. Bus speed is much faster than USB speed.

2. Did anyone achieve to get 30MB/s throughput using libusb 1.0 or is
this too ambitious ?  Should I better consider writing a kernel driver
which, I was told, would have much less latency ?

3. Is libusb 1.0 still depending on usbfs kernel driver ? If yes, is
their a constrain on its version ? I'm using Ubuntu with the linux
kernel 2.6.28-15 on which libusb 0.1-4 is installed.

Here is what I see in the message log file when I plug the controller:

Oct  9 11:37:50 xxx kernel: [330902.700012] usb 2-6: new high speed USB
device using ehci_hcd and address 11 Oct  9 11:37:50 xxx kernel:
[330902.833697] usb 2-6: configuration #1 chosen from 1 choice

Here is the output of lsusb -v

Bus 002 Device 011: ID 0fbb:0001 Bitwise Systems, Inc.
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  idVendor           0x0fbb Bitwise Systems, Inc.
  idProduct          0x0001
  bcdDevice            2.11
  iManufacturer           1 Bitwise Systems
  iProduct                2 QuickUSB QUSB2 Module v2.11.10 (Simple I/O)
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           32
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    255 Vendor Specific Subclass
      bInterfaceProtocol    255 Vendor Specific Protocol
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x86  EP 6 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          255 Vendor Specific Class
  bDeviceSubClass       255 Vendor Specific Subclass
  bDeviceProtocol       255 Vendor Specific Protocol
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

The product Id indicates simpleIO which is the firmware currently
installed. This one allows to reach 480Mb/s sending speed. I see two
endpoints. Apparently one for input and the other for outpu, each one
configured with 512 byte (words?) buffer. In the DX2 datasheet it is
stated that there should be four 512 buffers for each end point. What
does the 1x512 mean ? Does it mean there is only one 512 buffer ?

4. I downloaded libusb 1.0.3 I would like to give it a try. Is there
anything I have to do before being trying to use it ? For instance do
something with udev so it somehow attaches usbfs driver to it ?

5. I wrote a piece of code compiled with libusb 0.1(pre-installed on
Ubuntu) that locates the  QuickUSB device and get its description
informations. Apparently it can't get the product identification string
unless it is run with root privilege (sudo). Same for lsusb. Is this the
normal behavior ?

Ch. Meessen










------------------------------------------------------------------------
------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and
stay ahead of the curve. Join us from November 9 - 12, 2009. Register
now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Asynchronous callbacks for multiple tranfers

by Marko Cebokli :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

I am playing now with async transfers, and have one question:

If, since the last call of usb_handle_events, several transfers were
completed,

does the next call to usb_handle_events call the callback function once for
every transfer that was completed in the mean time, or only once, so that I
must I call usb_handle_events several times to process multiple completed
tranfers?

Marko Cebokli

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous callbacks for multiple tranfers

by Marko Cebokli :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

ops, one more question...

is it OK to re-submit a transfer from within the calback function?

Marko Cebokli


On Saturday 10 October 2009 17:08:20 Marko Cebokli wrote:

> I am playing now with async transfers, and have one question:
>
> If, since the last call of usb_handle_events, several transfers were
> completed,
>
> does the next call to usb_handle_events call the callback function once for
> every transfer that was completed in the mean time, or only once, so that I
> must I call usb_handle_events several times to process multiple completed
> tranfers?
>
> Marko Cebokli
>
> ---------------------------------------------------------------------------
>--- Come build with us! The BlackBerry(R) Developer Conference in SF, CA is
> the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> Libusb-devel mailing list
> Libusb-devel@...
> https://lists.sourceforge.net/lists/listinfo/libusb-devel



------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous callbacks for multiple tranfers

by Hans Petter Selasky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

On Saturday 10 October 2009 17:21:00 Marko Cebokli wrote:
> is it OK to re-submit a transfer from within the calback function?

Typically yes, at least on FreeBSD. Except in case the transfer is cancelled
by your software, you should not re-submit the transfer.

>does the next call to usb_handle_events call the callback function once for
>every transfer that was completed in the mean time, or only once, so that I
>must I call usb_handle_events several times to process multiple completed
>tranfers?

That depends on the implementation. You should call the usb_handle_events()
function as long as the libusb_wait_for_event() function returns and don't
make any assumptions about how many callbacks you get per usb_handle_events()
call.

--HPS

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Parent Message unknown Re: Asynchronous callbacks for multiple tranfers

by Tim Roberts :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

You wrote:
>
>I am playing now with async transfers, and have one question:
>
>If, since the last call of usb_handle_events, several transfers were
>completed,
>does the next call to usb_handle_events call the callback function once
>for every transfer that was completed in the mean time, or only once,
>so that I must I call usb_handle_events several times to process
>multiple completed tranfers?

The great advantage of open source software is that you have the complete source code, so you can go look up the answers to questions like this on your own.
--
Tim Roberts, timr@...
Providenza & Boekelheide, Inc.

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel

Re: Asynchronous callbacks for multiple tranfers

by Daniel Drake :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Marko Cebokli wrote:
> I am playing now with async transfers, and have one question:
>
> If, since the last call of usb_handle_events, several transfers were
> completed,
>
> does the next call to usb_handle_events call the callback function once for
> every transfer that was completed in the mean time, or only once, so that I
> must I call usb_handle_events several times to process multiple completed
> tranfers?

Don't read too much into this.. just call it in a loop until the events
that you are anticipating have occurred.

And yes it is fine to resubmit transfers from the completion handler.

Daniel


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Libusb-devel mailing list
Libusb-devel@...
https://lists.sourceforge.net/lists/listinfo/libusb-devel