Memos From the Cube

Good news to all MyUSB fans! As of now, MyUSB is now retroactively released under the following permissive MIT license:

Permission to use, copy, modify, and distribute this software
and its documentation for any purpose and without fee is hereby
granted, provided that the above copyright notice appear in all
copies and that both that the copyright notice and this
permission notice and warranty disclaimer appear in supporting
documentation, and that the name of the author not be used in
advertising or publicity pertaining to distribution of the
software without specific, written prior permission.

The author disclaim all warranties with regard to this
software, including all implied warranties of merchantability
and fitness.  In no event shall the author be liable for any
special, indirect or consequential damages or any damages
whatsoever resulting from loss of use, data or profits, whether
in an action of contract, negligence or other tortious action,
arising out of or in connection with the use or performance of
this software.

Which is a great step forward to getting the library into more commercial (and non-commercial) products. Essentially, this allows myself to retain the copyright and credit for my work, but allows the library to be integrated into commercial products without source code disclosure. This change will be visible in future library releases, however the above license now applies to previously released versions (including but not limited to the current release) with the exception of the Keyboard device demo and demos derived from the keyboard device demo. This exception is due to the keyboard demo being contributed by Denver Gingerich over at OSSGuy rather than myself, thus I do not have the power to relicence his contributions.

This exception may also (depending on Denver’s response to my enquiry about relicencing his portions) extend to the next release, but more will be known later.

University’s back on again and I’m going to be as busy as ever. Still, I appreciate all the feedback and questions I get by email, so don’t hesistate to drop me a line. If you’re an avid MyUSB or ButtLoad fan and want to give back to the developer (*ahem*, me!), don’t forget that I accept donations of any amount on my site via PayPal. Such donations allow me to get new toys for development and allow me to buy shiny things  while I slave away at Univeristy, so a big thanks to all those who have donated so far!

Ok, that’s my begging spiel over. Now on to the next related topic - an AVR32 MyUSB port. One of the guys behind this interesting MyUSB powered project contacted me this week enquiring about a port of MyUSB to the AVR32 line of AVR microcontrollers from Atmel. After having a look at the datasheets - I’ve not had access to any AVR32s yet - I’ve determined that it *may* be possible to expand MyUSB to support the UC3B (not the UC3A) microcontrollers using the same (or near identical) API, since the former has a USB controller very similar to the 8-bit USB AVRs.

Before I begin work on such an expansion, I’d like an interest check. If you are interested in an AVR32 UC3B series port of MyUSB, please send me an email or leave a comment so I can assess whether it is worth the development effort or not.

Today was spent doing what seems to be my new favourite hobby - reinstalling Windows on my laptop once again. This time the reinstall was sparked by something screwing in the OS chewing threw mountains of memory for no discernable reason; a good half gigabyte chunk of RAM was totally MIA and unaccounted for. After having the Aero interface kill itself (out of memory) while trying to do nothing more than play a DVD on my system with 2GB RAM, I backed up my data and started the system reinstall.

This morning I was actually doing my second favourite hobby (neither this one nor the above I enjoy, for the record) - fixing bugs. An email from a MyUSB user indicated that the current Mass Storage demo fails to work, so off I went to investigate.

First, I narrowed down the release revision which showed the symptioms - 1.4.1 (good) to 1.5.0 (bad). No surpise there; the differences between the two are striking due to the addition of DoxyGen documentation comments, and many code changes to make the library more uniform and inline with the USB specification names and mandates. Next I did my second MyUSB bug-fixing trick, transplanting the working demo code onto the new library (fixing up any changes in the process to make it compatible). After that too failed to work, I was able to determine that the fault lay in the library code itself, rather than the demo application.

My JTAG showed that the code was failing to get out of a spinloop, waiting for the Mass Storage IN data endpoint to become ready to write to. Further probing showed this was due to the endpoint being permenantly stalled, preventing it from ever becomming data-ready. Honestly, I’m a FIRM beleiver that all AVR users need a JTAG - those who cannot see the utility have obviously not had access to one for any useful length of time. The units high cost is quickly offset by the hours saved debugging using more primative methods such as serial logging or (gasp) LED indicators.

What puzzled me was that the storage demo only stalled the data IN endpoint when the command sent from the host was unable to be processed. Not a rare occurrence; the demo only supports a limited (mandatory) subset of the SCSI command set, while the host tries to use the more obscure (but more efficient) commands where possible to speed up the transfers. However, the spinloop the code was getting stuck in was after the loop waiting for the host to un-stall the endpoint via a standard Clear Feature endpoint request.

That, coupled with my previous experiment telling me the fault lay in the library pointed to the problem — the Clear Feature standard request handler code was broken. A SVN diff showed the cause; between the two versions I had altered the code slightly, removing a critical endpoint mask from one of the parameters from the host. Adding it back in fixes the problem.

For those unable to wait for the next release, you can fix the code manually. In MyUSB/Drivers/USB/LowLevel/DevChapter9.c, the second line of SB_Device_ClearSetFeature() should be changed from:

uint16_t wIndex = Endpoint_Read_Word_LE();

Into:

uint16_t wIndex = (Endpoint_Read_Word_LE() & ENDPOINT_EPNUM_MASK);

To make the code work once again.

Yesterday I made up my semester 2 university timetable, and it’s a doozy! With five subjects each with homework, I can unhappily kiss most of my leisure time goodbye.

Good news!
After sloughing through my own TCP/IP stack for a week now, I’ve finally got some usable results. I’m now able to connect to the device’s simple shell through TELNET, and see a basic webpage when directing to the fixed IP address. It’s got a lot of problems to fix yet (haven’t added the code to create multiple TCP transactions when a large amount of data is sent at once, the webserver doesn’t terminate the connection) but it works damnit!

The code is somewhat simplified by the lack of a proper ARP system (at the moment). Because the USB AVR enumerates to the host as an actual, seperate network adapter, implementing a full ARP table would be pointless, as only the host computer is able to communicate with it. Still, despite the lack of a proper ARP, the code allows for multiple ports to be opened, and applications to consume and create TCP data as needed.

I have to say I’m quite proud of it, despite the god-awful code. I’m going to try to clean it up a bit first, then complete it and put it as a demo in the next MyUSB revision.

Good news! About a week ago, I managed to get an initial implementation of Microsoft’s RNDIS specification working as a MyUSB demo, so that the USB AVRs could be enumerated as a network adapter on modern OSes (yes, Linux has RNDIS support now).

That implementation made it into the 1.5.1 MyUSB release two days ago. However, the demo was limited; it could only send/receive raw Ethernet frames, it didn’t perform any processing on them as that was left up to the application engineer. It also failed to work on older Linux kernels with the original RNDIS implementation, due to a flaw in the Linux code — the notification endpoint was ignored in favour of polling the control endpoint, causing a lockup while the device waited for the notification endpoint to become empty again.

Both of those issues have now been resolved in the latest working code, with a big thanks to AVRFreaks member Colin O’Flynn for his advice and debugging help. The demo now works on the older RNDIS Linux code, and it can do something with the Ethernet frames!
Currently, I’ve written up protocol handlers for Ethernet, ARP, IP and ICMP, with TCP a work in progress. That’s the bare minimum to get a pingable device:

Which is damn cool in my opinion — if only because I’ve spent a whole week getting this far. Once I get TCP working, I’ll be able to write up a simple webserver, with the ultimate goal of being able to control the board LEDs using a web browser.

What’s the utility of all this? Think how much easier a device would be to configure if you could just plug it into a laptop, navigate to it with a web browser (on any platform) and change configuration settings using native widgets. No more complex serial terminals!

Stay tuned for updates.

Additional: For those with good eyesight and inquisitive minds - the 300ms+ ping time is due to the USBKEY printing out the protocol details of each packet through the serial port for debugging.

I’ve decided to release MyUSB 1.5.1 today. This new version is a very minor update, mainly fixing errors in the demos — in fact, there are no migration issues at all from the previous 1.5.0 version. Other small enhancements to the library itself (including one or two fixes to long standing bugs) are present in the new version, but these are invisible other than flash code size savings.

Without further ado, the changelog for the new version:

• Changed host demos to enable the host function task on the firing of the USB_DeviceEnumerationComplete event rather than the USB_DeviceAttached event

• HID Usage Stack now forcefully cleared after an IN/OUT/FEATURE item has been completely processed to remove any referenced but not created usages

• Changed USB_INT_DisableAllInterrupts() and USB_INT_ClearAllInterrupts(), USB_Host_GetNextDescriptorOfType(), USB_Host_GetNextDescriptorOfTypeBefore(), USB_Host_GetNextDescriptorOfTypeAfter() to normal functions (from inline)

• Fixed USBtoSerial demo not sending data, only receiving

• Fixed main makefile to make all by default, fixed MagStripe directory case to prevent case-sensitive path problems

• ConfigDescriptor functions made normal, instead of static inline

• Pipe/Endpoint *_Ignore_* functions changed to *_Discard_*, old names still present as aliases

• Fixed ENDPOINT_MAX_SIZE define to be correct on limited USB controller AVRs

• Changed endpoint and pipe size translation routines to use previous IF/ELSE IF cascade code, new algorithmic approach was buggy and caused problems

• Bootloaders now compile with -fno-inline-small-functions option to reduce code size

• Audio demos now use correct endpoint sizes for full and limited controller USB AVRs, double banking in all cases to be in line with the specification (isochronous endpoints MUST be double banked)

• Added Interface Association descriptor to StdDescriptors.h, based on the relevant USB2.0 ECN

• Fixed MIDI demo, corrected Audio Streaming descriptor to follow the MIDI-specific AS structure

• Fixed HID class demo descriptors so that the HID interface’s protocol is 0×00 (required for non-boot protocol HID devices) to prevent problems on hosts expecting the boot protocol functions to be supported

• Added read/write control stream functions to Endpoint.h

• Fixed AudioOut demo not setting port pins to inputs on USB disconnect properly

• Added RNDISEthernet demo application

As always, email me if you have any suggestions or bug reports. Download the new library code from the project page. Also on that page is the latest library documentation, in downloadable or online HTML form.

A few days work, and what do I have to show for it? Only this:

Why is it showing as unplugged? I’ve no idea — I’m going to test it out on Ubuntu and use the dmesg tool to see if I can discover what’s going on. Definite progress has been made however, as there’s quite a bit going on behind the scenes. I can’t wait to get all the RNDIS wrapping completed so I can then start attacking the Ethernet frames. My goal for the moment is a pingable device (hard enough in itself) but eventually I want to have a micro HTTP server running on this thing. Lots of layers between me and that goal - Ethernet, TCP, IP, HTTP - but by taking it one step at a time it should be perfectly possible for myself to manage.

To some more good news; I’m very interested in the creations over at www.digitalsurveyinstruments.com, not because I want to become a geologist or surveyer, but because the devices are running - you guessed it! - MyUSB. I’ve had a ball talking to the engineer behind the products, discussing various ways to acomplish certain tasks, and ranting when it turns out that Windows doesn’t support what we want.

UPDATE: Found the source of the “Network Cable Unplugged” problem: RNDIS_STATUS_MEDIA_CONNECT isn’t the same as RNDIS_MEDIA_STATE_CONNECTED. It’s Ethernet time, baby!

Today I received a rather nice email from fellow AVRFreaks member Collin O’Flynn. In part:

Hi Dean,

I ended up getting RNDIS support to work on the Atmel USB chipset.

I used the Atmel application note as a base, due to the fact that you need a
lot of other crap for this to be useful. Other crap means TCP/IP stack of
course… and the Atmel application note was a bit easier to integrate
beacuse of this.

But I’m trying to get the code to interface to Linux now, as Linux does have
RNDIS support and should have working IAD support….

That’s rather neat - it means that a working RNDIS implementation is possible on the USB AVRs. I’ve been ITCHING for something to do the last two days, so I’m jumping in and cooking up my own implementation. Everything is from scratch — I’ve re-downloaded the RNDIS specification once again and started coding, basing the demo on the existing CDC MyUSB demo. Hopefully I’ll be able to make a working implementation on top of MyUSB without the use of any RTOS or dynamic memory.

I know I originally discounted the idea of Ethernet over USB due to the three competing standards (two of which are USB-IF certified, and once of which - RNDIS - is a Microsoft abomination) however since it appears that Linux supports it I can’t see the harm. In any case, it’s a sad fact that the majority of systems run Windows. And let’s face it, it’s many magnitudes more likely that Linux will implement RNDIS (quckly!) than it is that Microsoft will implement one of the true CDC Ethernet standards.

So far I’ve got my implementation to enumerate and show up in the Network Connections screen of my laptop, although the interface is currently dead (disabled permenantly) until I implement all the mandatory control commands and start sending some data.

I’m also excited to learn that the first 1000 board run of Donald Davis’ “Benito” Arduino programmer boards is going ahead. Excited because it’s the first large scale production of a device (to my knowledge) running MyUSB! If your project runs on MyUSB, please don’t forget to send me an email!

Right, soon 1.5.1 will be released - found a few more bugs which needed to be quashed, especially the faulty endpoint/pipe size to mask translation routine. One small new thing that will make it into the revision release will be the addition of the Interface Association standard descriptor to StdDescriptors.h.

The IAD is something I stumbled onto by chance while trying to find a link to a thread discussing getting CDC + anything to run as a compound device under Windows. In Windows, devices implementing several classes - called Compound devices - have each interface enumerated separately will different drivers. For most devices that isn’t a problem, but in the case of the CDC class it is disasterous, as the CDC class uses two separate interfaces for the one function.

Having a basic CDC class device is fine under Windows, as Windows then passes the entire device over to the usbser.sys driver file and all is well. However, under compound device mode, the two CDC interfaces are treated as separate functions and the result is a non-functioning CDC. People have posted Windows-specific workarounds based on the shoddyness of the drivers to get it to work, however that breaks functionality on other OSes.

While trying to find the thread which explained all that for the person enquiring about it, I found a mention of the “Interface Association Descriptor”, something I’d missed until now. It turns out that the IAD was a supplement to the USB2.0 standard and was posted as an Engineering Change Notice (ECN) after the specification, meaning that unless I knew to look for it, I wouldn’t have found it.

The IAD is designed to address exactly the above problem. It allows a compound device to specify which interfaces should be grouped and enumerated together under the one driver - allowing the two CDC interfaces to be linked, for example. Damn useful, with only two drawbacks:

1) It is only supported in Windows XP SP2 or higher, and in newer Linux releases
2) It requires the device to use a special class indicating that it is a IAD supporting device, meaning non supporting OSes will fail to enumerate the device completely rather than just some of the functions

However, it will be very useful in the future for all new devices, especially those for Vista and beyond. I’m really glad to have a real solution to the problem, so I’m adding in the descriptor to MyUSB.

You can read the IAD document here.

In case anyone is wondering, now is mid-year University exam time, which is the main reason for my lack of updates here. Actually, truth be told, discovering the (modern) Battlestar Galactica series has been the main contributor towards my laziness.

After tomorrow I’ll be free from Univeristy for a whole month. That’s great, but I’m going into my usual axiety spiral; not knowing what exactly to do any one one moment, causing me to do nothing and stress about wasting my free time. If my new contract job eventually gets of the ground I’ll have both a source of income (finally - ramen noodles, sleeping on burlap sacks and begging are tiresome!) and a sink for my creative *cough* tallent.

Soon I think I’ll release MyUSB 1.5.1, a very minor update fixing a couple of bugs in the demos and other non-critical aspects of the library like typos and the like. The library is for all intents and purposes done, so barring an ephiphany all updates in the near future will be minor only - no more frenzied development of new APIs.

It’s probably about time I got back to writing my USB book, which I’ve neglected. Hang on, I think I hear the next Battlestar Galactica episode calling me…

MyUSB 1.5.0 has been released onto the project page, including the new documentation. I completed the migration information I posted about a few hours ago - it is now a page in the online and downloadable documentation. Please notify me of any problems found with the new version.
Previous 1.5.0 BETAs have been removed in the wake of the new release and are no longer available for download. The full changelog for the new version is:

• Fixed MIDI demo, now correctly waits for the endpoint to be ready between multiple note messages
• Added CDC Host demo application
• Added KeyboardFullInt demo application
• Endpoint and Pipe creation routines now mask endpoint/pipe size with the size mask, to remove transaction size bits not required for the routines (improves compatibility with devices)
• Fixed AudioInput demo - now correctly sends sampled audio to the host PC
• Fixed AudioOutput demo oncemore — apparently Windows requires endpoint packets to be >=192 bytes
• Shrunk round-robbin scheduler code slightly via the use of struct pointers rather than array indexes
• Fixed off-by-one error when determining if the Usage Stack is full inside the HID Report parser
• Renamed Magstripe.h to MagstripeHW.h and moved driver out of the library and into the MagStripe demo folder
• Added preprocessor checks to enable C linkage on the library components when used with a C++ compiler
• Added Still Image Host demo application
• The USB device task now restores the previously selected endpoint, allowing control requests to be transparently handled via interrupts while other endpoints are serviced through polling
• Fixed device signature being sent in reverse order in the CDC bootloader
• Host demos now have a seperate ConfigDescriptor.c/.h file for configuration descriptor processing
• HostWithParser demos now have a seperate HIDReport.c/.h file for HID report processing and dumping
• Removed non-mandatory commands from MassStorage demo to save space, fixed SENSE ResponseCode value
• CDC demos now send empty packets after sending a full one to prevent buffering issues on the host
• Updated demo descriptors to use VID/PID values donated by Atmel
• Added DoxyGen documentation to the source files
• Fixed Serial_IsCharRecieved() definition, was previously reversed
• Removed seperate USB_Descriptor_Language_t descriptor, USB_Descriptor_String_t is used instead
• Removed unused Device Qualifier descriptor structure
• Renamed the USB_CreateEndpoints event to the more appropriate USB_ConfigurationChanged
• Fixed MassStorageHost demo reading in the block data in reverse
• Removed outdated typedefs in StdRequestType.h, superceeded by the macro masks
• Corrected OTG.h is now included when the AVR supports both Host and Device modes, for creating OTG products
• USB_DeviceEnumerationComplete event is now also fired when in device mode and the host has finished its enumeration
• Interrupt driven demos now properly restore previously selected endpoint when ISR is complete
• USB_HOST_TIMEOUT_MS is now overridable in the user project makefile to a custom fixed timeout value
• Renamed USB_Host_SOFGeneration_* macros to more friendly USB_Host_SuspendBus(), USB_Host_ResumeBus() and USB_Host_IsBusSuspended()
• Renamed *_*_Is* macros to *_Is* to make all flag checking macros consistant, Pipe_SetInterruptFreq() is now Pipe_SetInterruptPeriod() to use the correct terminology
• UnicodeString member of USB_Descriptor_String_t struct changed to an ordinary int array type, so that the GCC Unicode strings (prefixed with an L before the opening quotation mark) can be used instead of explicit arrays of ASCII characters
• Fixed Endpoint/Pipes being configured incorrectly if the maximum endpoint/pipe size for the selected USB AVR model was given as the bank size
• HID device demos now use a true raw array for the HID report descriptor rather than a struct wrapped array
• Added VERSION_BCD() macro, fixed reported HID and USB version numbers in demo descriptors
• Cleaned up GetDescriptor device chapter 9 handler function
• Added GET_REPORT class specific request to HID demos to make them complaint to the HID class
• Cleaned up setting of USB_IsInitialized and USB_IsConnected values to only when needed
• Removed Atomic.c and ISRMacro.h; the library was already only compatible with recent avr-lib-c for other reasons
• All demos and library functions now use USB standardized names for the USB data (bRequest, wLength, etc.)
• Added USE_NONSTANDARD_DESCRIPTOR_NAMES token to switch back to the non-standard descriptor element names