Memos From the Cube

This month’s “bizzare search terms which found this site” award has to go to whoever searched for the words “hack the gibson monkey” and found my site. Yes, I did mention all those terms on my blog at some point, but I’m still amazed that someone would be looking for websites about that…

MyUSB 1.5.2 has been released. This new minor release is *very* minor, only correcting one big mistake in the device chapter 9 handling code and enhancing the RNDIS demo to have a working (barely) TELNET and webserver for users to try out. Keep in mind that my TCP/IP implmentation isn’t the greatest — I wrote it as I learned about each layer — and so for proper projects it should be ripped out and replaced with a proper stack, such as Adam Dunkel’s uIP TCP/IP stack for embedded systems.

The real reason to release the new version with so little changed is because of the new licence. Denver Gingerich has agreed to relicence his portions of the library under the same new MIT license, so the latest release reflects the changes to the new open source license. I hope the change will appease the masses, including those shifty Germans who were suggesting that others use earlier library releases where the LGPL licence wasn’t strictly enforcable due to my failure to state the license restructions in every source file.

The changelog for the new version as as stated very minor, but reproduced for completeness:

• Fixed SwapEndian_32() function in Common.h so that it now works correctly (wrong parameter types)

• Updated RNDIS demo - notification endpoint is no longer blocking so that it works with faulty Linux RNDIS implementations (where the notification endpoint is ignored in favour of polling the control endpoint)

• Fixed incorrect Vendor Description string return size in RNDIS demo for the OID_GEN_VENDOR_DESCRIPTION OID token

• Added very basic TCP/IP stack and HTTP/TELNET servers to RNDIS demo

• Fixed DFU bootloader exit causing programming software to complain about failed writes

• Fixed DFU bootloader EEPROM programming mode wiping first flash page

• Fixed Clear/Set Feature device standard request processing code (fixing MassStorage demo in the process)

• Added support for the ATMEGA16U4 AVR microcontroller

• Library licence changed from LGPLv3 to MIT license

Notice there’s no migration information in the documentation, the reason for which is twofold:

1. I forgot to add it in before uploading the files

1. There isn’t any for this release anyway

So that should cement the devision to upgrade, as existing code will work *exactly* as it is with no changes whatsoever. That means no excuses for using older versions!

Also added in the new library as shown in the changelog is support for Atmel’s just-announced ATMEGA16U4 microcontroller. I only saw the new datasheet last night, but due to it’s similarity to the ATMEGA32U4 the changes required to implement it were extreamely minor. With any luck, someone will be able to test out MyUSB on either/both models when the silicon eventually starts shipping later this year.

Tomorrow I’ll be posting about my new magnetometer toy I received, powered by MyUSB…

Good news to all MyUSB fans! As of now, MyUSB is now retroactively released under the following permissive MIT license:

Permission to use, copy, modify, and distribute this software
and its documentation for any purpose and without fee is hereby
granted, provided that the above copyright notice appear in all
copies and that both that the copyright notice and this
permission notice and warranty disclaimer appear in supporting
documentation, and that the name of the author not be used in
advertising or publicity pertaining to distribution of the
software without specific, written prior permission.

The author disclaim all warranties with regard to this
software, including all implied warranties of merchantability
and fitness.  In no event shall the author be liable for any
special, indirect or consequential damages or any damages
whatsoever resulting from loss of use, data or profits, whether
in an action of contract, negligence or other tortious action,
arising out of or in connection with the use or performance of
this software.

Which is a great step forward to getting the library into more commercial (and non-commercial) products. Essentially, this allows myself to retain the copyright and credit for my work, but allows the library to be integrated into commercial products without source code disclosure. This change will be visible in future library releases, however the above license now applies to previously released versions (including but not limited to the current release) with the exception of the Keyboard device demo and demos derived from the keyboard device demo. This exception is due to the keyboard demo being contributed by Denver Gingerich over at OSSGuy rather than myself, thus I do not have the power to relicence his contributions.

This exception may also (depending on Denver’s response to my enquiry about relicencing his portions) extend to the next release, but more will be known later.

University’s back on again and I’m going to be as busy as ever. Still, I appreciate all the feedback and questions I get by email, so don’t hesistate to drop me a line. If you’re an avid MyUSB or ButtLoad fan and want to give back to the developer (*ahem*, me!), don’t forget that I accept donations of any amount on my site via PayPal. Such donations allow me to get new toys for development and allow me to buy shiny things  while I slave away at Univeristy, so a big thanks to all those who have donated so far!

Ok, that’s my begging spiel over. Now on to the next related topic - an AVR32 MyUSB port. One of the guys behind this interesting MyUSB powered project contacted me this week enquiring about a port of MyUSB to the AVR32 line of AVR microcontrollers from Atmel. After having a look at the datasheets - I’ve not had access to any AVR32s yet - I’ve determined that it *may* be possible to expand MyUSB to support the UC3B (not the UC3A) microcontrollers using the same (or near identical) API, since the former has a USB controller very similar to the 8-bit USB AVRs.

Before I begin work on such an expansion, I’d like an interest check. If you are interested in an AVR32 UC3B series port of MyUSB, please send me an email or leave a comment so I can assess whether it is worth the development effort or not.

Today was spent doing what seems to be my new favourite hobby - reinstalling Windows on my laptop once again. This time the reinstall was sparked by something screwing in the OS chewing threw mountains of memory for no discernable reason; a good half gigabyte chunk of RAM was totally MIA and unaccounted for. After having the Aero interface kill itself (out of memory) while trying to do nothing more than play a DVD on my system with 2GB RAM, I backed up my data and started the system reinstall.

This morning I was actually doing my second favourite hobby (neither this one nor the above I enjoy, for the record) - fixing bugs. An email from a MyUSB user indicated that the current Mass Storage demo fails to work, so off I went to investigate.

First, I narrowed down the release revision which showed the symptioms - 1.4.1 (good) to 1.5.0 (bad). No surpise there; the differences between the two are striking due to the addition of DoxyGen documentation comments, and many code changes to make the library more uniform and inline with the USB specification names and mandates. Next I did my second MyUSB bug-fixing trick, transplanting the working demo code onto the new library (fixing up any changes in the process to make it compatible). After that too failed to work, I was able to determine that the fault lay in the library code itself, rather than the demo application.

My JTAG showed that the code was failing to get out of a spinloop, waiting for the Mass Storage IN data endpoint to become ready to write to. Further probing showed this was due to the endpoint being permenantly stalled, preventing it from ever becomming data-ready. Honestly, I’m a FIRM beleiver that all AVR users need a JTAG - those who cannot see the utility have obviously not had access to one for any useful length of time. The units high cost is quickly offset by the hours saved debugging using more primative methods such as serial logging or (gasp) LED indicators.

What puzzled me was that the storage demo only stalled the data IN endpoint when the command sent from the host was unable to be processed. Not a rare occurrence; the demo only supports a limited (mandatory) subset of the SCSI command set, while the host tries to use the more obscure (but more efficient) commands where possible to speed up the transfers. However, the spinloop the code was getting stuck in was after the loop waiting for the host to un-stall the endpoint via a standard Clear Feature endpoint request.

That, coupled with my previous experiment telling me the fault lay in the library pointed to the problem — the Clear Feature standard request handler code was broken. A SVN diff showed the cause; between the two versions I had altered the code slightly, removing a critical endpoint mask from one of the parameters from the host. Adding it back in fixes the problem.

For those unable to wait for the next release, you can fix the code manually. In MyUSB/Drivers/USB/LowLevel/DevChapter9.c, the second line of SB_Device_ClearSetFeature() should be changed from:

uint16_t wIndex = Endpoint_Read_Word_LE();

Into:

uint16_t wIndex = (Endpoint_Read_Word_LE() & ENDPOINT_EPNUM_MASK);

To make the code work once again.

Yesterday I made up my semester 2 university timetable, and it’s a doozy! With five subjects each with homework, I can unhappily kiss most of my leisure time goodbye.

Good news!
After sloughing through my own TCP/IP stack for a week now, I’ve finally got some usable results. I’m now able to connect to the device’s simple shell through TELNET, and see a basic webpage when directing to the fixed IP address. It’s got a lot of problems to fix yet (haven’t added the code to create multiple TCP transactions when a large amount of data is sent at once, the webserver doesn’t terminate the connection) but it works damnit!

The code is somewhat simplified by the lack of a proper ARP system (at the moment). Because the USB AVR enumerates to the host as an actual, seperate network adapter, implementing a full ARP table would be pointless, as only the host computer is able to communicate with it. Still, despite the lack of a proper ARP, the code allows for multiple ports to be opened, and applications to consume and create TCP data as needed.

I have to say I’m quite proud of it, despite the god-awful code. I’m going to try to clean it up a bit first, then complete it and put it as a demo in the next MyUSB revision.

Good news! About a week ago, I managed to get an initial implementation of Microsoft’s RNDIS specification working as a MyUSB demo, so that the USB AVRs could be enumerated as a network adapter on modern OSes (yes, Linux has RNDIS support now).

That implementation made it into the 1.5.1 MyUSB release two days ago. However, the demo was limited; it could only send/receive raw Ethernet frames, it didn’t perform any processing on them as that was left up to the application engineer. It also failed to work on older Linux kernels with the original RNDIS implementation, due to a flaw in the Linux code — the notification endpoint was ignored in favour of polling the control endpoint, causing a lockup while the device waited for the notification endpoint to become empty again.

Both of those issues have now been resolved in the latest working code, with a big thanks to AVRFreaks member Colin O’Flynn for his advice and debugging help. The demo now works on the older RNDIS Linux code, and it can do something with the Ethernet frames!
Currently, I’ve written up protocol handlers for Ethernet, ARP, IP and ICMP, with TCP a work in progress. That’s the bare minimum to get a pingable device:

Which is damn cool in my opinion — if only because I’ve spent a whole week getting this far. Once I get TCP working, I’ll be able to write up a simple webserver, with the ultimate goal of being able to control the board LEDs using a web browser.

What’s the utility of all this? Think how much easier a device would be to configure if you could just plug it into a laptop, navigate to it with a web browser (on any platform) and change configuration settings using native widgets. No more complex serial terminals!

Stay tuned for updates.

Additional: For those with good eyesight and inquisitive minds - the 300ms+ ping time is due to the USBKEY printing out the protocol details of each packet through the serial port for debugging.

I’ve decided to release MyUSB 1.5.1 today. This new version is a very minor update, mainly fixing errors in the demos — in fact, there are no migration issues at all from the previous 1.5.0 version. Other small enhancements to the library itself (including one or two fixes to long standing bugs) are present in the new version, but these are invisible other than flash code size savings.

Without further ado, the changelog for the new version:

• Changed host demos to enable the host function task on the firing of the USB_DeviceEnumerationComplete event rather than the USB_DeviceAttached event

• HID Usage Stack now forcefully cleared after an IN/OUT/FEATURE item has been completely processed to remove any referenced but not created usages

• Changed USB_INT_DisableAllInterrupts() and USB_INT_ClearAllInterrupts(), USB_Host_GetNextDescriptorOfType(), USB_Host_GetNextDescriptorOfTypeBefore(), USB_Host_GetNextDescriptorOfTypeAfter() to normal functions (from inline)

• Fixed USBtoSerial demo not sending data, only receiving

• Fixed main makefile to make all by default, fixed MagStripe directory case to prevent case-sensitive path problems

• ConfigDescriptor functions made normal, instead of static inline

• Pipe/Endpoint *_Ignore_* functions changed to *_Discard_*, old names still present as aliases

• Fixed ENDPOINT_MAX_SIZE define to be correct on limited USB controller AVRs

• Changed endpoint and pipe size translation routines to use previous IF/ELSE IF cascade code, new algorithmic approach was buggy and caused problems

• Bootloaders now compile with -fno-inline-small-functions option to reduce code size

• Audio demos now use correct endpoint sizes for full and limited controller USB AVRs, double banking in all cases to be in line with the specification (isochronous endpoints MUST be double banked)

• Added Interface Association descriptor to StdDescriptors.h, based on the relevant USB2.0 ECN

• Fixed MIDI demo, corrected Audio Streaming descriptor to follow the MIDI-specific AS structure

• Fixed HID class demo descriptors so that the HID interface’s protocol is 0×00 (required for non-boot protocol HID devices) to prevent problems on hosts expecting the boot protocol functions to be supported

• Added read/write control stream functions to Endpoint.h

• Fixed AudioOut demo not setting port pins to inputs on USB disconnect properly

• Added RNDISEthernet demo application

As always, email me if you have any suggestions or bug reports. Download the new library code from the project page. Also on that page is the latest library documentation, in downloadable or online HTML form.

A few days work, and what do I have to show for it? Only this:

Why is it showing as unplugged? I’ve no idea — I’m going to test it out on Ubuntu and use the dmesg tool to see if I can discover what’s going on. Definite progress has been made however, as there’s quite a bit going on behind the scenes. I can’t wait to get all the RNDIS wrapping completed so I can then start attacking the Ethernet frames. My goal for the moment is a pingable device (hard enough in itself) but eventually I want to have a micro HTTP server running on this thing. Lots of layers between me and that goal - Ethernet, TCP, IP, HTTP - but by taking it one step at a time it should be perfectly possible for myself to manage.

To some more good news; I’m very interested in the creations over at www.digitalsurveyinstruments.com, not because I want to become a geologist or surveyer, but because the devices are running - you guessed it! - MyUSB. I’ve had a ball talking to the engineer behind the products, discussing various ways to acomplish certain tasks, and ranting when it turns out that Windows doesn’t support what we want.

UPDATE: Found the source of the “Network Cable Unplugged” problem: RNDIS_STATUS_MEDIA_CONNECT isn’t the same as RNDIS_MEDIA_STATE_CONNECTED. It’s Ethernet time, baby!

Today I received a rather nice email from fellow AVRFreaks member Collin O’Flynn. In part:

Hi Dean,

I ended up getting RNDIS support to work on the Atmel USB chipset.

I used the Atmel application note as a base, due to the fact that you need a
lot of other crap for this to be useful. Other crap means TCP/IP stack of
course… and the Atmel application note was a bit easier to integrate
beacuse of this.

But I’m trying to get the code to interface to Linux now, as Linux does have
RNDIS support and should have working IAD support….

That’s rather neat - it means that a working RNDIS implementation is possible on the USB AVRs. I’ve been ITCHING for something to do the last two days, so I’m jumping in and cooking up my own implementation. Everything is from scratch — I’ve re-downloaded the RNDIS specification once again and started coding, basing the demo on the existing CDC MyUSB demo. Hopefully I’ll be able to make a working implementation on top of MyUSB without the use of any RTOS or dynamic memory.

I know I originally discounted the idea of Ethernet over USB due to the three competing standards (two of which are USB-IF certified, and once of which - RNDIS - is a Microsoft abomination) however since it appears that Linux supports it I can’t see the harm. In any case, it’s a sad fact that the majority of systems run Windows. And let’s face it, it’s many magnitudes more likely that Linux will implement RNDIS (quckly!) than it is that Microsoft will implement one of the true CDC Ethernet standards.

So far I’ve got my implementation to enumerate and show up in the Network Connections screen of my laptop, although the interface is currently dead (disabled permenantly) until I implement all the mandatory control commands and start sending some data.

I’m also excited to learn that the first 1000 board run of Donald Davis’ “Benito” Arduino programmer boards is going ahead. Excited because it’s the first large scale production of a device (to my knowledge) running MyUSB! If your project runs on MyUSB, please don’t forget to send me an email!