Memos From the Cube

Happy new year everyone!

Now is supposed to be my free time of year, in my summer break from University. A time to kick back, relax, and code up until 3AM on a Wednesday night. That’s not been the case - I’ve actually been quite busy! These last few months since University finished for the year, I’ve been busy working at a neat place called the Center For Health Innovation (thats the current website, my current assignment is to upgrade it) which has led to some nice ongoing projects. I got to say, it’s been darn nice seeing my bank balance go up regularly instead of down! Working in a real environment doing real projects has proven to be quite rewarding (and educational), much more so than my original failed plan to get a job in retail for a few months during my holidays.

On that note, thank you all who have donated. I’ve been astonished at the level of generosity of people visiting my site and using my work.

I never expected such a response to my last post! There’s been more discussion on that unrelated rant, than on any of my many previous posts about my development milestones. For the record, I do not hate Linux in general, just the level of support on my modern hardware (a Toshiba Satellite A210, for those asking, with ATI graphics, Dual 1.9GHz AMD Turion CPU, 2GB RAM). That’s not all the Linux developers fault; my point is that it is unsuitable for general use by the general non-technical populous until driver support improves. Whether than means yelling at the hardware vendors or not is up for debate, but the summary would be:

• User expects OS to work on hardware
• Windows works on hardware
• Linux does not work on hardware
• User concludes Linux is bunk

I’ve actually gone ahead and ordered a the latest Ubuntu CD from the site, in the hopes that the last release fixed the issues for me in 7.10. I’ll repost when that arrives and I have time to give the new version a go.

On another unrelated note, the past weeks have seen a few neat new gizmos arrive in on my doorstep. Over from AVROpendous.org I received the latest incarnation of the board, the AVROpendous Mini (photo stolen from the AVROpendous page):

The new board is much smaller than the old, forgoing the previous SIL header strip - allowing it to be inserted vertically into a breadboard - for tiny female DIL headers of the same size as the USBKEY headers. Unlike the USBKEY however the AVROpendous Mini has the benefit of actually having said female headers mounted, removing the head-scratching problem of how to wire the darn thing up to anything. Standard solid-core breadboard wire will fit into the headers for easy prototyping to an existing breadboard. In case that’s not your cup of tea, there’s also a new adapter board which the Mini plugs into, to make it of the same form factor (save width) as the original AVROpendous, and an adapter to make the Mini easily wireable to a HD44780 based Alphanumeric LCD display.

Second, I’ve recieved some hardware from Denver Gingerich, over as OSSGuy, shown below after being Macgyver’d  into an old box of mine by myself to protect it:

Sporting a three-track magentic card reader and a custom board based on the AT90USB1287, my new firmware for it allows it to act as a USB keyboard and type the card data to the host. Denver’s new host-side software will be able to convert this read data into (hopefully) something meaningful, but for now I’m considering making it into an esoteric security system based on train tickets, which come with magnetic stripes here in Australia.

Much work to do - Dean out.

Wordpress is updated, and nothing’s blown up. Whee!

This evening I did something I haven’t done for quite a while, which might come as a shock to some egotists: I searched for references to myself and my past/present projects through Google. It’s quite an interesting trip down memory lane seeing how my contributions (I use that term very loosly) have spread out all over the web. More interesting is some of the more recent entries.

I am outraged to find that a guy has transformed my beautiful Simple Simon code into an abomination. It’s been ported for PICs. Good lord, strap on your helmets, because this world is going to hell in a handbasket. The Simple Simon code I wrote was actually very basic, and served to teach myself PWM and state machines — you can see the original, non PIC (bleah!) source here.

No no, I jest. I’m actually rather flattered that someone thought such simple code with such a simple purpose was worth porting to anything, and I’m a little saddend that there are no real-life pictures of the guy’s quite nice design, only drawings.

Also in my travells I found this blog entry, written by a guy experimenting with his own home-brew AVR USB boards and my LUFA (formerly MyUSB) library. According to that poor deluded fellow I’m a hero - a title shared with Smokey/Smileymicros/Joe Pardue at AVRFreaks - which is heartwarming to hear, if obviously untrue. Still, neat, and I enjoyed the semi-heart-attack I recieved from seeing delicate microcontrollers running from a breadboard sitting in a nice static-friendly plastic box.

It’s nice to see my name on the avr-libc contributors list, even for such small patches.  A while back I wrote the new ISR and Atomic block header files for avr-libc, which have been in the last few distributions. I actually think I made a mistake by making the new ISR macro backwards compatible with the old — if you haven’t seen the new functionality the extended macro offers, you really should read the header file’s documentation in the manual. Oh, and read the atomic block documentation as well, as it provides an ellegant solution to the age-long problem of atomic access.

Finally, on an unrelated topic - LUFA (formerly MyUSB) is now in Release Candidate stage - you’ve got a whole two days to alert me to any showstoppers before the next release. Essentially the code changes for the next release have been mostly wrapped up for a while, but the release has been delayed by me wanting to finish documenting each individual demo. Nuts to that I say, as the new changes are worth getting out there now, so I’ll be releasing the next version (now versioned by release date, rather than an arbitrarily increasing number) in the next few days with all the projects and demos documented, but only the device mode demos (minus the RNDIS demo) documented.

Don’t delay - grab a copy of the new version’s RC and get testing!

I have a cold. In summer. Go figure.

Yesterday I received yet another set of AT90USBXXX based boards for my prototyping and experimenting pleasure, this one called the Teensy. Rather than try to coax my cheap digital camera to take a psudo-macro shot, I’ll just steal the official image instead:

Essentially, the Teensy board (made by Paul) is similar to the AVROpendous board covered previously, except in a smaller DIP form factor reminiscent of the BASIC Stamp of yore. Much better that the latter in fact, as it’s AVR powered and doesn’t have an astronomical price, only an equally teensy US$19 - on par with the AVROpendous’ ~US$20 price tag. At this point, the two boards really only differ in physical form factor (DIP vs. SIL), with the Teensy sporting an onboard LED while the AVROpendous sports a good Wiki knowledgebase.

One area where the Teensy differs is in the bootloader and reset configurations; while the AVROpendous current uses two separate pushbuttons for the HWB and RST functionalities (allowing the user to perform either a full reset, or a reset-to-bootloader), the Teensy uses a single pushbutton. When pressed, the tiny button performs the reset-to-bootloader sequence - idiot proof, but less flexible than the AVROpendous.

The interesting reset system has an intentional side effect - the bootloader can be written to assume a completely reset AVR state. This has led to the Teensy’s proprietary “HalfKay” bootloader, a (as the name implies) half-kilobyte bootloader based on a proprietary communication protocol that runs over a standard HID device class connection. With the AT90USB162 only sporting 16KB of FLASH memory, this extra space savings can make all the difference.

This is one area where the Teensy could open up a bit — it would be nice to see the communication protocol published so that others can create compatible bootloaders and host-side software. As it stands however, its a neat trick, one performed using (shudder) raw assembly.

By contrast, the AVROpendous uses either the stock Atmel bootloader, or the LUFA (formerly MyUSB) DFU/CDC class bootloaders. The LUFA bootloaders are much bigger - 4KB a piece - however may be jumped to from the user application (with the processor in any state) and use the LUFA library as a backend, rather than a smaller set of custom USB code.

But all’s not standing still in the AVROpendous camp - there’s a new design in town. The new smaller design uses onboard female connectors in lieu of the original SIL configuration, so that the board may be jumpered into circuits via wires, rather than “plugged in”. The new design seems to be on sale as an alternative to the old, so that both designs will remain for sale and open hardware/software.

The good news about all this is of course the fact that USB AVRs are *finally* making it into the hands of hobbyists. The USBKEY is a great (if expensive) board, but remained (remains?) largely ignored by most it seems, due to the unpopular non-standard sized headers and complexity of the USB bus. With LUFA and one of the growing number of cheap USB board designs, perhaps more developers will be encouraged to drop their USARTS, throw away their USB-to-Serial converters and go with native USB.

Yesterday I proved to myself that I am at heart a geek; I found my Netbank’s “export” option, and used it to create graphs of my balance (delta and absolutes) over the past year or so since I got the account. The geeky maxim holds true once again: “If it can be represented on a computer, it can and must be spreadsheeted”.

More improvements are coming to the RNDIS demo of MyUSB. I still recommend against using my network stack in anything remotely important, as it’s more of a learning experience for me than a full blown, feature complete, zero-bug stack — after all, MyUSB is a *USB* library, not a TCP/IP stack. That said, I’ve fixed a lot of its problems, with only multiple simultaneous connections from the same computer on the same port still causing problems. I’m currently adding in UDP and DHCP protocol support, so that the next iteration of the demo won’t require any manual configuration of the adapter’s IP address to get it working - you’ll just plug it in and give it the driver INF file, and it will do the rest.

I’m also in a discussion with a few prominant AVR-USB people (Matt from AVROpendous, Paul from PJRC - rundown of Paul’s new “Teensy” board to come soon - and Donald from DorkbotPDX) about the possibility of future MyUSB versions to ship with simplified Wiriring interfaces for easy (if basic) control of the USB interface with no advanced programming knowedge required. Such an interface would be optional - i.e. the existing framework will remain identical - but would allow for complete newbies to get up and running with Virtual Serial ports for debugging and the like in only a couple of minutes. So far, so good, and once the details of the implementation are fleshed out I’ll start work on it.

Such an interface will not be present in the next 1.5.4 release of MyUSB, but will most likely exist in some form in the version following that. The release of 1.5.4 will be a new milestone, as it will reflect a mature framework with mature examples and much better documentation. It’ll also sport a new name: LUFA, the Lightweight USB Framework for AVRs. This name will come into use with the release of 1.5.4, and all documentation and site pages will be updated to reflect the name change.

A quick update, so that people actually notice it; I’ve created a new MyUSB suggestion box Wiki. Feel free to post your development suggestions there — anything (related) is fine, so go nuts! I’ll tag suggestions once they’re implemented, or I’ve read and decided against them (I’ll give the reason alongside the original suggestion).

Suggestions I deem feasable to implement will then be transfered over to the MyUSB bug tracker as an enhancement request, whereupon I’ll implement the proposed feature.

What are you waiting for? Go make yourself heard and leave a suggestion on the MyUSB Suggestion Box Wiki!

Woohooo - exams are officially over! I’m now a free man for the next few months, excluding any contract work I decide to take up. Yesterday I recieved my results for my Java Algorithms class, which was to design an efficient (non-database) transaction system for a ficticious phone company. Emphasis was placed on efficiency, which was a nice change from regular “whatever works” Java classes; I got to do some proper profiling of my Java code and ended up with something quite decent (the fastest in the class, in fact).

From that, I’ve learned one interesting thing: Exceptions really aren’t that bad. Last year when I first started out into the mystical land of Exceptions I honestly couldn’t see the appeal — it just seemed like an alternative to the tried and true method of returning an error code in a function. Of course, I’ve since seen the light and I realise how powerful exceptions are and how much clearer exception handled code can be in contrast to the old methods. The fact that my assignment was the fastest (slightly edging out my friend Alan, who took a very similar approach to mine minus all the exceptions) seems to reinforce that notion.

I’ve also discovered the fearesome power of hindsight, along with my other discovery that Javadoc/Doxygen comments don’t have to make your code into a complete an utter unreadable mess. I really wish I found that out for myself sooner, since I’m now stuck documenting all my demos in the MyUSB library — and there are a lot of demos. Still, with the addition of a few extra “articles” in the documentation on how the demos work from a higher level than the raw code, the next release should be sock-propelling-off-feet good. There are a lot of other changes to the library scheduled for the next release as well, of course, including a fixed and cleaned version of the DFU bootloader — no more bugs and wonky logic!

I’m thinking of setting up a freely editable “brainstorm” wiki, for people to leave suggestions for me - either new projects, interesting things to investigate, or ideas on inproving my existing projects. The comments I’ve recieved here and via email have been good, but I’m perpetually in danger of losing some of them, so a Wiki or something similar seems like a good idea to me.

UPDATE: I’ve started the new suggestion Wiki here: http://wikihost.org/wikis/myusbsuggestion/.

Next week I’ll be attending my final two exams for the year, first DDP (Digital Design and Programming — essentially, a beginner’s course to the VHDL language on CPLD chips) on Monday, then ALG (Algorithms) on Thursday. Neither are particularly difficult in the brains department, but both require a lot of memorizing. While I fully understand all the algorithms covered, I do think it’s a bit much to make students remember every possible case for every covered algorithm for repitition on demand — Red-Black Trees alone have a good five seperate cases for insertion. I’ve always been cursed with a poor memory, so I’m possibly doomed on that alone.

On a brighter note, Gary sent me a link to his own blog, detailing some of his AVR exploits, the most recent of which is MyUSB related. It’s always gratifying to see my work being used by both commercial and hobbyists - the latter was a great consern in the design of the library, since commercial outfits would use a bad library warts-and-all if no alternative was avaliable. It’s fantastic to see that people are using it for their personal projects, proving to me that it is at least somewhat accessible to the normal AVR folk out there.

During the course of our exam study the other day, myself and my techno-geek friends got to discussing absurd communication mediums. Honestly — the lot of us looked almost identical to the characters in the awesome show “The Big Bang Theory”, with a tablet PC connected to our TV, and iPhones and laptops flying every which way. That’s neither here nor there however, as the discussion moved from the mundane (TCP/IP over LEDs) to the outrageous (TCP/IP over SMS). This was my suggestion; correctly coupled with a free online SMS service, it would be theoretically possible to write a proxy which sent out TCP/IP packets over 150 byte long SMS messages.

I don’t think SMS supports much other than the basic latin alphabet, so that means that packets would be base-64 encoded. A typical TCP/IP data packet (as opposed to a non-data containing signalling packet such as an ACK, SYN or FIN) header is around 50 bytes, so there would be sufficient space for a tenth of a kilobyte or so of data to be transmitted per message. I can’t imaging the phone company being to happy about the torrents of messages it would produce, but the TCP layer would ensure data integrity on the recieving end (even for out-of-order packets) and would probably out-strip GPRS anyway ;).

I really should spend more time studying.

The first two of my exams are over, and I’m thankfully still alive - although somewhat battered. Tomorrow brings my Analogue Electronics exam, soon followed by my Calculus Maths exams, then my…on second thought, I need to have a lie down.

I’m still working (slowly) on the MyUSB documentation, but exams are taking priority — and there really is a mountain of work to be done, with each demo taking at least half an hour. On the plus side, I’ve taken the opportunity to clean up other areas of the documentation, so the next release will have more information pages.

Yesterday I used some downtime after an exam to work on the RNDIS MyUSB demo. The RNDIS (Remote Network Driver Interface Specification) demo is an implementation of Microsoft’s RNDIS protocol, used to create USB-over-Ethernet devices via a virtual network adapter. My previous efforts made a working implementation on top of which I hastily wrote a TCP/IP stack - but it left a lot to be desired. For one, you couldn’t close the connection from the application side, so the webserver demo left connections opened indefinetely.

My work yesterday was to implement proper application closing code, so that the established connection can be cut via a FIN message and a transition to the CLOSING and FIN WAIT 1 TCP states. I also took the opportunity to fix a few of the poor code which left it anything but a standards compliant TCP/IP stack. The result is still not clean nor fully complaint, and I recommend replacing it with a properly written stack like uIP for full functionality, but it serves as a working demonstration.

Aside from the documentation I’m really running dry on further development ideas. I’m going to investigate host-side driver development and the TMC USB class, but if you have any ideas of your own to share, leave a comment!

One thing I’ve learned in the last week - everyone loves a good ray tracer. Even my first, poorly written one. It seems that every man and his dog who has ever touched a computer has written one, and is able to give me advice. That’s a neat feeling; I’m following in the footsteps of many of those before me, and poorly reinventing the wheel for my own amusement and education.

I’ve just submitted my course enrollment for next year. It seems my fears about my failed calculus unit in first year preventing me from doing anything until I’ve completed the subsequent maths units is unfounded, as while I have different units to normal students in my course this and next year (due to the shifted maths units), I’ve still got a normal workload. It sucks that my course will be extended by at least six months at the end due to my inability to keep up in second semester last year, but it’s a part of life and something I cannot change.

For my specialties next year, I’ve decided to skip database fundamentals - I already have them, “DROP {Tablename}”, “TRUNCATE {TableName}”, “‘ OR 1=1″, *grin* - and go for the network engineering instead. That means I’ll be learning TCP/IP for real next year, which should cement the knowedge of TCP/IP I gained this year while writing my RNDIS Ethernet USB demo application for my MyUSB stack.

Speaking of the MyUSB stack, I’ve bitten the bullet and started to fully document all the demos with Doxygen. That means that the next release will contain function, macro, enum, file and global documentation for all of the demos, projects and bootloaders in the MyUSB library. It’s an absolutely enormous amount of work to be done, which will delay 1.5.4 for quite a while, especially due to my end of year exams which start in two days. Pray for my sanity, or better yet, donate!

My continued apologies to anyone whose messages to me I’m neglecting while I study for my exams. Please be patient, and above all understanding!

Many people have contacted me requesting I implement some class or other. I’ve tried to implement the majority of the standardized classes (or at least classes with well defined specifications) where possible, but sometimes I’ve deemed it just isn’t feasable. I thought I’d list the evaluated-but-not-implemented classes here, and get some feedback as to other classes to implement, or ways around the limitations I have found.

Printer Class (Device): Possible, but just not useful. This would require a complete printer output language such as PostScript to be implemented - a big job in itself. I can’t think of any real use for a printer device demo.

Printer Class (Host): Possible, but too many variations. The printer USB class is just a simple wrapper for the printer’s display language. Different printers implement one or more of many different display languages, and not all printers conform completely anyway. Mandating some requirements would work, but would limit the demo to only a small subset of all USB printers. I still might tackle this, if there’s enough interest.

Bluetooth Class (Device): No use, and too complicated. Enough said.

Bluetooth Class (Host): I’m still evaluating this one. There’s a lot of specification to go over but initial evaluation was not good; the host is required to support up to 64KB packets from the Bluetooth dongle - far more RAM than available. I do want to give this a go as I do see some real uses for it (Serial over Bluetooth for one) and I’ve been told that it’s possible to get away with much smaller packets.

Test And Measurement Class (Device): This is a standard USB class from the USB-IF, but most OSes don’t carry any prepackaged drivers or ways of utilizing a TaMC device. I’d need to find good existing (free) products for all major OSes before I do anything with this, as a demo with no host application is rather useless.

Test and Measurement Class (Host): It’d be neat to plug in a USB oscilloscope into the board and have it log information, but I haven’t got any TaMC devices to test this with.

SideShow (Device): Vista-only at present but hopefully will see more exposure in the future. Currently I’ve got a working implementation, but the need for a XML parser and the host ignoring the device’s limited capabilities means that this has been put on hold. Microsoft have indicated that changes to the SideShow system will be released in November which will make SideShow implementation on small microcontrollers much easier, so I’m holding all further development on this until then.

SideShow (Host): Again, no hardware to test this with. Might be neat for the future to have any sideshow device act as a nice GUI for projects.

CDC EEM/ECM (Device): There’s too many Ethernet-over-USB class choices. The current library implements Microsoft’s RNDIS class, which also works on current Linux kernels. I’ve heard CDC ethernet implementations are spotty across the board, so I’m not sure which other classes to go with. For now I’ve played it safe and used RNDIS which should work across all Windows versions in addition to newer *nix distributions.

CDC EEM/ECM (Host): Haven’t given much thought to this one. If my USB modem uses this class, I might have a go at writing up a host for it so I can make a public webserver from my RNDIS demo code.

Still Image (Device): Haven’t thought of a real need, but also might do this one in the future, interest permitting. If you want to see this or another class implemented, leave a comment or email me.

Video Class (Device): The Video class requires high bandwidth devices for fluid motion. I don’t think the AVRs with their Full Speed USB will be fast enough to make this worthwhile.

Video Class (Host): Same as the above. The USB AVRs just don’t have the USB speed to process video.