Memos From the Cube

Quite a while back I posted that I was working on my own ray tracer in my spare time, to learn the concepts behind modern realistic graphics. While the theory behind simple ray tracers was straightforward enough to myself, I had difficulty actually writing a working implementation, so the project languished. Fast forward to last week and in my spare time I started to look back over my code, and realised my errors which were preventing it from working correctly. Since then, I’ve gone from rendering two colored spheres onto the screen, to rendering many textured (image or mathematical based) spheres onto the screen with correct sphere-mapping, shadowing, reflections, omni lighting, sky lighting and super sampling.

While my small project will never develop to the point of anything capable of rending amazing images (like POVRay), it has allowed me to explore a new field which I’ve previously been interested in, but never examined. The best way to learn something is to write an implementation - be it USB, TCP/IP, Graphics or anything else technical related. Without formal study in those areas I won’t be able to place it in the skills section of my resume, but the personal satisfaction of knowing just how something works is good enough for me for now.

Here’s a small image of what it currently performs:

I’ll be exploring other possible features such as spot lighting and refractions at a later date. At the moment I’m currently in the pre-exam panic phase, with end of year exams being only two weeks away. Time for me to start the study processes that I hate oh-so-much. This means that I’ll be posting less, and email responses may be delayed by a day or two; if you’ve emailed, please be patient when waiting on a reply.

An old “Internet Aquintance” pointed out to me today that my blog isn’t very visible from the main site, existing only as a small link in the bottom of the navigation pane. That’s a good point, so I’ve added an extra banner to the top each page to try to promote it somewhat. I’ll see how the traffic stats change in the next few days to see if it is effective. While I realise that most of my blog posts here are not exactly gripping to the regular folk, I like to think that at least a few people are tracking my development of my projects.

Also recently I’ve been emailed by two different people asking the same question; can two endpoints of opposite data directions share the same endpoint number? To answer this, let’s first take a look at USB endpoints.
Each endpoint in a USB device is a logical, rather than a physical entity - that is, they exist only as an abstraction on top of the physical data cable. This is shown in the figure below, taken from my unfinished book on USB:

Endpoints serve the purpose of logically seperating several data channels in the device from one another, in a well defined (by the protocol) manner. Think of a USB Webcam; inside there are endpoints to handle both the video and the audio streams seperately. In order for each stream to be differentiated from one another, they are referenced by an endpoint address, an 8-bit number.

Endpoint addresses are themselves composed of two elements; seven bits indicating the endpoint number, and a single bit indicating the endpoint data’s direction. Endpoints are (for the most part) half duplex in that data may only flow in one direction, thus the most significant bit indicates which direction a given endpoint “faces” in the device (a logic “1″ for data IN to the host, or a logic “0″ for data OUT of the host).

This gives an interesting edge case; while every endpoint in a device must have a completely unique endpoint address (the 8 bits of direction and endpoint number), two endpoints of opposite direction can share the same endpoint number. This is commonly used for associated data streams (although such an association cannot be inferred by the host - that is the job of the device descriptors) so that related streams (like audio in and audio out in a USB sound card) share the addresses of 0b1xxxxxxx and 0b0xxxxxxx. This is entirely allowed by the standard, and some device drivers are hard-wired to expect the device to use such a scheme, for vendor-specific devices (devices made by a manufacturer which uses a manufacturer supplied driver rather than a generic class driver).

However, just because something is allowed in the standard doesn’t mean that the hardware supports it. This is the case with the Atmel USB AVRs. As the USB AVRs only contain a small amount of USB DPRAM for endpoints, Atmel made the decision to simplify the silicon and the device firmware by only allowing 4-bit endpoint numbers — the upper 7 bits of the endpoint number are implicitly fixed to all logic “0″. While the endpoint direction is still set via the device descriptors as per normal USB devices, the controller-application interface distinguishes between endpoints by only the four bit endpoint number, and not the full endpoint address.

For this reason, the Atmel USB AVRs are unable to accomodate two endpoints of opposite direction sharing the same endpoint number; both endpoint address 0b1xxxyyyy and 0b0xxxyyyy will be seen by the application and controller (in the application side, not the host side) by the shortened endpoint address of 0byyyy. The upper bits and direction flag in the address are discarded, thus imposing a second limitation - endpoint numbers cannot exceed 15 for the current USB AVR models.

The former limitation could be avoided by the addition of a direction flag in the USB registers, but this would complicate the firmware slightly. As it stands, the USB AVR microcontrollers are great little devices, but for full, absolute USB endpoint functionality, the bigger USB supporting AVR32 32-bit AVR line needs to be used.

Many people have contacted me requesting I implement some class or other. I’ve tried to implement the majority of the standardized classes (or at least classes with well defined specifications) where possible, but sometimes I’ve deemed it just isn’t feasable. I thought I’d list the evaluated-but-not-implemented classes here, and get some feedback as to other classes to implement, or ways around the limitations I have found.

Printer Class (Device): Possible, but just not useful. This would require a complete printer output language such as PostScript to be implemented - a big job in itself. I can’t think of any real use for a printer device demo.

Printer Class (Host): Possible, but too many variations. The printer USB class is just a simple wrapper for the printer’s display language. Different printers implement one or more of many different display languages, and not all printers conform completely anyway. Mandating some requirements would work, but would limit the demo to only a small subset of all USB printers. I still might tackle this, if there’s enough interest.

Bluetooth Class (Device): No use, and too complicated. Enough said.

Bluetooth Class (Host): I’m still evaluating this one. There’s a lot of specification to go over but initial evaluation was not good; the host is required to support up to 64KB packets from the Bluetooth dongle - far more RAM than available. I do want to give this a go as I do see some real uses for it (Serial over Bluetooth for one) and I’ve been told that it’s possible to get away with much smaller packets.

Test And Measurement Class (Device): This is a standard USB class from the USB-IF, but most OSes don’t carry any prepackaged drivers or ways of utilizing a TaMC device. I’d need to find good existing (free) products for all major OSes before I do anything with this, as a demo with no host application is rather useless.

Test and Measurement Class (Host): It’d be neat to plug in a USB oscilloscope into the board and have it log information, but I haven’t got any TaMC devices to test this with.

SideShow (Device): Vista-only at present but hopefully will see more exposure in the future. Currently I’ve got a working implementation, but the need for a XML parser and the host ignoring the device’s limited capabilities means that this has been put on hold. Microsoft have indicated that changes to the SideShow system will be released in November which will make SideShow implementation on small microcontrollers much easier, so I’m holding all further development on this until then.

SideShow (Host): Again, no hardware to test this with. Might be neat for the future to have any sideshow device act as a nice GUI for projects.

CDC EEM/ECM (Device): There’s too many Ethernet-over-USB class choices. The current library implements Microsoft’s RNDIS class, which also works on current Linux kernels. I’ve heard CDC ethernet implementations are spotty across the board, so I’m not sure which other classes to go with. For now I’ve played it safe and used RNDIS which should work across all Windows versions in addition to newer *nix distributions.

CDC EEM/ECM (Host): Haven’t given much thought to this one. If my USB modem uses this class, I might have a go at writing up a host for it so I can make a public webserver from my RNDIS demo code.

Still Image (Device): Haven’t thought of a real need, but also might do this one in the future, interest permitting. If you want to see this or another class implemented, leave a comment or email me.

Video Class (Device): The Video class requires high bandwidth devices for fluid motion. I don’t think the AVRs with their Full Speed USB will be fast enough to make this worthwhile.

Video Class (Host): Same as the above. The USB AVRs just don’t have the USB speed to process video.

This afternoon I tried my hand at building an AVROpendous board from the kit sent to me by the very generous Matt from the company of the same name. The “kit” contained the bare silkscreened PCB, plus all the components I needed to build it up - essentially, it’s the same finished board that everyone else can buy, except the parts came in a bag rather than attached to the board :).

Before today, I’ve only tried surface mount soldering once, on a half-dozen resistors needed for a microcontroller development board last semester (MSP430, containing a TI based microcontroller) in a reasonably large SM size.

This was certainly a learning experience. I got out my magnifying glass, my soldering iron, fine pitch solder, tweezers, solder wick and my components, and set to work. My verdict; SM soldering is fun, but at times quite frustrating. Sneezing and literally losing a component isn’t fun, so it was a good thing I had extras of many of the components. While I don’t know what the exact packge is for the components on the AVROpendous board, I do know that they’re DAMN TINY with the resistors measuring about 1mm lengthwise. It’s a good thing I had a quick lesson with my microprocessors lecturer before I started.

After making a reasonable mess of it but enjoying myself thoroughly, I had a finished board. While I ended up with far too much solder on most of the components, the end result wasn’t too shabby, for a first timer. Holding my breath, I plugged it in, and promptly got nothing.

Ever the optimist, I started running some tests. Stupidly I put in a LED directly to the 3.3V external rails to confirm the board was getting power, which it turned out it wasn’t. Bridging the external VCC to the VBUS line of the micro-USB connector finally resulted in a lit LED, and - what’s that?! - a working board.

Once I verified that bridging the supply worked, I went to grab the LED from the rails to put it away (its purpose as a quick go/no-go tester complete) and ended up almost succeeding in welding the plastic to my finger. Lesson to the wise; 5V + LED + Timer + 200mA supply + Finger != a happy Dean. Nevertheless, I poured over the schematic once more.

I found a likely source of the problem; the only point the board and VBUS meet is via L3, a small ferrite bead inductor. Knowing that a single point of failure usually means that it is the reason for a given failure, I bridged the inductor (this is safe, the inductor is for noise suppression purposes) and head Windows happily enumerate my board.

Once I re-soldered L3, the board was up and running sans a working power LED. That turned out to be a case of reverse polarity, a silly mistake which I do not feel at all silly about, since I haven’t a clue how to tell the polarity of a surface mount LED. As far as I can see, the darn thing’s completely symmetrical, which meant that good ol’ Moore caused me to mount it backwards.

That fixed, I now have my first working surface mount board, and a new addition to my AVROpendous family!

And so ends another enlightening experience. I think I’ll grovel to my lecturer solder the MEGA2561 onto the board John so kindly sent me (more on that in a later post) however, as I really don’t want to kill such an expensive chip.

Sometimes I wish there was a working “unsubscribe” link for life. People are just too damn difficult.

On a more positive note, it looks like the complaints I made about the current Windows SideShow implementation are actually resulting in some changes for the better. A Microsoft employee has stated that there are some new changes to the SideShow platform being unveiled at the WINHEC 2008 conference (damn my age and location!) which will make the platform much easier to implement on small embedded microcontrollers. With any luck, this means that consumers will start seeing new (low capability) SideShow devices popping up in six months or so with some useful functionality. Up until now, the platform has really been geared towards high-end microprocessors (such as the ARM series), making implementations difficult for RAM-sparse tiny 8-bit microcontrollers such as the USB AVRs.

A few weeks ago I tried my hand at a SideShow implementation on the MyUSB stack, with mixed success. After a lot of playing around I managed to get my device to enumerate correctly and even recieve XML content from the host. Where I got stuck was twofold:

1) Writing an XML parser to parse the content into a compact form for storage and display is quite involved

2) The GetCapabilities request was essentially never used for its intended purpose by the host (determining the device capabilities such as content caching, screen type, display, etc.)

I do think SideShow is a fantastic idea, and I applaud Microsoft for doing some real innovation. Where it falls down is the current system expects the SideShow device to be fairly interactive — being able to display pictures, many lines of text, take user input, cache data, display many colours, etc. How I envision SideShow is a little different; I see a host being able to push data to all manner of items with different capabilities, small or large.

For example, I wanted to make my demo device take in a Media Player widget application, and display the current song information to a 16×2 alphanumeric display. That serves as something that users can build quite easily and get a feel for the USB stack and the SideShow framework. What I got was a host which expected the device to be able to handle multiple applications at a time, and be able to display complex menus. Ideally, the host should be able to fully query the device capabilities, then push data to the display in a format suitable for that device. If the device is severely limited (such as having no user input, like my proposed song title display) customization of the widget/application should be offered on the host so that the desired data can be streamed to the device.

I’m very intereted in how this will pan out and, for once, I’m optimistic that it won’t turn out to be a complete disaster.

I discovered something interesting yesterday while recieving a private surface mount soldering lesson at University; do not drink caffinated beverages beforehand. After drinking an energy drink in the morning to keep myself awake (getting up at 6:30AM is a killer!) I was probably unable to solder through hole components, so bad was my hand shaking. On a positive note, once I purchase some fine-pitch solder and some solderwick, I’ll be able to complete the AVROpendous kit sent to me by Matt. A few days ago Matt finished a beta version of some new firmware for his board, turning it into a USB AVRDude-compatible AVR programmer. Since the firmware is MyUSB based, it is also compatible with any other USB AVR based board, so it’s useful to everyone rather than just AVROpendous board owners.

Today someone posted an interesting question to the MyUSB discussion board, asking if I plan on writing a USB Printer class demo for the AVRs. The USB printer class is simular to the Mass Storage class, in that it is essentially just a wrapper for an already established standard printer protocol (Postscript, or a few other alternatives). To be honest, I can’t see anyone implementing a complete PostScript parser on a small AVR, so I don’t see a pressing need for such a demo.

However, that caused me to download the PostScript specification and take a look, out of curiosity. I’m astonished; while I knew that PostScript was a scripting language capable of describing printable documents in a standard and defined manner, I never realised the language was Turing Complete - able to be programmed like other regular Turing complete languages such as BASIC or C. Why hasn’t anyone taken this a step further? Someone needs to make a LOGO to PostScript tool, so that entered LOGO programs could be not only printed to paper — but actually executed on the printer! Previews could be done with the regular print preview dialogs in most applications.

I love the idea of a printer co-processor, no matter how slow it might be. I need to device a postscript document which prints out the digits of Pi as they are computed. The sky’s the limit!

Some days I think I’m going crazy. Now, how does one send raw PostScript to a printer? Must find out…

Woooo - another MyUSB library release, version 1.5.3. This new release is mainly for compatibility and compliance, so all users should update both their library and their codebase, if one of the MyUSB demos was used as a starting point for a project. I’ve been really happy with the way things have been turning out; several people (Joerg, Chris, Akito, to name a few) have been very helpful in troubleshooting some obscure bugs in the library. Without too much further ado, the changelog for the new version is:

• Fixed CDC bootloader using pgmspace macros for some descriptors inappropriately

• Updated all Mouse and Keyboard device demos to include boot protocol support (now works in BIOS)

• Renamed bootloader directories to remove spaces, which were causing build problems on several OSes

• Removed serial number strings from all but the MassStore demo where it is required - users were not modifing the code to either ommit the descriptor or use a unique serial per device causing problems when multiple units of the same device were plugged in at the same time

• AudioOutput and AudioInput demos now correctly silence endpoints when not enabled by the host

• Added KeyboardMouse demo (Keyboard and Mouse functionality combined into a single demo)

• Added DriverStubs directory to house board level driver templates, to make MyUSB compatible custom board driver creation easier

• Extended MassStorage demo to support multiple LUNs, 2 by default

• Fixed incorrect device address mask, preventing the device from enumerating with addresses larger than 63

• Fixed incorrect data direction mask in the GetStatus standard request, preventing it from being handled

• Fixed incorrect GetStatus standard request for endpoints, now returns the endpoint STALL status correctly

• Added in new USB_RemoteWakeupEnabled and USB_CurrentlySelfPowered flags rather than using fixed values

• Added DualCDC demo to demonstrate the use of Interface Association Descriptors

• Added pipe NAK detection and clearing API

• Added pipe status change (NAK, STALL, etc.) interrupt API

• Fixed MassStorageHost demo so that it no longer freezes randomly when issuing several commands in a row

• Host demos configuration descriptor routines now return a unique error code when the returned data does not have a valid configuration descriptor header

• Added Endpoint_WaitUntilReady() and Pipe_WaitUntilReady() functions

• Stream functions now have software timeouts, timeout period can be set by the USB_STREAM_TIMEOUT_MS token

• All demos now pass the USB.org automated Chapter 9 device compliance tests

• All HID demos now pass the USB.org automated HID compliance tests

• Polling interval of the interrupt endpoint in the CDC based demos changed to 0xFF to fix problems on Linux systems

• Changed stream functions to accept a new callback function, with NO_STREAM_CALLBACKS used to disable all callbacks

• Mass Storage demo dataflash management routines changed to use the endpoint stream functions

• Added AVRStudio project files for each demo in addition to the existing Programmer’s Notepad master project file

• Re-added call to ReconfigureUSART() in USBtoSerial SetLineCoding request, so that baud rate changes are reflected in the hardware (change was previously lost)

Download the latest code from the project page here, then post any questions you have on the support/discussion group here, or bug reports to the tracker here. I’m also loving the uses people are comming up with for the library, so don’t forget to send me and email (or a donation!) if you have the time.