Extension Extravaganza

Recently, the final build of Atmel Studio 6.1 (the production ready one) was released, bringing with it some interesting new features. It also brought along some internal changes to the way the ASF extension works, which LUFA depends on, requiring me to build a new version of my extension. I thought this was a good opportunity to get back into the PC side of programming for a little while, so I pulled out my copy of Visual Studio, fired it up, and got to work. Well, tried to, as things have really changed since I developed with Visual Basic and WinForms. Damn you, XAML!

LUFA Extension

Atmel Gallery Link

The new LUFA release is much improved; one of the major issues I saw with the previous version was the overwhelming amount of “now what?” feedback from users. First impressions are very important, so if users install your product and receive no feedback on how to use it, they aren’t going to be a happy chappy. To combat this I’ve added an integration binary to the extension which presents a new first-run Getting Started page, which covers the fundamentals of how to create examples, add/remove LUFA modules and access the documentation:

Yes, I know my CSS and HTML skills are non-existent, so I kept it simple. Actually, it turns out the integrated web browser in Visual Studio 2010 is a modified IE7 engine, so any sort of fancy tricks wouldn’t work well anyway. I like the new page, and it’s got good feedback so far, so mission accomplished I think. You’ll see this on fresh installs of LUFA or – if you had the somewhat broken release I originally pushed – every darn time you add or remove any extension. Sorry about that, I blame poor Atmel Studio SDK docs and insufficient testing on my part.

Also new are the bootloaders, as the new ASF changes means I can finally add them to the extension without having to make one example for every single device and bootloader combination. Unfortunately I’ve still had to class them by target memory size, but it’s miles better than the alternative and much more friendly than command line compiling. They’re all included in the new release, as example projects.

Finally comes the help. I’ve previously wired up the module help to the online documentation, so that you can use the ASF Explorer window of Atmel Studio to view the documentation for a particular module. However, that’s not ideal, so the new release also comes with an integrated version of the manual inside the regular Atmel Studio Help functionality. You’ll be prompted to install it the first time the extension is installed (I’m not paying for a darn signing certificate, so users need to click a few confirmations instead) and once added the full LUFA manual will be available, including F1 look-up on (most) LUFA constants, variables, functions, enums and the like.

Making the integrated help was a task in itself – special thanks to my friend Morten, who’s authored the Docbook to HV1 transform needed to make it work. Over the last few weeks I’ve learned far more about XSLT than any many ought to, but it’s all worth it. The end result isn’t perfect, but it’s as good as automatically converted Doxygen XML->DocBook->HV1 is going to get.

Data Size Viewer Extension

Atmel Gallery Link

Second on my list of new Atmel Studio extensions is the Data Size Viewer, created in collaborated with the aforementioned Morten. This implements a basic working version of something I’ve wanted to see integrated into Atmel Studio for a long time, so I went ahead and made it myself. When installed, this will add a new tool window that will display the symbols (variables, functions, etc.) from the compiled ELF binary in a project, and their corresponding sizes within the compiled binary. Sorting this list can answer the question of “where the hell did all my FLASH and RAM go?” in an easier manner than manually reading through the LSS file, and symbols can be double-clicked to jump to their definitions in your source code. It’s a basic extension, but one I already find myself using quite a bit.

Build Taskbar Overlay Extension

Atmel Gallery Link

Ok, this one’s just a bit of eye-candy. However, at work it’s not uncommon for me to start a project build that will take several minutes to complete – the Windows ports of the AVR32 and ARM toolchains aren’t the speediest of things – and it’s nice to be able to minimize the program and do other things in the meantime. On Windows 7 machines this will add a small icon overlay to the Atmel Studio icon in the taskbar, showing when a build is in progress (clock icon), completed successfully (green tick icon) or failed (red cross icon).

Virtual Debug Channel Extension

Actually, this one’s a failed experiment, although I had a prototype working. Using nothing but a debugger I was able to print messages out from a running XMEGA inside Atmel Studio in a special build output window, using a tiny extension I wrote late Friday night. This works by shoving the data into a global array in the target, then using an inline assembly breakpoint instruction to signal my Atmel Studio extension to pick up the data, write it to the output window, and resume execution.

While it worked, I’m afraid it’s just not a good solution, since breakpoints cause the entire context (including watched variables, open debug windows, etc.) to be refreshed each time which is really, really slow. Perhaps in the future something better could be implemented by the real Atmel Tools team, perhaps using data breakpoints.

That’s it for now. If you have extension requests let me know; I’m always looking for new side projects.

 

I’ve got the magic in me…

Well, given that a number of people wanted ATMEGA32U4 and smaller support for my Mass Storage class bootloader, I’ve gone ahead and implemented the linker gymnastics required to make it work. A warning: it’s pretty ugly since I had to split the text segment into a secondary “Aux” bootloader space to make it fit on the smaller devices, but that’s just details. I’ve uploaded binaries of all the bootloaders to this location, including one for the Arduino Leonardo (ground IO13 on start-up to enter the bootloader).

I’ve also gone ahead and added EEPROM support to it, so the bootloader now exposes two virtual files, FLASH.BIN and EEPROM.BIN. Magic! Special thanks to Matt over at Opendous Inc. for his assistance with getting Linux support of the bootloader working, since my initial release had a few issues with write offsets.

Now, recently with the release of the LUFA VSIX extension to the Atmel Gallery for Atmel Studio 6.1, I’m getting increasing pressure to release ported demos for the XMEGA architecture, even if the backend implementation has a few known issues. A little while back I started work on porting things over in a separate branch (called “XMEGA-Demo-Ports”, for those playing at home) which you can try out inside AS6.1 using these instructions, or standalone if you just check out the SVN/GIT.

Right now I’m weighing up the XMEGA demo port branch, and I need a public opinion poll; given that the cross-architecture demo code is a little more complicated (since it needs #ifdef checks for architecture detection and hardware detection) I’m not totally convinced this is the right direction. On the plus side, the new ported demos can be compiled as-is for both XMEGA and AVR8 with no user-changes, and can be compiled for boards that lack some of the UI hardware the current demos expect (such as buttons and joysticks). An example of a ported XMEGA demo for a Device Mouse can be found here – contrast with the AVR8-only demo here. Note the addition of the #if ARCH=... and #if defined(BOARD_HAS_...) checks.

 

The new, new LUFA bootloader

Three blog posts in one week, after being so tardy in the last few months. Things are looking up!

After my last post detailing my latest and greatest abomination to the world of embedded engineering, Markus asked if I had considered making a Mass Storage class bootloader. Those are fairly common in the embedded world — the idea is that the device shows up like a normal disk drive, and you can reprogram it by dragging-and-dropping a new binary file to an emulated filesystem.

I had thought about this in the past, however somehow I never quite got around to looking into it properly. The FAT specification is one I’ve never really looked into before now, preferring instead to use the well known, proven FATFS library in my projects. Actually, I admit I previously thought it impractical on the USB AVRs, as for some reason I had it in my head that the file system had to be buffered in RAM before the new firmware was extracted and copied to FLASH. Revisiting this made me realize what a world class fool I am (hold your nods of agreement, please) when it hit me that I could just map the virtual filesystem to physical FLASH blocks.

As a result, I now have this working:

Mass Storage Bootloader

Which allows you to reprogram the AVR via a BIN file by overwriting the existing FIRMWARE.BIN file (which, when read, returns a binary image of the current FLASH contents). The emulated FAT code was actually surprisingly straightforward to write once I started reading some decent overviews on the subject, although I spent far longer than I care to admit (ok, ok, five hours) going back and forth over my code looking for a flaw, before finding out that the host OS was thinking my (valid) FAT16 partition was a FAT12 partition and becoming confused. That was the real piece of information I missed that is absolutely crucial; it’s the sector size and count that determines which of FAT12 and FAT16 the host interprets your partition size as, and not the obvious filesystem identifier string. Damn.

Actually, now that I know just how awful FAT is, I’m horrified to realize I used to store all my data on it in the 1990’s under Windows 95 and later Windows 98. It’s a terrible (if simple) filesystem where a single bit of corruption in the right place can cause horrendous data loss. I’ve no idea why Microsoft wants to patent it, since it’s…well…terrible. Objectively so.

Right now the new bootloader code is available on GitHub, if you have a device that is supported (currently just the 128KB and 64kB USB AVRs, since I need an 8KB bootloader section). Paul Stoffrogen’s made the great suggestion of remapping some of the routines to a custom section located just before the bootloader section in order to make it work on smaller devices, but I want to gauge the public opinion first. If you would really like to see a Mass Storage class bootloader on the ATMEGA32U4 or smaller devices, let me know.

In other news, my friend Morten and I went to see the local NTNU university last week, so he could give me a tour. I’ve put some photos up here, however one that stands out:

The room to the bottom-left of the main door is the Omega Verksted; the crazy place where Alf-Egil Bogen and Vegard Wollan designed the first AVR, the AT90S1200.

 

The new LUFA bootloader

So I’ve been feeling a little down lately, saddened by my lack of enthusiasm for developing new projects. Stuck in a rut, that sort of thing. A few weeks ago I thought I’d try something simple, new and fun to rekindle my love of developing — and who doesn’t love a good joke?

A long, long time ago I was thinking about Bootloaders for LUFA. Right now there’s three in the current LUFA distribution:

  • A DFU class bootloader, compatible with Atmel’s FLIP software, or the open source dfu-programmer code. Fits into 4KB, robust, EEPROM programming support and popular. On the downside, you need special host software and drivers.
  • A HID class bootloader, compatible with a modified version of Paul’s excellent cross-platform TeensyLoader command line utility. Fits into just 2KB on some devices, requires no non-standard drivers but it doesn’t really do EEPROM (unless you count my hack to sort of, kind of do it).
  • A CDC class bootloader, compatible with AVRDUDE as it implements the fairly popular AVR910 protocol. This is the one appropriated (and modified) by Arduino for their Leonardo board, and the one everyone else seems most eager to use in their own creations. Sadly due to Windows being Windows it needs a special driver file which requires a paid signing certificate as of Windows 8 to function.

But a while back, I had plans for a forth. A crazy, crazy idea: bootloading…over MIDI. Actually, MIDI bootloading turned out to be a rather old idea, and one that’s actually popular in a number of commercial instruments, using MIDI SYSEX commands. My idea was to encode the data as a series of note commands with hard-coded delays — that way the end-user could reprogram a board by literally just playing a MIDI file sent from the manufacturer, with no device-to-host feedback being required (thus no special host software). No drivers needed either, since all modern operating systems include them just like they do for a number of other standard classes.

Unfortunately, revisiting this the other week I was left disappointed; it seems Windows 7 simply doesn’t do proper MIDI mapping to hardware MIDI devices any more, and I’m not the first one to be disappointed by it. Of course you can use dedicated software for it, but that would take away from the simplicity of the implementation. Bang goes that idea.

However, as OK Go say, All is Not Lost. I had another idea up my sleeve; none of than the humble Printer class. Yes, I’m being totally serious. Let’s skip ahead a bit and start with the end result:

Printer Bootloader shown in Windows 7.

That might look like just a humble printer from the 1980’s, but in actual fact you’re looking at what might just be the first bootloader-over-USB-printer-class in the world. Need to reprogram a device? Just “print” a new HEX file to it. Whee.

That’s not to say the journey was easy; first I had to re-visit the specifications for the USB Printer class. It’s actually a deceptively simple specification, if only because it’s just like the Mass Storage one: a thin wrapper defining a generic data transport and a few error feedback mechanisms, and a “data sent is implementation defined” style catch-all. That’s fairly common in the USB world for class specifications – in many cases it’s only the transport that’s covered, and the actual encapsulated data is just whatever the hell the host and client agree to send to each other. In the real world, that can mean PostScript, Epson Command Language, pre-rasterized images, who knows.

So, first I had to make the host see a USB printer device. That’s just a matter of setting the Device Descriptor’s Class, Subclass and Protocol fields, plus a basic implementation of the status and error reporting control requests. I really, really didn’t want to mess with custom printer drivers, so originally I just use my own custom IEEE 1284 description string, and forced the driver manually under Windows to one of the basic generic PostScript drivers it has inbuilt, since that’s a well known standard printer language.

Let me tell you; PostScript is NOT for the faint-hearted. I was hoping to see something vaguely structured with the plain text of the printed source visible so I could pick it out, but alas the received data was just a mountain – seriously, several kilobytes worth – of pure PostScript craziness with no obvious payload in sight. Luckily thanks to receipt printers and ye-olde daisy-wheel printers of yesteryear, Windows also ships with a “Generic Text-Only” printer driver that does away with all the crazy and just outputs exactly whatever text you print with no formatting at all. Success!

Well, partially – with the new driver, I could get a completely plain text representation of whatever I printer from the host (pictures didn’t work, obviously). However, I still wanted the drivers to be plug-and-play, since I wanted the end-result to be as streamlined as possible. After a bit of searching, I ended up finding the answer buried on (embarrassingly) a Microchip support forum post. Plugging in the magic IEEE 1284 identification string now gave me an automatically installing printer under Windows 7 with the desired generic text only driver.

But, what to do now? Should I invent a crazy command format, and have users “print” it to the bootloader? Is there a better way? Of course there is:

Intel HEX format data, generated from a compiled application.

Yup, good old Intel HEX. In the AVR world (and in most embedded engineering circles) HEX files are king, and the file format is simple enough that making a parser for it isn’t too difficult. It’s also the native output format for most AVR build scripts and tools, and the one users are more familiar with.

I admit, the end result parser isn’t the prettiest code I’ve ever written, but it is (arguably) the most amusing. You’ve heard it here first folks; the world’s first AVR bootloader where you just compile, then print your HEX file in Notepad. On Linux, you can just cat the HEX file to the low level LPR printing subsystem to re-flash your board.

Sometimes the best jokes are the ones that turn out to have a somewhat practical use.

 

LUFA 130303 Released!

Some of you might be expecting something a little different here – I mentioned on my Twitter feed of my plans to write up my current tiny side-project (no hints here, either read the feed or be patient and all will be revealed) however I’m still trying to track down a small bug in it, so I thought I’d delay the post about it for now. Instead I bring good news; a new LUFA release!

This one is far, far overdue, but what with my month-long holiday back home, my recent move to a new apartment up in the mountains things got a bit behind. Actually, I was waiting for something else before I pulled the trigger; the release of the public Atmel Studio 6.1 beta.

The AS6.1 beta has a lot of the usual backend and front-end fixes, support for new devices…yadda yadda, you know the drill. However, it also brings support for something called the – well, uh, I’m not sure exactly, since it’s changed names internally half a dozen times – but I’ll call it the XDK since that’s the last monkier I heard for it. The XDK allows developers to package up your Atmel Studio 6.1 projects into dedicated extensions, so you can distribute them on the Atmel Gallery. It’s a little limited for now with plans for future expansion, but the important thing is that it’s given me the infrastructure I needed to finally offer a native LUFA extension.

That means no more crazy manual exporting and importing of the LUFA core, or trying to figure out what project options need to be set – all the demos and projects, as well as base templates, are made available as example projects, just like ASF. Configuration of the modules is done through the ASF wizard as well, so changing your board, device and adding/removing LUFA code modules can all be done through the GUI. It’s not perfect yet, but it’s a start, and I’d love for some public feedback from the new release package.

In any case, I’m not going through with a normal two week beta cycle for LUFA-130303, since the trunk code has been stable for some time now and pretty much continuously tested by myself and the community over the last few months. As a result, I feel very confident in going ahead with a full stable release immediately.

The Changelog for the new release is available here, and the new release downloads can be found here.

 
 
 

Vital Stats

  • 35 Years Old
  • Australian
  • Lover of embedded systems
  • Firmware engineer
  • Self-Proclaimed Geek

Latest Blog Posts

RSS