The new, new LUFA bootloader
Three blog posts in one week, after being so tardy in the last few months. Things are looking up!
After my last post detailing my latest and greatest abomination to the world of embedded engineering, Markus asked if I had considered making a Mass Storage class bootloader. Those are fairly common in the embedded world — the idea is that the device shows up like a normal disk drive, and you can reprogram it by dragging-and-dropping a new binary file to an emulated filesystem.
I had thought about this in the past, however somehow I never quite got around to looking into it properly. The FAT specification is one I’ve never really looked into before now, preferring instead to use the well known, proven FATFS library in my projects. Actually, I admit I previously thought it impractical on the USB AVRs, as for some reason I had it in my head that the file system had to be buffered in RAM before the new firmware was extracted and copied to FLASH. Revisiting this made me realize what a world class fool I am (hold your nods of agreement, please) when it hit me that I could just map the virtual filesystem to physical FLASH blocks.
As a result, I now have this working:
Which allows you to reprogram the AVR via a BIN file by overwriting the existing FIRMWARE.BIN file (which, when read, returns a binary image of the current FLASH contents). The emulated FAT code was actually surprisingly straightforward to write once I started reading some decent overviews on the subject, although I spent far longer than I care to admit (ok, ok, five hours) going back and forth over my code looking for a flaw, before finding out that the host OS was thinking my (valid) FAT16 partition was a FAT12 partition and becoming confused. That was the real piece of information I missed that is absolutely crucial; it’s the sector size and count that determines which of FAT12 and FAT16 the host interprets your partition size as, and not the obvious filesystem identifier string. Damn.
Actually, now that I know just how awful FAT is, I’m horrified to realize I used to store all my data on it in the 1990’s under Windows 95 and later Windows 98. It’s a terrible (if simple) filesystem where a single bit of corruption in the right place can cause horrendous data loss. I’ve no idea why Microsoft wants to patent it, since it’s…well…terrible. Objectively so.
Right now the new bootloader code is available on GitHub, if you have a device that is supported (currently just the 128KB and 64kB USB AVRs, since I need an 8KB bootloader section). Paul Stoffrogen’s made the great suggestion of remapping some of the routines to a custom section located just before the bootloader section in order to make it work on smaller devices, but I want to gauge the public opinion first. If you would really like to see a Mass Storage class bootloader on the ATMEGA32U4 or smaller devices, let me know.
In other news, my friend Morten and I went to see the local NTNU university last week, so he could give me a tour. I’ve put some photos up here, however one that stands out:
The room to the bottom-left of the main door is the Omega Verksted; the crazy place where Alf-Egil Bogen and Vegard Wollan designed the first AVR, the AT90S1200.
Wow, that Mass Storage bootloader is amazing. I would *love* to have that on ATmega32U4–it would really simplify the end-user process of updating firmware…no special drivers, just plug’n’play. How large would the bootloader theoretically be?
Right now it takes up approximately 6KB, so 2KB of that would have to be shifted to a special aux section before the real bootloader space, with a stub to jump over it in case the application section is blank. That would effectively shrink the space of the target device by 6KB, making it useful only for the 16KB and larger AVRs.
I’m still looking for ways to shrink the code a bit more without making it completely unmaintainable or break compatibility.
Cheers!
– Dean
I’d love to see it on the 32U4, regardless of code space usage (though of course, smaller would always be better). This is awesome news 🙂 What format is the FIRMWARE.BIN file? Is it just an Intel .hex file?
Not an Intel HEX, but an actual binary file – you can generate one manually from an ELF file using avr-objcopy or use the latest LUFA trunk.
Cheers!
– Dean
Dean, this looks nice!
I guess that many people in the embedded dev world are looking into new ways of fooling Windows8 into talking with their controllers.
Five years ago, I tried something similar (on a Microchip device), and getting into problems with the FAT looks *very* familiar. These days I managed to crash both XP and Linux by simply connecting my USB MSC device to my PC 😉
My plan was to give end-users access to alarm/parameter memory of an embedded control system through MSC, and to “serve” static files from the controller (e.g. HTML or an INF file for CDC installation). Today cheap picture frames do something very similar (e.g. by simulating a CD-ROM drive).
Currently I consider using an AT32UC3 family device in an redundant embedded control system. It’s high time for revisiting MSC based “driverless” access to firmware and configuration memory 🙂
Cheers,
Thomas
That Bootloaders/MassStorage is greit! My first thought was to add a ‘USBXYZ.INF’ file for Windows, or ‘USBXYZ.ZIP’ if it is big. ‘AUTORUN.INF’ to annoy people and a ‘README.TXT’ with version number, URL and other useful stuff. Read-only files I guess. A read/write file could be a ‘CONFIG.INI’ or ‘CONFIG.JSO’ (json) configuration file. The best thing would be if config files could be parsed on the fly and stored directly in EEPROM or RAM – sort of a pseudo file. This file system could make customer support much simpler like “load this file to your system and you are done”.
It does compile for atmega32u2 but I have not dare to try it yet.
Regarding the Microsoft patent thing, it is about long file names. But as embeded developers are rather stingy & frugal with flash-mem-usage it might not be a problem.
[…] given that a number of people wanted ATMEGA32U4 and smaller support for my Mass Storage class bootloader, I’ve gone ahead and implemented the linker gymnastics required to make it work. A warning: […]
that is super neat… congrats!
I’d personally love to use it on smaller uCs
only realized the date of the post now… I really need to go to bed
Thanks Dean !
I have a user board with atxmega64B3. It’s a pain in the *** to program. My friend can’t with his computer and so we can’t progress with our project. Do you think it’s a long task to port the MSD bootloader to XMEGA architecture ?
Regards,
Val’
Sure, I can make it work on an AT90USB162, you can get it working on the XMEGAs. The only real part that needs porting is the NVM routines, which need to be converted from the avr-libc routines (which don’t support the XMEGA) to one that does.
One issue will be that the XMEGAs’ dedicated bootloader section may not be large enough to fit the entire bootloader, but since I already use an AUX section workaround that shouldn’t give you too much trouble.
Cheers!
– Dean
Ok ok ok …
It is a great work … so amazing .. yet I think you guyz are just here to make guyz like me feel over depressed .. 🙂
The story is .. am a newbie .. actually I never yet programmed any MCu .. am from Jordan .. such a hope does not really exist which means I have to live with whatever I can find ..
So .. I found ATmega32APU and ATmega48PV … and I couldn’t find any AVR programmer .. to order one well cost me alot since I have only one way wich is ordering the programmer through that shop where I got the AVRs .. it’ll cost me something like 100$ .. (Hint my Monthly salary is less than 300$) .. Yes am poor yet my head is always up ..
I am searching the internet for a way to program the chips .. and get on this hobby trying to find a way around the need for a programmer … not only that but also a programmer that is already programmed ..
This is how I found you guyz .. first the printer method and then this one which is I like even more ..
The problem is .. you guyz are already in the tread .. and making you hobby even easier for you .. just a “Drag and Drop” .. for you who you already know what you are doing .. not for guyz like me who are challenging thier world to start and get on the train of Micro-controllers ..
So .. is there any room for people like me in your hearts??!! .. would any of you guyz take the initiative to put those new implementations in such a explanation that make sense for newbies????!!!
In another word … HEEEELLLLLLLLPPPP
At the end .. thank you very much .. even if you decided not to lend me a hand ..
Don’t be discouraged Mylo – there’s actually a TON of programming options for the AVRs, depending on how much effort you are willing to get into. Try searching on Ebay for “USBASP”, which is a very low cost but functional AVR programmer. It will require you to use command line tools to program the chips, but once that’s all set up it should be fine. You can also hack a programmer out of an old PC parallel port (not recommended) or a FTDI USB-to-Serial cable. Another option would be to buy an Arduino board which has a serial bootloader already programmed into it, and then use a legacy serial cable or USB-to-Serial cable.
Actually, best start with an Arduino or a clone, since then you have a known working board setup and a large community for support (and the programming problem is taken care of for you). Once you learn how to use that you can step up to writing native code for it, and then finally design your own boards.
Cheers!
– Dean
[…] I just haven’t felt like making large changes on the codebase. This release brings the new Mass Storage class and Printer class bootloaders I talked about in previous blog entries, support for a few new first […]