LUFA Extension

Atmel Gallery Link

The new LUFA release is much improved; one of the major issues I saw with the previous version was the overwhelming amount of “now what?” feedback from users. First impressions are very important, so if users install your product and receive no feedback on how to use it, they aren’t going to be a happy chappy. To combat this I’ve added an integration binary to the extension which presents a new first-run Getting Started page, which covers the fundamentals of how to create examples, add/remove LUFA modules and access the documentation:

Yes, I know my CSS and HTML skills are non-existent, so I kept it simple. Actually, it turns out the integrated web browser in Visual Studio 2010 is a modified IE7 engine, so any sort of fancy tricks wouldn’t work well anyway. I like the new page, and it’s got good feedback so far, so mission accomplished I think. You’ll see this on fresh installs of LUFA or – if you had the somewhat broken release I originally pushed – every darn time you add or remove any extension. Sorry about that, I blame poor Atmel Studio SDK docs and insufficient testing on my part.

Also new are the bootloaders, as the new ASF changes means I can finally add them to the extension without having to make one example for every single device and bootloader combination. Unfortunately I’ve still had to class them by target memory size, but it’s miles better than the alternative and much more friendly than command line compiling. They’re all included in the new release, as example projects.

Finally comes the help. I’ve previously wired up the module help to the online documentation, so that you can use the ASF Explorer window of Atmel Studio to view the documentation for a particular module. However, that’s not ideal, so the new release also comes with an integrated version of the manual inside the regular Atmel Studio Help functionality. You’ll be prompted to install it the first time the extension is installed (I’m not paying for a darn signing certificate, so users need to click a few confirmations instead) and once added the full LUFA manual will be available, including F1 look-up on (most) LUFA constants, variables, functions, enums and the like.

Making the integrated help was a task in itself – special thanks to my friend Morten, who’s authored the Docbook to HV1 transform needed to make it work. Over the last few weeks I’ve learned far more about XSLT than any many ought to, but it’s all worth it. The end result isn’t perfect, but it’s as good as automatically converted Doxygen XML->DocBook->HV1 is going to get.

Data Size Viewer Extension

Atmel Gallery Link

Second on my list of new Atmel Studio extensions is the Data Size Viewer, created in collaborated with the aforementioned Morten. This implements a basic working version of something I’ve wanted to see integrated into Atmel Studio for a long time, so I went ahead and made it myself. When installed, this will add a new tool window that will display the symbols (variables, functions, etc.) from the compiled ELF binary in a project, and their corresponding sizes within the compiled binary. Sorting this list can answer the question of “where the hell did all my FLASH and RAM go?” in an easier manner than manually reading through the LSS file, and symbols can be double-clicked to jump to their definitions in your source code. It’s a basic extension, but one I already find myself using quite a bit.

Build Taskbar Overlay Extension

Atmel Gallery Link

Ok, this one’s just a bit of eye-candy. However, at work it’s not uncommon for me to start a project build that will take several minutes to complete – the Windows ports of the AVR32 and ARM toolchains aren’t the speediest of things – and it’s nice to be able to minimize the program and do other things in the meantime. On Windows 7 machines this will add a small icon overlay to the Atmel Studio icon in the taskbar, showing when a build is in progress (clock icon), completed successfully (green tick icon) or failed (red cross icon).

Virtual Debug Channel Extension

Actually, this one’s a failed experiment, although I had a prototype working. Using nothing but a debugger I was able to print messages out from a running XMEGA inside Atmel Studio in a special build output window, using a tiny extension I wrote late Friday night. This works by shoving the data into a global array in the target, then using an inline assembly breakpoint instruction to signal my Atmel Studio extension to pick up the data, write it to the output window, and resume execution.

While it worked, I’m afraid it’s just not a good solution, since breakpoints cause the entire context (including watched variables, open debug windows, etc.) to be refreshed each time which is really, really slow. Perhaps in the future something better could be implemented by the real Atmel Tools team, perhaps using data breakpoints.

That’s it for now. If you have extension requests let me know; I’m always looking for new side projects.

I’ve also gone ahead and added EEPROM support to it, so the bootloader now exposes two virtual files, FLASH.BIN and EEPROM.BIN. Magic! Special thanks to Matt over at Opendous Inc. for his assistance with getting Linux support of the bootloader working, since my initial release had a few issues with write offsets.

Now, recently with the release of the LUFA VSIX extension to the Atmel Gallery for Atmel Studio 6.1, I’m getting increasing pressure to release ported demos for the XMEGA architecture, even if the backend implementation has a few known issues. A little while back I started work on porting things over in a separate branch (called “XMEGA-Demo-Ports”, for those playing at home) which you can try out inside AS6.1 using these instructions, or standalone if you just check out the SVN/GIT.

Right now I’m weighing up the XMEGA demo port branch, and I need a public opinion poll; given that the cross-architecture demo code is a little more complicated (since it needs #ifdef checks for architecture detection and hardware detection) I’m not totally convinced this is the right direction. On the plus side, the new ported demos can be compiled as-is for both XMEGA and AVR8 with no user-changes, and can be compiled for boards that lack some of the UI hardware the current demos expect (such as buttons and joysticks). An example of a ported XMEGA demo for a Device Mouse can be found here – contrast with the AVR8-only demo here. Note the addition of the #if ARCH=... and #if defined(BOARD_HAS_...) checks.

After my last post detailing my latest and greatest abomination to the world of embedded engineering, Markus asked if I had considered making a Mass Storage class bootloader. Those are fairly common in the embedded world — the idea is that the device shows up like a normal disk drive, and you can reprogram it by dragging-and-dropping a new binary file to an emulated filesystem.

I had thought about this in the past, however somehow I never quite got around to looking into it properly. The FAT specification is one I’ve never really looked into before now, preferring instead to use the well known, proven FATFS library in my projects. Actually, I admit I previously thought it impractical on the USB AVRs, as for some reason I had it in my head that the file system had to be buffered in RAM before the new firmware was extracted and copied to FLASH. Revisiting this made me realize what a world class fool I am (hold your nods of agreement, please) when it hit me that I could just map the virtual filesystem to physical FLASH blocks.

As a result, I now have this working:

Which allows you to reprogram the AVR via a BIN file by overwriting the existing FIRMWARE.BIN file (which, when read, returns a binary image of the current FLASH contents). The emulated FAT code was actually surprisingly straightforward to write once I started reading some decent overviews on the subject, although I spent far longer than I care to admit (ok, ok, five hours) going back and forth over my code looking for a flaw, before finding out that the host OS was thinking my (valid) FAT16 partition was a FAT12 partition and becoming confused. That was the real piece of information I missed that is absolutely crucial; it’s the sector size and count that determines which of FAT12 and FAT16 the host interprets your partition size as, and not the obvious filesystem identifier string. Damn.

Actually, now that I know just how awful FAT is, I’m horrified to realize I used to store all my data on it in the 1990′s under Windows 95 and later Windows 98. It’s a terrible (if simple) filesystem where a single bit of corruption in the right place can cause horrendous data loss. I’ve no idea why Microsoft wants to patent it, since it’s…well…terrible. Objectively so.

Right now the new bootloader code is available on GitHub, if you have a device that is supported (currently just the 128KB and 64kB USB AVRs, since I need an 8KB bootloader section). Paul Stoffrogen’s made the great suggestion of remapping some of the routines to a custom section located just before the bootloader section in order to make it work on smaller devices, but I want to gauge the public opinion first. If you would really like to see a Mass Storage class bootloader on the ATMEGA32U4 or smaller devices, let me know.

In other news, my friend Morten and I went to see the local NTNU university last week, so he could give me a tour. I’ve put some photos up here, however one that stands out:

The room to the bottom-left of the main door is the Omega Verksted; the crazy place where Alf-Egil Bogen and Vegard Wollan designed the first AVR, the AT90S1200.

A long, long time ago I was thinking about Bootloaders for LUFA. Right now there’s three in the current LUFA distribution:

• A DFU class bootloader, compatible with Atmel’s FLIP software, or the open source dfu-programmer code. Fits into 4KB, robust, EEPROM programming support and popular. On the downside, you need special host software and drivers.
• A HID class bootloader, compatible with a modified version of Paul’s excellent cross-platform TeensyLoader command line utility. Fits into just 2KB on some devices, requires no non-standard drivers but it doesn’t really do EEPROM (unless you count my hack to sort of, kind of do it).
• A CDC class bootloader, compatible with AVRDUDE as it implements the fairly popular AVR910 protocol. This is the one appropriated (and modified) by Arduino for their Leonardo board, and the one everyone else seems most eager to use in their own creations. Sadly due to Windows being Windows it needs a special driver file which requires a paid signing certificate as of Windows 8 to function.

But a while back, I had plans for a forth. A crazy, crazy idea: bootloading…over MIDI. Actually, MIDI bootloading turned out to be a rather old idea, and one that’s actually popular in a number of commercial instruments, using MIDI SYSEX commands. My idea was to encode the data as a series of note commands with hard-coded delays — that way the end-user could reprogram a board by literally just playing a MIDI file sent from the manufacturer, with no device-to-host feedback being required (thus no special host software). No drivers needed either, since all modern operating systems include them just like they do for a number of other standard classes.

Unfortunately, revisiting this the other week I was left disappointed; it seems Windows 7 simply doesn’t do proper MIDI mapping to hardware MIDI devices any more, and I’m not the first one to be disappointed by it. Of course you can use dedicated software for it, but that would take away from the simplicity of the implementation. Bang goes that idea.

However, as OK Go say, All is Not Lost. I had another idea up my sleeve; none of than the humble Printer class. Yes, I’m being totally serious. Let’s skip ahead a bit and start with the end result:

That might look like just a humble printer from the 1980′s, but in actual fact you’re looking at what might just be the first bootloader-over-USB-printer-class in the world. Need to reprogram a device? Just “print” a new HEX file to it. Whee.

That’s not to say the journey was easy; first I had to re-visit the specifications for the USB Printer class. It’s actually a deceptively simple specification, if only because it’s just like the Mass Storage one: a thin wrapper defining a generic data transport and a few error feedback mechanisms, and a “data sent is implementation defined” style catch-all. That’s fairly common in the USB world for class specifications – in many cases it’s only the transport that’s covered, and the actual encapsulated data is just whatever the hell the host and client agree to send to each other. In the real world, that can mean PostScript, Epson Command Language, pre-rasterized images, who knows.

So, first I had to make the host see a USB printer device. That’s just a matter of setting the Device Descriptor’s Class, Subclass and Protocol fields, plus a basic implementation of the status and error reporting control requests. I really, really didn’t want to mess with custom printer drivers, so originally I just use my own custom IEEE 1284 description string, and forced the driver manually under Windows to one of the basic generic PostScript drivers it has inbuilt, since that’s a well known standard printer language.

Let me tell you; PostScript is NOT for the faint-hearted. I was hoping to see something vaguely structured with the plain text of the printed source visible so I could pick it out, but alas the received data was just a mountain – seriously, several kilobytes worth – of pure PostScript craziness with no obvious payload in sight. Luckily thanks to receipt printers and ye-olde daisy-wheel printers of yesteryear, Windows also ships with a “Generic Text-Only” printer driver that does away with all the crazy and just outputs exactly whatever text you print with no formatting at all. Success!

Well, partially – with the new driver, I could get a completely plain text representation of whatever I printer from the host (pictures didn’t work, obviously). However, I still wanted the drivers to be plug-and-play, since I wanted the end-result to be as streamlined as possible. After a bit of searching, I ended up finding the answer buried on (embarrassingly) a Microchip support forum post. Plugging in the magic IEEE 1284 identification string now gave me an automatically installing printer under Windows 7 with the desired generic text only driver.

But, what to do now? Should I invent a crazy command format, and have users “print” it to the bootloader? Is there a better way? Of course there is:

Yup, good old Intel HEX. In the AVR world (and in most embedded engineering circles) HEX files are king, and the file format is simple enough that making a parser for it isn’t too difficult. It’s also the native output format for most AVR build scripts and tools, and the one users are more familiar with.

I admit, the end result parser isn’t the prettiest code I’ve ever written, but it is (arguably) the most amusing. You’ve heard it here first folks; the world’s first AVR bootloader where you just compile, then print your HEX file in Notepad. On Linux, you can just cat the HEX file to the low level LPR printing subsystem to re-flash your board.

Sometimes the best jokes are the ones that turn out to have a somewhat practical use.

This one is far, far overdue, but what with my month-long holiday back home, my recent move to a new apartment up in the mountains things got a bit behind. Actually, I was waiting for something else before I pulled the trigger; the release of the public Atmel Studio 6.1 beta.

The AS6.1 beta has a lot of the usual backend and front-end fixes, support for new devices…yadda yadda, you know the drill. However, it also brings support for something called the – well, uh, I’m not sure exactly, since it’s changed names internally half a dozen times – but I’ll call it the XDK since that’s the last monkier I heard for it. The XDK allows developers to package up your Atmel Studio 6.1 projects into dedicated extensions, so you can distribute them on the Atmel Gallery. It’s a little limited for now with plans for future expansion, but the important thing is that it’s given me the infrastructure I needed to finally offer a native LUFA extension.

That means no more crazy manual exporting and importing of the LUFA core, or trying to figure out what project options need to be set – all the demos and projects, as well as base templates, are made available as example projects, just like ASF. Configuration of the modules is done through the ASF wizard as well, so changing your board, device and adding/removing LUFA code modules can all be done through the GUI. It’s not perfect yet, but it’s a start, and I’d love for some public feedback from the new release package.

In any case, I’m not going through with a normal two week beta cycle for LUFA-130303, since the trunk code has been stable for some time now and pretty much continuously tested by myself and the community over the last few months. As a result, I feel very confident in going ahead with a full stable release immediately.

The Changelog for the new release is available here, and the new release downloads can be found here.

Late last year our team’s responsibilities changed from ASF development (that’s the giant framework inside Atmel Studio, also available as a standalone download) to NPI work – the “N” and “P” of which stand for “New” and “Product”, and I assume the “I” means “Introduction” – and that’s caused some massive shifted gears in my brain while the rest of me tries to catch up. It’s interesting work, and fun too since I get to argue discuss nitty-gritty details with my co-workers and really make a difference. At least, that’s how it seems now, but I guess ultimately time will tell.

Customer Visit

In September last year I was put on a difficult customer support case. One of those really nasty ones: “we stopped production because of a x% failure rate in y% devices, after around Z hours of use.” Several weeks, some lab testing and a veritable ton of code inspection later I was suddenly thrown into a brand new situation; the in-person Customer Support Visit (cue dramatic music). Sometimes you need to roll up your sleeves (actually, first I needed to obtain some branded Atmel shirts, so I would had sleeves to roll up) and get on-site to diagnose the really difficult problems.

Within 24 hours of the request from the customer my travel plans were approved, my flights booked and my hotels reserved, and I was on my way over to Scotland. To say I was nervous was an understatement; while I don’t mind traveling, I’m not known for my confidence in new and scary situations. Still my boss and my co-workers did a bang-up job in keeping my spirits up and my lunch down, and before long I was settling in on-site and seeing what I could do to help.

My dad used to travel quite a bit for his work, and I always remember him saying that business trips are boring, since all you get to see are airports and factories. Boy, was he right. I can now say I’ve been to Scotland, but the only sites I really got to see were the airport, three (!) hotels and a hospital-like customer site. I did see a “Postman Pat” style mail van which was quite endearing:

But other than that it was just hotel:

After hotel:

Although in the latter case, I did get to sit on the world’s largest cushion:

And try a local drink called “Irn Bru”, which I assume is Scottish for “Flurecent Orange Creamy Soda”.

And I also tried Haggis. Worryingly, it tasted exactly like the filling of an Australian meat pie. All in all, it was a great learning experience, and I got to see how a real-world factory works from PCB production right down to final assembly and packaging.

Visit Home

Right after my customer visit I was on to more traveling; this time, back to familiar places. I was selected to be a support engineer for the Australian “Tech on Tour” training sessions held in Sydney and my home town of Melbourne in November last year. This is the first year I am aware of that Atmel’s run first-party training sessions, so it was a fantastically happy coincidence as I was then able to take a holiday off for Christmas directly afterwards.

The ToT trainings went quite well, minus a few minor hiccups like the conference room Wifi falling over in the first 10 minutes (Melbourne) and the air conditioning being unable to cope with 20 people and 20 computers (Sydney). This was another great first experience for me as I got to interact directly with customers, helping them with issues and listening to feedback for my team. While in University I toyed with the idea of becoming a lecturer rather than a engineer in the field, so this was a chance to experience a bit of that without the drastic career change.

I was thrilled to see my fellow AVRFreaks friends in Melbourne again and I finally got to meet the (in?)famous John Samperi over in the Sydney training, who was must less grouchy in person that I was expecting (kidding John!). The Sydney event was completed with one of the attendees asking me to sign his JTAG-ICE3. I guess that means I’m finally popular – take that, highschool.

While home I got to eat good food again (finally!):

Spend Christmas with family:

And ride on the largest miniature railway in the Southern Hemisphere (not pictured: me):

All of which was good fun. While at home I also spent a little time helping out a friend with his LEAP robot:

Which he was building for our old University to be used as a marketing tool. I was tasked with making an Android frontend for it, so that users could control it via Bluetooth from their mobile phone. I kinda like Android development now that I’ve had a little more exposure to it, so perhaps I’ll explore it a bit more in the future.

Back in Trondheim

I was more than a little bit sad to leave Melbourne again, not only because I was once again leaving my family behind, but this time also my girlfriend. Living alone comes with an adjustment; the largest one is trying to find a new apartment (the building I am currently renting is being sold). That comes with a set of new challenges, although so far I’m coping fairly well.

Stay tuned for an update on LUFA soon, since we’re overdue for a release. I leave you now with a sneak preview of what I’ve been working on over the last few weeks:

Firstly, you need the current trunk version of LUFA – not the latest release. Not only does the latest trunk have board hardware definitions that the current release lacks, but it also contains a (preliminary) compatibility fix to the core to make it work with the Leonardo bootloader. Special thanks to Simon Inns for both his debugging help and the donation of an Arduino Leonardo board so that I can continue my development efforts with it. You can download the latest LUFA trunk here.

Secondly (Windows only), you’ll need to have downloaded the Arduino 1.0 IDE from the Arduino website. You’ll need this for two reasons; firstly, you’ll need the driver INF file it contains in the arduino-1.0\drivers directory, and secondly, you’ll need the Windows build of the AVRDUDE application that it includes in the arduino-1.0\hardware\tools\avr\bin. You need to add the latter to your system PATH variable so that you can run it from any folder from the command line. You can download the Arduino 1.0 IDE from here.

Thirdly, you’ll need to alter the project makefile that you want to compile. The relevant makefile magic here is:

MCU          = atmega32u4 # Change this
BOARD        = LEONARDO # And this
F_CPU        = 16000000 # And finally this
 
AVRDUDE_PROGRAMMER = avr109 # Add this
AVRDUDE_PORT = COM32 # And this too (use your correct COM port)

Those are the only lines that need to be changed from the standard LUFA makefile template. Note that on my system the bootloader appears as COM32, but on your system this will no doubt be different, so alter the AVRDUDE_PORT line to suit your own environment. If you use Linux, this line will most likely be /dev/ttyACM0 or another similarly named virtual USB TTY device.

That’s it – with the small changes to the project makefiles, you can then download a new application by pressing the reset button on the board, and running make avrdude from the command line shortly afterwards (while the L LED is still fading in an out to signal the bootloader is running).

This release has been one of the more tumultuous ones, as it’s the first made while I’ve had a full time job. In it I’ve tried to incorporate some of the new techniques I’m learning towards maintaining large software projects, but it’s also been significantly hampered by my lack of enthusiasm overall towards coding after a long day of, well, coding. I’m not saying I’m giving up by any means, but I’ve had to downgrade my overly enthusiastic future timelines to reflect reality a little. Lessons learned.

So, this release seems relatively short in the changelog area, but that’s somewhat deceptive as some of the changes are either back-end that don’t need changelog entries, or covered under a single line umbrella entry. There’s a new set of build tests (still not unit tests on real hardware I’m afraid, but the current smoke tests and static analysis still help) as well as the totally revamped build system, which paves the way slowly to a mufti-architecture future. The core now “kinda” works on the XMEGA platforms — but it is still not complete nor fully tested — and for once the limiting factor is revamping the board system (a comparatively simple, yet tedious task) and not the build system, demos or core. That’s a good thing. I’ve also reorganized the documentation to put the more useful bits right in the sidebar once again, to style it a little better, and to add new important documentation like a list of known issues for each supported architecture.

With this release I’m also moving the official GIT mirror from this GitHub entry to this one instead. I will continue to maintain the old repository mirror and you can still use that if you wish, but the new entry doesn’t suffer from the wrongly imported SVN repository issue that the older one has, which means simpler forks and smaller downloads. Also, automated tags are now actually a thing.

Thank you to the LUFA community for helping out, contributing and above all putting up with my relatively slow progress as of late. I realize people deserve better, thus I will attempt to redouble my efforts as much as I can in the future.

Without further ado:
LUFA 120730 Release Download
LUFA 120730 Prebuilt Documentation Download
LUFA 120730 Online Documentation Download

LUFA 120730 BETA Download
LUFA 120730 BETA Prebuilt Documentation
LUFA 120730 BETA Online Documentation
LUFA 120730 Changelog

Please download and test during the next two weeks, so that any unknown issues can be uncovered and patched before the full release is made. This release contains may bug fixes and new features, including some additional board drivers, a new “SerialToLCD” user project contributed by Simon Foster, the new LUFA build system, and fixes to the known broken demos and applications. Once again, thanks also to the great LUFA community!

Since it’s been so darn long since I posted about something technical that I thought I’d try doing that today.

So, I’ll admit it, I’ve been putting off proper LUFA development over the past few weeks, since the next step in my quest to refactor things to the point where multi-architecture demos can become a reality requires an awful lot of work. And not fun work either, mechanical “I would script this if it was practical” type of mechanical work. That’s not to say I’ve been sitting around idle, it just means I’ve been focusing my time learning some fun new things and performing other refactoring tasks (in addition to support and bug fixing). So here’s what I’ve been up to.

The LUFA Board System:

A problem lies in the current board driver system: currently each board has a folder full of individual hardware driver header files, with the actual driver code implemented as inline C functions for speed and size reasons. in the user application, you set your BOARD={Something} value in your project makefile, include the relevant <LUFA/Drivers/Board/XXXX.h> driver in your code and away you go. If the application is recompiled with a different BOARD setting, the driver dispatch header automatically picks the appropriate driver, or defers to a <Board/XXXX.h> header in your application directory. Wonderful.

Or at least, wonderful if you are using a board that has the same features as the USBKEY board I develop with, since that’s how I design the UI for the demos that ship with LUFA. If you use a different board without a Joystick for example and try to compile the Keyboard demo, you’ll get an error about not being able to find a Buttons.h and Joystick.h driver. Fixing this requires manual tweaking of the demos, which isn’t very nice to newcomers.

So, the plan I’ve come up with is to replace the current board system, leading me into the gigantic ball of apathy that you’ve all seen over the last month. This requires the creation of a new per-board board.h header file to act as the new board driver header, which will be include in all user projects. This header file will in turn include all the board specific drivers, but also include a new Board_Init() function to automatically initialize all the board hardware.

Doing this will allow all the user applications to just include the one board driver header file, rather than having to pick-and-choose the individual driver headers. It will also mean cleaner initialization code, but most importantly it will mean compile time detection in the user code of the board driver features. With this new system I will be able to change the demos to detect if a joystick, buttons or other hardware is available and automatically provide some sort of demo UI to suit the board being targeted. No more compile errors.

I’m tossing up whether to press ahead with a new LUFA release in the meantime or to wait until I find enough time to undertake this – does anyone have any thoughts on this?

The LUFA Documentation:

So, with the new release of Doxygen 1.8.1 I’ve done a bit of spurcing up of the LUFA documentation. I’ve gone and re-tweaked my small customizations to the standard theme – I can’t do visual design at all, but I found something I liked a bit more than the defaults – and fixed up the documentation index. In the previous 120209 documentation a lot of the helpful pages were shoved in a “Related Pages” node due to a Doxygen bug, which is no more in the new release. To commemorate that, I’ve cleaned up the ordering and structure a bit to make things a bit easier to read. I’ve uploaded a temporary export of what the new documentation will look like, where you can see the new index page list.

In addition, I’ve added some new pages listing known issues I need to fix up, the new LUFA build system (more on that in a moment), added more information giving the prerequisites for building on both Windows and Linux platforms and showing one way that you can export the library core for importing and use in Atmel Studio 6 projects.

No groundbreaking changes, but any improvements are always a good thing, and I think this is one small step towards making the project as a whole a little more accessible to new developers.

The LUFA Build System:

I finally got completely fed up with the current build system – having a gigantic 600+ line makefile in every application directory was killing me due to the maintenance requirements, and frustrating users due to the apparent complexity. The LUFA makefiles were built from the public domain template found in the old obsolete MFile project subcomponent of the now obsolete WinAVR project with a few additional tweaks to make them useful for LUFA based projects. However, inside each of these giant makefiles was a huge amount of repeated “magic” rules that most people will never touch, and a select few variables that need to be adjusted for different build targets. Other problems were crazy error messages if things went wrong in some way, untracked dependencies causing odd build behaviours if multiple make targets were invoked at the same time, and output files generated from stale object files in some cases.

To combat this, I’ve created my own – yes, I’ll wait a moment for you to finish cringing – build system for LUFA. This is actually just a set of loosely coupled makefile modules, which can be included in the application makefile to obtain additional functionality. I think this is the best solution, as it allows me to edit the modules all in one place, and have the changes propagate automatically out to all the library applications. It also means I can fix up all the problems in the above, so the new modules have proper dependency tracking, and give sane error messages when things go wrong.

Here’s an example of what a typical project makefile looks like, contrasted to what we have in the current 120219 release. It’s a big change, and I really think it’s a great improvement. You can now get some nice build help by running make help from any makefile that uses the CORE LUFA build system module, listing all the available targets and other relevant information. There’s also the aforementioned new chapter of the manual with detailed documentation on how to use it. Of course, the manual has also been updated to give an application makefile template and explain the new makefile variables.

I’m quite proud of this, and I’ve learned an awful lot of the ins-and-outs of GNU Make while on my quest to build it. I’ve been considering opening up the build system into a separate public-domain project so that it can be adopted everywhere, rather than the crazy makefile template everyone is currently using.

New LUFA Git Mirror:

When I first made the LUFA Git repository mirror, I made a bit of a boo-boo, as I forgot the “--stdlayout” switch when creating the initial GIT-SVN import. That has resulted in the current GIT mirror containing a literal import of the LUFA SVN repository layout, with tags and branches being located in the sole master branch of the GIT tree, something I didn’t notice until it was too late to try to change it.

After a couple of different people emailed me about it over the last few months – including one very helpful one from a LUFA fan with some great insights into how best to fix it – I’ve gone ahead and made a second public Git mirror with the imported layout corrected. This new repository will be switched over to become the official mirror soon once I’m satisfied nothing has broken, with the existing mirror still being deprecated but still maintained for those who depend on it. The new GIT mirror has the tags properly added as read Git tags, and this now results in downloads of the current HEAD of LUFA being around 5mb rather than the >30mb it used to be.

So that’s the latest LUFA news.