Lessons Learned

I’m back from my cruise – there’s truly nothing better than kicking around on a deck chair for 10 days drinking Mohitos and reading a pile of good books. I’ve now got a scant two days before I start at Blackmagic Design. I’m truly excited, exhilarated, anxious and terrified, all at once. This is the start of a new chapter, and hope the narrative winds up being another happy one like my time at Atmel.

So, it’s time for a tiny bit more reflection on the last 18 months. I think it’s important to get it down here just so I can organize my thoughts, and ensure I take what I’ve learned to heart. Here’s what I think – hopefully these will not be huge revelations to most folk.

 

Know what you want to build.
Engineers love to create, but as any engineer will tell you, if you don’t specify exactly what you want, you’ll end up with a dozen different implementations. It’s OK to not have every last detail on paper, but at a minimum a Product Requirement Document should be written and a Minimum Viable Product sheet should be created and agreed upon, to stop last minute “oh, why didn’t you include X” or “but I thought it would work like Y” situations.

The first pass is not the last.
I love to extend code, constantly making minor refactors while I fix bugs or add features. But a big lesson is that some times a clean break really is needed; the first design you come up with might not necessarily fit all that well a year later as product requirements constantly change. Don’t be afraid to redesign and start again once the tech debt starts to build up, but make sure you do it in parallel to ensure you can continue to push out updated in the meantime. Once the two versions reach pseudo-feature parity, develop new features on both versions for a short while to get a feel for the improvements that have been made, and to give you a redundant implementation for testing until the old version is dropped.

Don’t micromanage. Defer.
If you’re a manager, trust you have hired the right people to build what you’re after. Don’t hire skilled engineers, just to treat them as meat robots to great frustration by everyone. With a good team managers should focus on waving the baton, not strumming the strings. Tool choices should be made on technical merits (with a little fluidity) – don’t force a tool on a team (other than Git, of course) that doesn’t suit the job, just because you’re familiar with it. Ensure that a common minimal “supported” tool-set is agreed upon so that new engineers can get started more easily, but don’t force editors, compilers, diffing tools, etc. on others based on your own prejudices. Except no Eclipse. Ever.

Design for test. Automate.
This was a big one; I’ve learned much from some very skilled people on how to design for test; not just unit tests but also integration tests and automated physical tests. Testing is boring, so automate as much as you can. Ensure you have repeatable builds, a build server and a good test plan you strictly adhere to before each internal and public release.

Automated analysis is your friend.
There’s lots of great free tools around for checking your code – CPPCheck, scan-build (part of Clang), valgrind and more. Get to know at least some of them and use them regularly. No matter how good you think you are, you WILL make errors that these tools will catch.

Hire good people. Put them in positions that suit their skills, but don’t be afraid to let them branch out.
A good engineer likes to learn, and technology is a wondrous thing. If a Ruby programmer wants to have a go at firmware let them, or if a Objective-C programmer wants to see what this whole “programming” thing is all about by all means give them a look outside the locked basement for the day. Having cross-team expertise is a good asset and can help reduce integration issues between teams. Just make sure that any work done is properly scrutinized before it is merged and deployed, and that the engineer’s primary focus remains on their original job.

Engineers are not managers, and vice-versa.
This isn’t a hard rule, but so far it’s been almost universally true. It’s a rare engineer that makes a good manager and vice-versa, and chances are you haven’t found an exception to the rule. Don’t mistake engineering ability for the ability to mange people, as people are stochastic things much like raw NAND flash and are far more volatile.

Treat everyone kindly.
This should go without saying, but I now know better. Do this, or lose your entire engineering team in under a year. And make sure you treat them kindly everywhere; word gets around surprisingly fast, and you don’t want to be the employee everyone wants to throw under a (literal) bus because of those things you said about them you didn’t think they’d find out about. Respect is earned, but easily lost, and once lost becomes exponentially more difficult to regain.

Encapsulate business logic.
Copy-paste is bad when it comes to code. Don’t repeat yourself, as the old saying goes, but also abstract. Are a lot of your products similar? Do you intend on swapping out the underlying platform later on? Build simple abstractions and encapsulate all your business logic together in a reusable, retargetable library. Doing so will allow you to build emulators for your product (scalability testing, integration testing, memory analysis, yay!) but also will reduce the workload later on when you need to switch chip vendors or hardware out, as you can just alter the platform specific code leaving the abstract core intact.

DOCUMENT BUSINESS LOGIC.
There’s nothing worse than encountering “strange” logic in old code, the purposes of which are lost to time immemorial (or engineers that abandoned ship because they literally couldn’t take it any more). Without documentation on the reasons behind business logic, you’re merely building a “do what I mean” machine rather than a coherent piece of technology. Documentation shouldn’t be slapped on at the last minute; it should be done by the implementing engineer at the time of, or at least very soon after, the actual implementation is made.

Support arguments with evidence. Don’t force bad decisions on people just because you “have a feeling”.
We’re engineers, not witch doctors. If you think something is better, give a good technical argument as to why, and if possible back it up with numbers. “But that sounds slower” is a poor argument – without evidence your gut is often more wrong than right. Cache locality, internal bus speeds, network latencies and other factors can often surprise even the best of us.

Do what needs to be done, but maintain a work/life balance.
No matter how awesome or fun something is to work on, don’t routinely do it on your free time. Work/life balance is important, and believe me your efforts will not be appreciated later on, no matter how useful the work turns out to be. If it turns out to be really useful you can look forward to either having it appropriated by those who didn’t want it done on company time in the first place, or an increased workload in the future as your weekend time becomes factored into product schedules.

Use proven technologies.
Yes, new and shiny is great, but a successful company is – with the exception of Optus – one that has a product that is functional. Don’t select hardware based on business strategies or cost to the detriment of the product; saving 20 cents on the BOM to end up with a non-functional or only partially functional product is a fool’s errand. For the initial version at least select parts and software that have a proven track record, and only swap them out in future revision to reduce costs once you already have something successful. A consumer might not be happy with paying $5 more for your fabulous doohickey, but they’ll be much more unhappy if that slightly cheaper Frobulator 9000 doesn’t work the way they expected.

Admit mistakes.
Not everyone is right all the time, and everyone makes a mistake from time to time (sometimes blowing up previously perfectly good equipment, *cough*). Own up to it, and suggest a way to resolve it. Doing so will help earn the respect of your peers, and bring the team together towards a common goal. Don’t hide mistakes or try to cover them up; people will know, and they will not help or trust you in the future. Similarly, it’s OK not to know something, so admit it and try to learn rather than making up junk on the spot. Finally, don’t spend all your efforts pointing out the mistakes of others rather than getting stuff done.

 

 

Comments: 7

Leave a reply »

 
 
 

Hi Dean.

Some people take a whole lifetime to complete your “10 day Wisdom Cruise”. Well done. It would be worth delivering to your Alma Mater.

Cheers,

Ross

 

Dean, you’re a smart guy with good wisdom. I just stumbled onto this blog by finding your LUPA links in the Ardupilot github repo. I’ll have to follow you and your work more closely. I also glanced at your LinkedIn profile. Feel free to look at mine.

Perhaps someday we can work on a home project together. Someday I’d like to create a commercial product in my free time.

Embedded is powerful; it is fascinating. Microcontrollers are really cool. I’m about to make the big step over from 8-bit AVR to 32-bit ARM core. I looked at the datasheets. It’s a scary process. Any advice you have is welcome.

Here’s the compilation of my 8-bit background thus far (referring to the links at the bottom of this article…it kind of shows a map of what I’ve learned about them, as I’ve implemented various projects and things: http://www.electricrcaircraftguy.com/2014/01/the-power-of-arduino.html).

Talk to you later,

I look forward to seeing what you do.

Gabriel Staples
http://www.ElectricRCAircraftGuy.com/

 

Dean, I know this sounds like spam but it’s not (just read my last post…should be obvious).

Anyway, this post is so good I just came back to share it on Google+ or Facebook or something….and you don’t have any share buttons. 🙁 I hope you add some. It’s good for traffic.

One easy way is to use addthis.com. They make adding sharing buttons super easy.

So, meanwhile, I’ll just copy-paste the link.

 

I’ve always hated those silly share buttons – I thought everyone else thought the same, and just manually copy-pasted the links they wanted to share into their client/webapp of choice. If there’s enough interest I can see about adding them to my site.

– Dean

 

Hi,
I am an electronics engineer. I had just started working in an startup electronics company.
I was working on Arduino Bootloaders and headed to your website.
I really liked your thoughts.
Thanks for the work that you have done.
Aman

 

Hi Dean,

thank you for kindly sharing your learned lessons, as well as the other contributions you have done (tutorials, etc.). It’s really an inspiration to me.

I’ve also experienced the importance of nailing down the system requirements of a project.

In one of my past project the requirements were clearly written, but not nailed down, so after the development results were presented, one person who was just remotely involved, but actually appeared to be a sort of stakeholder, wanted to have different design and partly different functionality. What a bummer!

So I learned the importance of reviewing *and* to nail down the requirements with the *right* people.

I’d like to share another lesson I’ve learned in another project:
Make/help the customer to prioritize the requirements.
Some of the requirements, which sounded like “must”-requirements were actually not even “could”- or “nice-to-have”-requirements, but was actually just an idea, not a requirement at all! What a surprise! And efforts in vain…

Cheers,
Per

 

Dean Camera,
First of all, thank you for clarifying my view on the matter at LUFA.
I came here through a friend from Indonesian developer, I am a system analyst and I come every day finding out about embedded projects and IOT.
However here the lack of native material reduces the apredizado.

I would if possible, a note to understand the joy controller development process.

 

Leave a Reply

 
(will not be published)
 
 
Comment
 
 

 

Vital Stats

  • 35 Years Old
  • Australian
  • Lover of embedded systems
  • Firmware engineer
  • Self-Proclaimed Geek

Latest Blog Posts

RSS