« Back to home

The Race Timing Project - Some (hard) Lessons Learned

As I mentioned in a previous post I've been working on a small scale commercial project in collaboration with a friend, unlike most of the other projects I talk about on the site this one will be manufactured in quantity (for small quantities at least) and needed to be designed for end users who expect to be able to just use the device and are not the least bit interested in modifying or hacking it.

I had a number of additional things I wanted to explore with this project, above and beyond simply making a working device:

  1. Use it as a test bed for the real world use of the Clixx prototyping system. Up until now I had simply been designing and using Clixx peripherals in a testing environment - this was a good chance to put them to use developing an actual project.
  2. Designing a project for eventual production has some additional restrictions which make it a bit more challenging. Component cost, availability and ease of assembly all come into play and will influence the final design. In my real work environment many of these details are handled by a separate team - this project would give me exposure to the sort of decisions that need to be made in the design stage to satisfy these requirements.
  3. I have been toying with the idea of running one or more Kickstarter campaigns for various projects. Although hard numbers seem to be difficult to come by there seems to be plenty of anecdotal evidence of hardware projects delivering faulty products, very late or not at all. If I do run a Kickstarter I would really like to avoid those issues and be able to deliver what was promised and deliver it on time. This project will help identify the kind of issues that need to be accounted for to achieve that but with a lower risk to third parties.

    The project isn't complete yet (although it should have been) but I have learnt quite a bit through what I have done so far - this post explores those issues (both good and bad) and concludes with some potential ways to improve future projects. If you are thinking of making your own hardware in small production runs or through a Kickstarter campaign this is well worth a read - it could help you avoid a lot of problems in the long run.

Project Overview

The project itself is very simple - all it needs to do is measure the duration of a single individual in a competitive event. As sold the product will be used for timing school swimming events but it could be used for any event that involves multiple competitors.

There are three components to the system:

  1. The starter pistol. This device signals the start of the race and provides audio and visual cues to the competitors (a bang and a flash essentially).
  2. The timer units. These are handheld devices which perform the actual timing (down to 1/100s accuracy). The contain an LCD screen to provide some feedback to the operator about the current time, the lane number being timed and the current elapsed time for the race. A pushbutton is used to signal the end of a lap.
  3. A base station. This is a USB device that connects to the PC running the competition management software and allows it to monitor and control the other devices.

    The components communicate over a radio frequency network (not necessarily WiFi although that was an option being considered) with a competition management server running on a laptop or PC coordinating the interaction and keeping everything in sync. My part of the project only relates to the hardware devices and their associated firmware.

    In terms of production the plan is to build the devices on demand (they are sold in a set of 12 timers, 1 starter pistol and 1 base station - so 14 devices per order). My friend will be building them himself (they won't be produced in bulk by a fabrication house) so the design has to be easy to put together by hand for someone with fairly basic soldering skills.

    For casing requirements the plan is to use 3D printable cases. I'll print the cases for the initial set on my own printer, longer term we will use a service like Shapeways and just order the number of cases required.

    The target date for the first deliveries is the 31st of January, 2014 - we will need two completed sets ready to go by that stage. Usable versions will be needed before that date though - a handful of prototypes for field testing and later at least one set to do live demonstrations to potential customers.

    We first started working on this project about 10 weeks ago and there are about 4 weeks left before the first completed products are due to be delivered to the customers - at this stage I am not going to be able to meet that deadline.

What's Gone Right

To start things off on a positive note I'll look at some of the things that have been successful.

Clixx - An Excellent Prototyping Platform.

I've invested a fair bit of time and effort in the Clixx system over the past year and, apart from being a lot of fun, it has paid off handsomely in this project.

Clixx Docks

I used my modular docking stations connected to a central processor board to build up the initial prototypes. There were only a handful of tabs needed - a monitored power board, an adaptor for a Nokia 3110 LCD display, a wireless transceiver tab and finally, a simple IO tab with a push button and a buzzer on it.

I'd already created most of the tabs for previous projects so being able to re-use them help saved a lot of time and new modules I created for this project will be re-used in future ones as well.

Keeping to the Clixx format made it a lot easier to experiment with different solutions to a problem. The best example of this is the radio transceiver - I experimented with a range of different technologies for this - starting with the simplistic 433MHz remote controls up to full WiFi with a CC3000 802.11b/g module before finally settling on a NRF24L01 system for the final design. Migrating to a different technology was simply a matter of creating an appropriate Clixx tab and slotting it in to the dock - a few minor changes to the code and I was ready to test - no need to make major redesigns or layout changes to the circuit.

Overall the Clixx system has proven to be flexible, robust and very easy to use. I highly recommend it for the prototyping phase of any projects you have in mind.

Arduino as a Platform

There were a number of microcontrollers that would have been suitable for use in the project, about the only physical limitation I had was that the CPU had to be available in DIP format to simplify construction by hand. I had considered basing it around an 18 pin mid-range PIC chip (the 16F1827) to help reduce the board space required but selected an ATmega168 for the final design.

Clixx and Babyduino

The main reason behind this was to leverage the wide range of support available for the Arduino platform - the IDE, libraries and development tools. My partner in this project is an accomplished software developer but has limited exposure to embedded systems - making the device Arduino compatible makes the learning curve for him a gentle slope rather than a cliff of complexity.

You don't need to use an actual Arduino device to be able to use the tools and libraries available - it is extremely easy to build a very simple Arduino compatible system around a bare AVR chip. I used my Babyduino board as the main controller and developed the firmware using the Arduino IDE which now allows him to make modifications as he sees fit and be able to compile and upload new firmware all through the IDE with a minimum of extra tools required.

The popularity of the Arduino system means that there is no shortage of sample code available for him to look at to find out how to add extra features as he sees fit.

In terms of production all that is required is a cheap USB AVR programmer (with a suitable adaptor) to burn a bootloader on to the chip. I included a simple FTDI adaptor in the design to allow uploading or updating of the firmware over a simple serial interface. The FTDI port also allows debugging and diagnostic output with nothing more than some simple terminal software.

Despite the fact that the ATmega devices may not be the best solution hardware wise for a particular project the available infrastructure that has built up around the Arduino platform often mitigates this and has a lot of additional benefits that few other microcontroller platforms can offer.

What's Caused Problems

Of course the project hasn't gone as smoothly as I had hoped. At the time of writing there is a high risk that the project won't be ready by the deadline we aimed for. For a personal project this isn't really a problem but for a commercial design it's not at all good. Luckily we haven't taken any money for deliveries yet and we have a revised schedule in place that should keep all interested parties happy.

The majority of problems have been environmental rather than technical and I think this is a similar experience to what many hardware based Kickstarter campaigns face.

Component Availability and Delivery Times

I tend to buy a lot of components from eBay, usually from distributors based in China. The prices are great and if you take the time to look at the sellers history before ordering you can be assured that you products will turn up and will be in working order when you get them.

The downside is that it can take a while for them to arrive and it's not very predictable - I've had delivery times (to Australia) ranging from four days up to four weeks. If this is a cool new part you want to play with it's just a minor frustration but when you need parts to do a production run it can be a killer. Dealing with a four week lead time makes it very, very hard to schedule production runs.

Another downside is component availability - the suppliers on eBay tend to sell seconds or excess production, there is no guarantee that the supplier you used last time will have the same items available the next time around. In fact there is no guarantee that the items will be available at all at the time you need them. This introduces another risk to your production planning.

The final issue is consistency of the components. There is a risk that you will get faulty components (although that hasn't happened to me yet) but the larger and more common problem is that the components are not exactly the same as each other. The Nokia LCD devices used in this project are a good example - I've ordered three sets of them from different suppliers - each set is slightly different. The differences range from cosmetic (the brightness of the backlight for example), physical (one set has a slightly different sized casing to the other two which means the case I designed for the product is slightly too small for it) to electrical (all three sets have different pinouts, the pins are all there - just in a different order). These issues can all be worked around but add unnecessary complexity to the final design.

If you are planning to mass produce your project you need to take this into consideration. The best solution (and the one we've gone with) is to use a local commercial supplier for all the components we can. There is an increased cost but we have guaranteed delivery times and consistency in the components.

Identify High Risk Components.

The design of the timers was fairly straightforward and the majority of the components in it were things I had worked with before. The only part of the design with a question mark over it was the radio interface. Unfortunately that was the part of the design I left to last.

My reasoning at the time seemed sound - get as much working as possible so we could see how the system fit together and then plug in the radio component at the end. This would allow my collaborator to work with a mostly working system and give our potential customers the opportunity to see a version of the product earlier rather than later.

The problem was that differences in the various radio modules had flow on effects to the rest of the design. The size and antennae placement on the modules determined where it needed to be placed on the PCB as well as changing the requirements for the casing which delayed that process considerably. The complexity of the software required to implement the networking stack varied greatly and eventually resulted in stepping up from an ATmega8 to an ATmega168 to get the additional flash memory needed for the code. The additional RAM requirements for the network stack meant that other parts of the software had to be rewritten to make more efficient use of the available RAM as well. The flow on changes meant a lot of additional work was required above and beyond simply changing the module itself.

The take away point from all of this is simple - identify the risky components and find a solution for them first. Once the hardware is finalised the software can easily be changed as needed, any change in hardware can have dramatic effects on the entire system and greatly multiply the amount of work required to implement the change.

Building Portable Prototypes

Field testing the design required prototypes that were robust enough to be portable. Although Clixx is great for building up prototypes on the workbench the end result is not something that can be carried around easily and for field testing that is something that is needed. For full testing we need at least three of them to verify the radio range and the network protocol code.

I usually build up more permanent circuits on stripboard and that was my plan for this one. The trouble with stripboard is that it is not very scalable - a small increase in circuit complexity results in a far more complex board layout and the amount of time it takes to build and test the resulting design.

Rushing matters doesn't help either so the additional time pressures I covered above didn't make things easier - I'm actually still in the process of building some usable and portable prototypes more than three weeks after I had planned to have them ready.

PCB Design

It would have been better to make (or have made) a PCB to use as the prototype. Although PCB fabrication services are relatively cheap there is a time delay between ordering and delivery (usually about 3 weeks for the service I use). By this stage other delays had accumulated so much that that time simply wasn't available. I started to build prototypes on stripboard but it was soon evident that the time it took to build and create a single working prototype wasn't scalable - there was no way I could build them in the required numbers with the time I had available to me. Eventually I designed a simple single sided PCB that I could etch at home and I'm in the process of building the prototypes on them. This has sped up the process considerably but it's still not exactly what you would call quick.

Lessons Learned

There are a number of things I would do differently if I was starting the project now (and I will be applying this to future projects with any commercial aspect to them).

Clear Your Plate

At the time I took on the project I already had a number of other projects already in progress and prior commitments that I needed to meet. What I should have done was clear these away first and then dedicate myself to the commercial venture. Working on them in tandem seemed like a good idea at the time and it looked like the most efficient way to allocate my time. What actually happened was that delays in one project could introduce delays in another - these were essentially unpredictable and lead to scheduling chaos.

Make Pessimistic Predictions

Like many other developers I have a tendency to be overly optimistic in my schedules. You really need to take into account the worst case scenario and adjust your estimates as time goes on.

Another big problem I ran into was overestimating the time I had available to work on the project. Because I would be working on it over the Christmas holiday period I assumed I would have additional time to devote to the project that I would normally be spending at work - unfortunately family and social commitments not only took up all that spare time they seemed to take up even more than a normal workday would. I wasn't willing to become a total recluse and a family pariah for this particular project so far less work got done than I originally anticipated over this period.

Schedule Tasks to Match Resources

I have to carefully allocate my tasks to certain times. While I can do software development at any time outside of normal work hours my living situation (I'm in an apartment) limits any soldering, etching or use of power tools to the weekends. Building up the first prototypes using Clixx was fairly rapid - I could work on that every day. Building new Clixx boards or making modifications to existing ones was limited to the weekend and if I reached a stage in the middle of the week where I needed to do that the remainder of the week days were lost to the project. On the other hand there were weekends that I spent working on firmware for the project and didn't do any hardware development.

With a bit more planning I could have devoted the weekends to preparing all the hardware I needed for the next few weeks and kept the hardware side ahead of the most optimistic software schedule.

Get Assembled Prototypes Up and Running Early

Once the hardware design is working you should start making your assembled prototypes (as in components soldered to PCB's) put together as soon as possible. This project involved interaction between the devices so I needed more than one in order to do full testing but even if that isn't the case you should assemble more than one and test on all of them. There are a couple of good reasons for this - firstly it allows you to test for and handle variations between components and secondly it allows you to pass the device to others for testing. Never, ever depend solely on your own testing - you need someone who doesn't already know how they should use the device to use it.

Clixx was an excellent tool to get the initial hardware design up and working, once I got to that stage I should have stopped and built a few copies on PCB straight away. Unfortunately it was quicker and easier to simply put together more copies with Clixx - a fake time saver that turned around and delayed the schedule when I could least afford it.

Don't Take Money Until the Assembled Prototype Works

Luckily I didn't fall into this trap - we were aiming for a cash on delivery model from the start so we aren't left explaining to customers why they have to wait an extra month for something they have already paid for. I think a lot of Kickstarter projects fall into this trap - they are asking for money for what backers think is a working device while the prototype is still in development. Delays and costs accumulate and the project never makes it to manufacturing.

If you at least have working prototypes in the hands of a number of people the future issues you face are more related to manufacturing rather than design. These can be a bit easier to handle, and are certainly easier to explain to your backers.

Conclusion

As the project stands we can recover from the delays that have been experienced so far and the learning experience has been invaluable. I know that I certainly won't be taking anything to Kickstarter any time soon, not until I have at least one other project like this under my belt.