Estimating the size of any complex task in advance is difficult. Estimating the size of a complex task before it is properly defined, is even more difficult. Most consultancies of any description regularly have to provide both types of estimates, though the former is usually called a ‘quotation’
A ‘quotation’ from a consultancy is their commitment to what they will charge the client based on their complete understanding of the task. It of course is still effectively an estimate, albeit the task is fully defined so the estimation element is purely in the consultant’s prediction of the time the task will consume – attempting to anticipate the unanticipated – tasks that take longer than expected and/or the inevitable issues that are encountered along the way. Typically fixed price quotations are provided which means if a task does overrun, its the consultant’s cost to bear, not the client’s.
What we most commonly refer to as an estimate is the former, where costs are speculated in lieu of the full scope of the work. The estimator can lay out as many caveats as they like but the reality is, if their final quotation significantly differs from the estimate, questions of competency, integrity, or both – are likely to rear their heads and those caveats can be quickly forgotten.
Design Estimates are Necessary
For these reasons, no consultancy of any description really likes doing estimates – but they are often a necessary phase for both client and consultant.
For the client, they need an understanding that the development they are considering is commercially viable, prior to requesting or allocating a budget. If they plan to sell 100 a year at £100 each but the development will cost £100k, that’s 20 years of sales even at a 100% mark up to recover that cost.
For the consultant, initial comprehensive documentation such as a Design Proposal or a quotation, often involves a significant amount of cost and time investment so they are also looking to qualify the likelihood of this development realistically proceeding, especially in a world where many clients are hesitant to openly discuss available development budgets.
In our world, what we’re estimating the development effort of is electronic hardware, firmware/software, or both – but one is definitely less complex and risky, thus more likely to end up more accurate, than the other.
Hardware development estimation involves a combination of speculating the likely key components to be used and using past experience to speculate the effort involved in designing them into an electronic circuit. We can say ‘it will likely need a processor of X complexity which will likely have Y pins with Z complexity around length matching, placement, etc’ and so on for other key functions. We can look at past similar designs we’ve undertaken and the time functional sections involved – and add it all up.
The huge challenge in quoting any kind of firmware/software in lieu of a full functional specification is what you can observe from a datasheet or understanding of the operation of the final product can be the tip of the iceberg compared to what is happening beneath the surface.
A good analogy would be estimating the software involved in the entirety of the operation of a Smart TV by looking merely at the product box and the TV remote. We can indeed see a ‘Media player’ and a ‘Subtitle’ button so can consider the basics of achieving these functions.
‘Media player’ – what formats does it support (audio, photo, video?) – then which formats within each. Can it resize resolutions (stretch/pinch/zoom, etc, which ones), can it resync out of sync audio, can it update itself with new formats, does it support DRM.
‘Subtitles’ – Are these just in English, or multiple languages – if so, how many languages? Do they all use ASCII characters? Do they show at the top or the bottom, is this configurable? What colour is the text, does that change if the background is too similar? What font are they in, what size text, is this configurable?
Subtitles is a relatively simple function compared to most functions in complex devices but you can see from the above how, depending on the detail, this could end up multitudes of effort more than an estimation of a basic ‘subtitles’ function just seen from that remote control button.
What you’re effectively doing is just looking at top level functions with no understanding of how many sub-functions exist below it and how complex they are.
This method of estimating firmware/software from a basic functional understanding of what the final device must present gives a huge margin of error – we’ve seen scenarios in which the projected effort is double that original estimation – which is no good for the client who could have made decisions or allocated a budget based on this – and no good for the consultant whose professional integrity becomes under scrutiny though no fault of their own.
Different Approaches to Electronic Design Estimates
So what’s the solution? We’ve seen a few attempted solutions across various consultancies we’ve come across.
“No estimates” – a consultant could flatly refuse to speculate on any costs until they are in possession of the full specification. This protects the consultant in one respect, but this lack of willingness to provide even an indication may not be perceived well by the client.
“Consultancy phase” – this is where a client pays for the consultant to work with them to develop a specification sufficient that it can be formally quoted on. This can be unattractive to a client as they’re then having to spend money to potentially find out their development is commercially unviable at the end of it – though if the client is literally unable to get to the point of having a full specification on their own, often this is the only option.
“Caveats, caveats, caveats” – this is where most consultancies end up. They want to be helpful and engage openly with customers, but given those wide margins of error in estimations, managing expectations is critical. You will likely receive an estimate, potentially with a wide range of likely costs and listing all assumptions made when speculating those costs – then hoping that when the full firmware/software specification is ready – there’s no big surprises – as no one wants to have an estimate of £25k and be quoted £50k!