Now is a great time to talk about estimation. Software is notorious for being unpredictable and easy to blow out the budget. In a previous blog, I talked about why software costs so much, in this blog I want to talk about how we can get better at estimating the cost.

This is more important than ever as the world spends the next few years reaping the results of slamming on the brakes to stop the spread of covid-19.

Over the coming years, there will still be money being invested, but perhaps there will only be one round of funding instead of two, or perhaps the investment just needs to last a little longer, or perhaps there will be just a little bit less. Either way, its important that as software experts, we are able to predict what software development will cost so that our stakeholders can make good decisions.

‍

“An expert is a person who has few new ideas; a beginner is a person with many.”

‍

This awesome quote from Albert Einstein applies in any trade, if I want a new roof — then I want to talk to someone who has done it before many times and can tell me exactly what needs to be done. Its not just about the cost, its about having confidence that they know what they are doing and they aren’t going to be making it up as they go.

It also applies to pandemics. As I write this most of the world is in a lockdown and we’re all asking how long it will last. Who should we be listening to — our neighbours? Journalists with opinions? Or the experts … ? (ok, bad example because no-one seems to know)

‍

And it applies to software. The ability to predict, is the mark of an expert — a person who has few new ideas because they’ve done this before.

‍

Sometimes when we are delivering a discovery report, it can feel embarrassing to tell the customer (and charge them for) things that are plainly obvious to us. But our customers come to us for a discovery because we’ve done this before — they have their field of expertise and we have ours. Together, we are able to make a potent team — but the ideas they bring in their field of expertise are new to us, just as the ideas we bring are new to them.

As a product team, the goal isn’t to be better at estimating — its to create a culture of continual improvement, so you are always growing your expertise at what you do and being able to apply that expertise when looking at a new piece of work.

‍

Why Estimate?

Before we look at how to estimate, let’s look at why we should estimate. Other than the fact that our stakeholders need good estimates so they can make good decisions, as a software professional — why should estimates matter to you?

‍

Learning to estimate and measure your time against estimates will help you to grow as an expert.

‍

You are responsible for your work

As a team and individually, you are responsible for delivering software that solves real problems and makes users awesome. Before you can build customer driven long lived solutions that are both sturdy and maintainable you need to plan and design software to be this way. Otherwise it won’t be. The planning and the design (UI and back-end) is a vital part of your job.

If you feel like you’re unable to estimate a piece of work, then I would suggest you haven’t planned it well enough and if you are diving in to a piece of work that isn’t estimated, then I would suggest you aren’t taking responsibility for the outcome.

‍

Keep your batch size small

This is a pretty well understood principle of software development and is a key driver in managing the overall cost of software development. We know that smaller batches are faster and cheaper to get through the full lifecycle, they have lower risk, they allow us to iterate and learn sooner. Having smaller batches also helps us to track progress against our estimates and communicate to our stakeholders early.

‍

Know when work-in-progress is blowing up

Keeping your batch size small also has the benefit of being able to see when work is blowing up. We measure hours per story point and as soon as they go above 6, we flag the user story so the team can do something about it. Its time to have a discussion about why the estimate was insufficient and adjust the estimate or adjust the work being done. This can result in a user story being broken down, or the developer seeking help on a new technology, or the team just having a better understanding of the size and complexity of work being done and being able to communicate that early to stakeholders.

‍

How to estimate

Whether you are estimating a goal or project vs a user story, there are four principles to follow:

1. Divide and conquer (i.e. user stories, acceptance criteria)
2. Estimates are relative (i.e. story points)
3. Know your unknowns (i.e. complexity)
4. Zoom in for focus (i.e. epics)

‍

Divide and conquer

Our brain is wired to think, “I could do that in a weekend” or “I could do that in a couple of weeks” — so when asked for an estimate you’ll find that everything takes 2 days or 2 weeks. When giving an estimate, break the work down into a list then estimate the list. The list could be of anything — screens, database tables, user steps, acceptance criteria. If you’re estimating a goal, break it down into smaller objectives, then break each objective down into user stories, then each user story down into tasks. Eventually you’ll get to something that actually can be done in 1–2 days and based on that you can calculate the overall estimate.

‍

Estimates are relative

We instinctively compare things by default, but in order to estimate based on your experience you need to be able to track your actual time against estimates and you need to appreciate that everything looks smaller when its in the distance. When we are planning a sprint, Definition of Ready is an important way to make sure a user story is ready for the team to work on. It's also an important way to package up a unit of work and therefore make it easier to compare with other work and therefore easier to estimate and easier to learn from. Factor in your bias for simplicity and assume from your current perspective, you can only see half to two thirds of the solution.

For example take a look at this photo of the New York skyline. You might be tempted to estimate that the One World Trade Centre is about 300m tall compared to 17 State St. Its actually over 3 times the height.

‍

‍

Know your unknowns

This is a similar principle to the cone of uncertainty (below), but it matters even when we are right down the delivery end of the cone and estimating a user story. We like to look at the complexity of the story and we will assign the story a size and complexity. The three complexities we use are:

• Simple — we’ve done this before, its well understood.
• Tricky — there are lots of moving parts and it would be easy to miss something.
• Complex — there are unknowns involved which may be new technology, or unique UI

The idea is that not all user stories are equal and the team’s approach to delivering each should differ based on the complexity.

• Simple — just get on and do it
• Tricky — acceptance criteria and following a process are key
• Complex — when the story is picked up, more planning and discovery is needed and this should involve multiple people. Following the planning and discovery, the story should either be broken down into smaller stories, or it should be able to be downgraded to simple or tricky.

This also emphasises the need to involve the whole team in the estimation process and there has been a lot written about great ways to do this (i.e. planning poker). The goal is to get different perspectives, to leverage different people’s experience and to look out for differences of opinion, because this is where we will identify our blind spots.

This idea is further illustrated in the Johari window. By comparing what you know, to what the industry knows, you attempt to reveal your blindspots, hidden knowledge and unknowns. All of these can increase the risk in your estimates.

‍

‍

Zoom in for focus

This principle relates to estimating something bigger, like a goal or an epic. You’ve already broken it down into user stories (divide and conquer) and you’ve estimated each story. The question now is what gaps do you have? As I discussed in my blog Mind the gap, software is complex and its not uncommon to break it down into smaller parts, then pull the small parts together and realise it doesn’t look or work like what you had in mind. Zooming in is just a way to focus on a single objective.

‍

Communication

When it comes to communicating an estimate, there are three things to consider:

‍‍

Language matters

The cone of uncertainty is a well known idea that the only way to get more accurate estimates is to do work and based on this, we should be able to communicate the uncertainty in our estimates depending on where in the lifecycle we are.

‍

‍

The problem comes when we use the same language. We provide a high level estimate, a detailed estimate, a final estimate, we re-estimate. We estimate on the back of a napkin and we estimate with a whole team playing planning poker. Can you blame a stakeholder for hearing the word estimate and not appreciating the certainty.

So language matters and we should use different words to represent different types of estimate:

• Pace: If you want to deliver a beta in 6 months, you’ll need to move at a pace of 200 hours per sprint.
• Size: We’ve sized up the ‘Onboarding’ epic to be 2 weeks.
• Estimate: We estimate the ‘Welcome Screen’ user story to be 3 story points

‍

Precision matters

When people hear an estimate, they inherently assume some margin of error, so the precision in which you give your estimate will dictate the margin of error. For example:

• It will take 2 weeks …plus or minus a week?
• It will take 70 hours …plus or minus 10 hours?
• It will take 65 hours …plus or minus 5 hours?
• It will take 68 hours … plus or minus 2 hours?
• It will take 68.5 hours …plus or minus 0.5 hours?
• It will take 68.9 hours …plus or minus 0.1 hours?

‍

Summary

As a product team, the goal isn’t to be better at estimating — its to create a culture of continual improvement, so you are always growing your expertise at what you do and being able to apply that expertise when looking at a new piece of work. And when we start to achieve this, then reality will take us by surprise less often because a beginner sees a simple way forward and an expert knows how to make the way forward simple.

‍

‍

‍