The Psychology of Overconfidence

Dan Milstein has a nice write up of his thoughts on estimation: Coding, Fast and Slow: Developers and the Psychology of Overconfidence


To try to summarize,

  1. "Writing Software = Learning Something You Don’t Know When You Start"
  2. We are systematically, provably, overconfident.
  3. That said, we can get decent at estimating things that will take ~0-4 hours.
  4. But there's no way to get good at quick-estimation of things > 8 hours, because you need really quick feedback loops to hone this skill.
  5. Sadly adding up 100 4 hour tasks does not equal an accurate estimate of a large project.


So I totally agree that there's value to using 'System I' to make quick gut check estimates on small things. And I agree that spending very much time deeply estimating a project with 'System II' is pretty useless.

What I want to try to defend is the "2 week", "1 month" size gut check estimates. I'm definitely not going to argue that they're something to bet money on, but I think that used properly they can be useful.


Decoding the proverbial "Two Weeks"

So here's my secret decoder ring for programmer estimates. "Two weeks" really means "In two weeks, I'll be able to tell you what is really happening". Maybe 50% of the time that's because it you will have actual working software. But the rest of the time it means you'll have an engineer who now better understands why this will take another month. Or six. Or simply a day of cleanup.

"Two weeks" really means: "In two weeks, I'll be able to tell you what is really happening".

So how can this help you?

I may doubt an engineer's confidence in their estimation, but I feel a lot better about relying on their hatred of inefficiency. And I think that's part of what you're getting with a "2 week estimate." You're getting "Anything less than 2 weeks is going to be an inefficient use of my time."

So how do we translate a developer saying "2 months"? This is big. So big that it's going to take me two months to figure out the scope of what this thing is. That's right, you want an accurate estimate of Project X I've got news for you. If somebody says "2 months" they just told you it will take "2 months" to have an accurate estimate.

Is there any good news?

Yes! If we change what we hear I think estimates can actually speed up process and reduce churn from shifting priorities. Take the developer's "2 weeks" then don't bug them for 2 weeks. If you are going to ask them a question about schedule reduce it to this simple one: "what percent sure are you that this will be done". 80% means things are good. A seasoned developer will say 80% even if the code is written and tested and sales has signed off, and the CEO loves it, but marketing just wants to take "a quick look at the copy". Because a sesoned developer has seen 20% of their "done" projects still slip right here.

If the developer says they're only 50% sure you can start planning the reprioritizatuon meeting. But leave the engineers doing what they're doing. Make sure they're focussed on the totality of the problem. And at the end of their proverbial "two weeks" you will have a concrete and legitimate estimation about your project.


How to Frame "2 Week"+ Projects
So if we decide to be honest with ourselves and re-defined the nature of "two weeks" projects, is that the best we can do? Actually I think we can reap even greater rewards by clearly framing projects for developers in this light. If we say "You said '2 weeks'? Ok, do project X, you have 2 weeks" we are likely to get: a whole bunch of code, slapped together in the last half of the second week, a feature that appears to work, but has an unknown number of bugs and has not been user tested and may or may not really be on the right track.

However if we say "Work on project X, you have 2 weeks to report back to me what it's going to take to ship this with 95% confidence that it's a big win for customers". Well I think you get a really different result. I think you're going to have an engineer inclined to think critically about the problem, not about how to deliver "something" in "2 weeks". And what does that mean? Well if they're any good it means you're going to get a combination of design work, code spikes, feasability and a list of distinct manageable 4 hour tasks that aren't finished yet.

In my experience, estimates up to 2 developers & 2 weeks can be relatively accurate about getting 'something' shipped. But they absolutely require an "after-party" story to clean up. Baking this into the expectation from the outset can help engineers focus on the most critical bits first. Whether that's ensuring the API behaves properly, spiking out the critical path, or badgering the customer to figure out whether the feature is of any use at all.


In closing, I say to you "Go forth and shout your estimate from your hip!". (Just tell your PM what you really mean)





P.S. I should perhaps point out that this is all based on previous jobs. I haven't actually 'estimated' anything in the past 9 months at my new job. We just 'do' stuff. Crazy I know.