SlideShare a Scribd company logo
#NoEstimates
10 New Principles for
Software Development
(and testing!)
Vasco Duarte
@duarte_vasco
Tweet at:
#TAHelsinki
Vasco Duarte
@duarte_vasco
http://SoftwareDevelopmentToday.com
http://bit.ly/vasco_slideshare
Vasco.Duarte@oikosofy.com
http://NoEstimatesBook.com
#NoEstimates is for you if…
No estimates  - 10 new principles for testing
No estimates  - 10 new principles for testing
No estimates  - 10 new principles for testing
No estimates  - 10 new principles for testing
Principle #1
Trust your process, or change your
process
NumberofBugs
Timeline
Bug evolution in a non-agile project
Open
Closed
Submit
Development phase Desperately testing and fixing phase
Waterfall
Principle #2
Shorten the feedback cycle
Can estimates be accurate?
Accidental complication
Cost/Duration =
Essential Complication
x
Accidental Complication
No estimates  - 10 new principles for testing
Yet, some people still argue we
can be good at estimating… Let’s
look at the evidence
80% Late
or Failed
Source: Software Estimation by Steve McConnell
Reality: 80% is late or failed
Lets break that down
Chaos Report 1995
For the same combined challenged and impaired projects, over one-third also
experienced time overruns of 200 to 300%. The average overrun is 222% of
the original time estimate. For large companies, the average is 230%; for
medium companies, the average is 202%; and for small companies, the
average is 239%.
Time Overruns % of responses
Under 20% 13.9%
21 – 50% 18.3%
51 – 100% 20.0%
101 – 200% 35.5%
201 – 400% 11.2%
Over 400% 1.1%
~68% of
projects 51%
or more late!
Let’s continue to break that down
Chaos Report 2009
Average cost overrun: 45%
Average time overrun: 63%
Chaos Report 2011
Average time overrun: 63%
Just in case you don’t like the
CHAOS report
Source Gartner survey of project failure, 2012
Failure, means total
failure, not just late
Of the large systems that are comple
ted, about 66% experience schedule
delays & cost overrun
Source: “Project Management Tools and Software Failures and Successes” by Capers Jones
Crosstalk, the Journal of Defense Software Engineering
Traditional projects: 53% failed
or challenged
Source: Project success survey by Scott Ambler
Agile project: 40% failed of
challenged
Source: Project success survey by Scott Ambler
17 percent of large IT projects go so
badly that they can threaten the
very existence of the company
Source : McKinsey & Company in conjunction with the University of Oxford
Type of survey : Study on large scale IT Projects
Date : 2012
More data coming, sign-up at
NoEstimatesBook.com to receive
this report when available
Principle #3
Believe the data, not the estimates
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
Principle #4
Use alternatives to Estimate-driven
decision making
Comparison of 17 projects ending between 2001
and 2003. (Average: 62%)
Testing for value, a story…
Principle #5
Test for value first, then test for
functionality
Can #NoEstimates work in Real
Companies?
http://j.mp/NoEstimates-Book
Principle #6
Estimation is waste, reduce it’s
impact on your business
Principle #7
Measure progress only with
validated, running software
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
Source: Software Estimation by Steve McConnell
“Good”
estimates:
25% of
estimated
I AM GOING TO
GO AHEAD AND
ASK YOU TO
DELIVER 10
STORIES NEXT
SPRINT...
WTF!!!!!
!#%&!
0
1
2
3
4
5
6
7
8
9
10
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
velocity
Average=
LCL
UCL
Target
Actual, measured
throughput over 21
sprints
The 4 Estimates Dysfunctions
1. Estimate Bargaining
2. Internal Politics
3. Blame Shifting
4. Late Changes
5. Sunk Cost fallacy
Principle #8
The system where you work has
predictable outputs, learn to
understand the system
Wow! But I have a business to
run!
Is there a way do better than
that?
#NoEstimates delivers!
Counting Stories vs. Estimated
Story Points
Q: Which ”metric” is more
accurate when compared to
what actually happened in the
project?
A long project
24Sprints
Which metric predicted most
accurately the output of the
whole project?
a) After only the first 3 Sprints
b) After only the first 5 Sprints
Disclaimer...
This is only one project!
Find 21 more at:
http://bit.ly/NoEstimatesProjectsDB
After just 3 sprints
# of Stories predictive powerStory Points predictive power
The true output:
349,5 SPs
completed
The predicted
output: 418 SPs
completed
+20%
The true output:
228 Stories
The predicted
output: 220
Stories
-4%!
After just 5 sprints
# of Stories predictive powerStory Points predictive power
The true output:
349,5 SPs
completed
The predicted
output: 396 SPs
completed
+13%
The true output:
228 Stories
The predicted
output: 220
Stories
-4%!
Q: Which ”metric” is more
accurate when compared to
what actually happened in the
project?
But there is more...
What difference does a Story
Point make in a project that
used both Story Points and
#NoEstimates?
Next you will see the
forecasted release date when
using Story Points (values
1:3:5)
68
71 71 71 71 71 71 72 72 72 73 73
0 3 7 7 9 11 12 13
20 20 22 23
0
10
20
30
40
50
60
70
80
90
100
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on
20th October
2014
Next you will see the
forecasted release date when
using Story Points (values
1:2:3)
48
51 51 51 51 51 51 52 52 52 53 53
0 2 5 5 7 8 9 10
15 15 17 18
0
10
20
30
40
50
60
70
80
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on
14th October
2014
Next you will see the
forecasted release date when
#NoEstimates
(or, all stories = 1 story point)
28
31 31 31 31 31 31
32 32 32
33 33
0 1 3 3
5 5 6 7
10 10
12 13
0
10
20
30
40
50
60
1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7
Product Backlog Cumulative Flow Diagram
Remaining
Done
Linear (Remaining)
Linear (Done)
Release on 29th
September 2014
Conclusion...
All dates within 3 weeks of
each other in a 38 to 42 week
project (a margin of ~8%)
Data used with permission
from Bill Hanlon at Microsoft
”At that point, I stopped
thinking that estimating
was important.”
Bill Hanlon:
http://bit.ly/BHanlon
In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and
V.Y. Shen proposed that a good estimation
approach should provide estimates that are
within 25% of the actual results 75% of the time
--Steve McConnel, Software Estimation: Demystifying the Black Art
In this presentation you have seen examples that far outperform what
estimation specialists consider a ”good estimation”. In common they have
one way to look at work: #NoEstimates
#NoEstimates testimonial
The deviation between estimated and
actual velocity would have been
approximately 15% lower if we would have
used #NoEstimates.
We have analyzed data from 50 Sprints…
…at no time the story point based
estimation was better than #NoEstimates.
Principle #9
Don’t bet your company on poor
track record methods, use methods
with a proven track record
aka: Hope is a bad management
strategy
Carmen faces a very difficult situation…
The 7 Step Journey To NoEstimates
1. Start using Story Points
2. Stop estimating tasks
3. Limit the calendar duration for
Stories and Features
4. Reduce the number of
allowed estimates (say, 1,2,3
and 5 only)
5. Track your data
6. Use your data
7. Simply count the number of
stories
http://j.mp/NoEstimates-Book
http://j.mp/NoEstimates-Book
http://j.mp/NoEstimates-Book
Principle #10
The transformation starts with you…
Vasco Duarte
Vasco.Duarte@oikosofy.com

More Related Content

No estimates - 10 new principles for testing

  • 1. #NoEstimates 10 New Principles for Software Development (and testing!) Vasco Duarte @duarte_vasco Tweet at: #TAHelsinki
  • 8. Principle #1 Trust your process, or change your process
  • 9. NumberofBugs Timeline Bug evolution in a non-agile project Open Closed Submit Development phase Desperately testing and fixing phase Waterfall
  • 10. Principle #2 Shorten the feedback cycle
  • 11. Can estimates be accurate?
  • 15. Yet, some people still argue we can be good at estimating… Let’s look at the evidence
  • 16. 80% Late or Failed Source: Software Estimation by Steve McConnell Reality: 80% is late or failed
  • 17. Lets break that down Chaos Report 1995 For the same combined challenged and impaired projects, over one-third also experienced time overruns of 200 to 300%. The average overrun is 222% of the original time estimate. For large companies, the average is 230%; for medium companies, the average is 202%; and for small companies, the average is 239%. Time Overruns % of responses Under 20% 13.9% 21 – 50% 18.3% 51 – 100% 20.0% 101 – 200% 35.5% 201 – 400% 11.2% Over 400% 1.1% ~68% of projects 51% or more late!
  • 18. Let’s continue to break that down Chaos Report 2009 Average cost overrun: 45% Average time overrun: 63% Chaos Report 2011 Average time overrun: 63%
  • 19. Just in case you don’t like the CHAOS report
  • 20. Source Gartner survey of project failure, 2012 Failure, means total failure, not just late
  • 21. Of the large systems that are comple ted, about 66% experience schedule delays & cost overrun Source: “Project Management Tools and Software Failures and Successes” by Capers Jones Crosstalk, the Journal of Defense Software Engineering
  • 22. Traditional projects: 53% failed or challenged Source: Project success survey by Scott Ambler
  • 23. Agile project: 40% failed of challenged Source: Project success survey by Scott Ambler
  • 24. 17 percent of large IT projects go so badly that they can threaten the very existence of the company Source : McKinsey & Company in conjunction with the University of Oxford Type of survey : Study on large scale IT Projects Date : 2012
  • 25. More data coming, sign-up at NoEstimatesBook.com to receive this report when available
  • 26. Principle #3 Believe the data, not the estimates
  • 27. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art
  • 28. Principle #4 Use alternatives to Estimate-driven decision making
  • 29. Comparison of 17 projects ending between 2001 and 2003. (Average: 62%)
  • 30. Testing for value, a story…
  • 31. Principle #5 Test for value first, then test for functionality
  • 32. Can #NoEstimates work in Real Companies?
  • 34. Principle #6 Estimation is waste, reduce it’s impact on your business
  • 35. Principle #7 Measure progress only with validated, running software
  • 36. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art
  • 37. Source: Software Estimation by Steve McConnell “Good” estimates: 25% of estimated
  • 38. I AM GOING TO GO AHEAD AND ASK YOU TO DELIVER 10 STORIES NEXT SPRINT...
  • 40. 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 velocity Average= LCL UCL Target Actual, measured throughput over 21 sprints
  • 41. The 4 Estimates Dysfunctions 1. Estimate Bargaining 2. Internal Politics 3. Blame Shifting 4. Late Changes 5. Sunk Cost fallacy
  • 42. Principle #8 The system where you work has predictable outputs, learn to understand the system
  • 43. Wow! But I have a business to run! Is there a way do better than that?
  • 45. Counting Stories vs. Estimated Story Points Q: Which ”metric” is more accurate when compared to what actually happened in the project?
  • 47. Which metric predicted most accurately the output of the whole project? a) After only the first 3 Sprints b) After only the first 5 Sprints
  • 48. Disclaimer... This is only one project! Find 21 more at: http://bit.ly/NoEstimatesProjectsDB
  • 49. After just 3 sprints # of Stories predictive powerStory Points predictive power The true output: 349,5 SPs completed The predicted output: 418 SPs completed +20% The true output: 228 Stories The predicted output: 220 Stories -4%!
  • 50. After just 5 sprints # of Stories predictive powerStory Points predictive power The true output: 349,5 SPs completed The predicted output: 396 SPs completed +13% The true output: 228 Stories The predicted output: 220 Stories -4%!
  • 51. Q: Which ”metric” is more accurate when compared to what actually happened in the project?
  • 52. But there is more...
  • 53. What difference does a Story Point make in a project that used both Story Points and #NoEstimates?
  • 54. Next you will see the forecasted release date when using Story Points (values 1:3:5)
  • 55. 68 71 71 71 71 71 71 72 72 72 73 73 0 3 7 7 9 11 12 13 20 20 22 23 0 10 20 30 40 50 60 70 80 90 100 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 20th October 2014
  • 56. Next you will see the forecasted release date when using Story Points (values 1:2:3)
  • 57. 48 51 51 51 51 51 51 52 52 52 53 53 0 2 5 5 7 8 9 10 15 15 17 18 0 10 20 30 40 50 60 70 80 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 14th October 2014
  • 58. Next you will see the forecasted release date when #NoEstimates (or, all stories = 1 story point)
  • 59. 28 31 31 31 31 31 31 32 32 32 33 33 0 1 3 3 5 5 6 7 10 10 12 13 0 10 20 30 40 50 60 1/3 1/17 1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/10 10/24 11/7 Product Backlog Cumulative Flow Diagram Remaining Done Linear (Remaining) Linear (Done) Release on 29th September 2014
  • 61. All dates within 3 weeks of each other in a 38 to 42 week project (a margin of ~8%)
  • 62. Data used with permission from Bill Hanlon at Microsoft ”At that point, I stopped thinking that estimating was important.” Bill Hanlon: http://bit.ly/BHanlon
  • 63. In 1986, Profs. S.D. Conte, H.E. Dunsmoir, and V.Y. Shen proposed that a good estimation approach should provide estimates that are within 25% of the actual results 75% of the time --Steve McConnel, Software Estimation: Demystifying the Black Art In this presentation you have seen examples that far outperform what estimation specialists consider a ”good estimation”. In common they have one way to look at work: #NoEstimates
  • 64. #NoEstimates testimonial The deviation between estimated and actual velocity would have been approximately 15% lower if we would have used #NoEstimates. We have analyzed data from 50 Sprints… …at no time the story point based estimation was better than #NoEstimates.
  • 65. Principle #9 Don’t bet your company on poor track record methods, use methods with a proven track record aka: Hope is a bad management strategy
  • 66. Carmen faces a very difficult situation…
  • 67. The 7 Step Journey To NoEstimates 1. Start using Story Points 2. Stop estimating tasks 3. Limit the calendar duration for Stories and Features 4. Reduce the number of allowed estimates (say, 1,2,3 and 5 only) 5. Track your data 6. Use your data 7. Simply count the number of stories
  • 71. Principle #10 The transformation starts with you… Vasco Duarte Vasco.Duarte@oikosofy.com

Editor's Notes

  1. If you are bored with those endless estimation meetings
  2. You are bored and demotivated by the long, low-value meetings that end in endless discussions about the size of features that are not yet fully defined.
  3. If this is how it feels to present your estimates to your stakeholders
  4. If you want to focus on making customers happy, instead of producing even more software
  5. In 2014, JR Central reported that the Shinkansen's average delay from schedule per train was 54 seconds. This includes delays due to uncontrollable causes, such as natural disasters.[15] The record, in 1997, was 18 seconds. Their trick: Build a system that performs, don’t try to extract performance from an old system that was not built to perform.
  6. In a waterfall project no one is in control at the end (show bug curve at the end). Much overtime will be needed to get this product out, and that will be on your backs and on the backs of the product development group (be it software or other complex product). This typically leads to a pattern that some have called “Death March” (concentration camp picture), and that’s how it feels. It feels like we are marching to a concentration camp (rant about lives destroyed by this approach). You have a responsibility to avoid this pattern in your place of work, and you can do it. Here’s how.
  7. And this is only one of the reasons why Estimates don’t work. There are others.  Accidental Complication My friend JB Rainsberger talks about Accidental and Essential complication. In a short video available online he explains
  8. We use relative estimates (story points), but… When was the last time that you worked on a perfect code base, where the cost of the accidental complication (how messy the code base was) was either zero or the same multiple of g(e) as when you implemented feature A? He concludes that often the cost of accidental complication – dealing with organizational and technical issues specific to that code, and that organization – dominates (yes, dominates!) the cost of a feature. So if Accidental complication dominates the cost of a feature, and it cannot be predicted, what does that say about Estimates? Here’s another reason why estimates fail….
  9. Some people also take queuing very seriously…. (wait for reaction). I bet you do to. Have you fixed a bug that had been waiting for months in JIRA? So queues of work affect greatly the estimated time to complete. But wait time in queues are also unpredictable!
  10. One question comes from understanding that what we are trying to do here is learning to live without trying to impose control on the development system. The fact is that what makes sense to do will change as up learn more. It does not help to set a target outside a system's capability. In other words, if a system is stable and reliably churning out a certain output, it is of no use to set higher (or lower) targets for that system. Conversely, if the system is not stable, setting a target is useless because you cannot predict reliably enough the output of the system. Let me give you an example. Remember Office Space the movie that came out in 1999 (what a fitting year) and starred Jenifer Aniston, Ron Livingston and others?  
  11. Chaos report, 2004
  12. Source: http://www.csus.edu/indiv/v/velianitis/161/ChaosReport.pdf eWork and eBusiness in Architecture, Engineering and Construction: ECPPM 2012
  13. Sources: http:// people.senecac.on.ca/david.bath/BCS590/Powerpoints/05ch.ppt eWork and eBusiness in Architecture, Engineering and Construction: ECPPM 2012
  14. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  15. What was the most successful project in this chart? Success is not linked to meeting your schedule
  16. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  17. You come to the office and your boss catches you sitting down. ”You’re late Peter!” ”Oh sorry Mr Lumbergh. Traffic was really bad this morning, yeah” ”Sure, Peter. I’ll let this one slide. But say I would need you to deliver 10 stories in the next sprint. Do you think you can do that?” Did you see what he did just there?
  18. Milton knows best how to react to this kind of situations.
  19. Right, now back to the story. What Mr. Lumbergh did was Estimate Bargaining. Here’s what it means in practice….
  20. This an example of what I call Estimate Bargaining, only one of the 4 Estimate Dysfunctions I describe in the NoEstimates book
  21. One question comes from understanding that what we are trying to do here is learning to live without trying to impose control on the development system. The fact is that what makes sense to do will change as up learn more. It does not help to set a target outside a system's capability. In other words, if a system is stable and reliably churning out a certain output, it is of no use to set higher (or lower) targets for that system. Conversely, if the system is not stable, setting a target is useless because you cannot predict reliably enough the output of the system. Let me give you an example. Remember Office Space the movie that came out in 1999 (what a fitting year) and starred Jenifer Aniston, Ron Livingston and others?  
  22. Let’s take a look at one example of how we can use #NoEstimates in a real project and what were the results...
  23. Here are the questions that I started with...
  24. Here are the questions that I started with...
  25. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  26. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  27. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  28. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  29. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  30. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  31. Agile is alive and kicking. Take #NoEstimates and experiment! Be Agile!
  32. Just last week – as if prepared on purpose for Oredev…