SlideShare a Scribd company logo
how (fr)agilewe aremetrics in a complex worldGaetano MazzantiGama-Tech
???agilemetrics for a linear, deterministic worldtraditionalno metricscode & fixrigidprocesstop-downno processchaosorderedchaoticcomplex
product development is complex“self-organizing, non-linear,feedback systems areinherently unpredictable,they are not controllable“D.Meadows
however, we canwatch, learn and workwith the system
metricslearn & change
single loop learningresultsactionslead tohowwhich shape futureefficiencydoing things right incremental change
double loop learningresultsactionsvalues, assumptionsChris Argyrisguidehowwhylead to new/improved effectivenessdoing the right thingsefficiencydoing things right question assumptionsradical changeincremental change
learn, change,move onresultsactionsvalues, assumptionsdefinemetric*setexpirationdategoal ok orexpirationdate passed?metric*shared, simple, controllable, transparent, time-bound
metricsquadrants
inward & outwardlooking metricsoutwardloookinginwardlookingfeedbackR&DBusiness & Other Stakeholdersboundary objects
boundary objectsmetricbusinessR&Dboundary object [sociology]: something that helps different communities exchange ideas and information.could mean different things to differentpeoplebut allows coordination and alignment
metrics quadrantsBusinessoutward looking&feedbackProductProcessinward lookingTeam Maturity
metrics quadrantsBusinessboundaryobjectsProductProcessTeam Maturity
metrics quadrantsBusinessboundaryobjectsagileProductProcessfragileTeam Maturity
metrics quadrantsBusinessProductProcessTeam Maturity
metrics quadrantsBusinessLead TimeCycle TimeQuality of Service (SLA)ThroughputBusiness ValueRevenuesROICustomer SatisfactionBugs?ProductProcessWIPCadenceCI FailuresReworkImpedimentsRetrospectivesMoraleCode QualityTechnical DebtTest CoverageTeam Maturity
metrics quadrantsBusinesswhat!?no velocity?Lead TimeCycle TimeQuality of Service (SLA)ThroughputBusiness ValueRevenuesROICustomer SatisfactionBugs?ProductProcessWIPCadenceCI FailuresReworkImpedimentsRetrospectivesMoraleCode QualityTechnical DebtTest CoverageTeam Maturity
metrics quadrantsBusinessLead TimeCycle TimeQuality of Service (SLA)ThroughputBusiness ValueRevenuesROICustomer SatisfactionboundaryobjectsBugs?ProductProcessWIPCadenceCI FailuresReworkImpedimentsRetrospectivesMoraleCode QualityTechnical DebtTest CoverageTeam Maturity
metrics quadrantsBusinessLead TimeCycle TimeQuality of Service (SLA)ThroughputBusiness ValueRevenuesROICustomer SatisfactionboundaryobjectsBugs?ProductProcessWIPCadenceCI FailuresReworkImpedimentsRetrospectivesMoraleCode QualityTechnical DebtTest CoveragefragileTeam Maturity
metrics quadrantsBusinessLead TimeCycle TimeQuality of Service (SLA)ThroughputBusiness ValueRevenuesROICustomer SatisfactionboundaryobjectsagileBugs?ProductProcessWIPCadenceCI FailuresReworkImpedimentsRetrospectivesMoraleCode QualityTechnical DebtTest CoveragefragileTeam Maturity
fragility code quality   technical debtlack of advanced engineering practices(i.e. TDD, CI) => rework
code quality evolution
code qualityevolution
agilitybeing agile is not the goal,it’s a meanif you are really interested there are plenty of agility tests on the Internet:Nokia TestScrum Open Assessment - ScrumAllianceAgile Maturity ModelAgile Evaluation FrameworkComparative Agility Assessmentetc.
impediments, retrospectives, reviews# of questions answered# of questions asked # action items addressed# action items assigned(at previous meetings)# of WTFs?WTF!?WTF!?
metricsqueues
queues are badincreasecycle timeriskvariabilityoverheadreducequalitymotivationstop starting start finishing
cumulative flow diagramarrivalsqueue size(WIP)cumulativequantitytime in queue(cycle time)departures(throughput)timesource: Donald Reinertsen
cumulative flow diagram WIP is a leading indicatorcycle timeWIPcumulativequantitytime
cumulative flow diagramlarge batches large queuescumulativequantitytime
cumulative flow diagramsmall batches small queuescumulativequantitytime
Kanban boardWIPthroughputcycletime =backlogto doin progressdone23cycle time
no WIP limit -> queue!in progressreadybacklogto dodone23
no WIP limit -> queue!readybacklogto dodonein progress23flow= speed * density
Slack (%)optimize flowabsorb variation
flow related metricsactive WIP -buffered WIPtasks that are really in progress – task waiting to be handed-off (#,%,% of time spent)process efficiencyactive time / cycle timeBIPBugs In Processtechnical debtWIP / standard WIP# of projects a person works in parallel
visualizing tasks dynamicsbacklogto doin progressdone241234inactive taskdays
cumulative flow diagramnot so helpful?backlogto doin progress# user storiescycle timeWIPthroughputdonetime
single column dynamicsWIP
Kanbanboarddynamics
controlchartssource: SamuliHeljo
metricseasy but powerful42
Happiness Indexfeedback boardniko-niko calendar
Pizza IndexPizza = Overtime => not goodSteve Denning
how long since?you talked to a customerlast useful retrospectiveyou learned something at workyour boss last freaked outlast critical bug635221daysdaysdaysweeksweek
and don’t forgetbus factor# of key developers that need to be hit by a bus to kill a project

More Related Content

How (fr)agile we are. ALE2011

Editor's Notes

  1. See also law of requisite variety
  2. We can dance with the system – D.Meadows
  3. metrics as learning & change agentsMetricheaiutano a capire e prendereunadirezionepiuttostocheun’altraif you're not making mistakes & changing your mind, you're not learning. If you're not learning why are you iterating & collecting feedback?shift from “build, measure, learn” to “learn, measure, build” (Lean Startup)
  4. single is correcting an action to solve or avoid a mistake, double is correcting also the underlying causes behind the problematic actionLocal maximum, not global best … single = short termPDCAinspect. Single loop learning asks, “How can we do what we are doing better.”
  5. Kaikakukaizen, toglierepilotaautomatico!Argyris & Schon's Theory of Action, Double loop learning asks “Why do we think this is the right thing to do,” Fix the causes not the symptoms. Inspect and adapt, single loop?
  6. Focus on the positives. identifying the negatives and trying to fix them builds a wrong culture Metrics are needed to improve and not to punishBTW Failing is ok. self-definedsimplecontrollabletransparenttime-bound: created without extra effort, meaningful to all stakeholdersfocus on things that are actually controllable by the people being measuredLeading. prefer an imperfect forecast of the future to a perfect report on the past.visible and accessible without extra effortToo many metrics -> overload -> waste
  7. help to carry meaning, promote communication and assist with understanding. They serve as both containers and carriers. BOs are highly abstract and generalized form of knowledge organization with considerable reification. Classification systems, ontologies, paper forms serve as BOs (where they are used by diverse groups). An prime example of a BO is an ERP order entry process, a shared information space, a product tracking number, RFID tag
  8. Scorecard, sort of… Vanity metrics are things like registered users, downloads, and raw pageviews. They are easily manipulated, and do not necessarily correlate to the numbers that really matter: active users, engagement, the cost of getting new customers, and ultimately revenues and profits. The latter are moreactionable metricsThe only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Eric Ries
  9. Scorecard, sort of… Vanity metrics are things like registered users, downloads, and raw pageviews. They are easily manipulated, and do not necessarily correlate to the numbers that really matter: active users, engagement, the cost of getting new customers, and ultimately revenues and profits. The latter are moreactionable metricsThe only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Eric Ries
  10. Scorecard, sort of… Vanity metrics are things like registered users, downloads, and raw pageviews. They are easily manipulated, and do not necessarily correlate to the numbers that really matter: active users, engagement, the cost of getting new customers, and ultimately revenues and profits. The latter are moreactionable metricsThe only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Eric Ries
  11. Scorecard, sort of… Vanity metrics are things like registered users, downloads, and raw pageviews. They are easily manipulated, and do not necessarily correlate to the numbers that really matter: active users, engagement, the cost of getting new customers, and ultimately revenues and profits. The latter are moreactionable metricsThe only metrics that entrepreneurs should invest energy in collecting are those that help them make decisions. Eric Ries
  12. Outcome trumps output and activityNo measures related to individuals, “The basic building block of work is a team, not an individual” (esther derby)
  13. no clue abouteffectiveness/valueinducesextra effort/waste to correctly (!?) estimate -> don’t use it to predicteasy to be gamed ->don’t use it as a targetmeasure throughput instead (# of stories per iteration)
  14. Technical debt = 1 / design qualityRefactoring code that has no value is a waste of time
  15. Technical debt = 1 / design qualityRefactoring code that has no value is a waste of time
  16. Technical debt = 1 / design quality
  17. if nobody likes or wants to use your product, code quality does not really matter Technical debt = 1 / design quality Technical debt != bad codeWard CunninghamA little debt speeds development so long as it is paid back promptly with a rewrite.Every minute spent on not-quite-right code counts as interest on that debt.[Many] have explained the debt metaphor and confused it with the idea that you could write code poorly with the intention of doing a good job later.The ability to pay back debt [...] depends upon you writing code that is clean enough to be able to refactor as you come to understand your problem.
  18. Code quality leading indicator … successful products with poor quality, but …
  19. It takes longer to reach the front of a large line, this increases risk (customer & market), and variability (we move to a higher level of utilization where variability is amplified), more queues=more tasks & projects to track & report on, delayed feedback means bad assumptions live longer in code etc., no need to hurry if downstream activities will happen weeks later
  20. it isn’t easy to ignore a blocked and work on something else
  21. it isn’t easy to ignore a blocked and work on something else
  22. BIP Bugs In Process
  23. Check also pizza index (overtime = pizzas -> should drop to zero)
  24. Check also pizza index (overtime = pizzas -> should drop to zero)
  25. Simple, controllableSome of these mentioned by ArloBelshee