The Agile Manifesto has shaped software development for over 21 years. However, empirical research into its real-world effectiveness remains scarce. Recent findings highlight a pressing concern: 81% of business decision-makers in the UK and 89% in the USA worry about on-time software project delivery within their organisations.
A new study, conducted for the book “Impact Engineering,” reveals that 65% of software projects using Agile principles fail to meet deadlines, budgets, and quality standards. In contrast, projects employing the Impact Engineering approach show a failure rate of just 10%.
Agile emphasises principles including:
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
However, the new research indicates that projects with documented requirements before development are 50% more likely to succeed. Clear initial requirements boost success likelihood by 97%, and avoiding significant late-stage requirement changes increases success chances by 7%.
Additional findings highlight other success factors. Projects where software engineers felt psychologically safe to address issues promptly were 87% more likely to succeed. Accurate real-world problem-based requirements enhanced success probability by 54%.
Notably, the study found no significant difference in success rates between engineers working on single versus multiple projects, despite Lean methodology advocating reduced work-in-progress. However, previous research by Dr Ali showed that 83% of software engineers experience burnout—primarily due to high workloads.
The Horizon IT system, an early large-scale Agile project using Rapid Application Development, faced scrutiny in a public inquiry. Fujitsu witnesses Terence Austin and whistleblower David McDonnell condemned the lack of a robust requirements engineering process.
Technical expert Charles Cipione bluntly stated, “if you don’t have a good design, it’s not going to work properly.” This failure contributed to the Post Office scandal, described as Britain’s largest miscarriage of justice, linked to multiple suicides and wrongful imprisonments, including a pregnant woman.
The study also found UK software engineers are 13% less likely to feel able to discuss and address problems compared to their US counterparts. This aligns with November 2023 Engprax research showing 75% of UK software engineers faced retaliation for reporting wrongdoing.
Dr Junade Ali, author of Impact Engineering, commented, “With 65% of projects adopting Agile practices failing to be delivered on time, it’s time to question Agile’s cult following.
“Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout. This is fundamental to the philosophy of Impact Engineering.”
See also: Angel Montesdeoca, IBM: The role of AI in modern software development
Looking to revamp your digital transformation strategy? Learn more about Digital Transformation Week taking place in Amsterdam, California, and London. The comprehensive event is co-located with AI & Big Data Expo, Cyber Security & Cloud Expo, and other leading events.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
