In the world of software development, Agile is recognised for its flexibility, iterative feedback, and embrace of uncertainty. Paradoxically, most developers are still expected to estimate when a project will be completed.

In this article, I’ll explain why we estimate, how Agile was born, and how this paradox arises.

Why We Estimate

Software developers who are full-time employees will receive their monthly check whether the product they are working on is completed in six or twelve months, so estimates are usually not relevant for them.

Freelancers who are paid by project will struggle if the project takes twelve months and their earnings are insufficient, so estimates are quite relevant for them.

It is for this same reason that companies are interested in estimates. A project may be worth doing if it takes six months, but if it takes twelve months, they may prefer to work on something else.

So, basically, we estimate so that those who need to make critical decisions, usually related to money and time, have the information they need.

Once upon a time…

The history of software development has several true horror stories of software projects that went really bad. When I say bad I mean way about budget by millions of dollars, delayed by several years, and still never completed.

On those days, the predominant software development lifecycle was Waterfall, which looked like this:

Lots of things happened in the Requirements phase. Requirements would be captured in usually very long documents. Developers would extend the document with a solution analysis, and estimates of time and effort. The clients would sign off this document after some come and forth, essentially setting the requirements, due date and cost in stone. To this day, it’s still unclear if clients really read the documents, and whether those who did fully understood what they were signing.

Then we’d move to the Design phase and then the Development phase, in sequence. Once all development was ready, the software would be passed to the QA department or team to run tests (many times manual tests). Then, our software would be passed to the Ops team for deployment. And then we’d enter maintenance mode, were bugs would be reported and fixed.

Agile

As a response to this, In 2001, a group of 17 prominent software practitioners authored and signed the Agile Manifesto, fundamentally transforming software development practices.

Agile recognises that users often don’t know what they want until they see it, and that long-term estimates become unreliable due to unknowns. Instead of attempting to predict everything upfront, Agile structures work into short cycles with minimal bureaucracy. This creates a rapid feedback loop, allowing developers and customers to adjust course quickly when needed.

Robert C. Martin gives a great practical example of this in his book Agile Software Development, Principles, Patterns, and Practices. He explains how, at one point, he agreed to be paid a fixed sum upon completing the stories in a sprint, every sprint. The project had neither a fixed completion date nor a fixed cost. This approach prioritised flexibility and ongoing progress over rigid contracts, reinforcing the Agile principle of customer collaboration over contract negotiation.

In Scrum, we estimate in Story Points to determine how much work fits into a sprint, enabling better collaboration with stakeholders. If the team has been working together for some time, there will be enough data to measure its velocity, allowing us to forecast when the project might be completed. However, velocity is a trend metric, so it cannot guarantee a delivery date.

This adaptability extends beyond just estimating work: it applies to every stage of development, including design. While having all designs ready before starting a project might seem ideal, Agile teaches us that plans will evolve as we learn more. Trying to lock designs in place before writing stories is exactly what we did in the old days, and we know it doesn’t work. If we truly want to be agile, we need to embrace this uncertainty.

The Limits of Agility

Although Robert C. Martin’s example is quite good, in all my years as a software developer, I have never directly seen or heard of a truly agile company.

Companies are often accountable to a board of directors and stakeholders, many of whom may not fully understand the unique challenges of software development. They typically rely on a roadmap outlining major features or product releases, and missing those deadlines can undermine investor confidence, affect a company’s perceived value, and erode customer trust.

In my opinion, this is the main reason why companies can never be fully agile. While teams may work in an agile way at the team levels, agility fades as we move closer to shareholders. At that level, stock market analysts and executives prioritise predictability over adaptability: what matters is whether the company delivers on its commitments. The idea of “we’ll build in increments until the product is complete” simply doesn’t make sense at that scale, and I don’t think it ever will.

But there is still hope. We have lived with these paradigm for many year and we have delivered many successful products over the years. I will describe how we can do that in my next article.

Cheers!
José Miguel

Share if you find this content useful.
Follow me on LinkedIn to be notified of new articles.