Agile development: principles, methods, and when it actually works

Agile development is an iterative approach in which teams work in short cycles, deliver working software frequently, and continuously respond to feedback. It is not a methodology but a mindset — codified in four values and twelve principles in the Agile Manifesto (2001). In practice, that mindset takes shape through concrete frameworks like Scrum and Kanban.

Yet in many organisations, agile gets reduced to a set of ceremonies: a daily standup, a sprint review, a retrospective. The boxes get ticked, but the underlying principles — team autonomy, short feedback loops, genuine willingness to change — are absent. The result is an organisation that does agile without being agile.

Agile principles explained.

The Agile Manifesto was published in February 2001 by seventeen software developers at a ski resort in Utah. They articulated four core values in response to the then-dominant waterfall model, where large software projects ran for months or years with no visible output.

• Individuals and interactions over processes and tools.
• Working software over comprehensive documentation.
• Customer collaboration over contract negotiation.
• Responding to change over following a plan.

The manifesto is explicit about the nuance: 'That is, while there is value in the items on the right, we value the items on the left more.' Agile is not a license to skip documentation or ignore contracts. It is a reprioritisation: human interaction and working software carry more weight than paperwork.

The four values are supported by twelve principles. The most cited: deliver working software frequently (every two to four weeks), welcome changing requirements even late in development, and reflect regularly as a team on how to become more effective. According to Digital.ai's 17th Annual State of Agile Report (2023), 71% of surveyed organisations now use agile — though maturity varies widely.

Scrum vs. Kanban.

Scrum and Kanban are the two most widely used frameworks within agile. They are not interchangeable: each fits different types of work and team contexts.

Scrum operates in fixed iterations — sprints of one to four weeks. At the start of each sprint, the team selects a volume of work from the product backlog. At the end, working software is delivered and the process is evaluated. Scrum has three defined roles: the product owner (responsible for prioritisation), the scrum master (responsible for the process), and the development team.

Kanban has no fixed iterations and no defined roles. Work flows continuously from left to right across a Kanban board. The core mechanism is the WIP limit (Work In Progress): teams may only have a fixed number of items in progress at any time. This forces work to be finished before new work is picked up — and makes bottlenecks visible.

In practice, many teams combine elements of both. Scrumban — a hybrid approach — is popular with teams that want to keep sprint structure while using explicit WIP limits. There is no universally correct choice; the right framework depends on the nature of the work and the stability of requirements.

When agile works (and when it doesn't).

Agile is not a universal solution. It works well in contexts with high requirement uncertainty and products that need early validation with real users. It works less well — or actively against you — in contexts with fixed scope, strict regulatory requirements, or teams without genuine autonomy.

Where agile adds value.

• Requirements are uncertain or change frequently. Agile was designed for this scenario: short cycles make early course correction possible at low cost.
• End users are available for feedback. Agile requires customer involvement. Without a real product owner or user feedback, iteration becomes fictional.
• The team has ownership over the approach. Self-organising teams perform significantly better under agile than hierarchically directed teams.
• The domain is evolving. Software products in markets with rapid technological change — AI, platforms, consumer apps — benefit most from short feedback loops.

Where agile is a poor fit.

• Fixed price and scope (public tenders). Government contracts and procurement processes often require detailed upfront specifications. Agile conflicts structurally with this model — unless the client consciously adopts an iterative contract form.
• Strict documentation requirements. Medical devices, aviation software, and other regulated domains require extensive traceability. Agile can be adapted, but demands extra discipline.
• Simple, well-defined tasks. A clearly scoped migration or a small integration does not need sprint planning. Overhead without added value is an anti-pattern.
• Teams without psychological safety. Retrospectives and open feedback loops only work when team members feel safe raising problems. Without that foundation, the ceremonies become performance.

“Agile is not a silver bullet. It's a framework for managing uncertainty. If there's no uncertainty, you don't need agile.”— Dave Thomas, co-author of the Agile Manifesto

Agile beyond software teams.

Agile started in software development, but its principles transfer to other domains. Marketing teams use Kanban boards for campaign management. HR teams run sprints for recruitment cycles. Operations teams use WIP limits to surface overload.

According to Digital.ai's State of Agile Report, 18% of agile adoption now occurs outside IT departments. Most successful cases involve knowledge work — work where the output is hard to define upfront and where feedback determines quality. For repetitive, standardised work like manufacturing or logistics, lean principles are typically more effective.

The risks of agile outside software mirror the risks within it: cargo cult adoption. A marketing team that introduces sprints without understanding why — and without a product owner or real prioritisation — gains nothing. The structure becomes overhead, not an enabler.

Common pitfalls.

After more than two decades of agile adoption, the recurring failures are well documented. These five come up most often.

1. Cargo cult agile. The team follows the ceremonies — standup, sprint planning, retro — but does not act on the values. Requirements are still locked months in advance. The sprint review is a status update, not a genuine demonstration of working software. The form is there; the substance is not.
2. No real product owner. The product owner is central to agile: they prioritise the backlog, make scope decisions, and are available daily. In practice, the PO role is often a part-time add-on for a manager with five other responsibilities. Without mandate and availability, the team cannot iterate effectively.
3. Sprints as mini-waterfall. If the team expects detailed specifications for all items at the start of each sprint, and those specifications cannot change during the sprint, it is a waterfall project split into two-week blocks. The iteration is cosmetic.
4. Scaling too early. SAFe (Scaled Agile Framework) and similar frameworks are designed for large organisations with multiple teams. They introduce significant additional process and overhead. For teams of three to ten people, SAFe is almost always overkill — and a source of bureaucracy that undermines agility.
5. No psychological safety. Retrospectives are worthless if team members do not feel safe saying what is going wrong. A retro that ends with 'things are going well' every quarter is not a retro — it is theatre. Psychological safety is the foundation of every agile team. Without it, you are investing in the wrong thing.

Conclusion.

After twenty-five years, agile is no longer a trend — it is the standard in software development. Yet implementation continues to fail regularly. The reason is always the same: focus on form over substance. The ceremonies are there; the principles are not.

The core of agile is straightforward: deliver value early, listen to feedback, adapt. Scrum and Kanban are tools to structure that. Which tool you choose depends on your work — not on what is fashionable or what the book says. A team that ships working software daily and responds quickly to changing requirements is agile, regardless of whether they run sprints.