“Agile Methodologies” promise flexibility, speed of execution and the ability to deal with the multitude of scenarios in which development teams are called upon to operate in the digital transformation process.
A bit generic and perhaps even utopian. In fact, many organisations are ready to testify to the failure of these tools. But like any tool, agility must be applied to the right problem for it to be the solution.
Let’s address the elephant in the room: when people talk about Agile, they often do so improperly, referring to specific frameworks (Scrum being the most famous) or worse, to parts of them. The word “Agile” (with a capital A) actually refers to a mindset, an approach to organising projects, born in the field of software engineering.
The confusion is understandable and largely attributable to the very promoters of these organisational models who, in some cases and especially during the early periods of widespread adoption, spoke about them enthusiastically, as if they were magic wands capable of solving every delivery problem in one fell swoop, simply by adopting certain practices.
Today, agility is no longer optional. The software world moves at breathtaking speed and digitalisation is one of the main competitive differentiators for every product and service.
But is it really always needed? And how can Agile (with a capital A) help our organisation be agile (with a lowercase a), which is what we’re really interested in?
What is meant by Agile methodology?
An Agile methodology, broadly speaking, is a set of codified practices and rules whose purpose is to facilitate the migration of production processes from a waterfall logic to a more iterative and incremental approach.
In the waterfall approach, all design work is done in an initial phase. The focus of implementation is strict adherence to what was designed, and the result is verified at the end through acceptance tests that in turn aim to verify compliance with the specifications.
The authors of the Agile Manifesto, veterans of numerous unsuccessful projects, noticed that the main cause of problems was the complete lack of evolution in this equation.
If, in a long-term project, you don’t remove the constraint of strict adherence to the initial specifications, all the risk accumulates at the end. Unforeseen events, areas not covered by the analysis, incomplete or inadvertently incorrect requirements, changes in the market or corporate strategy, user feedback… everything is ignored until the project goes into production, which is when time and budget have run out (and almost always been exceeded).
In Agile software development, the main goal is to adopt practices and processes that help product teams focus on what truly delivers value, constantly improving their processes and the quality of the outcome.
The Agile Manifesto
So when we talk about Agile methodology, we are actually referring to an approach. A way of working that favours collaboration, adaptation and alignment of the product with shifting business goals, over adherence to pre-established and no-longer-negotiable plans.
Much like the Cloud Native model, the Agile movement began with the drafting of a manifesto that summarises the guiding principles for adopting and customising the framework in twelve key points. Here is an excerpt:
Our highest priority is to satisfy the customer through early and continuous delivery of valuable software […]
Welcome changing requirements, even late in development […]
Business people and developers must work together […]
Build projects around motivated individuals […]
Working software is the primary measure of progress […]
Continuous attention to technical excellence and good design enhances agility […]
Before delving further, let’s take a step back to understand what gave rise to this manifesto of values.
What are the goals of Agile methodology?
The Agile Manifesto, contrary to what one might think, is not a seminal product. It is a synthesis of the principles that drove, from the early 1990s, some developers to elaborate methodologies that would allow them to prevent the majority of software projects from failing within their organisations.
Agile is therefore not the answer to doing more with less, even though it is true that an Agile team, over the medium to long term, increases its efficiency thanks to good practices, efficient communication and trust at every level.
Agile is the answer to reducing the risk of a project, which sees in an empirical approach, based on continuous learning and process improvement, the solution to a potentially disastrous launch.
Generally, Agile organisations adopt the following practices to achieve this result:
- Frequent releases: Software is put into production very frequently, without waiting for perfection. Frequent releases of new features make it possible to track their actual adoption by customers and gather useful feedback to avoid building “cathedrals in the desert”.
- No technical debt: The organisation (not just the development team!) does not consider it acceptable to accumulate technical debt, as it would progressively make it impossible to maintain a rapidly changing codebase, preventing frequent releases. The team has the mandate to constantly improve the state of the code and their practices, without the race for new features taking over.
- Communication: Communication between the team and stakeholders on business topics is possible and indeed encouraged: an Agile team doesn’t just implement features (whose end-to-end behaviour is at best specified by requirements and use cases), but is also called upon to imagine solutions, especially in case of unexpected issues. Developers are therefore not mere executors, but must be results-oriented, able to make proposals and generate solutions.
This is an approach that may appear more expensive than the classic waterfall, but this impression is often false. How many projects seem perfectly on track until, close to the end, problems, delays and rework emerge?
An Agile approach seeks to bring testing and even adoption by real users forward as much as possible, so as to identify and address potential issues from the outset and get everything on the right track while it is still simple, quick and relevant to do so.
But is it always necessary?
What is the typical characteristic of Agile projects?
In terms of risk management, there are no projects that don’t benefit from an empirical approach. Frequently and promptly putting the product before its stakeholders, including end users wherever possible, can only bring out early and clearly the vital information needed to shape the software around real needs, investing in what truly delivers value.
Learning (not only technical, but also business) and adaptation are indeed the two aspects most rewarded by such an approach.
There are cases where a progressive approach, starting from scratch, is not applicable. Imagine wanting to create a new product to replace one already in production. It is unlikely that users would be satisfied with a subset of features, waiting for all the ones they are already accustomed to to be completed over time.
These cases seem like ideal candidates for a waterfall approach: we need to bring a well-established body of features into production. We know everything we need to do and how we need to do it, right?
Maybe.
First of all, in that case, why build a new product? Certainly there are purely technical reasons: key dependencies reaching end of life, legacy systems that are no longer maintainable, expiring licence contracts that force a technology change… but the opportunity remains a good one to look at the system with a critical eye and remain agile and discerning in the approach, to seize opportunities to increase the value of the outcome.
For example, the team can expose features to end users as they are completed, in a test environment. The organisation can then gather feedback, make adaptive decisions and adjust the roadmap to ensure time and budget are respected, anticipating any compromises, but also deciding to drop old features that were never used (or openly hated) in order to invest in improving the most critical ones or introducing new features.
In these scenarios, the architectural choices made in the past also influence the way and extent to which the Agile approach can help the product evolve. A microservices architecture, in fact, works optimally with an empirical and evolutionary way of conceiving change.
But above all, the focus on good practices and the quality of production processes ensures the creation of a maintainable and controlled product, both before and after going into production.
After all, which project doesn’t benefit from investing time and budget on the things that deliver the most value?
READ ALSO:
- How to introduce DevOps culture in your company
- Microservices: what they are and why use them
What are the Agile frameworks?
Alright, we’ve talked about mindset and principles, but in practice how do you reorganise to work in an Agile way?
As already mentioned, there are various methodological interpretations of the Agile mindset; in fact, the manifesto was born from the comparison of different organisational realities, distilling their common principles in retrospect.
Agile Frameworks are skeletons of already-consolidated practices that can facilitate the adoption and maturation of an Agile mindset within an organisation.
Why are they called frameworks? Because they provide guidance and “prescriptions” without imposing the operational details. For example, Scrum requires the use of an ordered, emergent and detailed product backlog; but it doesn’t say whether this backlog should be represented on a board or consist of a pile of sticky notes in a shoebox. Each team can implement its backlog in whatever way is most convenient and suitable.
Different frameworks are more or less suitable depending on the scenario and, although it is possible to get an idea of which is the most appropriate to start with, experimentation remains a very important component. Changing an organisational approach inevitably has effects on the context, and it is common for a framework that was suitable at a given point in time to end up transforming the organisation to the point where it becomes constraining, giving rise to further evolution.
The important thing is not to confuse frameworks (or methodologies) with agility: following the dictates of any Agile Framework to the letter, without developing an Agile value system, can lead to results very different from those hoped for and even worsen a team’s performance.
By way of example, let’s very briefly look at three examples of Agile frameworks, specifically: Scrum, Kanban and eXtreme Programming.
Scrum
Scrum is the most famous among Agile Frameworks, to the point that many now incorrectly use “Agile” and “Scrum” as synonyms.
The Scrum Guide defines it as follows: “Scrum is a process framework that has been used to manage complex product development since the early 1990s. Scrum is not a process or a technique for building products; rather it is a framework within which you can employ various processes and techniques. Scrum makes clear the relative efficacy of your product management and development practices so that you can improve”.
It is a framework oriented towards producing working software from the very first iteration and requires the entire organisation (not just the development team) to embrace its values in order to reap its benefits.
Besides being the most famous, Scrum is also the most misunderstood and poorly implemented, which has earned it the undeserved hatred of many disappointed developers. In the words of its creators, Scrum is easy to understand, but difficult to master.
It is a framework particularly suited for creating new products, where user feedback can make a real difference.
Kanban
This is a methodology inspired by the principles of Agile software development, the theory of constraints and the principles of the Toyota Production System. Developed by David J. Anderson in 2007 after a long gestation at Microsoft, it was formalised in the book “Kanban” published in 2010. Its operation is based on six principles and six practices:
- visualise the workflow;
- limit work in progress;
- measure and manage the workflow;
- make process rules explicit;
- establish feedback loops;
- establish cycles of process evolution and improvement.
Like Scrum, Kanban is also often misunderstood and reduced to one of its physical components: the board. The famous whiteboard populated with colourful sticky notes, diligently arranged in columns to represent the flow, is a very common aid for implementing the first practice (visualising the workflow). However, Kanban goes well beyond that, providing tools for identifying bottlenecks, measuring the effectiveness of process improvements and making accurate delivery forecasts.
Kanban’s focus is on creating the smoothest and most continuous delivery flow possible. It has the property of “starting from what the team is already doing and evolving based on evidence,” which makes it a very lightweight framework to adopt. However, to put it into practice for the first time, the help of a facilitator is often a key element to avoid getting lost in the many options and getting the right start.
Kanban is a scalable framework that can be adopted by a single team within a larger organisation, but is particularly suited for optimising systems composed of multiple interdependent workflows. It is an excellent entry point for organisations that want to develop an Agile mindset one step at a time, modifying their habits gradually and without sudden upheavals. For this reason, however, it requires a great deal of discipline.
Extreme Programming (XP)
Extreme Programming is a software development methodology that emphasises writing quality code and rapid response to changing requirements. It is a very rigorous framework that prescribes twelve practices that a development team must adopt in order to achieve the expected results.
XP has the great merit of having distilled and promoted many of the good development practices that have become common elements for every Agile team, regardless of the method they use. For example, it promoted:
- continuous integration;
- peer programming;
- iterative development and planning;
- the use of User Stories;
- the systematisation of testing and refactoring;
- the prohibition on developers writing code that is not strictly necessary;
- the emphasis on code clarity and simplicity; the preference for non-hierarchical management structures;
- finally, the importance given to direct and frequent communication between developers and the client, and among the developers themselves.
Extreme Programming, often criticised for being a model too centred on software engineering and for delivering more value to developers than to the business, is a development model that, in exchange for strict discipline and a process very focused on development practices, delivers very high technical quality.
READ ALSO: User Story Mapping: how to design a web product together with your team
Agile Methodology: a practical example
To get a more precise idea of Agile Methodology, a practical example can help us grasp its distinctive features. Let’s try comparing the way two different methodologies operate at various stages of the project lifecycle. For this example, we’ll compare a waterfall approach, generally contrasted with Agile, using the Scrum framework as an example.
We want to make clear that these are generalisations, deliberately polarised towards extreme cases. Reality is always more nuanced and depends very much on the degree of maturity in applying both approaches.
On the left side of the table we find the phases shared by the two approaches, while in the columns we find the respective way each phase is managed in the two different frameworks.
That’s great, but does it really work?
A concrete example of agility in action is represented by the Telethon success story that appears among our case studies.
The Telethon Foundation engaged SparkFabrik to improve the percentage of donations channelled through the website, which at the time of engagement stood at around 3% of total donations during the winter marathon.
Thanks to the Agile approach adopted by the entire organisation, the development of the donation website was completed a week ahead of schedule and without the need for an acceptance testing phase, since the client was able to validate the product increments at the end of each sprint.
Being able to review the application’s components every week with users and client-side managers made it possible to promptly revisit assumptions, adjust course and make decisions that were easy to implement, accelerating in the right direction. The increase in donations received through the new website was 37%!
Can you spot the elements discussed so far within this story?
The rapid release of the product; constant communication between the team and the client; timely and cyclical feedback and, of course, the quality and client satisfaction given the results achieved: all the elements promoted by the Agile Manifesto.
DISCOVER OTHER FRAMEWORKS:
- DevOps
- DevSecOps
- FinOps
Watch out for the pitfalls: how to avoid getting hurt
As we wrote, agility is first and foremost a mindset and requires a change of direction. Thinking you can adopt Scrum and then asking your PM to make sure things are delivered “exactly as planned at the end of each sprint” is no different from requiring that “the work is completed as per specifications, in time for the milestone review”.
This is simply a “relabelling” exercise and will bring nothing but frustration and poor results, as it does nothing more than accelerate the execution of the “plan”, often at the expense of product quality (the exact opposite of the goal!).
There is a term we use at SparkFabrik to describe this approach: Agile à la carte. It is a provocative way of naming the tendency, observed in several of our experiences, to adopt only the easy and immediately understandable aspects of methodologies and frameworks. These are usually the least substantial and most superficial elements — the so-called low-hanging fruit — the parts that are easy to stick onto existing processes, under the illusion that doing so changes their substance.
There is nothing particularly deplorable about this. Introducing new management principles is neither an easy nor cost-free process, and adopting a framework is not always the best way to “agilise” your business.
The adoption of Lean practices, already well-established in many areas including management, or the emergence of relatively recent and less talked-about approaches like Disciplined Agile (which defines itself as a toolkit and differentiates itself from frameworks by not wanting to be prescriptive) or SAFe (a set of practices and models for adopting Agile at enterprise scale that incorporates the concept of value stream at its core, borrowed from the Lean world), are all signs that the ways of practising agility are as numerous as the enterprises that want to adopt it.
Perhaps it is time to stop talking about Scrum or Kanban methodology, about tools, frameworks and pre-packaged solutions, and return to analysing the fundamental principles. There are, for example, interesting hybridisations between more traditional approaches to project management and Agile practices, complete with certification programmes for “Agilist” PMIs, which can provide an optimal solution for introducing a change in thinking, taking one step at a time in the right direction.
The journey from a “traditional” organisation to a fully Agile one is itself an empirical process, one that cannot be systematised (and beware of anyone who promises otherwise). And in any case, there is no moment when an organisation becomes agile. Agility, like all qualities, is not precisely measurable and sometimes it is only by observing other realities that we can tell whether and how we can improve.
It is wise to expect experimentation and the occasional failure, try different approaches and, above all, seek dialogue. In this respect, participation in communities and events (be they industry conferences such as Agile Business Day or so-called unconferences like Play14) plays a fundamental role in creating opportunities for exchange, growth and in identifying patterns of success and failure common to other organisations.
In essence, the paths and methodologies are numerous and constantly evolving; their names are often registered trademarks and serve in some way to make agility a business in its own right.
Is Agile methodology still relevant today?
More than two decades after the birth of the Agile Manifesto in 2001, the question of whether Agile is still relevant in software development is more than legitimate. The answer remains a resounding yes. The software world is in constant evolution, with new technologies and emerging requirements that demand flexibility and adaptability — intrinsic characteristics of the Agile approach. For this very reason, changes such as the introduction of new programming languages, cloud computing and other technological innovations do not risk making the Agile approach outdated; on the contrary, they provide the opportunity to apply the methodology’s principles, demonstrating their extreme necessity. That is why Agile continues to be a valuable methodology. Its fundamental principles — such as collaboration between teams and clients, iterative and incremental delivery, and the ability to react to change — will not only continue to be useful and relevant, but will be even more so in disruptive contexts and ever-new scenarios.
New trends
Agile is therefore, in every respect, a future-proof mindset. Will it remain unchanged in the years to come? It would be naïve to think so, and the already-mentioned hybridisations between different frameworks testify to the contrary. If anything, it will be the principles (those, yes) that will remain largely unchanged for a long time, while the frameworks will evolve significantly. We can cite, for example, the trend of the Obeya Management System, an approach that transcends and encompasses any Lean Agile methodology. Also derived from the Toyota Production System, Obeya is a Visual Management system that aligns strategy and execution through a collaborative architecture, without the need to certify scores of Scrum Masters or Agile Coaches. This system promotes collaboration, transparency, communication and accountability, offering a powerful team-centric management tool.
Another trend is linked to the growing spread of Agile principles, which have now broken beyond the boundaries of software development to flow into a wide variety of professional roles. Today, Agile skills are required in a great many business areas, constituting a fundamental requirement even for roles such as project managers, sales staff, business analysts and human resources specialists, to name but a few. This underscores the importance of a widespread Agile mindset throughout the organisation, at every level.
How can Agile methodology be useful to you?
In conclusion, we can say that Agile methodologies can be a fundamental ingredient in guiding companies towards a full digital transformation. A new way of working, ideal for putting into practice an efficient management of projects with a strong Digital and Cloud focus.
Agility is a quality that, once acquired, is reflected throughout the entire lifecycle of a project, bringing benefits in terms of product quality, customer satisfaction, monitoring and control capabilities, increased predictability of outcomes and risk reduction, greater flexibility and the promotion of continuous improvements and incremental evolutions.
Last but not least, successfully implementing an Agile methodology has a positive impact on team morale and significantly improves internal dynamics — fundamental aspects for maintaining a stimulating, creative and results-oriented work environment.
What matters most is developing the right posture towards the way the organisation approaches projects. Rather than trying to anticipate every possibility and hoping things go as planned from start to finish, accept that the world is complex and unpredictable, work iteratively with transparency and eyes wide open, and adapt your plans to maximise value, deploying resources to achieve the best possible outcome.
