From Wikipedia, the free encyclopedia - View original article
|It has been suggested that Agile Automation be merged into this article. (Discuss) Proposed since March 2014.|
|Software development process|
A software developer at work
Agile software development is a group of software development methods in which requirements and solutions evolve through collaboration between self-organizing, cross-functional teams. It promotes adaptive planning, evolutionary development, early delivery, continuous improvement and encourages rapid and flexible response to change. It is a conceptual framework that focuses on delivering working software with the minimum amount of work.
The Agile Manifesto, which first laid out the underlying concepts of Agile development, introduced the term in 2001.
Incremental software development methods have been traced back to 1957. In 1974, a paper by E. A. Edmonds introduced an adaptive software development process. Concurrently and independently the same methods were developed and deployed by the New York Telephone Company's Systems Development Center under the direction of Dan Gielan. In the early 1970s, Tom Gilb started publishing the concepts of Evolutionary Project Management (EVO), which has evolved into Competitive Engineering. During the mid to late 1970s Gielan lectured extensively throughout the U.S. on this methodology, its practices, and its benefits.
So-called lightweight agile software development methods evolved in the mid-1990s as a reaction against the heavyweight waterfall-oriented methods, which were characterized by their critics as being heavily regulated, regimented, micromanaged and overly incremental approaches to development.
Proponents of lightweight agile methods contend that they are returning to development practices that were present early in the history of software development.
Early implementations of agile methods include Rational Unified Process (1994), Scrum (1995), Crystal Clear, Extreme Programming (1996), Adaptive Software Development, Feature Driven Development (1997), and Dynamic Systems Development Method (DSDM) (1995). These are now collectively referred to as agile methodologies, after the Agile Manifesto was published in 2001.
In February 2001, 17 software developers (see below) met at the Snowbird, Utah resort, to discuss lightweight development methods. They published the Manifesto for Agile Software Development to define the approach now known as agile software development. Some of the manifesto's authors formed the Agile Alliance, a non-profit organization that promotes software development according to the manifesto's values and principles.
The Agile Manifesto reads, in its entirety, as follows:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over Processes and tools
Working software over Comprehensive documentation
Customer collaboration over Contract negotiation
Responding to change over Following a planThat is, while there is value in the items on the right, we value the items on the left more.
Kent Beck James Grenning Robert C. Martin Mike Beedle Jim Highsmith Steve Mellor Arie van Bennekum Andrew Hunt Ken Schwaber Alistair Cockburn Ron Jeffries Jeff Sutherland Ward Cunningham Jon Kern Dave Thomas Martin Fowler Brian Marick
© 2001, the above authors. This declaration may be freely copied in any form, but only in its entirety through this notice.
The meanings of the manifesto items on the left within the agile software development context are:
Introducing the manifesto on behalf of the Agile Alliance, Jim Highsmith commented that the Agile movement was not opposed to methodology:
The Agile movement is not anti-methodology, in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment. Those who would brand proponents of XP or SCRUM or any of the other Agile Methodologies as "hackers" are ignorant of both the methodologies and the original definition of the term hacker.—Jim Highsmith, History: The Agile Manifesto
The Agile Manifesto is based on twelve principles:
Later, Ken Schwaber with others founded the Scrum Alliance and created the Certified Scrum Master programs and its derivatives. Schwaber left the Scrum Alliance in the fall of 2009, and founded Scrum.org.
In 2005, a group headed by Alistair Cockburn and Jim Highsmith wrote an addendum of project management principles, the Declaration of Interdependence, to guide software project management according to agile development methods.
In 2009, a movement spearheaded by Robert C Martin wrote an extension of software development principles, the Software Craftsmanship Manifesto, to guide agile software development according to professional conduct and mastery.
|This section needs additional citations for verification. (December 2012)|
There are many specific agile development methods. Most promote development, teamwork, collaboration, and process adaptability throughout the life-cycle of the project.
Iterative, incremental and evolutionary
Most agile methods break tasks into small increments with minimal planning and do not directly involve long-term planning. Iterations are short time frames (timeboxes) that typically last from one to four weeks. Each iteration involves a cross-functional team working in all functions: planning, requirements analysis, design, coding, unit testing, and acceptance testing. At the end of the iteration a working product is demonstrated to stakeholders. This minimizes overall risk and allows the project to adapt to changes quickly. An iteration might not add enough functionality to warrant a market release, but the goal is to have an available release (with minimal bugs) at the end of each iteration. Multiple iterations might be required to release a product or new features.
Efficient and face-to-face communication
No matter what development disciplines are required, each agile team will contain a customer representative, e.g. Product Owner in Scrum. This person is appointed by stakeholders to act on their behalf and makes a personal commitment to being available for developers to answer mid-iteration questions. At the end of each iteration, stakeholders and the customer representative review progress and re-evaluate priorities with a view to optimizing the return on investment (ROI) and ensuring alignment with customer needs and company goals.
In agile software development, an information radiator is a (normally large) physical display located prominently in an office, where passers-by can see it. It presents an up-to-date summary of the status of a software project or other product. The name was coined by Alistair Cockburn, and described in his 2002 book Agile Software Development. A build light indicator may be used to inform a team about the current status of their project.
Very short feedback loop and adaptation cycle
A common characteristic of agile development are daily status meetings or "stand-ups", e.g. Daily Scrum (Meeting). In a brief session, team members report to each other what they did the previous day, what they intend to do today, and what their roadblocks are.
Specific tools and techniques, such as continuous integration, automated unit testing, pair programming, test-driven development, design patterns, domain-driven design, code refactoring and other techniques are often used to improve quality and enhance project agility.
Compared to traditional software engineering, agile development is mainly targeted at complex systems and projects with dynamic, undeterministic and non-linear characteristics, where accurate estimates, stable plans and predictions are often hard to get in early stages, and big up-front designs and arrangements will probably cause a lot of waste, i.e. are not economically sound. These basic arguments and precious industry experiences learned from years of successes and failures have helped shape Agile's favor of adaptive, iterative and evolutionary development.
Development methods exist on a continuum from adaptive to predictive. Agile methods lie on the adaptive side of this continuum. One key of adaptive development methods is a "Rolling Wave" approach to schedule planning, which identifies milestones but leaves flexibility in the path to reach them, and also allows for the milestones themselves to change. Adaptive methods focus on adapting quickly to changing realities. When the needs of a project change, an adaptive team changes as well. An adaptive team will have difficulty describing exactly what will happen in the future. The further away a date is, the more vague an adaptive method will be about what will happen on that date. An adaptive team cannot report exactly what tasks they will do next week, but only which features they plan for next month. When asked about a release six months from now, an adaptive team might be able to report only the mission statement for the release, or a statement of expected value vs. cost.
Predictive methods, in contrast, focus on analysing and planning the future in detail and cater for known risks. In the extremes, a predictive team can report exactly what features and tasks are planned for the entire length of the development process. Predictive methods rely on effective early phase analysis and if this goes very wrong, the project may have difficulty changing direction. Predictive teams will often institute a Change Control Board to ensure that only the most valuable changes are considered.
Risk analysis can be used to choose between adaptive (agile or value-driven) and predictive (plan-driven) methods. Barry Boehm and Richard Turner suggest that each side of the continuum has its own home ground, as follows:
|Agile methods||Plan-driven methods||Formal methods|
|Low criticality||High criticality||Extreme criticality|
|Senior developers||Junior developers(?)||Senior developers|
|Requirements change often||Requirements do not change often||Limited requirements, limited features see Wirth's law|
|Small number of developers||Large number of developers||Requirements that can be modeled|
|Culture that responds to change||Culture that demands order||Extreme quality|
One of the differences between agile and waterfall is that testing of the software is conducted at different stages during the software development lifecycle. In the Waterfall model, there is always a separate testing phase near the completion of an implementation phase. However, in Agile and especially Extreme programming, testing is usually done concurrently with coding, or at least, testing jobs start in early iterations.
In a letter to IEEE Computer, Steven Rakitin expressed cynicism about agile development, calling an article supporting agile software development "yet another attempt to undermine the discipline of software engineering" and translating "Working software over comprehensive documentation" as "We want to spend all our time coding. Remember, real programmers don’t write documentation."
This is disputed by proponents of Agile software development, who state that developers should write documentation if that's the best way to achieve the relevant goals, but that there are often better ways to achieve those goals than writing static documentation. Scott Ambler states that documentation should be "Just Barely Good Enough" (JBGE), that too much or comprehensive documentation would usually cause waste, and developers rarely trust detailed documentation because it's usually out of sync with codes, while too little documentation may also cause problems for maintenance, communication, learning and knowledge sharing. Alistair Cockburn wrote of the Crystal method:
Crystal considers development to be a series of co-operative games, and the provision of documentation is intended to be enough to help the next win at the next game. The work products for Crystal include use cases, risk list, iteration plan, core domain models, and design notes to inform on choices...however there are no templates for these documents and descriptions are necessarily vague, but the objective is clear, just enough documentation for the next game. I always tend to characterize this to my team as: what would you want to know if you joined the team tomorrow.—Alistair Cockburn[attribution verification needed]
Well-known agile software development methods and/or process frameworks include:
The agile methods are focused on different aspects of the Software development life cycle. Some focus on the practices (e.g. XP, Pragmatic Programming, Agile Modeling), while others focus on managing the software projects (e.g. Scrum). Yet, there are approaches providing full coverage over the development life cycle (e.g. DSDM, IBM RUP), while most of them are suitable from the requirements specification phase on (FDD, for example). Thus, there is a clear difference between the various agile methods in this regard.
Agile development is supported by a bundle of concrete practices suggested by the agile methods, covering areas like requirements, design, modeling, coding, testing, project management, process, quality, etc. Some notable agile practices include:
The Agile Alliance has provided a comprehensive online collection with a map guide to the applying agile practices.
In the literature, different terms refer to the notion of method adaptation, including 'method tailoring', 'method fragment adaptation' and 'situational method engineering'. Method tailoring is defined as:
A process or capability in which human agents determine a system development approach for a specific project situation through responsive changes in, and dynamic interplays between contexts, intentions, and method fragments.
Potentially, almost all agile methods are suitable for method tailoring. Even the DSDM method is being used for this purpose and has been successfully tailored in a CMM context. Situation-appropriateness can be considered as a distinguishing characteristic between agile methods and traditional software development methods, with the latter being relatively much more rigid and prescriptive. The practical implication is that agile methods allow project teams to adapt working practices according to the needs of individual projects. Practices are concrete activities and products that are part of a method framework. At a more extreme level, the philosophy behind the method, consisting of a number of principles, could be adapted (Aydin, 2004).
Extreme Programming (XP) makes the need for method adaptation explicit. One of the fundamental ideas of XP is that no one process fits every project, but rather that practices should be tailored to the needs of individual projects. Partial adoption of XP practices, as suggested by Beck, has been reported on several occasions. Mehdi Mirakhorli proposes a tailoring practice that provides a sufficient road-map and guidelines for adapting all the practices. RDP Practice is designed for customizing XP. This practice, first proposed as a long research paper in the APSO workshop at the ICSE 2008 conference, is currently the only proposed and applicable method for customizing XP. Although it is specifically a solution for XP, this practice has the capability of extending to other methodologies. At first glance, this practice seems to be in the category of static method adaptation but experiences with RDP Practice says that it can be treated like dynamic method adaptation. The distinction between static method adaptation and dynamic method adaptation is subtle.
|This section needs additional citations for verification. (August 2010)|
Agile methods have much in common with the Rapid Application Development techniques from the 1980/90s as espoused by James Martin and others. In addition to technology-focused methods, customer-and-design-centered methods, such as Visualization-Driven Rapid Prototyping developed by Brian Willison, work to engage customers and end users to facilitate agile software development.
In 2008 the Software Engineering Institute (SEI) published the technical report "CMMI or Agile: Why Not Embrace Both" to make clear that the Capability Maturity Model Integration and Agile can co-exist. Modern CMMI-compatible development processes are also iterative. The CMMI Version 1.3 includes tips for implementing Agile and CMMI process improvement together.
While agility can be seen as a means to an end, a number of approaches have been proposed to quantify agility. Agility Index Measurements (AIM) score projects against a number of agility factors to achieve a total. The similarly named Agility Measurement Index, scores developments against five dimensions of a software project (duration, risk, novelty, effort, and interaction). Other techniques are based on measurable goals. Another study using fuzzy mathematics has suggested that project velocity can be used as a metric of agility. There are agile self-assessments to determine whether a team is using agile practices (Nokia test, Karlskrona test, 42 points test).
While such approaches have been proposed to measure agility, the practical application of such metrics is still debated. There is agile software development ROI data available from the CSIAC ROI Dashboard.
One of the early studies reporting gains in quality, productivity, and business satisfaction by using Agile methods was a survey conducted by Shine Technologies from November 2002 to January 2003. A similar survey, the State of Agile, is conducted every year starting in 2006 with thousands of participants from around the software development community. The State of Agile survey tracks trends on the benefits of agile, trends, lessons learned, and preferred practices and agile methodologies. From the 2013 results released in January 2014, the survey concludes that 73% of respondents say agile software helps them complete software projects faster; 92% say agile improves their ability to manage changing customer priorities; and 87% say agile improves their development team's productivity. Yet another survey conducted in 2006 by Scott Ambler, the Practice Leader for Agile Development with IBM Rational's Methods Group reported similar benefits. Others claim that agile development methods are still too young to require extensive academic proof of their success.
Large-scale agile software development remains an active research area. Agile development has been widely seen as being more suitable for certain types of environment, including small teams of experts.:157 Positive reception towards Agile methods has been observed in Embedded domain across Europe in recent years.
Some things that may negatively impact the success of an agile project are:
The early successes, challenges and limitations encountered in the adoption of agile methods in a large organization have been documented.
In terms of outsourcing agile development, Michael Hackett, senior vice president of LogiGear Corporation has stated that "the offshore team ... should have expertise, experience, good communication skills, inter-cultural understanding, trust and understanding between members and groups and with each other."
Teams implementing Agile often face difficulties transitioning from more traditional methods such as Waterfall development. Below are problems often faced by teams implementing Agile processes:
Adding stories to a sprint in progress
Adding stories to a sprint in progress is detrimental to the flow established by Agile. From Ilan Goldstein's Sprint issues – when sprints turn into crawls, "'Isn't the ability to change course on the fly what Scrum is all about?' Well not quite. Scrum certainly provides provision to change product backlog priorities mid-project however this needs to occur between sprints and not during them."
If there is an issue which arises which requires additions to a sprint, Ilan's recommendation is to perform an abnormal sprint termination. This does not mean that a user story cannot expand. Teams must deal with new information which may result in additional tasks for a user story. If the new information prevents the user story from being production ready during the sprint, then it should be carried over into the following sprint. However, during the next sprint planning, the user story should be prioritized in comparison to all remaining user stories. The information found requiring new tasks may have altered the priority of the user story.
Lack of sponsor support
Agile is often implemented as a grassroots efforts in organizations by software development teams trying to optimize their development processes. By not having sponsor support, teams may face difficulties and resistance from business partners, other development teams and management. "Carrying on without an effective sponsor raises the probability of project failure..." 
Additionally, the project sponsor is responsible for ensuring the team has appropriate funding and resources.
A survey performed by Version One found respondents cited insufficient training as the most significant cause for failed Agile Projects Teams have fallen into the trap of assuming the reduced processes of Agile compared to other methodologies such as waterfall mean there are no actual rules for Agile. Agile is a prescribed methodology and training/practice is a requirement.
Product Owner Role is not properly filled
A common mistake is to have the product owner role filled by someone from the development team. According to Johanna Rothman this is a mistake, "When the business is unaccountable, the agile ecosystem breaks down." Having the development team fill this role results in the team making its own decisions on prioritization without real feedback from the business. Additionally, business issues are either attempted to be solved internally or there are delays as the team must reach outside the core group for input. These issues can result in finger pointing and divert from the collaborative process directed by Agile.
Teams not focused
The Agile Process requires teams who are focused on the project to meet the project's commitments. It is expected during a sprint, for a resource who has the capacity, to take-up a task potentially outside of their subject area. If team members have multiple projects it will be difficult for any spare capacity to be available to help complete the sprint. "While having information developers working on multiple scrum teams is not ideal, it can be done with some proper planning and judicious evaluation of which meetings you should attend."
Teams may fall into the trap of spending too much time preparing or planning. This is a common trap for teams less familiar with the Agile process where the teams feel obligated to have a complete understanding of all user stories or a detailed design. Teams should leverage the ability for Sprints to act as a method discovery and moving forward with the information they do know. As more information is gained it should be applied to the next Sprint.
Problem-Solving in the Daily Scrum
The Daily Scrum is meant to be a focused, timely meeting with all team members to disseminate information. If problem-solving occurs it often can only involve certain team members and potentially is not the best use of the entire team's time. If during the Scrum the team starts diving into problem-solving, it should be tabled until a sub-team can discuss immediately after the Daily Scrum completes.
One of the benefits of Agile is it empowers the team to make choices as they are the closest to the problem. Additionally, choices are to be made as close to implementation as possible compared to a traditional waterfall approach allowing more timely information to be used in the decision. If team members are assigned tasks by others or too early in the process then the benefits of localized and timely decision making can be lost.
Another tendency is for assigners to box team members into certain roles (for example, team member A must always do the database work) which hinders cross-training. Team members themselves can choose to take on tasks which stretch their abilities and provide cross-training opportunities.
ScrumMaster as a contributor
Another common pitfall is for the ScrumMaster to act as a contributor. While not prohibited by the Agile Methodology, the ScrumMaster needs to ensure they have the capacity to act in the role of ScrumMaster first and not working on tasks for the project. A ScrumMaster's role is to facilitate the Scrum process. "Facilitating meetings such as a daily scrum, sprint planning, sprint reviews and sprint retrospectives is part of this. A technical contributor's role is to work with the other team members to figure out how to get the work done and to do it." 
Having the ScrumMaster also multitasking may result in too many context switches to be productive. Additionally, as a ScrumMaster is to remove roadblocks for the team, the benefit gained by individual tasks moving forward may not outweigh any roadblocks which are deferred due to lack of capacity.
Lacking test automation
Due to the iterative nature of Agile multiple rounds of testing for a project is often needed. "Having an automated testing framework, which takes care of both system and integration tests, adds a lot of firepower to such a team. It not only acts as a safety net against regressions caused by new development, but more importantly frees up a lot of precious developer and tester time - allowing them to focus on the things they do best." 
Test automation also supports continued refactoring required by iterative software development. Allowing a developer to quickly run tests to confirm refactoring has not modified the functionality of the application may reduce the workload and increase confidence that cleanup efforts have not introduced new defects.
Allowing technical debt to buildup
Focusing on delivering new functionality may result in increased technical debt. The team must allow themselves time for defect remediation and refactoring. Technical debt hinders planning abilities by increasing the amount of unscheduled work as production defects distract the team from further project progress.
As the system evolves it is important to refactor as entropy of the system will naturally increase. Over time the lack of constant maintenance will result in increasing defects and development costs.
Attempting to take on too much in a sprint
A common misconception is Agile allows continuous change, however a Sprint backlog is an agreement of what work can be completed during the Sprint. Additionally having too much Work-In-Progress(WIP) can result in inefficiencies due "to avoid the penalties of wasted time, effort and resources." 
A possible issue is the team being pressured into taking on additional work. "An important point to reiterate here is that it's the team that selects how much work they can do in the coming sprint. The product owner does not get to say, 'We have four sprints left so you need to do one-fourth of everything I need.' We can hope the team does that much (or more), but it's up to the team to determine how much they can do in the sprint."
Fixed time, resources, scope and quality
Agile fixes time (Sprint duration) and resources while the scope and quality remain variable. The customer or product owner often pushes for a fixed scope for a Sprint. However, teams should be reluctant to commit to locked time, resources and scope (commonly known as the project management triangle). Efforts which attempt to add scope to the fixed time and resources of Agile may result in decreased quality.
Agile methodologies can be inefficient in large organizations and certain types of projects. Agile methods seem best for developmental and non-sequential projects. Many organizations believe that agile methodologies are too extreme and adopt a hybrid approach that mixes elements of agile and plan-driven approaches.
The term "agile" has also been criticized as being a management fad that simply describes existing good practices under new jargon, promotes a "one size fits all" mindset towards development strategies, and wrongly emphasizes method over results.
Alistair Cockburn organized a celebration of the 10th anniversary of the Agile Manifesto in Snowbird, Utah on February 12, 2011, gathering some 30+ people who’d been involved at the original meeting and since. A list of about 20 elephants in the room (“undiscussable” agile topics/issues) were collected, including aspects: the alliances, failures and limitations of agile practices and context (possible causes: commercial interests, decontextualization, no obvious way to make progress based on failure, limited objective evidence, cognitive biases and reasoning fallacies), politics and culture. As Philippe Kruchten wrote in the end:
The agile movement is in some ways a bit like a teenager: very self-conscious, checking constantly its appearance in a mirror, accepting few criticisms, only interested in being with its peers, rejecting en bloc all wisdom from the past, just because it is from the past, adopting fads and new jargon, at times cocky and arrogant. But I have no doubts that it will mature further, become more open to the outside world, more reflective, and also therefore more effective.
Agile methods have been extensively used for development of software products and some of them use certain characteristics of software, such as object technologies. However, these techniques can be applied to the development of non-software products, such as computers, motor vehicles, medical devices, food, clothing, and music; see Flexible product development.
Agile development paradigms can be used in other areas of life such as raising children. Its success in child development might be founded on some basic management principles; communication, adaptation and awareness. Bruce Feiler has shown that the basic Agile Development paradigms can be applied to household management and raising children. In his TED Talk, "Agile programming -- for your family", these paradigms brought significant changes to his household environment, such as the kids doing dishes, taking out the trash, and decreasing his children's emotional outbreaks which inadvertently increased their emotional stability. In some ways, agile development is more than a bunch of software development rules; but it can be something more simple and broad, like a problem solving guide.
|Wikibooks has a book on the topic of: Software Engineering with an Agile Development Framework|