Requirements Engineering (RE) can be summarized as a set of activities that ensures a team is building the right product. In the field of Software Engineering, hence, it seeks the correct and suitablesoftware deliverable.
Now, what do correct and adequate mean? Correct is a deliverable that meets stakeholder expectations, i.e. that it realizes their vision of the product, while suitable relates to whether it is fit for purpose.
RE consists of 4 core processes:
Elicitation, where business needs are gathered.
Analysis, also called business analysis (BA), organizes business (domain) knowledge and desired capabilities (requirements) in such a way that inconsistency, incompleteness, ambiguity and inaccuracy are minimized.
Verification & Validation, which checks the right product was built.
Requirements Management ensures most important requirements are delivered first and in accordance with scope control. It is executed throughout the entire cycle.
The main role involved and is usually the owner of these is the Business Analyst.
RE in the industry today
Despite how much RE has progressed over recent decades, mostly fueled by the booming software industry, its initial challenges remain. Even today, virtually all software projects have suffered from the issues mentioned above, with its consequent cost.
Software has evolved from the automation of business processes in existing industries to becoming business platforms in their own right, so agile methodologies have blossomed and have provided tools to support innovation and step into the unknown, delivering business value upfront and, therefore, improving ROI.
RE has had to evolve as well. Agile methodologies provide a framework for performing all the aforementioned processes following agile values and principles, while collaboration tools boost communication within and between teams, as information can flow freely and business knowledge can spread more rapidly.
Requirements Engineering in Scrum
Out of all roles, the Product Owner (PO) is the most heavily involved in RE without a doubt, as they are the “voice of the customer” throughout the cycle.
As described in this article, a Business Analyst can assist with the PO’s RE-related tasks (in different team configurations). A more detailed analysis of these tasks can be found below.
Maintain a visible and clear product backlog
For that matter, constant requirements Elicitation and Analysis is required. The PO can then take advantage of “old school” techniques, such as focus groups or brainstorming for gathering requirements, whereas ad-hoc modelling (such as UML) can be used for a better understanding of these requirements. Of course, software that works is always more important than extensive documentation, so models have to be used when they add up and as long as they make sense using. Once knowledge has been transferred to the team, they lose their value.
Most common technique for Product Backlog documentation is through User Stories, as they combine simplicity and flexibility in a somewhat structured manner.
Negotiate priorities and scope
Clearly a Requirements Management activity. It is fundamental that the PO understands what is most important at all times. Measuring business value and weight against cost is a core task of the role, as well as understanding the difference between the urgency of a request (how quickly it should it be done for it to make sense to exist) vs its value (importance). A useful tool from traditional RE is the urgency/important matrix, or even applying Pareto law.
One of the key activities of the Scrum Master involves helping the PO to maintain the Product Backlog in a way that ensures that the needed work is well understood so the team can continually make forward progress. Knowledge of requirements Analysis techniques can help greatly in achieving this task, as described above.
To be able to commit to a Sprint Backlog, the Development Team must be able to fully understand and flesh out each potential item from the Product Backlog. Requirement Analysis techniques will become handy in this joint team effort to understand items and their scope, as the Scrum Team will be wearing the “analyst hat” a fair amount during this exercise.
The Sprint demo is the most important milestone when it comes to requirements Validation, as stakeholders can validate the release (or potentially shippable increment, PSI) and provide first-hand feedback.
Backlog refinement (usually referred to as grooming) is actually not a core Scrum practice, but has been widely adopted as an extension to it in many teams.
A desired side-effect of backlog visibility is giving stakeholders the opportunity to validate requirement items early in the process as well as performing analysis in advance and, hence, lowering the chance of flawed requirements to enter the development cycle.
Other useful practices
“Good enough” Product Backlog and a state-of-the-art Sprint Backlog
Agile methodologies have adaptive lifecycles, which means they are meant to adapt to changes rapidly rather than provide long-term predictability (that is what predictive approaches are for). They can, but they are designed for the former.
The Product Backlog provides input for long-term predictions, but is it worth it? By choosing an adaptive lifecycle, we’re assuming change will be the rule rather than the exception and, hence, the further in the future our predictions are, the least likely they will be. Thus, the Product Backlog should be good enough to make informed decisions, yet there is a point where further refinement becomes inefficient.
On the other hand, Sprints are the opposite. They must be as predictable as possible. Therefore, the Sprint Backlog should be precise and fully understood before kicking off its development.
The bulk of the analysis effort should be focused on the upcoming stories. The higher a story’s priority is (i.e. the nearer in time it will be developed), the more time we should spend understanding it. This makes effective prioritization particularly critical in agile environments for project success.
Communication as primary tool
One of the 4 agile values states that working software is always preferred over extensive documentation. This is the most visible difference between traditional and agile RE: communication in the former is always through specific artifacts, mostly system models (structured, UML, etc.), while in agile information is related to each story and the system as whole will only be documented ad-hoc.
Successful teams communicate requirements and domain knowledge effectively, and put special focus on this, including suitable modelling tools, and applying user story writing best practices.
Inexperienced teams usually fail at establishing an acceptance criteria for them, resulting in requirements that cannot be tested. RE provides extensive literature on the subject, as well as proven best practices for capturing, assessing, and communicating them effectively.
Another challenge of non-functional requirements is the fact they commonly affect multiple features or even the system as a whole, making them difficult to prioritize. Literature and experience concur in that in these cases, they should be managed as a quality attribute that must be “built into” the features delivered.
There is a point in a solution’s lifecycle, where all further requirements in the Product Backlog provide little value, this might mean the “true scope” has been achieved.
Agile teams must look after customer’s pocket. The PO must constantly monitor the Product Backlog and be able to identify this scenario and act in consequence. The team must be capable of proposing improvements and extensions to system features, leveraged by domain knowledge gained during the project, and/or deeper architecture revisions that would enable the system to grow in new, different ways, or further improve their maintainability/ease of operation.