Introduction
Building software is hard. Engineering a valuable, scalable and robust digital solution is even harder. This is the process I’ve used in the past with different teams to solve this problem.
This page is a living document, and I intend to update it as new ideas, practices and reflections emerge to reshape the way I approach software engineering work.
Keep in mind that my perspective is biased. I am an engineer by trade. That might explain the slant toward engineering details and processes rather than other disciplines.
I also primarily work in Agile, with Scrum and Kanban my preferred flavors.
The Process
graph TD
BusinessContext["Business Context"]
TechnologyLandscape["Technology Landscape"]
UXStrategy["UX Strategy"]
DigitalProductVision["Digital Product Vision"]
RoadmapOfEpics["Roadmap of Epics"]
Epic["Epic"]
TechnicalDesign["Technical<br>Design"]
UXResearch["UX Research"]
UserStoryMap["User Story Map"]
Wireframes["Wireframes"]
HighFidelityDesigns["High Fidelity Designs"]
UserStories["User Stories"]
ProductBacklog["Product Backlog"]
AcceptanceCriteriaGherkin["Acceptance Criteria<br>with Gherkin"]
BDDandTDD["BDD and TDD"]
Code["Code"]
BusinessContext --> DigitalProductVision
TechnologyLandscape --> DigitalProductVision
UXStrategy --> DigitalProductVision
DigitalProductVision --> RoadmapOfEpics
RoadmapOfEpics --> Epic
Epic --> UserStoryMap
UserStoryMap --> TechnicalDesign
UXResearch --> UserStoryMap
UserStoryMap --> Wireframes
Wireframes --> HighFidelityDesigns
UserStoryMap --> UserStories
HighFidelityDesigns --> UserStories
TechnicalDesign --> UserStories
UserStories --> ProductBacklog
ProductBacklog --> AcceptanceCriteriaGherkin
AcceptanceCriteriaGherkin --Glue between<br>requirements<br>and code--> BDDandTDD
BDDandTDD --> Code
click DigitalProductVision "./#digital-product-vision" "Open Digital Product Vision"
click RoadmapOfEpics "./#roadmap-of-epics" "Open Roadmap of Epics"
click UserStoryMap "./#user-story-map" "Open User Story Map"
click Wireframes "./#wireframes-and-high-fidelity-designs" "Open Wireframes and High-fidelity Designs"
click HighFidelityDesigns "./#wireframes-and-high-fidelity-designs" "Open Wireframes and High-fidelity Designs"
click TechnicalDesign "./#technical-design" "Open Technical Design"
click UserStories "./#user-stories" "Open User Stories"
click ProductBacklog "./#product-backlog" "Open Product Backlog"
click AcceptanceCriteriaGherkin "./#acceptance-criteria-with-gherkin" "Open Acceptance Criteria with Gherkin"
click BDDandTDD "./#bdd-and-tdd" "Open BDD and TDD"
click Code "./#code" "Open Code"
Table of Contents
- Digital Product Vision
- Roadmap of Epics
- User Story Map
- Wireframes and High-fidelity Designs
- Technical Design
- User Stories
- Product Backlog
- Acceptance Criteria with Gherkin
- BDD and TDD
- Code
Digital Product Vision
The first thing we will notice about the process is that Business Context, the Technology Landscape and UX Strategy work together to inform the product vision. None of these functions can define the product vision alone, because knowledge from each cross-functional field are critical to developing a valuable, scalable product that serves users.
Without knowledge of the business context, designers risk designing something pointless and engineers risk building something useless.
Without knowledge of the technology landscape, business and design risk wasting time on infeasible solutions that turn out to be excessively costly.
Without knowledge of the user experience, business and technology teams commission and build unusable products that turn users away.
The product vision provides a direction, but is too big picture to wield effectively. To make it actionable, we need to break them up into workable chunks: Epics.
Roadmap of Epics
Epics
Epics are chunks of work that deliver clear business value. An epic is a well-defined concept in Agile / Scrum.
For business value, a hazy sense of “this will help users do X” is not enough. Goals need to be defined using SMART, so that the business value to be delivered is clear, measurable, and therefore comparable between epics.
Each epic also needs to consist of actionable development work that can be deliverable in a reasonable time frame. Epics that are too large risk being unreleasable and/or over-engineered.
A business and UX perspective alone cannot define the epics, because they have no ability to define how large or small individual pieces of work are. For instance, to decide whether an epic needs to be broken down into multiple smaller epics that each deliver their own pieces of business value requires engineering experience.
An engineering perspective alone cannot define the epics, because it lacks the lens of the kind of business value that can be delivered.
As always, all 3 disciplines need to collaborate and work together.
Roadmap
A roadmap is a list of epics, ordered by priority. The keyword here is “priority”.
Because the business value and estimated size of an epic are made transparent during the epic definition process, comparisons can be made to decide what needs to be worked on first.
Stakeholders are often not technical and primarily concerned with business value. This makes the roadmap an effective tool for communicating the status of development with stakeholders, because it tends to be very high level. This allows stakeholders to have a sense for which items are being worked on and prioritized, to assist in business planning decisions.
We need to also keep in mind that roadmaps are not about deadlines, but about priorities. Development work is fluid, with often unforeseeable blockers and complexities. Roadmaps can be used to shift the conversation from “when something will be delivered”, to “what are the blockers and how do we unblock them?”
User Story Map
A user will always take a series of actions in order to achieve a goal on a digital product â what we often call the user journey.
A user story map is a tool in Agile meant to map out this series of actions. This concept looks very similar to a User Flow diagram seen commonly in UX design.
In our context, a user story map takes a single epic, and breaks it down into a series of user tasks. This series of tasks take place on the digital product’s surface, and are meant to end with a user (or a set of users) achieving their desired outcome.
Based on personal experience, user story maps should always drive discussions about the product. It’s not just a matter of putting users front and center. It is also about building a shared view of the product at a useful fidelity.
The process of building a product is essentially a series of communications using various mediums (high fidelity designs, user stories). In communication, things that are not made explicit are unwittingly made implicit. And implicitness is a dangerous thing when communicating across domains.
By centering discussions on the user story map, we have (1) a sphere of understanding where all disciplines have the ability to think in, (2) an explicit rather than implicit understanding of the user’s intended goals and behaviors, and (3) a single source of truth that collects the shared decisions made by all parties. This allows for highly productive discussions where all functions can contribute effectively.
A user story map also integrates well with a Customer Experience Map, which is useful in mapping out the omnichannel experience of a customer.
Wireframes and High-fidelity Designs
On the design front, no design work starts without the user flows being discussed and agreed upon.
Wireframes are then employed for quick and dirty visualisation, which are then used to drive our high-fidelity designs. This approach allows us to move from a low-fidelity view to a high-fidelity view in terms of the UI/UX design process.
It might seem like extra steps, and some of these might seem unnecessary, but the principle is simple: low fidelity views abstract away the details and allow for richer and more constructive discussions.
Through the use of wireframes, conversations shift away from look-and-feel toward the actual functionality of the product. How often have we been drawn into unproductive discussions about colors and font sizes, while user needs are being sidelined? These are important considerations â but should come later in the design process. User goals must always come first.
Low fidelity views are also much lower cost, and easier to iterate based on feedback. The closer we are to the actual implementation, the more expensive it is for changes to be made.
Technical Design
Once the user story maps are complete, we can start the technical design process. Here we think about the modifications to the software architecture that will be needed to support the epic.
The following diagrams are useful for my teams and I when we work out software architectural changes:
- Flowchart Diagrams to map out the relationship systems and components.
- Entity Relationship Diagrams (ERD) to visualise the impact on relational data that needs to be persisted.
- Sequence Diagrams to visualise the flow of data and API calls between systems.
As you can likely tell, I am a student of Unified Modeling Language (UML). I am also a big fan of Mermaid, a tool that allows easy use of UML diagrams in Markdown. Yes, I am also a big fan of Markdown đ.
User Stories
User stories are my favorite way to start to define and flesh out requirements. User stories are written with the following format:
- As a [who],
- I want [what],
- So that [why].
In a user story, the user is the first thing that we have to define. As with the rest of our process, they are the centerpiece. Some examples of user stories are:
- As a airplane technician, I want to check out my tools from the toolstore, so that I can check if any tools are missing at the end of the work day.
- As a high-tech farmer, I want to check the soil moisture level in a field, so that I can know when to water my crops.
Who are they? What do they want to do on our platform? Why do they want to do it? These are all questions that a user story provides context into.
Good software engineering requires context. Lots and lots of it. And much of that context comes from the users and their perspectives. It is far easier to develop software that drives user needs when a developer understands the user and what they hope to achieve.
Context also helps to drive an understanding of the domain, which allows the use of approaches like domain-driven design that lead to scalable, robust implementations of software.
Product Backlog
A product backlog is an emergent, ordered list of what is needed to build a product. In our context, it is a list of user stories that are ordered according to priority.
Best practice dictates that the product owner has the sole accountability for the product backlog and its prioritization. They can delegate some of these responsibilities, but they hold the ultimate accountability.
Priotitization is generally decided by the product owner’s evaluation of each user story’s business value, weighed against engineering’s estimated effort required for delivery.
I cannot stress this enough: Ordering is critical to a product backlog.
On the business side, business needs are infinite, and requirements never stop coming in a product with high stakeholder interest. Ordering ensures that discussions with stakeholders focus on prioritization. If a stakeholder wants something first, they have to be able to justify why what they need is more important than what has already been prioritized in the product backlog. Like the roadmap, the product backlog acts like a shield for engineering, focusing discussion away from unrealistic deadlines (that are fatal for medium- to long-term engineering productivity), to what a product is really about, delivering as much value as possible.
On the engineering side, the product backlog dictates the order in which work is done. Without exception. Software developers need to respect the product backlog’s prioritization, and deliver work in the order decided on by the product owner. If there are any disagreements, the solution is not to ignore prioritization directives, but to raise them in conversation with the product owner. Healthy teams will be able to have these conversations in a productive and respectful manner.
Acceptance Criteria with Gherkin
Acceptance criteria refer to specific behaviors that a system needs exhibit, in order for a requirement to be considered “accepted”. In our context, they are parked under user stories.
User Story Maps make user journeys explicit, and user stories make user tasks and goals explicit. Acceptance Criteria take this explicitness and amp it up several notches. Low-fidelity to high-fidelity.
In my teams, we define acceptance criteria using Gherkin, which use the Given, When, Then keywords to give structure and meaning to executable specifications. These keywords generally represent a transition between states of a system being developed:
- Given the initial state of the system
- When the user performs a specific action
- Then the system should transition to another specific state
This way, the behavior of the user is made explicit in its relation to the code. Because of how Gherkin puts the user behavior at the front and center of the development process, it serves as a very powerful unifying language between requirements (user stories) and code (executable specifications).
BDD and TDD
BDD represents Behavior-driven Development, and is a software development practice that puts the expected behavior of a system at the heart of the development process. It is also frequently represented using Gherkin, which we use to define our acceptance criteria.
TDD represents Test-driven Development, and is an approach that puts testing at the heart of software development. Common industry practice is to first write code, then follow this up with tests. TDD insists that tests are written first â so that they can be used to drive development.
Both practices combined give us our glue between requirements and code. Our approach is as follows:
- User stories have their ACs defined according to Gherkin.
- Developers pick up the user stories, and write automated E2E test cases for those user stories first, following TDD practice.
- These test cases should adhere to Gherkin sytax, often assisted by testing libraries like CucumberJS, that encode Gherkin keywords into code.
- Developers then build the software that will pass these E2E test cases.
One can imagine that the transition from requirements to code is nearly seamless with this approach.
Code
This section is perhaps self-explanatory. Here software engineers do what they love best, code.
Supported by a process with the primary objective of driving communication across all functions, they build something useful, valuable and delightful to users.
Everyone wins.