Today, I will share how I like to review a design produced by my design process:
I call this a ‘semi-formal’ design review, because it leverages key aspects of a ‘formal design review’ such as:
- the design is built from a structured template tailored to our problem space and is written to be read and reviewed
- it is supported by a design template and list of idea generating questions tailored to our problem space
- reviewers are provided dedicated time to read and focus on the design
- reviewers are prompted with ‘critical questions’ they can use to channel or inspire their feedback
- review feedback is captured in written and/or video form
But I wear a t-shirt while running it and I’m not tracking defect detection data to confirm it is as effective as the ‘Formal Design Review’ method described in Code Complete, 2ed.
This process is based on twenty years of experience gathering feedback on software designs and my understanding of how Amazon does design reviews. The objective of this review is to get subject matter experts and stakeholders to help you to identify about half of the latent defects. You can use this information to improve the design, adjust the project or plan, etc.
The mechanics
The review process is simple, but has a twist that makes it work for busy people.
Prepare for the meeting:
- Determine who you need to get feedback from
- Tidy up the design doc from those reviewers’ perspective
- Request a meeting with required and optional attendees and a conference room or video conference
Set the meeting’s agenda to:
Objective: Read, discuss, and gather feedback on ‘design name’
Design doc: url to doc
Agenda:
- Read and annotate design doc with questions and notes in silence (20-30min)
- Discuss questions, concerns, and alternatives (about 30min)
- Summarize feedback and any decisions (5 min)
Critical questions:
- Problem & Solution Definition: Does persona ‘X’ really have this problem? Will this solution help them? How much?
- Architecture and Impl: What’s the simplest thing that could possibly work? What could we remove, defer, or implement with a simpler technology and still satisfy the user’s needs?
- Security: Does this design control access to data, services appropriately at each component in the architecture? Suggestion: Identify them on the physical architecture.
The ideal setting for the meeting is a physical meeting room. You can also use a video conference tool. In that case I recommend recording the meeting. Print a copy of the design for each reviewer on paper before the meeting and get a box of pens. Reviewers will mark-up the document in pen. Put computers away if in a physical room or ask people to minimize all apps except the video conference tool and enable ‘Do not disturb’ mode. Now is the time to focus. Suggest to reviewers that in addition to the expertise they bring to the table, there are some ‘Critical questions’ you’d like them to consider during this review.
And then run through the agenda above.
Yes, the first half of the meeting is reserved for reading and thinking about the design in glorious silence. Finally — reviewers have some time to think.
Reading a design document in silence may feel weird at first, especially on a video conference. However, this gives your reviewers a chance to immerse in your problem and solution space and navigate through your document on their own terms. This is much more informative, interactive, and intense engagement in the material than the author leading the group through slides or even the design doc.
Check if reviewers are ready to go at the 20 minute mark and transition to discussion by 30 minutes.
Kick off the discussion by asking each reviewer what questions they have in turn. This will help fill in ambiguities in the document and also signal to authors where the broader design discussion will go. The first few times I started a design review this way, I was amazed at how good the questions were (no longer amazed). They were specific, grounded in context, and usually straightforward to answer. With this approach, reviewers do not have to guess at what is going to be introduced later. They’ve already seen the context you’ve provided and were supported with prompting questions embedded in the design template.
Next, move on to a broader discussion about key elements of the design. I usually follow a progression like:
- problem and solution definition
- general reviewer concerns and suggestions
- risks
- security
- availability
- scalability
Why it works
Thirty minutes of discussion is enough time to cover the most important concerns or record them for follow-up in my experience. This is possible for a few reasons.
First is that the design’s authors’ knew this was coming and had to think through and communicate the most important elements of the design quickly.
Second, the reviewers started off with uninterrupted, quiet time where they were able to focus on the design and think critically about it.
Third, authors and reviewers are supported by the list of ‘critical questions’ to help generate constructive feedback.
At the close of the meeting, the design authors should summarize the feedback and decisions. They should also collect the reviewers’ annotated designs or obtain the video conference recording.
Now the authors can integrate the feedback into the design and run through the development cycle once more.
#NoDrama
