The Testable Design (Case Study)

This is a case study for use in a class like Software Design. It deals with moving from validated learning on customer/user needs to testable user stories.

A Season of Change & Disruption

“HVAC in a Hurry” (HinH) is now itself in a hurry. After decades of happy existence as a prosperous regional supplier of heating & air conditioning services for commercial properties, its industry is consolidating. The CEO, Mary Condor, feels they need to either scale, combine with another firm, or gradually be out-competed into insolvency. Scaling is her first choice.

She’s brought in Frangelico DeWitt to lead a change management and digital transformation program. His charter was to take the best of what HinH has learned and to use it to improve existing operations and facilitate rapid expansion into new regions (possibly through a franchising model). While resources are scarcer than he would like, DeWitt feels up to the challenge (with a first name like Frangelico, you have to be resilient).

Learning What to Build

It’s Friday afternoon, and the team is closing out a one-week design sprint to understand the company’s key roles/personas and the relative importance of their various tasks/problem-scenarios. Based on what they learned in the interviews, the team thinks the most valuable early wins will involve improving the interface between dispatch and the HVAC field technicians. The dispatchers work at headquarters and field calls and emails from customers as well as technicians about HVAC repairs. Technicians are dispatched to customer sites to execute jobs (installation, maintenance, and repair). Basically, the idea is to take what individuals at the the company have learned works well and to standardize and automate it (where possible) with updated software.

The team has tuned their current charter to focus on three design challenges:

1. 1. How might we improve the troubleshooting and problem resolution dispatch is able to do over the phone to reduce site visits and improve customer retention?
   2. How might we improve the preparation of technicians when they go on site for a job to reduce time on site and the number of visits per job?
   3. How might we improve the self-service capability of technicians on site to reduce downtime on site, the number of visits per job, and customer retention?

Solving the Right Problem with the Right Solution

To close out their current sprint, the team plans to focus on developing user stories for both of the first two design challenges (#1 & #2 above), since they’re so closely related. In terms of the specific problems and propositions where they think there’s a likely win with some new software, they’ve sketched out the following focal points:

Problem Scenario (JTBD) + Current Alternative	Proposition	Outcome Metrics
PS) Resolving Customer Problems Remotely Alt.) Dispatch does this. The level of know-how and success varies a lot between dispatchers.	A) If we create a best-practice troubleshooting tool for common problems, dispatchers will use it and it will improve resolutions over the phone. B) If we create a post-mortem and review infrastructure, dispatchers and tech’s will engage on improving and expanding the troubleshooting tools.	– Increase in % Issues Resolved Remotely – Increase in Customer Retention (Av. Contract Length)
PS) Finding Available Technicians with the Right Skills Alt.) Dispatch has a means to see availability with the current system.	(There might be a win around more sophisticated filtering and selection based on skills and experience with a particular customer, but it doesn’t look like as easy or obvious a win as the other items we’re working. It’s on the product backlog for now.)	– Increase in Customer Satisfaction per Job – Increase in Customer Retention (Av. Contract Length)
PS) Preparing Tech and Customer for Job On Site Alt.) Right now, it’s just free-form notes and the tech’s feel they’re encountering the same kind of problems over and over- missing information about accessing the site and the nature of the problem so they can bring the right parts and equipment.	A) If we create a set of adaptive checklists based on the information on hand in the account and the type of job, dispatchers will use it and tech’s will be better prepared for jobs.	– Decrease in Time on Site – Decrease in Site Visits per Job – Increase in Customer Retention (Av. Contract Length)

From Validated Problem to Testable Solution

To close their design sprint, the team wants to draft a set of user stories and rough prototypes to make sure they’ve thought through the specifics of how they might deliver the propositions they’ve identified. The basic idea is just to make sure they’ve thought through a practical, realistic user experience that would deliver on the propositions.

To make sure they’d really thought through the delivery of their propositions, the team started with storyboards.

Storyboard 1: Handling Customer Calls

This storyboard is a take on the process the team has learned about where the dispatch is taking an initial call and figuring out what to do next. This take is mostly focused on repairs (calls for installation and maintenance differ some). Her goal is to either resolve the issue on the call or make a specific conclusion about whether they need a site visit and what needs to be done.

Panel	Notes
A	Here Danielle the Dispatcher take an inbound call from a customer who has a time-sensitive HVAC issue. Danielle needs to bring up their account for background and to make sure they’re a current customer.
B	Danielle the Dispatcher steps through a few questions, isn’t able to resolve the issue.
C	Danielle has an idea about the fix and thinks it might be something where of the technicians could suggest a remote fix. She asks for a snapshot to send to a technician who’s been working at the site.
D	Carl the customer takes the requested photo and sends it.
E	Danielle asks Tess if she can take a look at the issue. Tess the Technician gets the photo and a question via text from Danielle.
F	Tess the Technician responds with a possible fix but thinks the fix will probably require a visit.
G	Danielle the Dispatcher gets back in touch with the customer, who tries the fix. The fix doesn’t work and Danielle determines that they’ll need to schedule a site visit.

Storyboard 2: Scheduling & Prepping a Site Visit

In this storyboard, Danielle (or Daniel) the Dispatcher has concluded that they need a site visit to complete a repair. Her goals is to schedule the site visit with the best available tech, making sure both the tech and the customer are ready to do what needs to be done to complete the repair.

Panel	Notes
A	Daniel has made the determination that HinH needs to do a site visit to make the repair and needs to arrange this with the customer.
B	Danielle needs to schedule a time that works for the customer and where there is an appropriate technician available.
C	Danielle wants to make sure the address and contact information is current and to update the technician if there are any particulars for this visit (ex: alternate contact).
D	Danielle wants to make sure all the right contacts at the customer know when the visit is happening, the scope of the visit, and can easily update it if they require changes.
E	Danielle wants to add a new alternate contact to the account for future reference.
F	After 48 hours, Carl the customer needs to move the visit to the next day.

Towards a Testable Design

As the team sits down to draft stories, they have a few key priorities. First, they want to be detail-oriented- not about the specifics of the implementation, but about the experience a user would have that delivers their target propositions. This is a key feature of user stories: they are great at working out user experience but avoid the mistake of over-specifying the implementation details prematurely. They’re a focal point for discussing implementation; not a specification.

Second, they want to make sure they think about the ‘messy’ parts of the experience- times when things don’t go as planned. They know from experience that glossing over these situations is often what causes users to abandon or underutilize a system. This doesn’t mean they plan to cover every theoretical corner case- just that they won’t ignore the inconvenient details they’ve learned about in their interviews.

Third, they want to make sure their user stories are testable. Probably the best way to make sure you have a coherent user story is to ask yourself ‘Could I mock up an interface for this, prompt a user to try to execute the action in the story, and then specifically observe whether or not they achieved the target goal/reward?’. For example, if you were making a mobile app for salespeople to log their customer appointments, you might ask them to show you they’d add a note for XYZ customer that says (something) and then ask them if they think it was saved successfully.

Working from the storyboards, the team plans to do the following:

1. Draft an ‘epic user story’ to summarize the interaction described by the storyboard. What initiates or triggers this story? When is it done and successful from the perspective of the user? 
2. Detail individual ‘child user stories’ within the epic to detail the component interactions in more detail. Do you think the storyboard is missing important details? It’s normal and healthy to swivel between the storyboard and the user story as you develop the details. Do you see any details they might have missed?

The Littlest Tutorial on Agile User Stories

TL;DR User stories are a generally accepted tool for modern software development. They have the format ‘As a [persona], I want to [do something], so that I can [achieve some reward/conclusion].’.

User stories have a specific format:

 “As a [persona],
I want to [do something]
so that I can [realize a reward]”

This format is designed to help the story writer be descriptive about the target user experience. The purpose of user stories is to drive better discussions about implementation with the rest of their team. Done right, the format helps prompt the following important questions:

Stories are generally organized around ‘epics’. These are user stories (same format as any other story), but they summarize the user experience where the balance of the user stories (I refer to them as ‘child’ user stories) detail the individuals components of the experience.

For example, let’s say you had this storyboard of an HVAC technician finding that they need a replacement part to complete a repair:

An epic that summarizes the interaction might be something like: ‘As Ted the HVAC technician, I want to identify a part that needs replacing so I can decide my next steps.’

From there, you’d move on the detailing individual user stories. If you’re starting from storyboards (which is a great idea!), don’t force a 1:1 relationship between the storyboard squares and the user stories. You may find some squares require more than one user story and others are just background and don’t really need their own story.

Here’s an example set of user stories that detail the interaction and test cases with supplemental detail and notes:

EPIC: As Ted the HVAC technician, I want to identify a part that needs replacing so I can decide my next steps.’

CHILD USER STORY	TEST CASES
I know the part number and I want to find it on the system so I can figure out next steps on the repair.	Make sure it’s possible to search by part number. Make sure descriptive info. appears as the search narrows (photo?) to help avoid error.
I don’t know the part number and I want to try to identify it online so I can figure out next steps on the repair.	Make sure it’s possible to search by make/model of units Make sure it’s possible to search by type
I don’t know the part number and I can’t determine it and I want help so I can figure out next steps on the repair.	Make sure an estimate of the turnaround time for an expert to review is available
I want to see the cost of the part and time to receive it so I decide on next steps and get agreement from the customer.	Make sure it’s possible to dispatch a request by email to the customer in case they order their own parts and/or carry their own inventory of spares. NOTE: How would the customer respond so we can help structure the next steps as we would otherwise?
	Make sure it’s possible to indicate priority Make sure cost associated with priority delivery are available
I want to order the part so that I can complete the repair.

For more on stories, check out the online tutorial: Your Best Agile User Story.

The INVEST Checklist for Better User Stories

Writing great user stories and (more importantly) driving great discussion and development with them takes practice. Below are a few key focal points for you to consider as you draft your stories. These are based on the INVEST checklist by renowned agilist Bill Wake.

Individual

Is the story integral- does it have all three clauses ([1: As a persona], [2: I want to [do something]] so that I can [3: achieve some kind of reward/conclusion.]?

Does the story make sense as a stand-alone item? Could you sit down right now and prototype it? Making sure you’ve broken down the experience into discrete pieces will make your design better. Making sure you can work in small batches will make your practice of agile better.

Negotiable

Is the story about the experience you think the user should have vs. how that experience should be implemented? This is key to your functioning effectively with your team and generally making sure you stay focused on doing what actually makes sense for the user vs. checking off items on a to-do list (which is easier but not effective).

Here’s an extreme example: ‘As a user, I want to push a red button, so I can submit the contact form.’. This story has no reason for existing- if somehow you’re just utterly convinced you need a red button, do a prototype that shows how it would look. Instead, this story should be more about what this user is hoping to achieve but submitting the form and how they’ll know if they’re going to get it.

Also, avoid terms like ‘easily’, ‘quickly’. No one is every tried to build software that’s difficult or slow- it happens all the time but it’s not because someone forgot to add ‘quickly’ to their user story! If there’s some kind of benchmark you ultimately need to reach with users, that’s a good item to add to your test cases.

Valuable

What validated learning do you have that suggests this user story is worth building? What measurable outcome would it improve? See the above section Learning What to Build for more depth on this.

Estimable & Small

These aren’t the same, but they’re tightly related. If it’s hard for a developer to roughly estimate the level of effort to implement a story, it’s very likely that the story is too vague (IRL, this is what I observe 92.4% of the time). It’s also possible (though far less likely) that the developer or team is thinking of an approach to implementation that’s more expansive than is necessary or product. For perspective on this, check out Bill Wake himself discussing ‘YAGNI’: Bill Wake on Yagni.

Testable

Personally, this is my number one with a bullet. Why? A highly testable story is very likely to score well on all the elements above (in my experience). Key to this is the third clause of the story: ‘…so that I can [achieve some kind of reward/conclusion].’. Making sure you can think through the user experience to a testable conclusion is not only key to moving forward with implementation and launch of this functionality, it will help you be more thoughtful about your design.

Better Brainstorming by Dot Voting

The basic idea with dot voting dovetails with a core tenant of design thinking and the maker/doer culture in high-tech: ‘Let’s think and sketch for a minute before we all just start saying stuff to each other.’ For example, instead of having five people talk at someone who’s trying to write a single user story on a white board, we could probably have all five people write their own version of a story in the same amount of time. Then, not knowing who’s story is whose, we could vote in which stories (or storyboard or prototypes, or whatever) we think are best.

The idea is not to elect a single story (or whatever) and obey it as the supreme leader of all other stories (or whatever). The idea is to look at which items got a lot of votes and discuss what it is about them everyone liked. Then you can either use them as-is or (equally good and maybe better) revise them based on what you discussed.

A standard recipe is to issue 1/3 + 1 dots per set of items. For example if the HinH team had six people and they all did an epic and a set of user stories + prototype(s) and the team was going to vote on those as a set, everyone would get 3 votes.

Creating Prototypes with Balsamiq

For notes on using Balsamiq, check out:

a) Balsamiq Section of Prototyping Tutorial

b)  Creating Interactive Prototypes in Balsamiq.