Agile- Just the Basics

If you’re wondering what you need to know about agile, you’ve come to the right place. We’ll also talk about how agile relates to the dizzying constellation of ideas and terms that orbit it.

Does this one page have everything you might need to know? No. Does it have everything you need to get started and focus in the right places? Yes! (At least, I’d say so.) Let’s start.

What is agile?

Agile-Team--s-200pxIn 2001 a bunch of software developers (to the best of my knowledge, all dudes) got together in Snowbird, UT. They thought there was probably a better alternative to the big huge plans they were using to build software. In hindsight, they were right. (Thanks guys.)

They published the Manifesto for Agile Software Development. It sounds big, but the core of it is only 68 words.

Basically, it says:

Individuals
Interactions
 
Processes
& Tools
Working
Software
 
Comprehensive
Documentation
Customer
Collaboration
 
Contract
Negotiation
Responding to
Change
 
Following a
Plan

Side Note– I often encounter misunderstandings on two of the terms above:
1. ‘Documentation’
When they talk about ‘documentation’, they mean big, long specification plans. If you feel you need to deliver documentation, help, etc. to your user, agile doesn’t suggest you do otherwise.

2. ‘Contract’
By ‘contract’, they don’t just mean a legally binding contract. They also mean implicit contracts, like the idea that we should make sure product management writes sufficiently detailed specifications and then we should make sure that engineering writes ‘correct’ software for that specification that never breaks. Yes, it turns out that approach isn’t very functional and structured collaboration is better. More on that soon.

That’s the core of it and that’s about the only part of agile that everyone will agree is agile.

On the one hand, I feel like the manifesto is perfect. On the other hand, I totally get that it’s not actionable. This is where practicing agile starts.

Basically, your job as a (de facto) agile coach is to do three things:

  1. Pick and choose the methods you think will help you and your team
  2. Discuss and adjust with your team and make sure you all are applying those methods (within the limits of good judgement)
  3. For each iteration, assess (with said team) how those methods performed and go back to #1

Simple, right?

Like most ideas of its type, it’s simple in concept and complex in practice. Practicing agile is hard. If you want to cross things off your list, confident you did ‘your job’, don’t do agile. It’s hard. Hour for hour, you’ll be thinking harder and digging deeper to make sure you understand your colleagues. All that said, agile is the best way to do meaningful work and drive to innovative, valuable outcomes with a team.

Get to the Point: How do I start with agile?

If you’re joining or forming an agile team, I really would try to at least skim the contents of this page. That said, if you just want to get started as quickly as possible, here are my top three favorite practices.

The User Story

agile-user-story-150-pxThe user story is the primary input to the development process. It has a specific syntax with 3 clauses:

‘As a [persona/user role],
I want to [do something]
so I can [achieve some benefit/reward].’

Use this syntax to focus your understanding of what’s going to be valuable to the user. Don’t design or build anything you can’t link back to a user story. For more, see the related section below. For even more, see the User Story Tutorial here on the site.

Narrative Collaboration

narrative-collaboration-150pxStories themselves are not a substitute for ‘requirements’/big plans up front. Stories + discussion, frequent narrative collaboration using stories, is a substitute for requirements. This is where agile fundamentally differs from traditional processes: to do it in a serious way involves fundamentally changing the way you work. If your product managers are slamming user stories into JIRA and then seeing what development builds weeks later, you have lots of room to improve.

Small Batches

agile-small-batches-150pxBuild your product in 2-4 week iterations (sometimes called sprints) and discuss your work together in brief, but focused ‘daily standups’ (see section below on that).

At the end of that iteration, you should have working software that is ‘potentially shippable’ and that you can evaluate against what it’s supposed to do for the user (even if that user is internal, even if it’s an API/service another developer will use). Super important here is decomposing your work into ‘thin’ end-to-end slices. If your feature/area has the basic flow 1) search 2) edit 3) share, then don’t do all the search stuff in iteration #1 and then move on to edit. For more, see the section below ‘Agile and Story Mapping’

If you’re rearing to go, is it really OK to give agile a try with just these three items? Personally, I would spend a little more time skimming the sections below and digging in where you see the need, but, sure—it can work. The key thing will be that you remain very observant about what’s working well, and at the end of your iterations you and your team consider how to evolve your practice of agile to make it more effective.

The Agile User Story

Why it’s Important

There’s a reason I’m starting the topics here: you can run a perfect project, work super hard, and if your understanding of what will be valuable to the user isn’t strong, the only thing all that work will amount to is good practice on building things. And this isn’t just a theoretical truism: we software folks build a lot of software that users don’t like and don’t use. The applicability of the various figures are, of course, debatable, but it may be as high as ⅔ (low or no use) for existing products and IT and a 90% failure rate for new products.

So, understanding your buyers and user is really important. If you’re struggling to make the case, think about it this way: Let’s say the portion of features your users use very heavily is 40%. What if you could take that to 60%? It would be like increasing your development budget by 50%. Amazing!

What it Is

Maybe you saw this above, but the user story has a specific syntax with 3 clauses:

‘As a [persona/user role],
I want to [do something]
so I can [achieve some benefit/reward].’

The first clause establishes who you’re talking about, the second clause what they want to do, and the third clause why you assume they want to do this thing and how you’ll know if they were successful. The user story is a ‘testable narrative’. Testable means you could sit down a user, ask them to attempt to achieve a goal, and observe whether they think they did it. For example, let’s say the user is saving a record of some type. When they’re done, are they clear that it’s saved and OK (or not)?

How it Works

User-Story-200pxIt may seem a little ‘loose’ to organize all your development around a bunch of stories. Mostly, I’d say try it out—I think you’ll find it works well. But also there is some layering and organization. Epic stories broadly describe an interaction, and child stories detail the individual steps. Test cases then offer a place for more detail on the stories.

Size matters and stories should be small. Particularly if you’re new to product design, you’ll probably start by writing stories that are way too big and broad. When you discuss the stories with your team, hopefully they ask you about facets of the story that (likely) will make you want to decompose it into more detailed stories. If that happens, don’t worry— it’s natural and healthy.

For more detail and examples, see the User Story Tutorial here. To learn more about how to improve your User Stories, check out my agile course on Coursera: Agile Meets Design Thinking.

User Stories and Personas

Why it’s Important

How do you write great stories? Most importantly, you have to know your user. How do you do that? Well, you have to go talk with them in a relatively specific way and organize your observations into personas and problem scenarios. These will create a nice strong foundation not only for your user stories, but all the other questions you’re bound to have about how to build, sell, explain, etc. whatever it is you’re making.

What it Is

Think-See-Feel-Do-200pxThe persona is a humanized view of your customer, be they buyer and/or user of your product. In addition to a general description (ideally supplemented by photos or even a day in the life narrative), it’s common practice to lay out what your persona thinks, sees, feels, and does in your area of interest.

The problem scenario is a statement of something the persona wants to do—it may be a job/task or a habit/desire. The idea is to make sure you know what problems the persona wants to solve so you don’t build something that doesn’t matter to them. This also sets you up to ask: ‘What alternative are they using today and how do I know what we’re doing is better?’ But more on that when we get to the section on Lean Startup.

How it Works

Personas and problem scenarios aren’t something you check off the list. If they are, I’d recommend not investing the time. Personas are a durable asset you use to ask, answer, and test the right questions about what’s valuable to your user. Ideally, you work on them iteratively and they get better over time, kind of like this:
personas-process
I find that drafting first helps teams surface all the things they didn’t realize they need to know. For this, I use my personas template on Google Docs. Following that, I recommend drafting the interview guide and then interviewing subjects.

From an agile perspective, the purpose of personas is to write better, more valuable stories to discuss with your team. As you may remember from the last section, the purpose of those stories is to create a strong understanding within your team about what’s valuable to the user. They’re only a substitute for that bad old practice of ‘big requirements’ when they’re used to drive regular narrative collaboration within the team. The next section deals with a great tool for facilitating that: the story map.

Other than the personas tutorial I mentioned above, the Customer Discovery Handbook is a good guide. If you want something with more structure, course 1 of my Coursera class (Agile Meets Design Thinking) steps through the process, including live dramatizations!

User Story Mapping

Why it’s Important

As I mentioned above, if your product managers are slamming stories into JIRA (enterprise software for development) and then waiting around to see what development produces, you have lots of room for improvement. The job of your stories is to drive interdisciplinary collaboration on an everyday basis. Everyone should constantly be asking ‘Why?’ and ‘How do we know/will we know if the user cares?’ and ‘What have we learned so far in this area?’.

You should literally do every single thing you find helps spur these questions. Jeff Patton formalized and popularized one method I particularly like: the user story map.

What it Is

The user story map has various stripes. The first one has a storyboard and the subsequent stripes sets of user stories:
User-Story-Map
The job of Stripe 1 is to showcase the team’s shared understanding of what’s happening with the user (via various personas) over the course of the relevant journey. In terms of stories, this will almost certainly span multiple epics.

The job of the subsequent stripes is to organize and prioritize slices of the narrative in Stripe 1. If your feature/area has the basic flow 1) search 2) edit 3) share, then don’t do all the search stuff in iteration #1 and then move on to edit. You’ll end up running out of time and not have something functional you can ship or demo and it won’t be because you didn’t code fast enough: it will be because your prioritization was broken.

How it Works

Great news—if you’ve been following the tutorials and/or agile course above, you already have the inputs for a story map. All you need is a big space to post up your storyboards and some discriminating judgement about how to ‘slice’ your stories.

And, yes, I can’t help but mention: you can find out more about this in the user story tutorial and my UVA/Darden School of Business online agile course at Coursera.

User Stories vs. Use Cases

This is one of my least favorite topics, but it merits coverage: What’s the differences between a ‘use case’ and a ‘user story’? In terms of raw frequency, I usually hear ‘use case’ is used to mean very generally (too generally, I would say) ‘something a user might do’. You might hear a CTO or product manager talk generally about a ‘use case’.

While it’s good to think about things from a user perspective, 98.6% of the time I’ve heard this term used, the situation would benefit from a little more structure and specificity about who the user really is and what they want. I know, I said I was not orthodox, but in this particular matter I have always found the user story (with all 3 clauses carefully considered) to be the best prescription.

There is also a clinical definition of use case as an alternative to user stories (Wikipedia page).

Agile and Prototyping

Why it’s Important

Agile-Prototype-Wireframe-500px“A prototype is worth a thousand meetings.” You may have heard that one, and, yes prototypes are a really great vehicle for arriving at the shared understandings that are so crucial to agile collaboration.

They’re also important for iterating to good outcomes. The Manifesto emphasizes ‘working software’, but what they really mean is something that you can meaningfully test. Testing early and often is core to the practice of agile. Prototypes are a good way to try different approaches without over-investing in something that either isn’t well liked by users or isn’t as great as it could be.

What it Is

Prototype is a pretty general term. If you draw up a screen on a whiteboard with your team, that’s a prototype. If you use a wireframing tool like Balsamiq to make an interactive prototype for user testing, that’s a prototype. If you do a similar thing in Keynote (or Power Point), also a prototype. If you do the same thing in HTML/CSS/JS, still a prototype.

How it Works

There are a lot of ways to use prototypes, but if you’re looking for something specific and relevant to try with your team I recommend the following:

  1. Pick a user story. It’s not super important which one, but make sure it’s relatively specific.
  2. Choose a prototyping tool. Balsamiq is a good one, but there are many fine alternatives, pencil and paper being one of them. One thing I would recommend is making sure to stick to established UI elements as you sketch- drop-down boxes, text input fields, etc.
  3. Pick four different comparables or patterns. These are sites or UI patterns that are functionally relevant to whatever it is you’ve established/assumed the user wants to do in the user story.
  4. Create four parallel prototypes, one for each comparable/pattern. Yes, four. This will give you a strong and practical view of what prototyping for innovation is about: trying lots of alternatives so you can find one that’s really great.

From there, you might want to develop them out and make them interactive for testing. Generally speaking, some kind of customer/user testing to establish the best direction is the best next step.

If you want an expansion of the above, there’s a prototyping tutorial here on the site. If you want a deeper dive into more structured understanding of prototyping, my Coursera course Agile Meets Design Thinking steps you through the process starting with a user story.

Agile and Acceptance Testing

Why it’s Important

Arriving at a shared understanding of what your team means by ‘acceptance testing’ is important because in the kind of development agile hopes to improve, it is strongly associated with ‘contract negotiation’ (one of the bad things from the Manifesto).

Asking ‘How will we know when it’s done?’ is a loaded yet super constructive question in agile—and innovation, in general. In agile, the ‘working software’ we’re after is ‘potentially shippable’, meaning it breaks minimally and/or predictably enough that as a product team you think it would be OK to put in front of real users. But is that it? Does it have to be usable? Used heavily by users? Should it ultimately create some kind of observable outcome for the user (lose weight, get a job, etc.)?

Unfortunately, usually (not always) the term is used to mean that some ‘business user’ says that the software is OK and therefore engineering is done and the contract has been satisfied. That kind of interdisciplinary ‘contract negotiation’ is something the agile Manifesto fundamentally suggests you avoid. Instead, an interdisciplinary team is working together toward a target outcome they can break into a series of observable, measureable tests and events. They will probably have some do-overs. They will have to instrument observation into what they do and wait for results from observations in the field. It will be messy, but it will be worth it.

What it Is

I checked, and as of today this is the first sentence on the Wikipedia definition of ‘acceptance testing’:
‘In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met.’

Often in agile, there’s the idea that on the back of the card (physical or virtual) where you write your user story, you write test cases. I think this is a good idea as a way to add additional detail to the story and surface important questions for discussion. As agile thought leader Bill Wake once said to me, ‘Sometimes, you just need to establish that x + y should equal 14 to make sure everyone knows how it should work.’ (Or something similar—I’m paraphrasing).

Those kind of tests are good. But saying, ‘You have to tell me now the test we’ll run to call this done, and then that’s it’, is setting everyone up for a hard working environment and the user for something they probably won’t like.

But how can we get the businesspeople to do their homework and give engineering good inputs? Good question. They should get outside the building and talk to users and generally know that the software engineering is about to build is a good investment. That’s their job. Furthermore, asking for tests and examples may be a good way to show them the level of thoughtfulness and specificity the team needs. But, overall, you’re much better off creating a collaborative environment for that with a shared understanding of the target outcomes. It’s hard. Agile is hard (but worthwhile).

How it Works

Basically, it doesn’t work this way. Examples are good, tests are good, but setting specific criteria up front to call something totally done is not a good idea.

One simple formulation I like is that idea that there are 0, 30, and 90-day test criteria.

0-Day: It tests as usable by way of structured, phase-appropriate user testing (see below on that).
30-Day: Users are using it regularly/at roughly the frequency you established to call the feature ‘relevant’
90-Day: It’s contributing to whatever is the target outcome for the user- selling more, collaborating better, etc.

Note: None of this deals with whether the software ‘breaks’. For that, see the sections below on TDD (test-driven development) and continuous delivery.

If you want to learn more about testing against an outcome, I step through answering these three questions in my agile course Testing with Agile:

  1. Should we build it? (And then: Did it matter?)
  2. Is it usable?
  3. Did it break?

Agile and User Testing

Why it’s Important

The basic idea with agile is to create a shared understanding about what constitutes a valuable outcome for the user and to iterate towards that in small batches. Usability is always part of getting to that valuable outcome, so most agile teams agree (at least in concept) about the importance of testing usability early and often.

In practice, many teams struggle to accommodate regular usability testing. You’ll often hear something like, ‘We can’t test it until it’s done, and, by then, we’re out of time.’ However, with a little practice, anyone can do user testing, and they can do the early testing (most critical) without a line of code.

What it Is

Fogg-CurveUsability deals with how much effort it takes your user to achieve their goal. A usable product is not necessarily a popular product—yes, it’s sad but true. There’s also motivation. Motivation deals with how motivated your user is to use your product/proposition to achieve their goal. They’re different and, believe me, I see tons of beautifully executed products that users aren’t motivated to use.

To explain the difference while still bearing the relationship in mind, I generally turn to my favorite curve: The Fogg Curve. (Thank you, BJ Fogg). Basically, what the Fogg curve shows is that there’s a relationship between motivation and usability: enough of either will get you action from your user; the reverse is also true. If you think about a point on the curvey in the upper left, that’s a case where the user is very motivated and you’ll get action even if the action is hard (usability is low). If you think about a point in the bottom right, that’s a situation where the user isn’t that motivated, but if it’s really easy (usability is really high), they’ll act.

The key point here is a working definition of usability. For more on motivation, see the section below on ‘Agile and Lean Startup’. Usability testing deals with controlling for motivation and testing usability. So, if you have a software that paints virtual walls, you do not ask your test subject, ‘Would you like to paint a virtual wall?’ You don’t even ask them ‘What color would you like to paint this wall?’ Both of those deal with motivation. If you’re testing usability you just put them in front of your interface and ask them, ‘Show me how you’d paint this wall yellow.’

How it Works

I said in the beginning of this section that it’s important to test usability early and often. It sounds incredible, doesn’t it? A statement kind of like ‘You should eat plenty of vegetables.’ There’s a reason.

As we discussed in the section above on prototyping, in practice you get the best outcome by testing several meaningful alternatives for an interface or interface element. Since the cost of any given approach grows geometrically as you move from concept to production, it pays to do as much testing and validation as you can early in the process.

There are three general phases to usability testing that map to that progression from idea to production-ready:
User-Testing-Type
In exploratory testing, you’re evaluating approaches, hopefully several if the interface element is important. Your test items should be anchored in user stories as opposed to specific solution details. I highly recommend using interactive prototypes at this stage for the reasons I mentioned in the section on prototyping.

Once you have an approach you think will work (or you’re ready to invest in), you move on to the ‘assessment’ phase. Here you’re generally testing a single approach, but still with a focus on user stories vs. specific interface elements, and probably still with interactive prototypes vs. code.

Most companies I talk to think of what I’d describe as validation testing when they discuss usability testing. This is the final phase where think you have an approach that’s ready to go and you want to validate that. (Obviously, if you haven’t done the prior testing, your odds of being wrong about that are a lot higher.) You’re probably using coded software, you’re timing the user to see how long things take. Your results should match the results from your field analytics relatively closely.

For a tutorial on doing this kind of testing (including a test plan template in Google Docs that anyone can use), see the Usability Hypothesis section of the Customer Discovery Handbook. For even more depth, see the section on usability sprints in my Coursera course Running Product Design Sprints as well as the final course in that specialization, Testing with Agile.

Agile and Venture Design

Why it’s Important

COMING SOON

What it Is

COMING SOON

How it Works

COMING SOON

Agile and Sprints (or Iterations)

Why it’s Important

Working in small batches with ‘potentially shippable’ or otherwise conclusive output is core to the practice of agile. Very few teams would say they’re practicing agile if they don’t work in 2-4 week sprints (or iterations, the more generic term). To answer your question, yes, >4 week iterations is not great and probably means the process needs work.

Watching what happens at each iteration is what allows you to improve faster, both in the way you work and the way you get to a valuable outcome for the user. I can’t tell you how many questions I hear where I think one of my top answers is ‘Keep the iterations short, the observation level high, and then hold a solid retrospective at the end to see what worked and how to improve.’ That could be anything from the design process to the test process to the supporting infrastructure to the location and composition of the teams.

There are a four key things that need to work well for iterations to help:

  1. Soluble Ideas. As a friend of mine said ‘You can’t take a 9-month idea, slice it into two week iterations and get agile outcomes.’ Management must take their big ideas and break them into small, testable ideas.
  2. Problem-Focused Charter. Because of the uncertainty inherent in innovation (even at the very tactical level), team charters should focus on solving a problem instead of building a particular solution.
  3. Retrospectives. If you and your team don’t invest the time at the end of each iteration to discuss how things went and what you’d like to try changing to improve, your agile is born stale.
  4. Whole Teams. Dedicated, interdisciplinary teams are the new normal for any company that wants productive outcomes and innovation. Siloed organizations with resources split between projects are the old normal (that mostly didn’t work so well).

What it Is & How it Works

Since iterations are themselves a process, these two facets are tightly coupled. Starting at the top, the input to an iteration is a prioritized list of user stories (backlog) and the output is really two things:

1) working software (or conclusions, if it’s a design sprint), and
2) a completed retrospective on the sprint itself.

Let’s step through the process. The diagram below uses terms from Scrum, a specific methodology described below, but most of the iteration elements are common to agile practice at large.
Scrum-Plan
From a consolidated list of everything you think might be a good idea (product backlog), someone chooses a subset to discuss with the team that is a little bit larger than what they think will fit in the next iteration. Once you get to discussion, you’ll probably find that some stories aren’t well enough understood or are for some other reason not a great fit for the iteration—that’s why you want some extras.

From there, you discuss the stories to arrive at a shared understanding. Doing that with the whole team (which hopefully has all the major disciplines in some form) is pretty important—you want to get a dev., a test, a systems perspective on all of them. Some teams also do very rough estimates at this point to help with prioritization, some don’t.

Then the team arrives at a working sprint/iteration backlog which is a prioritized list for the iteration. From there, some teams will break the stories into sub-tasks for managing the work, others don’t. Now you’ve got a planned sprint (iteration).

Now you’re working the sprint for the next 2-4 weeks with a process that looks something like this:
Daily-Standup
In high-functioning agile teams, individuals self-organize to divvy out the work in an optimal way, help each other, and manage flow (making sure, for instance, not everything lands in QA at the last minute). They almost always have a brief, internal meeting where everyone answers these three questions:

  1. What did I accomplish yesterday?
  2. What will I accomplish today?
  3. What obstacles are impeding my progress?

If a question or discussion arises out of these, the parties concerned meet afterward. The idea is that each of these should take only a few minutes and with a team of 7 or so people, that’s a nice short meeting.

If a big issue with the sprint arises, the team may decide to cancel and re-plan the sprint. Otherwise, they work through, completing as much working software as possible. Whatever’s done at the end is done—sprints have fixed time, not fixed content. At the end, the team will usually hold one or more demos to talk about the content (sometimes one internal and one with a larger set of stakeholders, for instance). The working software should be ‘potentially shippable’, but will not necessarily be released. Some teams will complete multiple iterations before they release; releasing is closely related to sprint completion but not the same thing in agile.

The last step is very important if you want to realize agile outcomes: the retrospective. The whole team will spend a substantial amount of time, say a few hours (varies with sprint duration) discussing the sprint and what practices and methods they want to change for the next sprint. Typical agenda items are:
Iteration-Retrospective-750px
The main thing is honest discussion vs. the perfect format. One popular technique is the ‘five why’s’. When something doesn’t go right, answer the question ‘Why?’ five times to really get at the root of the issue. The output of the retrospective is a set of decisions on how to run the next iteration.

I think what’s most important about running successful iterations is thinking about the jobs you need done and how you’ll measure success (and don’t get too caught up with what the various methodologies specify).

If you want to learn more, I step through team development techniques in my Coursera course ‘Managing an Agile Team’.

Agile and Design Sprints

Why it’s Important

Design-Sprint-Calendar-300pxThe amount of stuff you’re supposed to do to innovate is kind of overwhelming: design thinking, personas, UX, HCD, Lean Startup, customer discovery/development, prototyping. Wow.

If you have any familiarity with customer behavior, you know what customers do when faced with too many choices: nothing. The important thing about design sprints is that they give you a specific, actionable structure for applied innovation. It’s something you’ll be able to implement, and, in turn, train your colleagues to do as well.

What it Is

Venture-Design-Sprints-300pxBasically, a design sprint is a time-boxed iteration with a specific format created to facilitate some (design-oriented) objective. All the formats I’ve seen are time-boxed at 5 days—enough time to do something substantial, but not enough time to get lost in the details. The sprint formats prescribe the methods you should use on at least day-by-day, AM vs. PM basis to help you keep on track. Google Ventures has popularized a 5-day format focused on solutions for relatively well-understood problems (and it’s very well put together). The teams I work with have a broader range of design questions (explicit or implicit) and so I use four sprint types, depending on what’s most urgent for the team.

How it Works

I have four design sprints I like to use, and they map closely to the Venture Design process I use to structure the innovation process for new ventures and products.
Venture-Design-Sprint-Process-1000px
THE PROBLEM SCENARIO SPRINT
The ‘Problem Scenario Sprint’ deals with discovering what makes your customer tick and what problems/jobs are really worth solving for them.

THE MOTIVATION SPRINT
The ‘Motivation Sprint’ deals with validating if and which propositions will be valuable to your customer/user, better than their current alternatives.

THE USABILITY SPRINT
The ‘Usability Sprint’ deals with validating which are the best approaches to making that proposition highly usable to your user.

THE ARCHITECTURE SPRINT
The ‘Architecture Sprint’ deals questions of approach on implementing your solution.

I step through the use of these sprints in this tutorial on Venture Design Sprints and in more detail in my Coursera course ‘Running Product Design Sprints’.

Agile and Lean Startup

Why it’s Important

Lean is about avoiding waste and Lean Startup is about doing that in the context of startups/innovation. (Lean Startup defines ‘startup’ as any project that hasn’t found a scalable fit between product and market). Practitioners do this by structuring and testing their ideas/propositions to avoid overinvestment in solutions that aren’t valuable to users.

This is important, and it’s why Lean Startup is one of the most popular new topics in agile: metrics vary, but if you factor in new products and features, it’s likely that the majority of software goes effectively unused. Some of that is inevitable—innovation is messy. But a lot of what we’ve learned about startups and innovation in the last few years, and a lot of what the most successful companies are doing, is about avoiding overinvestment in solutions that aren’t relevant. If agile is about driving to valuable outcomes for the user, Lean Startup is one of its best friends.

Avoiding overinvestment/waste doesn’t mean starving new product teams for resources. It means giving them more more shots at a successful outcome. Let’s just say that a new feature has a 1 in 5 chance of being a hit with users. What if you could find that out in half the time/money you’re spending now? That would double your output and number of successful features!

What it Is

Essentially, Lean Startup is about applying the scientific method to new ideas. In the first step (01), you formulate your hypothesis. I strongly recommend doing this with validated problem scenarios so you make sure you’re proposition is relevant to a job or habit that actually exists for your customer.

In the second step (02), you declare your hypotheses (or assumptions) in a testable format, usually, ‘If we do [something] for [segment or persona], then they will [respond in a certain, measurable way].’ In the third step (03), you design experiments to test the hypotheses—the best ones take a week or less. Then you execute those experiments (04) and evaluate your results (05). If your results were above your target threshold, ‘persevere’ (6b) and scale up the idea. If they don’t, ‘pivot’ (6a) and try something else before you waste your resources on a proposition the customer/user doesn’t want.
Scientific-Method-Lean-Startup-1000px
Note: In the book ‘The Lean Startup’ the author Eric Ries describes a ‘Build-Measure-Learn’ loop. I think that’s fine; I just prefer the above for drawing attention to the things I’ve found are important.

Fogg-CurveAnother Note: Remember when we talked about separating out questions of motivation and usability using the Fogg Curve? Well, this is how you figure out customer motivation and I would focus on that before you invest (much if at all) in usability for the reasons I described above in the section on user testing.

How it Works

Any practicing scientist will tell you that the best way to get a positive result out of your experiment is to go into it with a strong hypothesis. If you just do Lean Startup and neglect your understanding of the relevant personas and problem scenarios (see also above), you’ll invest in testing weaker hypotheses/ideas. This is one of the things I emphasize in the Venture Design process: even though Lean Startup type experiments are way cheaper than building product, they’re still more expensive than talking to customers (relative to getting an actionable result). So, make sure you’re testing a proposition that at least solves a problem the customer has.

Make sure your hypotheses have causality implied in them (if we do x, then the customer will do y), otherwise their relationship to your experimental design is likely to be murky. When you design your experiments, make sure to include threshold metrics (pass/fail threshold). Also, I highly recommend sketching several experiments before you decide. They will probably vary a lot in their relative cost and duration.

Finally, make a habit out of experimenting and sharing the results. The most successful teams I know include them in weekly team meetings about outcomes.

For more on practicing Lean Startup, see the Lean Startup tutorial here on the site. That tutorial also links to a Lean Startup template in Google Docs. If you want more depth, I cover running ‘Motivation Sprints’ around it in my Coursera course ‘Running Product Design Sprints’ and go into more depth on managing experiments in ‘Testing with Agile’.

Agile and Project Management

Why it’s Important

OK, let’s have the talk. Agile is not about project management. At least, it’s not about project management in the sense the term is typically used. You might ask: But what about all the agile books and training for people who are career project managers? Isn’t that what agile is?

Kind of. Commercially speaking, yes. But a whale’s not a fish. Sometimes those sort of distinctions don’t matter, but in this case they do. The reason it’s important is that traditionally project management is about making big (long term) plans, managing known constraints, optimizing around those, and generally treating people like interchangeable parts (not to be callous, but just because it seems like that’s what best way to help everyone do their jobs and get the best possible outcome).

Agile is different in a way that matters a lot in practice—it’s fundamentally about atomic, self-organizing teams. That matters because if you shim agile practices into the same old approach to managing projects, you (just about always) end up setting expectations where you’ll fall short. Then everyone thinks agile stinks and things usually get worse not better. Employees feel like they’re getting extra work and constraints dumped on them from above and management is frustrated that the company isn’t innovating and/or getting better at releasing product. Barf.

What it Is

Agile is a kind of un-project management. If you’re coaching a team on how to do it, your job is to do help the team drive to valuable outcomes through self-organization. You can measure your success in that by how much ‘managing’ you have to do—telling people what to do, asking them about progress, etc. High-functioning agile teams are interdisciplinary and self-organizing.

Also, to achieve these valuable outcomes, you need what the XP methodology (see below) calls a ‘Whole Team’. This means an interdisciplinary team that is colocated and collaborating readily on a day by day basis. What does that mean, ‘interdisciplinary’? When I work with teams, I break it down across the five jobs below:
Software-Development-Business-v4-1000px
Now, you may think: ‘A lot of those are not my job.’ But it is if you’re responsible for the project’s success. Unpacking and fixing that ‘not my job’ statement is, in a sense, a lot of what agile is about. Siloed departments with partially dedicated resources are the hallmark of an ossifying corporation that’s (probably) getting disrupted by more able competitors. Interdisciplinary teams with problem-focused charters are the hallmark of innovators that are able to create value for customers. If you’re tired of taking my word for al this, check out this item on Spotify’s engineering culture.

That’s the interdisciplinary part. Let’s talk about the self-organizing part. As I mentioned above, your job as an agile coach is to do three things:

  1. Pick and choose the methods you think will help you and your team
  2. Discuss and adjust with your team and make sure you all are applying those methods (within the limits of good judgement)
  3. For each iteration, assess (with said team) how those methods performed and go back to #1

At the core of making this successful is a culture of experimentation, hence the ‘hypothesize, experiment, learn’ loop you see above. This is critical because:

a) nothing works perfectly, not even close, and this mindset brings the need to try things to the foreground; and
b) it puts the emphasis on evaluating results rather than whose individual opinion prevails.

As the coach or team lead, your job is to set up those preconditions and then let the experiment run. You can measure your success by the frequency and depth of incidences where you have to ask or tell team members what to do. The reverse is also true: the more you see the team helping each other and using the methodology to progress the team to a valuable outcome, the better you’re doing (agile-wise).

How it Works

If you’re a titular or acting project manager, you’ve likely been given a charter that’s narrower than what I described above. That’s normal—organizations change slowly. A lot of how businesses operate is still leftover from managing widget factories in the 1950’s. Rest assured, whoever your manager is, somewhere up their chain of responsibility, there’s someone that just wants you to make them some money. (Unless you’re a non-profit, in which case substitute the org’s target outcome for money.)

The implication of this is that, above all, you should focus on making your project successful. If you need change or you need something from someone else, your best bet is to frame your request in those terms. Will you then automatically achieve a beautiful, harmonious, interdisciplinary paradise? No. But, you’ll be taking steps towards it, and that’s what’s important. Create small wins, show (with practice) that your approach is working, and you’ll be fine.

I realize this ‘How it Works’ is kind of general. Partly this is because the concept of ‘project management’ is pretty expansive if you apply it to agile. But, fear not, the sections below go into more detail about specific methods.

If you want to learn more about the jobs of proposition and product design, see the related sections above on personas and user stories. The Venture Design page has more detailed tutorials and my Coursera courses ‘Agile Meets Design Thinking’ and ‘Running Product Design Sprints’ step through the processes in even more detail.

Roles and Jobs in Agile Software Development

Why it’s Important

This section extends the section above in that we’ll discuss how to relate the fundamentals of agile to actual practice specifically to the job of software development. As you practice agile, you may meet folks from two different extremes: pro-manifesto vs. pro-methodology. It’s a huge oversimplification, but bear with me.

The ‘pro-manifesto’ camp will tell you the Manifesto is all you need to know about agile. The rest you should figure out. I’d say they’re not exactly wrong, but it’s a hard place to start.

The pro-methodology camp (often bearing an agile or project certification of some type) will tell you that (usually) scrum is agile and it says you should do such and such. Now, no real person has such a simplistic viewpoint, but you will probably run into this conundrum. The reality is that, yes, the fundamentals are the most important, but you need practical ways to apply them. At the same time, it’s easy to get arbitrary about the relative simplicity of prescribed methodologies and lose site of what fundamentally constitutes a good outcome and if/how you’re getting there.

So, what do you do? Well, taking a page out the design thinking playbook we discussed above, I’d say you should anchor your practice of agile in problem scenarios, core jobs to be done. This will keep you anchored to the fundamentals while you explore and test different methodologies.

What it Is

I’ve divided these fundamentals into jobs and roles. These are both specifically for the job of ‘Software Development’ in the diagram you saw in the last section.

JOBS
What are the basic jobs of ‘Software Development’? There are a lot of different ways to look at it. I’ve organized them into these four main areas:
Agile-Jobs
Here are some notes on what I mean and what constitutes success.

1. Learning
How is your team learning what’s valuable to the user and getting better at translating that into testable deliveries? As a reminder, this is specifically for the job of ‘Software Development’, per the diagram in the section above. The jobs of ‘Proposition Design’ and ‘Product Design’ I mentioned above and in the earlier sections. Collaborating together on the job of ‘Product Design’ and keeping that closely linked to development is one of the most important success factors here.

2. Deciding
How is your team deciding what to build, who’s going to build it (and test it and deploy it), and in what sized batches? Is that process resulting in overall more valuable deliveries, measured in terms of user outcomes? Is the work environment getting better for the team?

3. Building
How is your team actually building, testing and deploying the software? Are the methods in place contributing to progressively higher output velocity, lower bug incidences, and smoother deployment/operation? Do the team members feel they’re learning and improving their craft?

4. Managing
How is the team being managed and managing its interface to the rest of the organization? Do team members feel that they have meaningful discretion to self-organize and do what they think makes sense? Do they understand and buy in to the team objectives in a way that allows them to see the value of their work? Are there surprises from outside the organization (last minute product changes, surprising assessment results from outside stakeholders)? And, last but not least, how much time is everyone having to spend in meetings of >2 people vs. doing work?

ROLES
The sections on agile methodologies below describe their view on roles. I’ve made the role descriptions below generic and functional specifically to avoid oversimplification on roles and overemphasis on their prerogative vs. what they should be doing for the team. If you’re familiar with the Stanford Prison Experiment, you know that terrible things can happen when you put otherwise reasonable people in the wrong roles relative to each other.

It is fine for an individual to act in multiple roles. Popular practices like Devops and Behavior-Driven Development specifically focus on the integration of these roles. While being ‘full stack’ and interdisciplinary is often useful and is popular (with good reason) in agile, I have heard horror stories from large corporations where everyone was arbitrarily required to periodically do everything. That’s not good. While I generally recommend developing against narrative (vs. tasks) as a way to stay focused on user value, everything requires moderation and good judgement.

Here are some ideas on thinking generally about roles.

1. Answering Questions about Users
Someone needs to take the lead role on getting these questions answered. That said, teams rarely get good outcomes when one person is the ‘designated expert’—everyone has good ideas.

This person’s job is to conduct investigations but also organize them for the team. For example, I recommend the whole team participate in design sprints in order to improve the result and cultivate a strong shared understanding of what’s valuable for the user. Other tasks here might include designing and monitoring analytics.

2. Finalizing Split Decisions on Design
Techniques like dot voting are a great way to get everyone’s opinions out on the table efficiently and stimulate constructive discussion. This may seem like a paradox but whole you want to draw on the wisdom of the whole team, but it’s important that someone is in a position to finalize product decisions that are not obvious or split. Otherwise, you end up with something that’s averaged out and crummy or uncertain and shaky. Remember, everyone’s solution is temporary and your emphasis should be making the best possible decision under the circumstances, building it, and then testing it.

3. Creating Human Interfaces
One or more people on the team may do this, but probably not everyone. This area in particularly is tightly tied to the related job of ‘Product Design’.

3. Creating Software
This role involves not only coding but writing unit tests, evaluating alternative approaches (including use of 3rd party apps, modules, and services). I’d also lump tasks like version control and build automation here, though these are highly interdisciplinary.

4. Testing Software
Tightly tied to the previous job, this includes unit, integration, and system tests (or whatever you call the various levels of testing—see below on that). This includes design for testability (though that could equally be placed in #3), creating test infrastructure, as well as executing tests.

5. Designing and Operating Systems
This role includes designing the system parameters for the software as well as creating automation and containerization. It includes executing and monitoring of OS and application updates, and, yes, the much beloved role of fixing systems when they break.

6. Dealing with Logistics
In any project, there are ground balls—infrastructure that’s needed or broken, stuff you need to buy and a purchasing process to follow, questions for other groups and areas. Unfortunately, most of this can’t easily be delegated to a strictly admin role, because usually to get these things done, you have to have a pretty good idea about the project. (How much memory did we need on this server again? What plan do we need from [such and such SaaS service]?)

How it Works

How do you get started? Well, the prevailing answer is ‘Pick one of the major methodologies, and start with that.’ For most teams that do so, that’s a methodology called ‘scrum’, which we’ll review below. Scrum is fine, and its main virtue is that it attends to details you need to have on hand to practice agile. Very often, the teams gets caught up in the orthodoxy of scrum and lose sight of what’s really valuable about their outcomes.

What I suggest teams and managers do in parallel is think about these fundamental jobs that need doing to deliver value to the customer and how they’ll define success in those areas. Many successful teams start with scrum and tune from there, and that’s fine. Just make sure scrum’s working for you and not the other way around. We’ll step through these jobs in a little more detail in the next section.

If you want to learn more about managing with agile, I step through it in my Coursera course ‘Managing and Agile Teams’.

Agile and Scrum

Why it’s Important

To many people, scrum is agile. Terms it originated like ‘sprint’ are more or less standard agile terms. The reason for this is that scrum emerged early on as a full-baked, ready-to-use agile methodology. And it’s pretty good. And it’s important that there is complete methodology out there and that you don’t necessarily have to assemble one yourself to get started with agile.

But it’s easily misused as an arbitrary set of rules, and there are methods beyond scrum that you will need in order to maximize the success of your team. The successful practice of agile is mostly about knowing what methods are out there and how they’ve worked across situations comparable to those of your (vs. being a super duper expert about scrum).

What it Is

Scrum is an agile methodology. It originated as a product development methodology in the mid-80’s and was later adapted to software and agile principles by Ken Schwaber and Mike Beedle. They formalized those ideas in a book, Agile Software Development with Scrum, which was released in 2001, the same year the Agile Manifesto was published. Since then, many additional books, classes and certifications have emerged to support it.

How it Works

I’ll describe the basic practices of scrum in sections that roughly mirror the preceding section on roles and jobs.

JOBS
Note: These correspond to the section above, but are not in the same order.

1. Deciding
Like all agile methodologies, scrum is organized around iterations, called ‘sprints’. The guideline for these used to be 4-6 weeks, where now, with shortening product cycles, 2-4 weeks is more prevalent.

The process of deciding what to build in the iteration involves a few steps, shown below:
Sprint-Planning-Scrum-1000px
Scrum describes a ‘product backlog’, which is basically all the user stories you’d like to develop. Before each sprint, the product owner (see ROLES below) selects a set of user stories to discuss with the team in a sprint planning meeting: the ‘draft sprint backlog’. The team together estimates these stories (using a size proxy called ‘story points’) after which the product owner re-prioritizes them, based on their assessment of their value relative to the cost expressed in the estimate.

The now prioritized list is truncated, if needed, based on the estimates and this becomes the ‘sprint backlog’. The team creates a headline for the sprint expressed as the ‘sprint goal’, which is useful not only for internal cohesion but also for communicating updates to external stakeholders.

Some teams find it useful to decompose the ‘sprint backlog’ into tasks for granularity and configuring workload. Others prefer to develop against integral user stories (one of the big issues being the management of flow which we’ll talk about in the section on kanban).

Now, you’re running the iteration:
Scrum-Daily-1000px
The goal is self-organization. In scrum, the daily standup is called the ‘daily scrum,’ but it has the same format we discussed in the section on iterations. If there’s an issue with the backlog, mainly, if there’s new content required, scrum says stop the sprint and restart it with a new backlog, etc.

Scrum describes a progress tracking method called burndown:
Scrum-Burndown-1000px
The idea is that during the estimation process, points are assigned to the various stories as a proxy for size (I’ll skip that in the interest of keeping this short, but it’s in the Coursera course on managing with agile). The point at the top of the y-axis is the sum of all those story points. The x-axis is the days in the iteration and the straight line is just a rough guide to what perfectly linear process would look like (more a guide than a specific goal).
From these, you mark your actuals each day, decrementing the story points of completed (done, tested, shippable) stories. That’s the more wandering curve you see. Some teams use burndown, some prefer to focus on measuring velocity based on actuals.

Whether or not burndown will be useful for your team is probably a function of a) the decisions you want to be able to make during the sprint based on its values and b) the predictability of what you’re doing. If your team is working on a brand new problem that’s not that well understood, burndown is likely to add stress. If you’re doing another batch of something you do all the time, it will be relatively more accurate and more likely to be useful.

2. Learning
At the end of the sprint, there’s a demo. Often, the team holds its own demo to discuss the gory details and another to present to external stakeholders. While this isn’t a substitute for the kind design research, usability testing, or motivation testing we discussed above, it’s a good opportunity to vet the state of the team’s shared understanding and its success understanding what constitutes a valuable outcome with external stakeholders. Suffice to say that big surprises during the demo point to a problem in the practice of agile.
Scrum-Demo-1000px
Like most agile methodologies, the process deals mostly with the process of building of software itself. While agile has always been very design-friendly, linking the overall practice of agile with best practices in product design and design research is one of the more recent, faster evolving areas of agile.

3. Managing
To conclude a sprint, the team conducts a retrospective:
Scrum-Retro-1000px
Possibly the fastest way to predict how likely a team is to improve their practice of agile is to ask about the nature of their retrospectives. Is it a boring meeting everyone feels they have to sit through while someone goes through the motions? Is it a dreadful blame-fest? Or is it an honest, sincere appraisal of how things went and what could be better where everyone’s #1 priority is to help the whole team do better on the next one? The questions you see above are common but a retro. for, say, a 2-week sprint should be at least a couple of hours. Another popular technique is the 5-Why’s—peeling back the root of why certain things happened.

4. Building
Other than a few details, scrum doesn’t so much deal with the specifics of how to build the software. This is one of its major points of contrast with XP, which we’ll describe next.

ROLES
There are three major roles in scrum.

Scrum Master

Scrum-Master-300The scrum master has several jobs. One, as I mentioned above, is to ‘enforce’ the rules of scrum. Their job is also to deal with logistical issues and impediments as well as ‘buffer’ the team from external distractions (if they really are distractions). They also work with the product owner (up next) to make sure the team understands the product backlog and how to validate completeness. I will editorialize here and mention that if this ends up being predominantly one specialist’s job, then the inputs and interdisciplinary collaboration need work. The flip side of this is that, sure, if some particular facet of a user story needs more investigation, it’s perfectly OK for a couple of folks to go sort that out and get back to the team.

Product Owner

Scrum-Product-Owner-300pxThe job of the product owner is basically everything attendant to the product backlog—all the work that would go into creating valuable user story and validating their completion as valuable to the user. I will editorialize here and say that while I think it may be good for one individual to take the lead on investigating and facilitating related activities (like design sprints), I think it’s generally a mistake to ‘buffer’ the development team from exposure to users and stakeholders.

Development Team

Development-Team-Scrum-300pxThis is basically a placeholder for everyone else.

If you’d like to know more about scrum and how it fits into the jobs of software development, I cover that in my Coursera course ‘Managing an Agile Team’.

Agile and XP

Why it’s Important

Like scrum, Extreme Programming (XP) is a fully articulated agile methodology. One of its major points of contrast with scrum is that it is more prescriptive regarding actual programming practices. XP is closely associated with high-performance programming practices like test-driven/test-first development and pair programming. Because it generally requires a deeper level of buy-in and practice from developers, you may frequently get the sense that there’s a certain reverence for XP as an agile practice. It has contributed substantially to advancing software craftsmanship.

Understanding the basics of XP is important for agile practitioners even if they’re not themselves practicing coders. It is a large and important body of work in the area that continues to be a common feature of successful teams. That said, because of the level of depth and specificity of the practices I would always look to the development team for a decision on whether they want to adopt some or all of the XP practices.

What it Is

XP-MethodologyLike scrum, XP preceded the Agile Manifesto itself. Agile thought leader Kent Beck was collaborating with a team at Chrysler (including Ron Jeffries and Don Wells, who created the diagram you see). After that work and various other collaborations came two things: 1) the site http://www.extremeprogramming.org/ and 2) Kent Beck’s book ‘Extreme Programming Explained’.

The diagram to the right describes the contents and orientation of the methodology. Like scrum, XP has releases separate from iterations and uses daily stand up’s to facilitate self-organization. Below the stand up, you’ll notice a few other things. Let’s talk about how those work.

How it Works

Like scrum but unlike kanban (see next), XP is a fully articulated agile/SDLC (software development life cycle) methodology. Let’s step through this one relative to the jobs we discussed above.
XP-Principals
Like scrum, XP is mostly focused on the process of building software. For Learning, it uses stories, which are a great way to tie your implementation discussions back to your work in design and design research.

For Deciding, weekly iterations and quarterly planning/big picture cycles are popular (but not mandatory). Slack is an important concept- basically the idea that you should leave time for the unexpected so that you can do things right vs. get boxed into a schedule and accumulate technical debt that slows down your ability to innovate later.

Building is one of XP’s strong suites. In case you were wondering, yes, many teams use some of these methods in conjunction with an agile practice based on scrum. High-functioning agile practice is focused on outcomes and fundamentals, not orthodoxy. In pair programming, two developers work simultaneously, one typing with the other directing or advising. Many developers prefer the immediacy of this practice to before the fact design discussions and/or after the fact code reviews. The idea with the ten minute build and continuous integration is find code-related issues early, and instill the confidence try new things and refactor, knowing that the developer won’t have to wait long to see if/how their change breaks the build. Both of these practices require unit tests to work- a set of very atomic code-level tests that run whenever the developer wants but particularly when they commit code and build software modules. Finally, incremental design is that idea that rather than building an underlying framework to support the functionality you think you’ll need in the future, solve the problems/needs you have at the moment and then look to refactor and update the code as you learn more about what you actually want it to do. The outcome from this is an ‘emergent design’ as opposed to a more premeditated/planned design.

Managing is another strong suite of XP as it is relatively prescriptive in the area of work practices. The practice of Sit Together is what it sounds like- the team should be working together in a common physical space. The Informative Workspace practice suggests that the team use ‘information radiators’- charts, storyboards, etc. that help cultivate and maintain a strong shared understanding of how the team is working together and what constitutes a valuable outcome. Whole team refers to the idea that XP teams should include members from every major discipline, and they particularly emphasize having someone on hand all the time that’s in tune with what’s valuable to the customer. (I’ll editorialize briefly that I think everyone should know that to a degree and that while this isn’t what XP specifically intends, the idea that there should be someone that helps ‘keep the customer away from the developers’ is both outmoded and part of the reason we build so much useless software.) Energized work refers to the idea that the team should be managed in a way where output is maximized, not long hours where the team gets worn down.

If you’re interested in learning more about XP, I step through it and the ‘jobs’ of software development above in my Coursera class ‘Managing an Agile Team’. Another great resource for XP is Bill Wake’s blog XP123. Bill is a thought leader in agile and I have leaned on his deep experience and perspective on XP in my Coursera class, Darden classes (where he’s a guest speaker), and my work in general.

Agile and Kanban

Why it’s Important

Kanban deals with the concept of ‘flow’, the evenness of your team’s output/work-in-progress (WIP) across the development process. If you’ve ever gotten to the end of a development cycle and seen that QA is overloaded and/or there isn’t enough time to test everything, then you already understand the importance of flow.

This concept is closely related to lean. If you’ve read The Lean Startup, you may remember Eric Ries’ story about folding 600 letters into 600 envelopes. Do you a) fold all the letters first and then seal them into envelopes or b) fold one letter, seal it in an envelope? From the context you probably guessed ‘b’, but that’s not most people’s first guess (I know because I use it in class all the time). It turns out that if you fold all the letters first, you end up with a lot of wasted time shuffling around the piles of paper you create and other overhead.

It turns out most of us have a natural bias towards just grinding away on one thing vs. moving it all the way through the process and that can result in waste. It probably has something to do with the mental overhead of switching context. That waste might come in the form of:

    • everything piling up for testing right at the end of a cycle
    • not being able to review/discuss/test working software
    • realizing certain components don’t work together
    • and other stuff like that

.

Hence, it turns out kanban is pretty useful in the process of software development.

What it Is

Taiichi Ohno developed the concept of kanban at Toyota in the late 1940’s. It basically dealt with managing the amount of inventory/work-in-progress on the factory floor so that there wasn’t too much stacked up in one place and also not places that went idle because they had no inputs.

In the process of software, for a basic application of kanban I’d recommend doing three things. First, model your process visually- probably on a whiteboard. Lay out each step from coding to release. Be sure to model it as it actually is vs. how you would like it to be. You can change it, but the application of kanban will only work well if you execute it against your actual process. Second, find a place where you can model that so everyone can see it- a bulletin board or a digital system like Trello. Third, establish work-in-progress limits (WIP) at each step. Then you use that board to track your work during an iteration. There are books on kanban and many available refinements to the practice, but those basics will get you going (more on that in the next section).

There is a surprisingly widespread misconception that kanban is an agile/SDLC (software development life cycle) methodology like scrum or XP- that you need to choose between kanban, scrum, or XP. That’s basically incorrect. Kanban is a method you can apply within either of those methodologies, but it is not itself a complete methodology. It doesn’t have all the ‘features’ of a fully articulated methodology and so while it’s a perfectly great practice it isn’t an alternative to scrum or XP.
Agile-Methodologies-Chart

How it Works

You’ve probably seen a kanban board like the ones here (and if you haven’t just Google it or check out my course on agile management):
Trello-Agile-Kanban-1000px
Trello is the most popular tool I see, though there are many that I’m sure are fine. These are easy to set up and pretty easy to use in practice. The main thing is to make sure that you’ve followed the steps above, working with your whole team to make sure the steps/phases on your board match your actual process and assigning WIP limits. In the example above, I added the WIP limits in parentheses. There are plug-in’s for Trello to help with this if you want something more full bodied.

The most important thing with the WIP limits is just to have an explicit definition. If you’re looking for a place to start, one idea is that each developer should take a user story all the way through the process before they start on something else- so that would mean the WIP limits at each stage would be equivalent to the number of developers. If you have hand-offs, to QA for example, then you’ll need to model that. Just take your best guess (as a team), and then be sure to consider how these are working in your retrospectives.

If you’d like to know more about kanban and how it fits into the jobs of software development, I cover that in my Coursera course ‘Managing an Agile Team’.

Agile and Test-Driven Development

Why it’s Important

COMING SOON

What it Is

COMING SOON

How it Works

COMING SOON

Agile and Continuous Delivery + DevOps

Why it’s Important

In the olden days, when everyone was trapped in big long siloes according to their department, there was a process where:

  1. The ‘business people’ (product managers, etc.) would write ‘requirements’ for a product and then hand them to the engineers
  2. The engineers would be software based on their interpretation of these requirements
  3. The engineers would hand this software to ‘quality assurance’ to make sure it was OK and didn’t break
  4. The organization would hand the software to ‘operations’ to install and operate

The reality is many of us still exist in what will hopefully soon be thought of as olden days and, if you’ve ever tried the alternative, you know this process performs very poorly.

Whereas agile has traditionally dealt mostly with ‘a’ and ‘b’, the practice of continuous delivery (CD) and the DevOps movement deal mostly with ‘c’ and ‘d’. Don’t get me wrong- they deal with it by integrating thoughtful work across the whole process.

In your happy place in CD is that a developer can commit code, have it tested and automatically roll it to production with confidence if it passes all your tests. With product cycles decreasing and the growing realization that rapid experimentation is the key to successful innovation, this is of obvious importance.

Your happy place with DevOps is that in your product design and development work, you readily consider and handle questions from development, test, and operations angles. With the emergence of SaaS and recognition that siloed product development is as close to a complete failure as any general process can be, many teams are embracing the DevOps movement.

What it Is

Central to the practice of continuous delivery is the idea of a ‘continuous delivery pipeline’:
Continuous-Delivery
The input to the pipeline is new code and the output is released product- released in the sense that it’s up and running in production. Given the breadth of the CD pipeline, the practice of CD naturally covers a lot of ground, topically speaking. Big topics include:

  1. automation of different types of testing
  2. automation of build and deployment processes
  3. standardization and abstraction of deployment environments (servers, etc.)

DevOps is a movement- closely related to CD but different. DevOps promotes a set of practices and methods to encourage interdisciplinary collaboration in the area above and more thoughtful, disciplined work in the areas of testing, releasing, and operating software.

How it Works

While I’ve dealt with the related topics in my various roles over the years (and even implemented a bunch of tests in Selenium that broke a lot), I am no kind of expert in these emerging areas of CD and DevOps. When I talk with experts, they just about all mention the same first step: as a team, diagram your whole code to release process together.

Just creating an explicit view of the process and talking about it as a team is a valuable first step, not to mention foundation for what follows. From there, decide where the biggest pains lie. Don’t just assume you need to do more test automation because that’s common- it may or may not be the best initial CD project for your team. Consistent improvement is likely to be more important than a big tear down and rebuild of how you do things. The reason is that it’s hard to get everything right in one big go.

If you’re successful working on CD, it’s likely you’re doing a lot of what the DevOps folks would recommend. One thing I hear a lot is that teams should avoid the fallacy of recruiting a ‘DevOps expert’. Yes, experience in the area is super valuable, but since it’s fundamentally an interdisciplinary practice, cultivating an emerging expertise across your team or teams is most important.

There are a lot of great resources online in both these areas. In the area of continuous delivery, Jez Humble maintains a site with a lot of free (and paid) resources: https://continuousdelivery.com/. Jenkins is a popular open source automation server, if you want to try out a tool. There are a ton of great resources online for DevOps, the site http://devops.com/, being one easy place to start. If you’re interested in a general introduction to the topics, I also cover them in my Coursera course ‘Testing with Agile’.