Learn as you go

I’m a big fan of projects that deliver value in small, incremental steps. Unfortunately, some of the projects I worked did not turn out that way. After one of the not-so-great projects, I decided to do a personal retrospective. Instead of listing all the things that went wrong, however, I decided to write about what could have been.

The following story is fictitious. Its purpose is to describe a North Star: something to strive for in future projects.

We started with “why”

A while ago, the organization I worked for at the time had just released a new product. We wanted to turn that new product into a success story, but we didn’t quite know what our target audience looked like. Of course, we had a few assumptions and ideas. But that was it. So our product managers decided that something should be done about it.

Armed with nothing but this clear mission, our team was involved. Our team had been in a stable configuration for over a year, which had allowed the members to develop strong relationships built on trust and respect.

Project inception

We began by getting the whole team together. In the same room we had a product manager, a designer, an analyst, a content designer and a couple of developers. The product manager began to describe the “why” of the project, but left out the “what” and the “how” - he knew that the best outcome could be achieved by involving the entire team.

Over a series of sessions we aligned on the mission and built a shared understanding of the project. You could notice the excitement, because everyone felt like they could contribute an important part to this project.

What’s more, involving everyone right from the start helped us to avoid a few dead ends. We found out that one of our proposed solutions could not possibly work early on. Because of that, we saved an enormous amount of time that would have otherwise been spent creating designs, writing copy or developing code.

We took away three things when the inception came to an end:

  1. An early idea of what we wanted to do (mostly sketches)
  2. A set of questions that we wanted to answer
  3. A set of small, equally-sized user stories, ordered by perceived value

Iteration 1: Setting up

We used the first iteration to:

  • set up the tools we needed for this project
  • spike a few of the unknowns
  • flash out our designs a bit more
  • invite experts to talk about the parts of the system we didn’t quite understand

This took us an entire iteration in which we delivered no customer value at all. However, things changed with the next few iterations.

The first few iterations

The first set of stories we worked on all focused on learning as much as possible.

We decided to add a simple survey dialog to our website that would ask visitors a few questions.

As the survey dialog might be perceived as annoying to annoying, we only rolled it out to about 5% of users. Even a small set of responses would help us to learn. As we kept learning more, we improved the initial questionnaire.

The final couple of iterations

Once the questionnaire got into a good enough state, the team began working on the next set of stories. Most importantly, we began to develop a hypothesis and to A/B test different designs with visitors, to see what they were most curious about.

We also began to make the survey useful to our visitors. Once they had answered the survey, we were able to direct them to the area of our website that had the most relevant information to them. That’s when we decided to roll it out to about 20% of visitors.

Soon afterwards, we had a pretty good idea of what our target audience looked like. The insights gathered by our team were shared with the entire organization. We were quite happy to see that our insights actually started to shape our organization’s roadmap.

Retrospective time

At the end of the project we all got together to learn what had gone well. A few positive points stood out:

  • During the entire project, we felt like we owned the work. Consequently, we took responsibility for the work.
  • We shipped something something valuable every week. We started with the most important things first, and ended with the least important ones.
  • We didn’t need to know everything from the beginning. Instead, we deferred critical decisions for as long as possible.

After the retrospective, we felt ready to take on our next project.