John Obelenus

The ROI of a Solved Problem

We build software to solve problems. The solution to the problems must be valuable enough for a customer to pay for them. There is a fundamental assertion here that we have to make—we, as a collective group of people, are extremely bad at solving valuable problems. I feel very comfortable making that statement because 90% of all start ups fail. And incumbent companies that already sell valuable solutions also go out of business. VCs are willing to burn money rolling one hundred dice just to get one that hits.

The foundation of my thinking here rests on this article, of the same name, by Dorian Taylor. To summarize the article: when we attempt creative problem solving, we cannot know the maximum value the solution holds, and therefore cannot estimate return on investment. To quote directly:

…if you pick a problem you can solve today, and close out the day with that problem solved, you have an asset.

If 90% of everything is crap, the obvious strategy should be to produce as much crap as you can afford, because the other 10% will be spectacular.

We all have gut instincts, especially about problems we suffer from. It is why engineers making products for other engineers has a higher success rate. But instinct is not evidentiary knowledge. And we talk with potential customers, “Of course I would pay for that”, which I will call hearsay. Until the money has transferred hands, it is not evidence. Even thoughts like, “I’ve made products like this before and it worked last time, so it will work this time” has an incredibly short half life. The world is changing faster and faster, the preconditions for people’s behavior to remain the same does not last very long.

There is some good news with framing the situation in this way: when you have a valuable solution that people pay for, you have a tangible asset. I have witnessed companies that keep chugging along based on the work that people did five years ago. Their product works, the maintenance cost is pretty low. Every year, for four years, in January at the company kickoff, the same problem is trotted out in different words. Every year they tried new things, unable to solve the new problems they faced. But, even in the face of those failures, the company kept on chugging.

As you may have guessed, I am extremely comfortable with the word failure. I think we often associate failure with a bad character, or negative morals. Failure, entropy, is simply the regular state of the world. It takes skill, perseverance, and luck to succeed. Nature tends towards disorder. We should not feel bad when we fail to produce value, that is just going to get in our way when we try again.

How should we approach creative problem solving in a world where failure is common, the value we produce is unclear, and it is impossible to determine the return on investment? That is the framing in which I look at Product Engineering organizations.

This is my answer to the question:

  1. Define the problem to solve in great detail
  2. When you lack details to define the problem make the smallest investment to get more information
  3. When everyone agrees on the problem, brainstorm multiple solutions
  4. Attempt an MVP solution with the least investment required
  5. Put the solution in front of real people with the problem and see how valuable they find it
  6. Iterate. Over, and over, and over. Until you’re very tired of it.

This is where I bring up the phrase I wrote down in my previous article:

The answer is not in this room. If it was, we would have solved the problem already. The answer is out there somewhere.

No. 5 cannot be answered by anyone in your organization. Only actual customers can answer that for you. As a reminder, it's extremely likely that you will fail to produce actual value every time. Failure is the default. You ought to have every freedom to decide “this is the wrong problem,” or “this is the wrong solution, let's try another.” If you’re following this pattern you are going to gain lots of data along the way, you ought to be rethinking how much investment specific solutions require. And which problems are valuable.

No. 6 I think is where the rubber meets the road and what separates organizations that take this kind of approach seriously from those who do not. Iteration is what turns failure to success. There are a million reasons I have seen organizations not iterate. The engineering and product managers are too busy; “Just tell me when you’re done with this Epic, I don’t need to be involved”. Or, “we all assume what we decided is correct and all the users will love it, so we don’t need to check!” To define a problem in great detail and choose an MVP you are going to iterate on is difficult, and not without conflict. Which is one more reason I have seen teams avoid it. You end up with a large project that takes up lots of time, and is incredibly difficult to determine if it was successful in its goal or not. While no one was uncomfortable in a moment of conflict, I’m willing to bet almost everyone was confused at one point or another as to what we’re really trying to do here.

If you asked me to paint a picture of what this looks like across a Product Engineering organization I would say:

Product Engineering is not all of Engineering, but it is the organization that needs to spend its energy exploring. Exploration is its own specific kind of activity with its own modus operandi. The rest of Engineering has its own MO, neither is it a cost center either. These organizations must mutually reinforce each other.