From Product Idea to Machine Learning Problem

How to Frame AI Features the Right Way — Without Confusing Your ML Team

Have you ever said something like, “Let’s use AI to make the product better”?
It sounds good. But for your machine learning team, it’s like asking a mom to “cook something tasty” without telling her what ingredients you have or what you like to eat.

As a product manager, your job is to turn vague chaotic ideas into clear instructions. We will discuss how to turn a simple product idea into a machine learning (ML) problem including the basic things your team needs to build the right solution: inputs, outputs, labels, and success metrics.

Let’s start with a simple truth:
Not every product idea needs machine learning.

Step 1: Make the Product Goal Clear

The first step is to take your idea and make it more specific. A clear goal helps everyone understand what the user is trying to do and what the product should deliver.

At the start of the 2024, I was thinking about ideas where we can apply machine learning and AI. We wanted to simplify airlines fare rules. Usually the airlines fare rules are long clunky text which only trained support team can understand. We just wanted to simplify the same.

That sounds nice, but it’s too general. A clearer version could be:
Extract key informations (data points) like Refundable, Cancellation Charges and Reschedule Fees.

Now the goal is tied to a clear product outcome. This gives your ML team something to work with.

Step 2: Match the Goal to a Machine Learning Type

Different problems use different kinds of machine learning. Here are some of the common types:

If your product idea doesn’t match one of these types, you probably don’t need ML. You might be able to solve it with simple logic or filters.

Step 3: Don’t Use ML When You Don’t Need To

Sometimes it’s better to use simple rules instead of machine learning. Rules are easier to build and more reliable when the logic is clear.

Here are some examples:

If you can write a rule to solve the problem, use it. Don’t overcomplicate things. Machine learning is powerful, but it should be used only when needed.

Let’s try a couple of examples.

Example 1:
"Highlight hot destinations for each user."
This is a recommendation problem. The model should show different destinations based on user behavior and trends.

Example 2:
"Auto-reply to support chats."
This is a mix of classification (to understand the user’s intent) and generation (to write the reply).

Once you learn to spot these patterns, you’ll be able to guide your ML team much better.

Define the Key Building Blocks

Once you’ve identified that you do need ML, the next step is to define four key things: Inputs, Outputs, Labels and Metrics

Inputs and Outputs

Inputs are the data the model uses to make a decision. Outputs are the result it gives you.

Ask yourself: What do we already know about the user? And what are we asking the model to predict?

Labels (The Correct Answers)

Labels are important. They tell the model what the right answer is. The model learns by looking at examples where the label is known.

If you’re recommending destinations, your label might be the place a user clicked or booked. That shows what they liked.

If you don’t have labels, you can’t train a model. You’ll need to either collect them or find a way to use other data.

Also, make sure your labels are clean. If the data is messy or wrong, your model won’t work well.

Success Metrics

It’s easy to think a model is good because it has a high score. But that score might not mean anything for your business. You need to track both model performance and business impact.

Here’s a simple comparison:

Let’s say your model shows the top five destinations.

• The ML metric could be: How often is the correct one in the top five?

• The product metric could be: How many users clicked on a destination?

Both are important. A model that scores high but doesn’t help users is not useful.

Example: Let’s say your feature is to recommend sightseeing tours. Here’s how you could define everything your ML team needs:

With this setup, your data team can get started quickly. They don’t have to guess. And you stay in control of the product vision.