Create AI products systematically with this 9-step framework

These days, every product is “AI-powered.” But what does it take to make a truly great AI product that stands out, especially when it seems like the technology is always changing? 

It takes a methodical approach to AI product development. Shub Agarwal, author of Successful AI Product Creation: A 9-Step Framework, explains how to build AI products systematically.

Get all of Shub’s insights in the on-demand webinar.

The MIT study The GenAI Divide: State of AI in business 2025 found that 95% of generative AI pilots fail to deliver ROI. In other words, most AI projects never move beyond the demo stage.

“A demo should be built with a goal to achieve a business outcome,” says Shub. “It acts as a building block towards the end goal. These demos are failing because there's not a systematic methodology. There's no path to production. And that's essentially the dilemma we are facing right now.”

To create successful AI products, organizations need to approach AI product development like their business strategy—with a systematic approach.

“We need to understand what the user needs are. We want to understand the product vision and strategy. We need to be exceptional at execution, so that we can build something meaningful for our customers and for our business. That remains. Those fundamentals remain. But AI product creation brings new complexities,” says Shub.

Here are some of those new complexities to consider:

Creating AI products relies on expertise across domains, and it’s blurring the lines between traditional roles. 

“Your role is defined by impact and not title,” explains Shub. “Very soon, as a product leader, you will have several agents reporting to you. You are going to orchestrate those agents, and then you have to build products. 

“Essentially, everyone becomes a manager in this new era. Instead of managing people, you're managing agents. . . . You have to think about those complexities, and it requires a new set of skills, a new set of knowledge. The best product creators lead strategic vision, user empathy, and deep technical insight to create intelligent products that people trust.”

Build your team’s AI skills with hands-on upskilling.

“Experimentation in the AI era goes on steroids,” explains Shub. “You're embedding experimentations throughout the lifecycle of creation of a product. New cross-functional teams emerge that you never thought about before. There's a lot more change management happening.

“You don't have a checklist of requirements. You have probabilities. You have to define what good looks like and negotiate with your stakeholders because it can be different in different business contexts.”

In other words, everything is constantly in motion. Managing execution and stakeholders will be a lot harder without a framework and methodology to work from. 

“The next generation of products won't behave like startup software. They won't have fixed requirements. They will evolve more like living organisms,” says Shub.

“AI products don't have a final state. They exist in perpetual improvement, always learning, always evolving, and always surprising,” says Shub.

As a result, measuring the success of AI products requires more than traditional KPIs. In addition to metrics like engagement, customer satisfaction, and revenue, organizations need to look at AI and GenAI-specific metrics.

For AI-specific metrics, look at the model’s:

• Accuracy
• Precision
• Fairness
• Recall

For GenAI-specific metrics, understand:

• Is the model output coherent?
• Does it hallucinate?
• Does it have any contextual relevance?

It’s imperative that you measure both business and AI metrics for AI products. Says Shub, “If you only measure the business metrics, you risk building something that is not useful and usable. And you could be in the headlines for bias and hallucinations. 

“If you only measure the AI metrics, you can create something that no one cares about, and there's no adoption. So, that is why you have to balance and measure both metrics over here.”

“Generative AI gives product creators superpowers. But the real skill isn't using it. It's shaping raw potential into real products. And that is what requires a systematic way to think about it,” says Shub.

His framework for AI product development helps organizations approach AI systematically.

The first layer of the framework starts with building a Strategic Foundation. This is key to building valuable products that align with business goals. The second layer, Implementation & Integration, bridges the gap between AI research and reality. The final layer is Sustainable Excellence & Innovation, which focuses on continuous, responsible AI innovation. 

At a high level, the 9-Step Framework for AI Product Creation includes:

1. Mapping problems to business goals for AI products
2. Curiosity to learn AI use cases and emerging technical ML concepts
3. Experimentation mindset and room in the roadmap to innovate
4. Integrating MDLC with SDLC
5. Scaling research to production
6. Acceptance criteria in the world of AI
7. Patience and plan to surpass human-level performance
8. Model explainability, interpretability, ethics, and bias
9. Model operations: Model drift management

To dive deeper into each step, explore Shub’s book. 

“The future belongs to those who see products not as something to build once, but as evolving organisms that never stop learning, changing, and surprising,” says Shub.

Stay ahead of the curve—get the insights you need to create successful AI products in the on-demand webinar with Shub.