Building the ultimate synthetic model

Our experimentation in building synthetic models models began years ago, utilizing our in-market transactional data for volumetric forecasting and trend detection. With more than 20 years of experience building patented AI and machine learning solutions—not to mention our vast stores of consumer-panelist data—we’ve always been ideally positioned to bring such a tool to market. But even as the recent emergence of GenAI has propelled new possibilities forward, our experimental approach has remained a deliberate one, prioritizing client value over hype. Above all, we’ve been laser-focused on ensuring output accuracy through testing and validation and are unwilling to compromise quality over speed to market. This strategy has been affirmed time and again, not only through our findings but also in conversations with clients who have noted that the early-market technology they adopted did not deliver as promised.

Although there are numerous complexities that go into building the ultimate synthetic respondent solution, we’ve identified three simple and non-proprietary principles that distinguish best-in-class models from their counterparts. It should come as no surprise that each of these begins and ends with real, human-provided consumer data..

The dream of every product development team is to “lift and shift”—that is, build scalable mechanisms that drive effective results across every category, market, and circumstance. In fact, this touted simplicity-of-use is often what makes many of the early-market synthetic models so attractive: Who doesn’t want the ability to type in a few details about consumer demographics and preferences and generate concept feedback for skin care, cleaning solutions, and dog food?

But can such an approach create output that accurately reflects human decision-making?

We know that consumers are predictably irrational when it comes to decision making, and that value drivers are often shaped by context. What a consumer wants in cereal packaging will be different from their preferences for skin care; their price sensitivity for teeth whitening products will be different than that for chewing gum. Synthetic personas similarly demonstrate varied preferences among categories but in different ways from humans. Model biases can also show up in different roles across categories—for example, in our experimentation, we observed that the synthetic respondents seemed to care more about human health than the actual humans did.

Controlling for these factors requires continual adjustment of methodology, instructions, and even prompting order to arrive at truly authentic feedback. Learning as we go, we’ve adapted our testing procedures accordingly and, more importantly, validated that output against feedback from actual human consumers, leveraging our vast experience calibrating consumer-panelist scores to in-market sales from retail sales data.

Importantly, we’ve also worked to ensure our methodology prioritizes the protection of our clients’ intellectual property—meaning their proprietary content or data will never be used to train models. While our process might seem tedious to some, we’ve found there are no shortcuts for protecting data quality or our clients’ competitive information. These continual refinements and the ability to leverage our data assets for input and validation give us confidence in the accuracy and utility of our output, resulting in the business outcomes our clients have come to expect from us.

Think back to Spring 2020. Up to then, did you need multi-packs of disinfecting wipes? Did you ever strategize about your toilet paper supply beyond expecting house guests or finding your favorite brand on sale? The pandemic is perhaps a dramatic but relevant example of how quickly consumer behavior can shift. Even subtle movement in trends over time can lead to feedback variation, which is why we have prioritized prompting models with the latest in-market behavioral data rather than training them on historical preferences.

But not all consumer data is created equally. We have spent decades curating our datasets to ensure they are both controlled and calibrated to accurately reflect what consumers are both interested in—and actually buying—today. Leveraging these behavioral datasets in our prompts ensures that synthetic feedback is predictive of current and emerging trends instead of reflective of preferences from 5 or 10 years ago. From a time and resources standpoint, this approach also alleviates the need to constantly retrain models as information is updated and technical capabilities evolve, ensuring our platform is always utilizing the latest and greatest data and technology.

If you’re going to invest in market research to narrow down concept ideas, it’s essential that those insights point you in the right direction—otherwise, you’re no better off than simply picking the “team favorite.” By leveraging our extensive and proprietary scope of recent market data to prompt our synthetic personas, we found a clear divergence in output quality to validate our methodology and offer reassurance about our models’ ability to stand the test of time.

You input the prompt, ran the query, generated the output … now what?

For most synthetic models models currently on the market, functionality ends once the output is generated. Unfortunately, this often leaves users holding the bag on next steps. Just how good is a “good” concept? Can a “bad” idea be optimized with a few tweaks, or should it be discarded? What is the next phase of idea development, and how can it be elevated all the way through launch?

As experts in curating unique datasets, we are building our own database of synthetic respondents. This repository will not only indicate exactly how well a concept performs but also place that performance in comparative context against innovators’ other ideas. By removing this guess work, innovators can determine which initiative is a better bet, or pinpoint improvements instead of going back to the drawing board.

Developing and launching a successful innovation is a lengthy process that goes well beyond idea vetting. Whether your consumer feedback is synthetic or human, providing that information in context to navigate next steps is crucial and, in our minds, what differentiates a best-in-class tool from an average one. The ability to evaluate your early-stage idea with The Full View™ of NIQ insights can make the difference between a product that simply launches and one that ultimately thrives.