Is Synthetic Data Good Enough? Seven Criteria for Product and AI Teams

The idea is rarely the problem. The problem is the lack of data. Sometimes, the necessary information simply does not exist yet. Sometimes, it cannot be used for data protection reasons. And sometimes, the rare situations a system is expected to handle reliably are precisely the ones that are barely documented.

Synthetic data offers a way forward. It can be generated, modified, and scaled in a controlled manner, allowing teams to start working before the real world has produced enough data. But this new ability to act should not be mistaken for empirical certainty.

The key question, therefore, is not whether a synthetic dataset appears realistic enough. What matters is which decision it is meant to support, which properties of reality need to be preserved for that purpose, and how the underlying assumptions can later be validated.

Almost every product and AI team knows the situation: the idea is there, and a first prototype can be built. But the data the eventual system will depend on either does not exist yet, cannot be used, or fails to capture precisely those rare situations that matter most.

The obvious response is to look for more data or for a way to simulate it. Both approaches can help. But neither is useful unless it is clear beforehand what the data is actually needed for.

The key question is not whether synthetic data looks realistic. It is which decision the data is meant to support.

The momentum is real, not just a hype narrative. In 2021, synthetic data accounted for around 1% of the training data used for AI models. By 2023, Gartner analysts were already predicting that this share would rise to more than 60% by the end of 2024, with noticeable effects across healthcare, financial services, and retail (see Tech Monitor, 2023). The economic benefit becomes particularly apparent in four recurring situations.

The product or process does not exist yet, so no real-world data is available. Teams building a new product category therefore simulate usage scenarios and data flows instead of waiting for the first real traffic to arrive.

Real-world data cannot always be used freely for legal, ethical, or organizational reasons. In insurance underwriting, for example, data protection requirements and gaps in historical data can limit risk assessment. Insurers therefore generate synthetic customer profiles to test their risk models on a broader yet privacy-compliant basis and improve the accuracy of their risk evaluations (see adesso).

Certain errors, anomalies, or edge cases occur only rarely in real-world datasets, yet can be highly relevant from a business perspective. In the financial sector, this is a core challenge in fraud detection: fraudulent transactions are extremely rare compared with legitimate ones, which is why models are additionally trained on synthetically generated fraud patterns (see Tech Monitor, 2023). The same principle applies in industry to equipment failures. Components with very long lifespans produce little real-world failure data. Rather than waiting for the first actual defect to occur, production teams model the wear process and use these simulations to train AI systems for predictive maintenance (see ingenieur.de, 2025).

Data models, user interfaces, visualizations, and response logic can be tested before a resource-intensive data collection phase begins. This shortens the time between the initial idea and the first prototype that can actually be validated, especially in cases where real-world usage data only becomes available after launch.

Four situations, one common denominator: the real advantage lies in the time and room for experimentation a team gains, not in the dataset itself.

The four situations above show where synthetic data can, in principle, be useful. Whether it is suitable in a specific case depends on seven key criteria.

"We want to use synthetic data" does not yet describe a concrete use case. More precise formulations would be:

The final question requires significantly more evidence than the first four.

Depending on the purpose, different qualities matter:

A dataset can be structurally sound and still be socially implausible.

For a dashboard test, a simplified simulation is usually sufficient. But when decisions affect people, services, safety, or health, that is no longer enough. What matters is the scope of the consequences, not the technique used to generate the data: a false test signal may cost a sprint. A false assumption about real-world user groups can cost trust, budget, or, in the worst case, people’s health. The greater the potential impact, the earlier Criterion 6 needs to be addressed.

For this purpose a compact documentation is required:

This documentation requirement is now also embedded in regulation: the EU AI Act expects companies to be able to demonstrate the provenance, processing, and quality assurance of their training data, including measures to prevent bias and data gaps (see secjur, 2026). Teams that take Criterion 4 seriously are effectively addressing this documentation requirement at the same time.

Synthetic datasets often appear particularly clean and consistent. Real-world environments, by contrast, involve contradictions, missing information, and ambiguous patterns. When a scenario is designed from the drawing board, it almost inevitably incorporates the designers’ own assumptions about what is normal and can therefore overlook precisely those cases that later cause surprises in real-world operation. A review by domain experts who are familiar with the actual edge cases encountered in day-to-day practice can expose this gap early, before it becomes apparent during the pilot phase.

A robust validation path ideally consists of the following five stages:

A synthetic result and an empirical claim belong to two different categories. The traffic-light framework in the next section shows how to draw this distinction in practice.

A simple traffic-light framework categorizes use cases according to how much additional validation they require.

Recent practitioner guidance on synthetic data proposes a three-pillar model: fidelity, meaning how closely the data reflects real-world distributions; utility, meaning how useful it is for the intended training objective; and privacy, meaning protection against re-identification. These dimensions are assessed, among other methods, using Train-on-Synthetic-Test-on-Real benchmarks and nearest-neighbor distance to real-world datasets (see Fixstars, 2026). This aligns with Criteria 2 and 6: without testing performance on real-world data, any fidelity score remains only a claim.

Market research draws a similar line when it comes to product testing. A model cannot experience consumer behavior in the way a human can, which means synthetic simulation remains a complementary tool in this context, it does not replace real-world testing (see marktforschung.de, 2025).

This is especially important when working with synthetic behavioral data. Whenever people (and therefore social beings) are involved, particular caution is required. Because synthetic data is often generated according to predefined rules, it can reproduce biases or appear far more clear-cut than real-world data ever would. Whenever synthetic data is used to accelerate processes such as user testing, it is therefore essential to take a critical view of the resulting insights and evaluate them carefully.

As a rule of thumb, it helps to think of a five-step ladder.

Synthetic data can prepare the ground for the first stages. It cannot replace the later ones.

To get started with your own project, the framework can be captured on a single page. The Synthetic Data Readiness Canvas brings together ten fields: development question, purpose of the data, required properties of reality, data generation, assumptions, risks, validation path, permissible claims, transition to real-world data, and responsibilities.

Synthetic data changes when a team can start working. It does not change how much evidence a decision ultimately requires. Teams that distinguish between these two questions early can avoid debates that would otherwise emerge only during the pilot phase. When, for example, a domain team realizes that the available data does not provide sufficient evidence to support its conclusions, this becomes very useful.

The framework of seven criteria, the traffic-light model, and the evidence ladder can be applied to any initiative, from an initial prototype to a production-ready AI application. It does not replace case-specific expert assessment, but it helps product, data, and domain teams establish a shared language early on: What exactly needs to be demonstrated, and what kind of data is required to provide that evidence? Establishing this clarity at the beginning of a project is far less costly than correcting course later.

Want to develop an AI or product prototype but do not yet have a reliable data foundation? In a Synthetic Data Readiness Workshop, we work with you to determine which development questions can be addressed through simulation, what data structure you need, and at what point validation with real-world data becomes indispensable. As part of the endeavors in our R&D Lab, we have built up considerable expertise in the use of synthetic data.

adesso: Methoden und Tools fürs Underwriting. Prozessunterstützung für eine schnelle Bearbeitung.

Fixstars, 2026: Synthetic Data for AI Model Training: A Decision Framework for ML Engineers.

ingenieur.de, 2025: Wie synthetische Daten die KI in der Produktion trainieren.

marktforschung.de, 2025: "Synthetische Daten bieten für Produkttests viele Vorteile."

secjur.com, 2026: EU AI Act: Warum Datenqualität der Schlüssel zur Compliance ist.

Tech Monitor, 2023: The majority of AI training data will be synthetic by next year, says Gartner.