15 Sept 2026 · 5 min read
Assessing data quality before an innovation decision
Review the relevance, freshness and coverage of your evidence, then decide which gaps require action.
A data-quality review should establish whether the evidence is suitable for the decision being considered. That includes its source, age, coverage, relevance and consistency with other findings.
For a concept review, current consumer research might sit alongside a provisional production estimate and an incomplete competitor audit. Each raises a different question about what the team can reasonably conclude and what it should investigate next.
This article sets out a practical review process for those differences. It includes an illustrative evidence table, guidance on handling incomplete assessments and a way to assign follow-up work.
Identify missing, outdated, mismatched and conflicting evidence
We propose four labels for reviewing the important inputs to an innovation decision.
Missing means the relevant information has not been obtained. The business may not yet have a supplier quotation for the intended volume.
Stale means the evidence exists but the conditions may have changed. The issue is not its age alone. It is whether a material assumption has moved since the observation was made.
Mismatched means the information concerns a different population, offer, geography, period or decision. A finding can be valid in its original context and still be a poor basis for this choice.
Contradictory means relevant evidence points in different directions. A positive stated preference and weak observed purchasing should prompt investigation, not an automatic average.
These are proposed working labels, not a new standard. Their purpose is to help a team choose the right response to each problem.
Assess coverage against the decision requirements
The same evidence can be enough for one commitment and insufficient for another.
A preliminary feasibility discussion might justify a small production trial. It would not necessarily justify committing to supply at scale. An exploratory interview can help formulate a hypothesis without validating the size of a market.
Describe coverage against the question: "What supports the next step?" Avoid implying that an evidence-coverage measure is a probability of product success.
The distinction is especially important in a visual interface. A green status can easily be interpreted as permission to proceed, even when it only means a document was uploaded. Name what the status represents and make the unresolved consequence visible.
An example of a concept evidence review
Imagine a team considering a new pack format. This example is fictional.
Input: Shopper problem
Evidence state: Relevant exploratory research
Decision consequence: Supports investigating the format, not a sales forecast
Next action: Test the proposed offer with the intended audience.
Input: Production cost
Evidence state: Missing for the proposed volume
Decision consequence: Commercial comparison remains provisional
Next action: Obtain a quotation under specified conditions.
Input: Competitor prices
Evidence state: Older channel snapshot
Decision consequence: Price positioning may have changed
Next action: Refresh the relevant comparison set.
Input: Consumer response
Evidence state: Tested at a different pack size
Decision consequence: Does not directly validate the new proposition
Next action: Check whether the change affects the research question.
Input: Repeat expectation
Evidence state: Stated intention only
Decision consequence: Actual repeat remains unobserved
Next action: Plan measurement after a suitable market test.
The table does not assign a universal severity to each row. Severity depends on the commitment under review. The production quotation may be a priority before further commercial work, while repeat requires a later observation window.
Handling missing inputs in concept scoring
A common design temptation is to calculate an overall assessment using only the inputs that happen to be available. That can be appropriate for a carefully specified model, but the missingness must still be disclosed and its consequences evaluated.
For a decision interface, an absent feasibility check should not simply vanish while strong consumer indicators dominate the display. Nor should the absence automatically be interpreted as poor feasibility.
The honest state is narrower: feasibility has not yet been established under the stated conditions.
That leaves the team with an actionable question. It also prevents two concepts with very different evidence foundations from appearing equally well supported just because their headline scores are similar.
Assign responsibility for resolving important gaps
A list of gaps can become another passive report unless it directs work.
For each material gap, record who can resolve it, the likely source, the cost or effort of the next check and the decision it could change. State whether the team should proceed with a limitation, commission research, change the option or pause the commitment.
Do not demand more research simply because uncertainty exists. Ask whether resolving this uncertainty could alter the decision. A low-impact missing detail may not deserve attention before a decisive production or price assumption.
Keep the reason for the response in the record. Another person should be able to see why a gap was accepted rather than assume it was overlooked.
Make evidence quality visible in the working assessment
An innovation workspace should make the relationship between a claim and its evidence inspectable. Useful questions include when an input was checked, which context it covers and which decisions rely on it.
Where AI is involved, the NIST AI RMF's mapping and measurement functions offer a broader reference for considering use context, limitations and evaluation. Our evidence-review table is an editorial proposal, not a NIST checklist or a compliance certificate. [1]
For TasteForge, the design implication is that data coverage belongs beside the decision, not only in an administrative screen. A user should be able to understand the limit without having to inspect the entire data pipeline.
That is a direction to build and test, rather than a claim that a filled-in matrix makes every conclusion reliable.
The next time a team asks for more data, ask which decision the missing information could change. Start there.
Source
[1] NIST, AI Risk Management Framework 1.0, Core. Reference for context and evaluation principles. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/