31. How would you establish a data-quality program from scratch?
For a new data-quality program, describe how business requirements become rules, thresholds, ownership, monitoring, and escalation.
I would start with the business decisions and reports that depend on the data, then define the expected schema, grain, source of truth, and quality dimensions such as completeness, accuracy, consistency, validity, uniqueness, and timeliness. I would translate those requirements into measurable rules with agreed thresholds and owners, add reconciliation and lineage checks, and automate monitoring. When a rule fails, the response should depend on business impact: critical data may be blocked or quarantined, while lower-risk exceptions may continue with alerts. I would keep an audit trail, verify recovery, and add prevention or regression checks so recurring problems are fixed at the source.
A data-quality program is a repeatable way to make sure data is trustworthy enough for the decisions that depend on it. I would not begin with a monitoring tool or a large checklist. I would first understand which decisions and outputs matter, what each dataset represents, where the data comes from, and what failures could make it unsafe to use. Then I would turn those expectations into measurable rules, agree on thresholds, assign owners, automate monitoring, and define what happens when checks fail. The goal is controlled, visible, and auditable quality where errors could affect decisions.
- Which business decisions, reports, or analytical outputs are most important to protect?
- What are the authoritative source systems for those outputs?
- What are the expected population, row grain, schema, units, reporting interval, and refresh timing for the important datasets?
- Which quality failures would make an output unsafe to use, and which could be tolerated temporarily?
- Who owns the business definition, source data, transformation logic, and downstream output?
- Are there existing expectations for data arrival, refresh timing, alerting, or issue resolution?
- What evidence is required for audit, incident review, or recovery approval?
I would organize the program as a closed loop from business requirement to measurable rule, from rule to monitored control, and from failure to remediation and prevention.
First, I would identify the critical data products and decisions. Data quality is contextual: a problem matters because it can change a report, analysis, operational action, or business decision. For each important output, I would document its purpose, expected population, row grain, units, reporting interval, required fields or metrics, and authoritative source of truth. I would also record assumptions and known data limitations instead of leaving them implicit.
Second, I would define the main quality dimensions in plain language. Completeness asks whether required data is present. Accuracy asks whether values correctly represent the authoritative source or business reality when that can be verified. Consistency asks whether the same concept follows compatible definitions and values across systems or processing stages. Validity asks whether values follow agreed formats, domains, and business rules. Uniqueness asks whether records that should be unique occur only once at the defined grain. Timeliness asks whether data arrives and becomes usable when the business needs it. These dimensions matter only when they are tied to a specific requirement.
Third, I would translate business requirements into explicit quality rules. Each rule should state what is being checked, at what population and grain, against which expected condition or reference, how failure is measured, and who owns the rule. Examples of control types include null checks on required fields, duplicate checks on expected keys, range or domain checks, schema-change detection, key validation, and referential-integrity checks. I would also perform data profiling for nulls, duplicates, ranges, unexpected categories, and schema behavior so I understand the current data before setting controls.
Fourth, I would establish reconciliation controls across important processing boundaries. Reconciliation means checking that data has not been unintentionally lost, duplicated, or altered as it moves from source to report. I would compare row counts before and after transformations when count preservation is expected and reconcile meaningful control totals back to the source system when comparable totals exist. The comparison must use equivalent populations, grain, units, definitions, and time windows. A mismatch is evidence to investigate, not automatic proof of a defect, because filters, joins, deduplication, late-arriving records, or intentional grain changes can legitimately change counts or totals.
Fifth, I would document data lineage. Lineage shows where an important field or metric originated and which transformations it passed through. If a downstream metric looks wrong, I can trace it backward through the transformation chain to determine whether the problem began in the source system or was introduced later. This helps prevent teams from repeatedly fixing downstream symptoms while the upstream cause remains.
Sixth, I would define thresholds and severity levels based on decision impact. A threshold should represent an agreed level of risk, not an arbitrary technical value. Some failures may make an output unsafe to release and should trigger safe failure, quarantine, or withholding of affected data. Lower-severity exceptions may allow processing to continue while creating an exception report and alert for review. The threshold, severity, allowed behavior, and escalation path should be agreed before an incident whenever possible.
Seventh, I would assign ownership. Each important dataset and quality rule needs clear accountability. Depending on the organization, ownership may be divided among a source-system owner, transformation or pipeline owner, business-definition owner, and downstream consumer. The important point is that a failed rule has a known person or team responsible for investigating it and a known decision owner for determining whether affected data is safe to use. An alert without an owner is only noise.
Eighth, I would automate monitoring at appropriate points in the data flow. For each execution, I would retain the rule evaluated, expected condition, observed condition, status, affected scope, execution time, and ownership information. I would monitor individual failures as well as trends where gradual deterioration matters. Schema changes and unexpected categories deserve explicit checks because they can silently change downstream behavior even when a pipeline technically completes.
Ninth, I would define exception handling and escalation before failures occur. Invalid records should not be silently dropped, overwritten, imputed, or repaired. If a business-approved rule permits a correction, the transformation should be reproducible and auditable. Otherwise, affected records can be quarantined with a reason when isolation is safe. Critical failures should escalate to the appropriate owners and affected consumers. The incident record should state what failed, the affected scope, whether the output should be trusted, and what is still uncertain.
Tenth, I would keep an audit trail. I would retain quality-rule definitions, threshold changes, check results, exceptions, approved overrides, remediation actions, and recovery evidence. This provides accountability and makes later investigation easier. If a threshold or rule changes, the team should be able to explain what changed, why it changed, and which outputs may have been affected.
Finally, I would focus on prevention. After a defect is understood, I would classify the root cause as a source defect, schema change, transformation defect, definition problem, or missing control. Then I would add the appropriate upstream prevention or regression check. Recovery is not complete merely because a job reruns successfully; I would rerun the relevant quality checks, reconcile counts or control totals where appropriate, and verify the downstream output again.
The main tradeoff is coverage versus maintainability. Hundreds of weak checks can create alert fatigue and unclear ownership. Too few checks leave important risks invisible. I would therefore begin with critical decisions and high-impact failure modes, make those controls reliable and actionable, measure whether they detect useful issues, and expand coverage based on evidence. That creates a program that improves data trust without becoming an unmanageable collection of alerts.
- Identify the critical business decisions, reports, and data products that require trustworthy data.
- Document each important dataset's authoritative source, expected population, row grain, schema, units, reporting interval, lineage, assumptions, and known limitations.
- Define the relevant quality dimensions: completeness, accuracy, consistency, validity, uniqueness, and timeliness.
- Translate business requirements into measurable quality rules with explicit scope, expected condition, failure measurement, owner, and downstream impact.
- Profile nulls, duplicates, ranges, unexpected categories, keys, and schema behavior to establish the current state.
- Add appropriate null, duplicate, range, domain, schema, key, and referential-integrity controls.
- Add source-to-report reconciliation controls such as row-count checks and control-total comparisons using equivalent populations, grain, units, definitions, and time windows.
- Define business-approved thresholds, severity levels, and safe failure behavior for important rules.
- Assign ownership and define alerting, investigation, quarantine, exception reporting, and escalation paths.
- Automate monitoring and retain rule results, failures, threshold changes, overrides, and remediation evidence in an audit trail.
- Use data lineage to trace failures to their source and distinguish source defects from transformation defects.
- After remediation, rerun the relevant checks and reconciliations, verify downstream recovery, and add prevention or regression controls for recurring problems.
The main cost is operational rather than algorithmic. Every quality rule needs a correct definition, data access, an owner, monitoring, investigation, maintenance, and a response when it fails. Too few controls leave important problems invisible, while too many low-value controls create alert fatigue. I would prioritize high-impact data first, keep rules understandable and actionable, and expand coverage only when the team can maintain the controls and respond to their failures.
The interviewer is testing whether I can turn broad expectations about trustworthy data into an operating program with measurable rules, business-aligned thresholds, clear ownership, monitoring, escalation, validation, and prevention instead of treating data quality as a one-time cleanup task.
Common mistakes are starting with a tool instead of business requirements; defining generic checks without preserving population, grain, units, definitions, or time windows; checking only nulls and duplicates while ignoring reconciliation and lineage; assuming the source system is automatically correct; setting thresholds without business ownership; treating every failure as equally severe; creating alerts without clear owners or escalation paths; silently dropping, overwriting, imputing, or repairing invalid records; failing to distinguish source defects from transformation defects; changing rules or thresholds without an audit trail; and declaring recovery complete without rerunning controls and verifying downstream outputs.
Explain the program as a closed loop: business requirement -> quality rule -> threshold -> owner -> monitoring -> safe response -> escalation -> root cause -> prevention. Emphasize that controls are tied to decision impact and that invalid data is never silently changed without a justified, auditable rule.







