Four failure modes account for most of what goes wrong in a large migration, and none of them is the transfer of records.
Modern tooling moves data reliably. The difficulty sits earlier, in what nobody established about the source before the plan was committed, and later, in whether anyone trusts the destination once it arrives.
Sampled profiling against exhaustive profiling
The choice between these two sounds like an engineering detail. It decides the shape of the project.
What sampled profiling gives you
A sample finds the shapes that repeat. It tells you what the data usually looks like, which is genuinely useful when the goal is to understand the structure of a system. It is fast, it is cheap, and it is a reasonable compromise for a design exercise.
Its exceptions get discovered during the move, when the schedule has no room left for them.
What exhaustive profiling gives you
Profiling every record finds the ones that do not fit, which is where migration effort actually goes. That is a deliberate cost and it buys one specific thing: the exceptions are known before the plan is set rather than during execution.
A migration fails on what the data rarely looks like. A method designed to find what it usually looks like is optimized for the wrong question.
Failure one: nobody knows what the source contains
Long-lived systems accumulate logic in places nobody documents. Business rules end up enforced in custom code because that was the fastest way to solve something in 2011, and the person who wrote it has moved on.
That logic is invisible to a schema comparison and invisible to a sample. It surfaces during user acceptance testing, when a calculation that has been quietly correct for a decade starts producing a different answer and nobody can explain why the old system did it differently.
The exhaustive pass surfaces those rules before they become blockers, identifying which are being enforced, which downstream processes rely on them, and which migration decisions they constrain.
Failure two: identity was never resolved
The same customer exists under several account numbers across company codes, entered by different teams under different conventions across a decade. The same part exists under several material records depending on which plant created it and when.
A migration that carries those duplicates forward has moved the problem rather than solved it. A migration that merges them without evidence has created a worse one, because a wrongly merged customer is harder to unpick after go-live than before it.
Entity resolution matching records across the whole source instance, without requiring matching keys or a pre-existing governance program, is what makes this tractable. It also means master data does not have to be fixed before the migration can be scoped, which removes the dependency that stalls most of these programs at the planning stage.
Failure three: dependencies were assumed rather than mapped
A system gets scheduled for retirement because it looks peripheral on an application inventory. Three months later a process nobody associated with it stops working.
Lineage discovery traces every connection between a candidate system and its consumers before anything is scheduled: data flows, process triggers, interfaces, reporting chains, and derived datasets, however many hops removed. The analysis follows every hop rather than the first one.
Five questions get answered for every candidate. Which systems receive this data, directly or indirectly. Which business processes depend on that flow, at any remove. Which domains would become unavailable if it stopped. Whether the data is generated anywhere else in the estate. Whether consumers could carry on with a substitute source.
There is an enormous difference between a system that can be switched off with rework and a system that cannot be switched off without destroying a business capability. Both look similar on an application inventory, and only one of them is a candidate.
Failure four: reconciliation followed the move
Counts get compared after cutover, differences get logged, and the work of explaining them competes directly with the work of running a business on an unfamiliar system.
Running source and destination namespaces side by side through cutover changes that sequence. An exception surfaces and closes in the same reconciliation cycle rather than in the month-end that follows, which is the difference between a defect and an incident.
What the exhaustive pass actually surfaces
Four groups, in practice.
Undocumented business rules enforced in custom code. Duplicated master data across company codes and plants. Open items, meaning purchase orders, sales orders, and deliveries carrying years of accumulated inconsistency, which are the records that stall a cutover because they are live and cannot simply be reshaped. And orphaned references pointing at entities that no longer exist, harmless in a system that has learned to ignore them and a hard failure in a destination with a stricter data model.
Each of those is a question about the data that should have been answered before the plan was committed.
Why the destination’s trust posture is the real deliverable
Poor data quality already costs organizations an average of $12.9M annually according to Gartner, and a migration is the single event most likely to add to that figure.
Scoring at the attribute level rather than the record level is what changes the arithmetic. A weak field is visible as a weak field rather than dragging an otherwise sound record down with it, and every individual value in the destination carries its own confidence from the first day of go-live. After go-live the destination namespace becomes the operating fabric, so the work done to move the data is the same work that governs it afterwards.
Migration is a moment. The trust posture is permanent.
Map it before you move it
The PolyPhaze white paper Migration Is a Moment covers exhaustive profiling, the four scope decisions, lineage-based impact analysis, and continuous reconciliation through cutover. Download the full data migration ebook for the complete method.
Frequently asked questions
Why do data migrations fail?
Rarely because data cannot be moved. Failures cluster in four knowledge problems: undocumented logic in the source, unresolved identity across duplicate records, dependencies assumed rather than mapped, and reconciliation performed after cutover instead of continuously through it.
What is exhaustive data profiling?
Exhaustive profiling examines every record in the source rather than a representative sample. It costs more and finds the records that do not fit, which is where migration effort concentrates, so exceptions are known before the plan is set rather than discovered during execution.
Do you have to clean master data before a migration?
No. Entity resolution can match records across the whole source instance without matching keys or an existing governance program, which means duplicate master data can be identified and scoped as part of the migration rather than treated as a prerequisite that delays it.