The myth of the clean weekend cutover
When agencies pitch catalogue migrations, they routinely draw a simple linear timeline: extract legacy data, transform rows, load into Shopify, and switch DNS over a weekend.
For a boutique store selling two hundred items, this naive strategy works. When a catalogue spans 28,000 products across tens of thousands of variant combinations, the linear model collapses entirely: GraphQL rate limits require twelve to eighteen hours of sustained processing, staging merchandising work is at risk of being wiped out, and mid-market merchants cannot freeze their live business for three weeks.
A resilient cutover is not an event; it is an ongoing, automated data pipeline. The initial migration establishes the baseline; scheduled delta synchronisations maintain parity until the final cutover takes less than fifteen minutes.
The staging concurrency problem: source edits vs Shopify enrichments
To avoid clobbering staging progress, your migration architecture must establish explicit Field Ownership Boundaries: the legacy ERP remains authoritative for baseline SKUs, wholesale/retail prices, inventory counts, and weights, while the Shopify staging environment remains authoritative for enriched HTML copy, media galleries, custom metaobjects, curated collections, and SEO metadata.
A common disaster during catalogue replatforming occurs when migration scripts fail to partition field ownership. If a legacy export includes an empty description column, a naive update mutation will overwrite the enriched HTML description meticulously crafted by the marketing team.
Furthermore, as explored in our architectural guide on large catalogue migrations in Shopify, GraphQL productSet treats variant lists as declarative state. If an update payload includes only the variant whose price changed and omits the product's remaining variant IDs, Shopify interprets the omitted variants as deleted and purges them immediately from the database.
A delta sync engine must be surgically selective: patching only source-authoritative fields while preserving destination-authoritative structures and existing variant entity IDs.
Deterministic content hashing: isolating dirty records from no-ops
In a catalogue of 28,000 products, typically fewer than three to five percent of records actually change between Friday evening and Sunday night. Iterating through all 28,000 products and firing unconditional GraphQL productSet mutations consumes massive API throughput, triggers rate-limit throttling (HTTP 429), and creates thousands of unnecessary webhook events downstream.
To achieve near-instantaneous delta syncs, the engine evaluates records using Deterministic Content Hashing: normalizing source attributes, strictly excluding non-functional updated_at timestamps or database primary keys from the hash, generating an MD5 hash of core commerce attributes, and comparing it against the target state.
If the hashes match, the record is flagged as NOOP and skipped entirely, consuming zero GraphQL API points.
Pure delta reconciliation engine: from source feed to mutation batches
The delta reconciliation engine must be pure and deterministic. It evaluates incoming source records against existing Shopify products, detects genuine modifications, protects staging enrichments, preserves variant IDs, flags dropped items, and chunks mutations into manageable GraphQL batches:
export interface SourceProductRecord {
sourceId: string;
sku: string;
title: string;
vendor: string;
productType: string;
price: string;
compareAtPrice?: string;
barcode?: string;
weightGrams: number;
tags: string[];
}
export interface ExistingShopifyProductRecord {
shopifyId: string;
sourceId: string; // Stored in custom.source_id metafield or external ID
handle: string;
status: "ACTIVE" | "DRAFT" | "ARCHIVED";
variantId: string;
sku: string;
title: string;
vendor: string;
productType: string;
price: string;
compareAtPrice?: string;
barcode?: string;
weightGrams: number;
tags: string[];
// Staging enrichments that must be preserved
enrichedDescriptionHtml?: string;
customMetafields?: Record<string, string>;
seoTitle?: string;
seoDescription?: string;
}
export type DeltaActionType = "NOOP" | "CREATE" | "UPDATE_PATCH" | "TOMBSTONE_ARCHIVE";
export interface DeltaAction {
action: DeltaActionType;
sourceId: string;
shopifyId?: string;
variantId?: string;
sku: string;
patchPayload?: Partial<SourceProductRecord>;
preservedEnrichments?: {
descriptionPreserved: boolean;
metafieldsPreservedCount: number;
};
reason: string;
}
export interface DeltaSyncReconciliationResult {
summary: {
totalSourceRecords: number;
totalExistingShopify: number;
noopCount: number;
createCount: number;
updateCount: number;
tombstoneCount: number;
};
actions: DeltaAction[];
chunkedMutationBatches: DeltaAction[][];
errors: string[];
}
/**
* Computes a deterministic MD5 hash of core source-controlled commerce attributes.
* Excludes transient timestamps and destination-owned styling/metafields.
*/
export function computeSourceContentHash(record: SourceProductRecord | ExistingShopifyProductRecord): string {
const norm = {
sku: record.sku.trim().toUpperCase(),
title: record.title.trim(),
vendor: record.vendor.trim(),
productType: record.productType.trim(),
price: parseFloat(record.price || "0").toFixed(2),
compareAtPrice: record.compareAtPrice ? parseFloat(record.compareAtPrice).toFixed(2) : null,
barcode: (record.barcode || "").trim(),
weightGrams: Math.round(record.weightGrams || 0),
tags: [...(record.tags || [])].map((t) => t.trim().toLowerCase()).sort(),
};
return crypto.createHash("md5").update(JSON.stringify(norm)).digest("hex");
}
/**
* Pure delta reconciliation engine for large catalogue cutover migrations.
* Compares incoming legacy source feeds against live Shopify staging catalog,
* isolating genuine updates, protecting manual staging enrichments,
* guarding against productSet variant deletion, and handling discontinued items.
*/
export function reconcileCatalogDeltaSync(params: {
sourceFeed: SourceProductRecord[];
existingShopify: ExistingShopifyProductRecord[];
batchSize?: number;
}): DeltaSyncReconciliationResult {
const { sourceFeed = [], existingShopify = [], batchSize = 50 } = params;
const actions: DeltaAction[] = [];
const errors: string[] = [];
const existingBySourceId = new Map<string, ExistingShopifyProductRecord>();
const existingBySku = new Map<string, ExistingShopifyProductRecord>();
for (const item of existingShopify) {
if (item.sourceId) {
existingBySourceId.set(item.sourceId, item);
}
if (item.sku) {
existingBySku.set(item.sku.trim().toUpperCase(), item);
}
}
const processedSourceIds = new Set<string>();
// 1. Evaluate incoming source records against existing Shopify products
for (const source of sourceFeed) {
processedSourceIds.add(source.sourceId);
const existing = existingBySourceId.get(source.sourceId) || existingBySku.get(source.sku.trim().toUpperCase());
if (!existing) {
// Record does not exist in Shopify -> New product create
actions.push({
action: "CREATE",
sourceId: source.sourceId,
sku: source.sku,
patchPayload: source,
reason: "New source SKU discovered during cutover delta feed.",
});
continue;
}
// Check if source attributes actually changed
const sourceHash = computeSourceContentHash(source);
const shopifyHash = computeSourceContentHash(existing);
if (sourceHash === shopifyHash) {
// Content is identical -> No-op, preserve rate limits
actions.push({
action: "NOOP",
sourceId: source.sourceId,
shopifyId: existing.shopifyId,
variantId: existing.variantId,
sku: source.sku,
reason: "Source attributes identical. Zero mutation emitted to preserve API points.",
});
continue;
}
// Attributes changed -> Generate selective update patch
// Crucial: Only mutate source-controlled fields, preserving Shopify staging enrichments!
const patchPayload: Partial<SourceProductRecord> = {};
if (parseFloat(source.price) !== parseFloat(existing.price)) {
patchPayload.price = source.price;
}
if ((source.compareAtPrice || null) !== (existing.compareAtPrice || null)) {
patchPayload.compareAtPrice = source.compareAtPrice;
}
if (source.title !== existing.title) {
patchPayload.title = source.title;
}
if (source.weightGrams !== existing.weightGrams) {
patchPayload.weightGrams = source.weightGrams;
}
if ((source.barcode || "") !== (existing.barcode || "")) {
patchPayload.barcode = source.barcode;
}
const metafieldCount = Object.keys(existing.customMetafields || {}).length;
actions.push({
action: "UPDATE_PATCH",
sourceId: source.sourceId,
shopifyId: existing.shopifyId,
variantId: existing.variantId, // Preserves variant in productSet
sku: source.sku,
patchPayload,
preservedEnrichments: {
descriptionPreserved: Boolean(existing.enrichedDescriptionHtml),
metafieldsPreservedCount: metafieldCount,
},
reason: "Source pricing/specifications modified. Emitting selective patch while isolating staging enrichments.",
});
}
// 2. Identify products discontinued or dropped from source feed
for (const existing of existingShopify) {
if (!processedSourceIds.has(existing.sourceId)) {
// Product was dropped in source ERP/catalogue.
// Do NOT hard-delete (which breaks 404s/redirects); mark as tombstone for unlisting/archiving.
actions.push({
action: "TOMBSTONE_ARCHIVE",
sourceId: existing.sourceId,
shopifyId: existing.shopifyId,
variantId: existing.variantId,
sku: existing.sku,
reason: "SKU absent from source feed. Flagged for status ARCHIVED rather than hard deletion to protect URLs.",
});
}
}
// 3. Summarize outcomes
const summary = {
totalSourceRecords: sourceFeed.length,
totalExistingShopify: existingShopify.length,
noopCount: actions.filter((a) => a.action === "NOOP").length,
createCount: actions.filter((a) => a.action === "CREATE").length,
updateCount: actions.filter((a) => a.action === "UPDATE_PATCH").length,
tombstoneCount: actions.filter((a) => a.action === "TOMBSTONE_ARCHIVE").length,
};
// 4. Chunk active mutations (creates, updates, tombstones) into manageable GraphQL batches
const mutatingActions = actions.filter((a) => a.action !== "NOOP");
const chunkedMutationBatches: DeltaAction[][] = [];
for (let i = 0; i < mutatingActions.length; i += batchSize) {
chunkedMutationBatches.push(mutatingActions.slice(i, i + batchSize));
}
return {
summary,
actions,
chunkedMutationBatches,
errors,
};
}The tombstone dilemma: handling discontinued products without 404s
What happens when a SKU present in your initial staging export is absent from the cutover delta feed? In many instances, the supplier discontinued the product during the six-week staging window. Naive migration scripts react by calling productDelete.
Hard deletion is an organic search catastrophe. As detailed in our analysis of Shopify redirects when deleting products, deleting a product completely obliterates its URL, immediately returning a 404 HTTP status to Googlebot and destroying historical SEO equity.
Our delta reconciliation engine treats omitted records as Tombstones: the record is retained in Shopify, transitioned to ARCHIVED or UNLISTED status (removing it from search and sitemaps), and mapped to parent collections via automated 301 redirects.
Similarly, as demonstrated in the Lantern Indie audiobooks platform migration, heavy media assets (such as multi-gigabyte audiobook archives and high-resolution textile swatches) must be decoupled from transactional catalogue metadata. Ingesting media through dedicated cloud pipelines ensures that media assets move independently of volatile product price and inventory updates.
What native Shopify migration tools cannot do
Off-the-shelf CSV imports and generic migration apps on the Shopify App Store fall short during enterprise cutovers:
- Cannot perform two-way field partitioning: Standard CSV imports overwrite every mapped column. If your CSV includes a blank description column, Shopify will overwrite your enriched staging description with whitespace.
- Cannot execute selective content-hashed diffs: Standard tools process records linearly from top to bottom, making 28,000 mutations regardless of whether only twelve prices changed, burning API throughput.
- Cannot preserve variants in productSet declarations: Standard API scripts sending partial variant rows trigger automatic variant deletions due to Shopify's declarative schema model.
- Cannot automate tombstone lifecycle transitions: Generic CSV tools cannot detect that a product missing from the new file was present in the old file, leaving discontinued items stranded as active stock.
When this needs an engineer
You do not need a custom engineering build if you are migrating three hundred products from WooCommerce with no staging enrichments and can afford an hour of maintenance mode. In that scenario, Matrixify or standard Shopify CSV files handle the transfer adequately.
You need an engineer when your migration operates at scale: when catalogues exceed twenty thousand SKUs, when editorial and marketing teams spend weeks enriching data on Shopify staging while the legacy store continues trading, when GraphQL rate limits require content-hashed delta diffing, and when a botched cutover would wipe out weeks of copywriting and destroy search engine indexation.
Explore our catalogue migration case studies to see how large-scale product datasets are structured, or inspect The Fabric Co. 28,000-product migration case study to see how complex supplier-driven inventories were migrated safely. You can also explore our Shopify migration services.
Send us the store and the symptom. We will audit your source datasets, inspect your staging schema, and build a deterministic delta cutover engine that launches your new platform without downtime, data loss, or SEO regressions.