Track · 2:32 · Liner note
Just the New Rows
Delta loading moves only the rows that are new or changed since the last run, identified by a timestamp, an increasing ID or a change log, instead of copying the entire table.
Why copy a table that mostly did not change? Just the New Rows is the delta idea in four words: identify what is new or altered since the last run, by a timestamp, an increasing ID or a change log, and move only that. Runtimes drop from hours to minutes and the source database gets a quieter night.
Deltas assume you can see every change. Updated rows that do not bump their timestamp, and deleted rows that leave no trace, slip through. Reconcile counts against the source now and then, and keep a full reload ready for the day they disagree.