Track · 1:20 · Liner note
Truncate and Trust
A truncate-and-load pattern empties a table and reloads it completely from the source on every run, which is simple and self-healing but costly at large volumes.
Wipe the table, load everything, go home: the simplest pipeline in the world. Truncate and Trust has no watermark to lose, no merge to get wrong and no history of partial failures. Whatever the source says now is what you have now.
That suits small tables and dimensions that change often. It fails at scale and it leaves a window where the table is empty, so load into a new table and swap it in. Trust it where it is cheap, and graduate to incremental loads when the runtime says it is time.