Flink has consistency guarantees and efficient checkpointing model which make it a good fit for Uber’s money-related use cases, such as driver incentives. However, Flink’s time-progress model is built around a single watermark, which is incompatible with Uber’s business need for generating aggregates retroactively. The talk covers our solution for on-demand backfilling. It also outlines other abstractions and features we expect Flink to support as it matures.

Slides: Maxim Fateev – Beyond the Watermark- On-Demand Backfilling in Flink

Video on YouTube