Abstract

The extensive set of high-level Flink primitives makes it easy to join windowed streams. However, use cases that don’t have windows can prove to be more complicated, making it necessary to leverage operator state and low-level primitives to manually implement a continuous join. This talk will focus on the anomalies that present themselves when performing streaming joins with infinite windows, and the problems encountered operating topologies that back user-facing data. We will describe the approach taken at ResearchGate to implement and maintain a consistent join result of change data capture streams.

Slides: Sanjar Akhmedov – Joining Infinity – Windowless Stream Processing with Flink.pdf

Video on YouTube

Speaker

Sanjar Akhmedov
Senior Software Engineer, ResearchGate

Details