Abstract

Apache Flink offers unparalleled powered and control for streaming applications at scale, wrapped in developer-friendly Java and Scala SDKs, simple to scale up and monitor even for high-throughput applications. It’s a beast (an adorable beast), hungry for data, and we must feed it. Apache Streams (incubating) unifies a diverse world of digital profiles and online activities into common formats and vocabularies, and makes these datasets accessible through standard interfaces, hiding the complexity of the underlying APIs and making the data they generate inter-compatible by default. Streams is the perfect tool to ingest interesting datasets to feed Flink, and embedding Streams components within Flink pipelines is dead simple and downright pleasant. People Pattern uses Flink and Streams to grow and maintain our database of profiles (200M and growing), plus posts and connections from 10+ social networking services including Twitter, Instagram, Google+. Come learn how we do it.

Slides: http://sblackmon-apache-org.herokuapp.com/flinkforward2016.html#/

Video on YouTube

Speaker

Details