Autoscaling Streaming Applications in Cloud Dataflow with Scotiabank (Cloud Next '18)

This talk provides and overview of how Cloud Dataflow autoscales streaming applications on GCP. Often the production streaming pipelines are over provisioned in order to handle peak loads. Autoscaling improves resource utilization by scaling up or down based on the load. Better understanding of various factors that influence its decisions in a pipeline is helpful in development and management of production pipelines. We also talk about how common streaming sources like PubSub and Kafka interact with autoscaling.


Event schedule →

Watch more Data Analytics sessions here →
Next ‘18 All Sessions playlist →

Subscribe to the Google Cloud channel! →

Duration: 38:2
Publisher: Google Cloud
You can watch this video also at the source.