This talk provides and overview of how Cloud Dataflow autoscales streaming applications on GCP. Often the production streaming pipelines are over provisioned in order to handle peak loads. Autoscaling improves resource utilization by scaling up or down based on the load. Better understanding of various factors that influence its decisions in a pipeline is helpful in development and management of production pipelines. We also talk about how common streaming sources like PubSub and Kafka interact with autoscaling.
DA222
Event schedule → http://g.co/next18
Watch more Data Analytics sessions here → http://bit.ly/2KXMtcJ
Next ‘18 All Sessions playlist → http://bit.ly/Allsessions
Subscribe to the Google Cloud channel! → http://bit.ly/NextSub
Duration: 38:2
Publisher: Google Cloud
You can watch this video also at the source.
