Troubleshoot streaming job not scaling down in Dataflow

Would you like to learn how to troubleshoot autoscaling issues in Dataflow ? In particular, are you observing that your streaming jobs are not scaling down? Would you like to know more about the autotuning features in Cloud Dataflow?

Check out this first episode to learn about the Autotuning features Horizontal autoscaling, Vertical autoscaling, and Dynamic Work Rebalancing. Watch this video to learn about the possible causes for streaming jobs not scaling down and how to resolve the same. It also covers the streaming engine and its benefits.

0:00 – Intro
0:16 – Autotuning features
1:44 – Problem: Streaming job not scaling down
1:59 – Validation of the criteria for autoscaling
2:20 – Limitations of streaming appliance
2:34 – Example of limitations imposed by streaming appliance
3:05 – Cause for streaming job not scaling down
3:20 – Solution for the problem
3:33 – Benefits of using the streaming engine
3:57 – Documentation and conclusion

Autotuning features in Dataflow →

Streaming Engine →

Subscribe to Google Cloud Tech →

Duration: 00:04:26
Publisher: Google Cloud
You can watch this video also at the source.