google cloud platform – GCP PubSub to GCS dataflow

I have one use case to create files in GCS buckets from PubSub with the below conditions:

  1. Window size limit
  2. Published message size to PubSub topic exceeds 500MB

For eg,
i) If we specify window size as 15 mins, and the messages published to PubSub exceeds 500MB before the window size of 15 mins, then we can load data to GCS buckets
ii) If the messages published to PubSub does not exceed 500MB even after the window size of 15 mins, then it that case too, it should start data load to GCS Bucket since the window size is reached.

Can anyone please help me with code reference for the above use case?

I tried creating a function to get the size of messages published to PubSub but unable to get the size

Read more here: Source link