Google Cloud error 429: The request was aborted because there was no available instance

I have a function in Cloud Run that takes files from a bucket, loads them to BigQuery and then deletes them.
It is activated using the Cloud Scheduler and an HTTP call.
To avoid processing the same files several times, I’ve used:

autoscaling.knative.dev/maxScale: '1'
containerConcurrency: 1

I set up a minimum of 1 container to avoid cold starts.

The function runs for about 2 minutes (can see log in beginning and end)
The scheduler calls it every 5 minutes

After a successful run, and sometimes when I redeploy the function, I see a 429 error:

“The request was aborted because there was no available instance.”

It happens several times in a row in 5 minute intervals (total of 10-30 minutes)
Then I get a successful run and after that the errors return and it goes on like that.

Other solutions I found talk about load spikes (which I don’t have) or cold starts, which I also eliminated AFAIK.

How do I fix this?

Read more here: Source link