How to avoid duplicate Batch Id’s in a table in Azure SQL db using python and Azure Blob Trigger

For context, I’ve got files that are uploaded daily to an Azure Storage Container. When files are uploaded at the same time, I have trouble avoiding duplicate Batch Id’s inside of my load log table in my Azure db.

I’m generating Batch Id’s by querying the load log table. Basically,

BatchId = SELECT TOP 1 MAX(BatchId) FROM ["Load_Log"]

Then I just increment it by one and send it back to the load log with some other data.

The problem comes when files are loaded at the same time. I’ve enforced the scale out limit on my Function App to 1, but it seems it still runs more than one instance if files are uploaded at the same time. Which causes duplicated Batch Id’s

How can I avoid duplicate Id’s whether that’s through somehow controlling the amount of instances at one time on the function App, or some other method?

Additional context: The table allows for duplicate Batch Id’s because one “Batch” consists of an xml and csv file and I put file names inside of the load log. Since they relate to each other I have the same Id for both.

Read more here: Source link