azure sql database – Handle 4000 upserts per second on Sql Server (MSSQL)
I have a 6 vCore Azure MSSQL database where I’m consuming from a Kafka topic at 4000 messages per second. No matter what I try it seems that this is pushing the limits of what’s possible with this setup. My indices get’s fragmented almost instantly and Log I/O is hammering at 100%. I’m using a MERGE statement to upsert the data, but I need to have this thing insert data at at least 4001 entries per second, and then be able to query the data on some of the fields (so having an index on a key that’s random).
What’s some common optimization techniques that could help speed this up?
Read more here: Source link
