Unable to access Azure SQL database from Windows server

I have created a new Azure SQL database which I am able to access from my local development machine using SQL Management Studio after I added my client IP to the firewall rules. We have a Windows server that also needs access to this new database, but even though I have added the server’s IP address to the firewall rules in Azure the connection is failing when I try to connect from the Windows server using Management Studio.

I have checked that port 1433 is open (both incoming and outgoing) on the Windows server, but the connection is still failing. Is there something else that needs updating to allow the server to reach the SQL instance in Azure?