Why does nginx respond to IP address without a default_server directive?

I have an nginx webserver with two sites that listen to port 80 for two domains: dom.net and dom.se

In sites-enabled I have files for each of these, dom.net.site and dom.se.site, which are identical but for the server_name directive. It is either server_name dom.net *.dom.net; or server_name dom.se *.dom.se;. The rest is virtually the same but for a location to help me debug and it looks like this (for the .net site):

  location /check {
    return 200 'dom.net';
    add_header Content-Type text/plain;
  }

What I don’t understand is why when I go to my-server-ip/check I get the same response as if I go to server.dom.net/check. I have no default_server directive in any of the site files and still nginx responds to the ip-address.

And why did it pick server.dom.net and not server.dom.se?

Very confused.