linux – Nginx autoindex is blocking IO on listing files

I recently encountered this issue where,

  1. I served a data path using auto-index, within it there is a directory with a LOT of files (I have a directory with 1228240 files on btrfs). Accessing an individual file is fine but listing files take 2 minutes.
  2. When you try to access that path directly through an HTTP request, nginx freezes (yes, entirely freezes, all virtual hosts times out) until listing is done, that’s 2 minutes of downtime.

However, I can’t just disable auto-index because other paths depend on its json output.

Is there a way to resolve this issue? Or is it something that needs to be reported to nginx.

Read more here: Source link