express – Node.js Error: connect ECONNREFUSED / connect ETIMEDOUT

I am trying to run a load test on my dev environment, where my client is sending multiple api calls to my server parallely. The initial stage of the load test runs smoothly, without any issues. Errors start occurring towards the end of the load test.

For example, I am running a test with 100k records to be processed with each record performing about 20-25 api calls.
I start getting these ECONNREFUSED / ETIMEDOUT errors roughly after 75k-80k records have been processed successfully.

My server is running on nodejs and I am using single axios instance to make api calls.
I have also implemented back-off retries for api calls which are failing due to these but it is not helping much.

Below is my axios config (I think I have set it to high values which is not a good practice but I am trying to see which configuration is helping me avoid these errors) with which I am running tests but still I am getting these errors. Anything I can do on code level?
I have checked my cpu utilisation and it is not utilising much of it.

const Agent = require('agentkeepalive');
const HttpsAgent = require('agentkeepalive').HttpsAgent;
import axios from 'axios';

const Cacheable = require('cacheable-lookup');

let cacheable = new Cacheable.default({ maxTtl: 300 });

let instance;
const keepAliveAgent = new Agent({
  maxSockets: 1500,
  maxFreeSockets: 256,
  timeout: 5 * 60000,
  freeSocketTimeout: 5000,
  keepAlive: true,
  keepAliveMsecs: 60000,
});

// HTTPS agent
const httpsKeepAliveAgent = new HttpsAgent({
  maxSockets: 1500,
  maxFreeSockets: 256,
  timeout: 5 * 60000,
  freeSocketTimeout: 5000,
  keepAlive: true,
  keepAliveMsecs: 60000,
});

// Attach cacheable-lookup to the agents
cacheable.install(keepAliveAgent);
cacheable.install(httpsKeepAliveAgent);


const axiosInstanceGenerator = () => {
  if (!instance) {
    instance = axios.create({
      timeout: 30 * 60000,
      httpAgent: keepAliveAgent,
      httpsAgent: httpsKeepAliveAgent,
    });
  }

  return instance;
};

export const axiosInstance = axiosInstanceGenerator();

Read more here: Source link