elasticsearch – Cannot send logs from Logstash to Elastic Search

I deployed ELK Stack to k8s by using helm. In the cluster, Elasticsearch, Kibana and Filebeat are running. I also configured Logstash to send Filebeat logs and logs from external resource.

My external resource is running in another server so I created logstash service as nodePort from 30123. Here is my values.yaml for logstash.

logstashConfig:
  logstash.yml: |
    http.host: 0.0.0.0  
  pipelines.yml: |
    # This file is where you define your pipelines. You can define multiple.
    # For more information on multiple pipelines, see the documentation:
    #   https://www.elastic.co/guide/en/logstash/current/multiple-pipelines.html
    - pipeline.id: logstash
      path.config: "/usr/share/logstash/pipeline/logstash.conf"
    - pipeline.id: devopsdashboard
      path.config: "/usr/share/logstash/pipeline/devopsdashboard.conf"
#  log4j2.properties: |
#    key = value

# Allows you to add any pipeline files in /usr/share/logstash/pipeline/
### ***warn*** there is a hardcoded logstash.conf in the image, override it first
logstashPipeline:
  logstash.conf: |
    input {
      beats {
        port => 5044
      }
    }
    filter {
    }
    output {
      elasticsearch {
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        hosts => [ "elasticsearch-master:9200" ]
      }
    }
  devopsdashboard.conf: |
    input {
       tcp {
         host => "0.0.0.0"
         port => 30123
         codec => "json_lines"
       }
    }
    filter {
       mutate {
          remove_field => ["host", "port"]
       }
    }
    output {
      elasticsearch {
        index => "logstash-%{+YYYY.MM.dd}"
        hosts => [ "elasticsearch-master:9200" ]
      }
    }

When I want to create index on Kibana, I can see filebeat index but cannot see logstash. If curl to elastic inside of logstash pod, my index is created but if I try to send logs by tcp via logstash, nothing happens. Do you have any ideas that how I can send logs from logtash to elastic?

I tried a python script to send logs by tcp from my local and I received “400 Bad Request”. I do not know what I am doing wrong. Here is my python script.

import socket
import json
import logging
from datetime import datetime
import sys

print("starting to send data to Elastic search")
# Create TCP/IP socket
print("Creating TCP/IP socket")
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
message = []
try:
    # Connect to port where server is running
    server_address = ('Cluster_IP', 30123)
    sock.connect(server_address)
    data = {'@test' : 'test1', '@message': 'python test message', '@tags': ['python', 'test']}
    sock.sendall(json.dumps(data).encode())
    print("Sent")
    print(sock.recv(1024)) 
except socket.error as e:
    sys.stderr.write(str(e))
finally:
    sock.close()

Thanks a lot!