apache kafka – Route message based on IP origin with SMT’s to a specific destination topic

admin

I have to produce messages based on virtual iP’s (Those ip’s are targetting the same Kafka cluster behind) .
So i need to extract the IP from the url (producer request) to route the mmessage to a specific topic before the message is persisted to Kafka .

**Example **

Static IP’s on host machine available :
192.168.0.2
192.168.0.3
192.168.0.4
192.168.0.5
Destination topics

  • Dest02 for IP 192.168.0.2
  • Dest03 for IP 192.168.0.3
  • Dest04 etc….
  • Dest05

So i Publish a record 001 to topicA (Virtual IP set to producer config > 192.168.0.2) in the service
=> record001 is routed to Dest02 destination topic

If you wonder why i want to route my message this way its because i cannot change the upstream service (producer) nor the downstream service neither (consumers) .

One more thing , i need to base this logic on the virtual IP as its used as my discriminant element to take a decision . otherwise i would not be able to know where to rouge my message

Thanks for your help

I am investigating on SMT’s with HTTP source connector to try to catch the message before its written in Kafka brokers but maybe its not a good approach .

Read more here: Source link