How to connect local machine with Docker Container?

I'm trying to use kafka on my local computer (Windows 10) as producer and a spark streaming on a docker container (ubuntu) as a consumer. My examples work together in the container but i need to create the stream on my local computer.

I'm running zookeeper at port 2181 in the host (local computer) and kafka server at 9092.

And i start a broker at 9092 to create the stream (local computer)

producer = KafkaProducer(bootstrap_servers='localhost:9092') while True: message='Hola' producer.send(1,'test', message) time.sleep(1)

On the other side in the container i'm listen to port 2181 to recieve the stream.

ssc = StreamingContext(sc, 5)

kvs = KafkaUtils.createStream(ssc, "127.0.0.1:2181", "spark-streaming-consumer", {'test': 1})

The problem is that i don't know how to link both (Windows Kafka and Ubuntu)

I hope you can help me. Thanks.

apache-sparkdockerstreamingapache-kafka

Answers

answered 1 year ago shahin #1

while starting the docker container expose port 8080 to 8080 and 7077 to 7077 , which can be done by providing -p 8080:8080 -p 7070:7070 in docker run command.

Then In kafka you can configure using localhost or 127.0.0.1

comments powered by Disqus