Cannot connect to Kafka from inside Docker

JimF Source

I have a Kafka server running and I have a program which consumes messages. The programs run on different physical servers. The Kafka consumer program runs any a physical server just fine. But, from inside of docker it produces

2018/03/13 17:45:40 Recv loop terminated: err=EOF
2018/03/13 17:45:40 Send loop terminated: err=<nil>
2018/03/13 17:45:40 Failed to join consumer groupgroup.testingkafka: client has run out of available brokers to talk to (Is your cluster reachable?)

I execute the program with:

[[email protected] c2s]# ./c2s.sh
oracleuser=c2s
oraclePassword=someOtherPassword
oracleHost=x.x.x.51
oraclePort=1521
oracleServiice=orcl.protoeffect.cxm
kafkaConsumerAddr=x.x.x.36:2181
zookeeperAddr=x.x.x.36:9092
topicName=c2s
2018/03/13 13:31:33 Starting loadIFL
2018/03/13 13:31:33 Exiting loadIFL
2018/03/13 13:31:33 Start consume.
2018/03/13 13:31:33 Starting Consumer
2018/03/13 13:31:33 Connected to x.x.x.36:2181
2018/03/13 13:31:33 Authenticated: id=99676750877491229, timeout=6000
2018/03/13 13:31:33 Re-submitting `0` credentials after reconnect
2018/03/13 13:31:33 Waiting for messages

That is exactly what I'd expect. And, it processes messages. I build the docker image with:

FROM centos:latest
COPY c2s /c2sp
COPY c2s.sh /
COPY ./instantclient/oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm /
COPY ./instantclient/oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm /
COPY ./instantclient/oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm /
RUN /usr/bin/mkdir /config
RUN /usr/bin/mkdir /c2s
RUN chmod 777 /c2s.sh /config /data
RUN yum install libaio -y
RUN rpm -i /oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
RUN rpm -i /oracle-instantclient12.2-devel-12.2.0.1.0-1.x86_64.rpm
RUN rpm -i /oracle-instantclient12.2-sqlplus-12.2.0.1.0-1.x86_64.rpm
ENTRYPOINT /c2s.sh

And, I run the program with:

docker run -it --net=bridge -v /go/src/c2s:/config -v /c2s:/data c2s:0.0.1

Docker produces the following:

oracleuser=c2s
oraclePassword=someOtherPassword
oracleHost=x.x.x.51
oraclePort=1521
oracleServiice=orcl.protoeffect.cxm
kafkaConsumerAddr=x.x.x.36:2181
zookeeperAddr=x.x.x.36:9092
topicName=c2s
2018/03/13 17:45:39 Starting loadIFL
2018/03/13 17:45:39 Exiting loadIFL
2018/03/13 17:45:39 Start consume.
2018/03/13 17:45:39 Starting Consumer
2018/03/13 17:45:39 Connected to x.x.x.36:2181
2018/03/13 17:45:39 Authenticated: id=99676750877491230, timeout=6000
2018/03/13 17:45:39 Re-submitting `0` credentials after reconnect
2018/03/13 17:45:40 Recv loop terminated: err=EOF
2018/03/13 17:45:40 Send loop terminated: err=<nil>
2018/03/13 17:45:40 Failed to join consumer groupgroup.testingkafka: client has run out of available brokers to talk to (Is your cluster reachable?)

But, I know the cluster is reachable. I connect to it from several servers including my docker server. When I log into the running docker image, I can ping the kafka server so I know the network connection is available. It's like the return port is being blocked.

dockernetworkingapache-kafka

Answers

comments powered by Disqus