今天我在cetos上安裝好了開啓了kafka,網上找了幾個例子,想用c++寫一個生產者和消費者模型的例子.
然後踩了幾個坑,公佈出來,希望大家以後不要再踩
我是用它自帶的sh工作作爲生產者,配置好了主題和partition等信息,如下所示
[root@localhost bin]# ./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test11959
Created topic "test11959".
[root@localhost bin]# ./kafka-console-producer.sh --broker-list ${IP ADDRESS}:9092 --topic test11959
> 此時等待你的輸入
然後我啓動了我的消費者程序(用的是librdkafka這個庫),但是出現了兩個問題
首先出現的現象是這樣的:
(這一條是網上的:Connecting to IPv6 addresses: Connect to ipv6#[::1]:9092 failed: Connection refused)
%3|1567166247.579|FAIL|rdkafka#consumer-1| [thrd:${IP address}:9092/bootstrap]: ${IP address}:9092/0: Connect to ipv4#${IP address}:9092 failed: Connection refused (after 1ms in state CONNECT)
%3|1567166247.579|ERROR|rdkafka#consumer-1| [thrd:${IP address}:9092/bootstrap]: ${IP address}:9092/0: Connect to ipv4#${IP address}:9092 failed: Connection refused (after 1ms in state CONNECT)
解決方法是 :
cd到kafka的安裝路徑,然後到config目錄下.在文件server.properties 將下面這一項給打開,他的說明中解釋的很清楚了
# Hostname and port the broker will advertise to producers and consumers. If not set,
# it uses the value for "listeners" if configured. Otherwise, it will use the value
# returned from java.net.InetAddress.getCanonicalHostName().
advertised.listeners=PLAINTEXT://172.16.100.121:9092
然後出現的問題是:
%3|1567165624.868|ERROR|rdkafka#consumer-1| [thrd:app]: rdkafka#consumer-1: testTopic [1]: desired partition does not exist in cluster (Local: Unknown partition)
最終我找到了這個鏈接:https://github.com/edenhill/librdkafka/issues/327 他裏面有一層樓是這樣說的
If you create a topic with 4 partitions then those partitions are numbered 0..3,
you seem to try to consume partition 4 which is indeed an Unknown partition.
看懂了嗎?假如partitions 是4 則可用的partition是從0 開始計數的:0,1,2,3 所以,我的生產者的設置的partition=1 ,我縮寫的客戶端用的也是1 就導致客戶端找不到partition了
希望大家少走彎路!!!