Clickhouse在Docker中的編譯、安裝和集羣部署

Clickhouse是俄羅斯Yandex公司開源的OLAP系統，單表性能最好，最近兩年發展很快。百度開源的Doris是另外一個優秀的OLAP系統，我上次記錄了百度開源OLAP系統Apache Doris在Centos7.0下編譯和安裝，這兩個系統因爲源碼編譯和安裝比較繁瑣，估計很多人會卡在這一步，不利於這些系統的推廣和使用。

源碼編譯安裝

請先閱讀官方文檔，但是不詳細
https://clickhouse.tech/docs/zh/getting_started/install/
https://clickhouse.tech/docs/zh/development/build/

Docker鏡像

CK在Ubuntu下安裝更加友好，可以使用官方Ubuntu的鏡像Docker鏡像地址

$ docker pull yandex/clickhouse-server

下載完畢後，啓動鏡像：

docker run --name map-clickhouse-server -it -v $HOME/ck_database:/var/lib/clickhouse -v $HOME/code/ClickHouse:/ClickHouse yandex/clickhouse-server /bin/bash
下次ps一下，找到CONTAINER ID，docker attach就進去了
docker ps
docker attach CONTAINER ID

更改源

因爲官方docker鏡像裏面各種軟件都沒有安裝，所以，先更改一下源，比較快，可以使用阿里雲的源，命令行生成sources.list文件（因爲鏡像中沒有VIM）

sudo mv /etc/apt/sources.list /etc/apt/sources.list.bak
sudo echo "deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse" > /etc/apt/sources.list
sudo echo "deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse"  >> /etc/apt/sources.list 
sudo echo "deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse"  >> /etc/apt/sources.list
sudo echo "deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse"  >> /etc/apt/sources.list

安裝依賴

sudo apt-get update
sudo apt-get install git
sudo apt-get install pbuilder
sudo apt-get install debhelper
sudo apt-get install lsb-release
sudo apt-get install fakeroot
sudo apt-get install debian-archive-keyring
sudo apt-get install debian-keyring

安裝編譯工具

安裝cmake等編譯工具

sudo apt-get install git cmake ninja-build

安裝PPA包，這裏主要是爲了安裝G++9

apt-get install software-properties-common
sudo apt-get install software-properties-ommon
sudo apt-add-repository ppa:ubuntu-toolchain-r/test
如果添加PPA包失敗，執行如下命令
sudo apt-get install --reinstall ca-certificates
sudo -E add-apt-repository --update ppa:ubuntu-toolchain-r/test
//update一下，然後安裝gcc9和g++9
sudo apt-get update
sudo apt-get install gcc-9 g++-9

設置CC和CXX的GCC版本

vim ~/.bashrc
export CC=gcc-9
export CXX=g++-9
//souce一下生效
source ~/.bashrc

下載源碼

git clone --recursive [email protected]:ClickHouse/ClickHouse.git

或者分開下載

git clone https://github.com/ClickHouse/ClickHouse.git
// 下面命令下載第三方包，如果下載失敗，請刪除contrib目錄下對應的目錄，重新執行
git submodule update --init --recursive

如果網速慢，可以下載zip包

wget https://github.com/ClickHouse/ClickHouse/archive/master.zip ./

源碼編譯

git clone --recursive https://github.com/ClickHouse/ClickHouse.git
cd clickhouse
git submodule update --init --recursive
mkdir build
cd build
cmake ../
ninja

編譯完畢，在build/programs目錄下，會生成可執行文件

[map@server build]$ ls programs/ -l
total 5219304
-rwxr-xr-x 1 root root 3387833288 Apr 22 17:38 clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-benchmark -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-client -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:39 clickhouse-compressor -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-copier -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-extract-from-config -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-format -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-local -> clickhouse
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-obfuscator -> clickhouse
-rwxr-xr-x 1 root root 1956718008 Apr 22 17:36 clickhouse-odbc-bridge
lrwxrwxrwx 1 root root         10 Apr 22 18:38 clickhouse-server -> clickhouse

配置文件在clickhouse源碼目錄的lprograms/server/下

[map@server ClickHouse]$ ls programs/server/*.xml -l
-rw-rw-r-- 1 lihaibo lihaibo 22942 Apr 16 14:23 programs/server/config.xml
-rw-rw-r-- 1 lihaibo lihaibo  5328 Apr 13 18:08 programs/server/users.xml

遇到的問題

如果cmake過程中提示缺少第三方包，請刪除contrib下相關目錄重新下載

git submodule update --init --recursive

無法下載Fuzzer

路徑 /ClickHouse/contrib/grpc/third_party/bloaty/third_party/libFuzzer
想辦法下載下來：https://chromium.googlesource.com/chromium/llvm-project/llvm/lib/Fuzzer/
cp ../ck_patch/Fuzzer/ -r ./contrib/grpc/third_party/bloaty/third_party/libFuzzer

找不到ZLIB
修改contrib/CMakeLists.txt，把CMAKE_DISABLE_FIND_PACKAGE_ZLIB標誌位設置爲0，讓他從標準路徑上查找

$ sudo apt-get install zlib1g
$ sudo apt-get install zlib1g.dev

找不到Z3

Could NOT find Z3: Found unsuitable version "0.0.0", but required is at least "4.7.1" (found Z3_LIBRARIES-NOTFOUND)

下載鏈接：https://github.com/Z3Prover/z3/releases

mkdir build
cd build
cmake -G "Unix Makefiles" ../
make -j4 # Replace 4 with an appropriate number
apt-get install python3-distutils
make install
/etc/ld.so.conf

沒有atomic

Host compiler must support std::atomic!
vim /etc/ld.so.conf
/usr/lib/gcc/x86_64-linux-gnu/9/
vim /etc/profile
export C_INCLUDE_PATH=/usr/lib/gcc/x86_64-linux-gnu/9/include/:$C_INCLUDE_PATH
export CPLUS_INCLUDE_PATH=/usr/lib/gcc/x86_64-linux-gnu/9/include/:$CPLUS_INCLUDE_PATH
如果提示缺少atomic.h文件，找到拷貝到/usr/include/中
清理一下build目錄，重新cmake ../

按包安裝

下載相關安裝包，包都打好了，只需要自己解壓執行安裝腳本即可。
自己編譯，需要把包、配置文件等自己整理好

可以參考這篇文章clickhouse + chproxy 集羣搭建

wget https://repo.yandex.ru/clickhouse/tgz/clickhouse-common-static-20.2.1.2183.tgz
wget https://repo.yandex.ru/clickhouse/tgz/clickhouse-server-20.2.1.2183.tgz
wget https://repo.yandex.ru/clickhouse/tgz/clickhouse-client-20.2.1.2183.tgz
wget https://repo.yandex.ru/clickhouse/tgz/clickhouse-test-20.2.1.2183.tgz

下載後解壓，install目錄的doinst.sh安裝，三個包都安裝一遍

cd clickhouse-server-20.2.1.2183/install/
./doinst.sh

單機啓動

日誌文件 /var/log/clickhouse-server/
配置文件/etc/clickhouse-server/config.xml
配置文件/etc/clickhouse-server/user.xml
狀態文件/var/lib/clickhouse/status

服務端和客戶端啓動

啓動服務器
寫個小腳本 sudo vim start_ck_server.sh

sudo -u clickhouse clickhouse-server --daemon --config-file=/etc/clickhouse-server/config.xml
sleep 3
sudo cat /var/lib/clickhouse/status

客戶登陸

sudo clickhouse-client

啓動時的權限問題

Clickhouse用clickhouse的用戶啓動，需要設置權限，可以寫一個腳本，如果啓動失敗，可以看一下是否clickhouse用戶沒有目錄或文件權限

sudo vim chown_clickhouse_dir.sh

sudo chown -R clickhouse /etc/clickhouse-server/
sudo chown -R clickhouse /var/log/clickhouse-server/
sudo chown -R clickhouse /var/lib/clickhouse/
sudo chown -R clickhouse /usr/bin/clickhouse
sudo chown -R clickhouse /data/clickhouse
sudo chown -R clickhouse /bin/clickhouse-server
sudo chown -R clickhouse /etc/metrika.xml

啓動時的ulimit -c unlimited的問題

Poco::Exception. Code: 1000, e.code() = 0, e.displayText() = File not found: /etc/clickhouse-server/config.xml0. 0xbc7786c Poco::FileNotFoundException::FileNotFoundException(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int)  in /usr/bin/clickhouse

//設置一下
ulimit -c unlimited

集羣搭建

需要把安裝包拷貝到每個服務器上，這次測試

1個ZK節點
3個CK節點

Zookeeper集羣配置

因爲Clickhouse沒有節點一致性管理能力，所以必須安裝zk來管理，zk的作用有三個

節點管理
元數據管理
副本管理

參考資料
ClickHouse - 多卷存儲擴大存儲容量（生產環境必備）
clickhouse + chproxy 集羣搭建
 clickhouse 部署+介紹

下載ZK

ZK的配置較爲簡單，可以部署一個ZK集羣，這裏因爲測試，所以ZK就部署了一個節點。
下載地址https://zookeeper.apache.org/releases.html
在ZK的啓動腳本中設置一下JAVA_HOME，不設置啓動時會出錯，也可以在~/.bashrc腳本中設置

export JAVA_HOME=sudo ./zkServer.sh start

ZK的配置文件zoo_sample.cfg拷貝一個修改一下

cat ../zookeeper/apache-zookeeper-3.6.0-bin/conf/zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2989
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

## Metrics Providers
#
# https://prometheus.io Metrics Exporter
#metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
#metricsProvider.httpPort=7000
#metricsProvider.exportJvmInfo=true

每個節點的Clickhouse的配置，配置後重啓一下生效

sudo vim /etc/metrika.xml

<yandex>
<clickhouse_remote_servers>
    <perftest_3shards_1replicas>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>xx.xx.42.37</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>xx.xx.42.38</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <internal_replication>true</internal_replication>
            <replica>
                <host>xx.xx.42.40</host>
                <port>9000</port>
            </replica>
        </shard>
    </perftest_3shards_1replicas>
</clickhouse_remote_servers>
<!--下面是ZK的IP和端口-->
<zookeeper-servers>
  <node index="1">
    <host>xx.xx.42.37</host>
    <port>2989</port>
  </node>
</zookeeper-servers>

<!--
<macros>
    <replica>10.198.42.37</replica>
</macros>
-->

<networks>
   <ip>::/0</ip>
</networks>

<clickhouse_compression>
<case>
  <min_part_size>10000000000</min_part_size>             
  <min_part_size_ratio>0.01</min_part_size_ratio>
  <method>lz4</method>
</case>
</clickhouse_compression>
</yandex>

查看集羣情況

sudo clickhouse-client登陸

XXX : ) select * from system.clusters;
SELECT *
FROM system.clusters

┌─cluster───────────────────────────┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user────┬─default_database─┬─errors_count─┬─estimated_recovery_time─┐
│ perftest_3shards_1replicas        │         1 │            1 │           1 │ xx.xx.42.37 │ xx.xx.42.37 │ 9000 │        1 │ default │                  │            0 │                       0 │
│ perftest_3shards_1replicas        │         2 │            1 │           1 │ xx.xx.42.38 │ xx.xx.42.38 │ 9000 │        0 │ default │                  │            0 │                       0 │
│ perftest_3shards_1replicas        │         3 │            1 │           1 │ xx.xx.42.39 │ xx.xx.42.39 │ 9000 │        0 │ default │                  │            0 │                       0 │
└───────────────────────────────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴─────────┴──────────────────┴──────────────┴─────────────────────────┘

然後，就可以愉快的玩耍了：）

Clickhouse在Docker中的編譯、安裝和集羣部署

源碼編譯安裝

Docker鏡像

更改源

安裝依賴

安裝編譯工具

下載源碼

源碼編譯

遇到的問題

按包安裝

單機啓動

單機啓動

服務端和客戶端啓動

啓動時的權限問題

啓動時的ulimit -c unlimited的問題

集羣搭建

Zookeeper集羣配置

下載ZK

查看集羣情況

[轉帖]使用NMT和pmap解決JVM資源泄漏問題原創

Python實現大麥網搶票的四大關鍵技術點解析

Python 安裝庫指令大全

salesforce零基礎學習（一百三十八）零碎知識點小總結（十）

一款開源的.NET程序集反編譯、編輯和調試神器

關於接口協議，你必須要知道這些！

2020年上半年數據庫系統工程師考試

基於 Milvus + LlamaIndex 實現高級 RAG

【2024-05-21】以茶會友

VMware 7.1 手動模式安裝Centos 5.3

g++undefined reference可能是鏈接庫的順序不對

研發管理的歷史文章目錄

百度開源OLAP系統Apache Doris在Centos7.0下編譯和安裝

Google Python 風格指南

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結