ITPub博客
首页 博客 专栏 学院 社区 IT新媒体评选
发博

首页 > 数据库 > NewSQL > Centos7下Clickhouse安装+Cluster配置

Centos7下Clickhouse安装+Cluster配置

原创 NewSQL 作者：gaia 时间：2020-09-19 14:08:25 0 删除 编辑

  Clickhouse集群官方推荐使用Zookeeper去实现集群副本之间的同步。也可以使用应用程序去实现，往所有节点副本里面去写入数据来实现同步，不过官方不推荐此方法，不能确保集群所有副本的数据一致性。官方解释如下:


  To enable native replication ZooKeeper is required. ClickHouse will take care of data consistency on all replicas and run restore procedure after failure automatically. It's recommended to deploy ZooKeeper cluster to separate servers.ZooKeeper is not a strict requirement: in some simple cases you can duplicate the data by writing it into all the replicas from your application code. This approach is  not recommended, in this case ClickHouse won't be able to guarantee data consistency on all replicas. This remains the responsibility of your application.


 1.每台主机上安装clickhouse软件

 这里准备了三台主机:

 clickhouse01:192.168.137.20

 clickhouse01:192.168.137.21
 clickhouse01:192.168.137.22

 在三台主机上分别安装clickhouse


1.1.检查操作系统是否支持

[root@clickhouse01 ~]#  grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported"
SSE 4.2 supported


1.2.添加官方存储库


[root@clickhouse01 ~]# yum install yum-utils
[root@clickhouse01 ~]# rpm --import https://repo.yandex.ru/clickhouse/CLICKHOUSE-KEY.GPG
[root@clickhouse01 ~]# yum-config-manager --add-repo https://repo.yandex.ru/clickhouse/rpm/stable/x86_64
Loaded plugins: fastestmirror, langpacks
adding repo from: https://repo.yandex.ru/clickhouse/rpm/stable/x86_64
[repo.yandex.ru_clickhouse_rpm_stable_x86_64]
name=added from: https://repo.yandex.ru/clickhouse/rpm/stable/x86_64
baseurl=https://repo.yandex.ru/clickhouse/rpm/stable/x86_64
enabled=1


1.3.安装clickhouse server和client

[root@clickhouse01 ~]# yum install clickhouse-server clickhouse-client


1.4.启动和停止clickhouse-server


[root@clickhouse03 clickhouse-server]# service clickhouse-server start
Start clickhouse-server service: Path to data directory in /etc/clickhouse-server/config.xml: /var/lib/clickhouse/
DONE
[root@clickhouse03 clickhouse-server]# service clickhouse-server stop
Stop clickhouse-server service: DONE


 2.部署zookeeper

 需要在每台主机上安装并配置zookeeper


2.1.创建目录

[root@clickhouse01 ~]# mkdir  /apps
[root@clickhouse01 ~]# mv zookeeper-3.4.14.tar.gz  /apps/


2.2.安装zookeeper
[root@clickhouse02 apps]# tar -zxvf zookeeper-3.4.14.tar.gz
2.3.修改配置文件

[root@clickhouse01 apps]# cd zookeeper-3.4.14/
[root@clickhouse01 zookeeper-3.4.14]# ls -l
total 1676
drwxr-xr-x.  2 2002 2002     202 Mar  6  2019 bin
-rw-rw-r--.  1 2002 2002   97426 Mar  6  2019 build.xml
drwxr-xr-x.  2 2002 2002      77 Mar  6  2019 conf
drwxr-xr-x.  2 2002 2002    4096 Mar  6  2019 dist-maven
-rw-rw-r--.  1 2002 2002    1709 Mar  6  2019 ivysettings.xml
-rw-rw-r--.  1 2002 2002   10742 Mar  6  2019 ivy.xml
drwxr-xr-x.  4 2002 2002    4096 Mar  6  2019 lib
-rw-rw-r--.  1 2002 2002   11970 Mar  6  2019 LICENSE.txt
-rw-rw-r--.  1 2002 2002    3132 Mar  6  2019 NOTICE.txt
-rw-rw-r--.  1 2002 2002   31622 Mar  6  2019 pom.xml
-rw-rw-r--.  1 2002 2002    1765 Mar  6  2019 README.md
-rw-rw-r--.  1 2002 2002    1770 Mar  6  2019 README_packaging.txt
drwxr-xr-x.  3 2002 2002      22 Mar  6  2019 src
-rw-rw-r--.  1 2002 2002 1515359 Mar  6  2019 zookeeper-3.4.14.jar
-rw-rw-r--.  1 2002 2002     836 Mar  6  2019 zookeeper-3.4.14.jar.asc
-rw-rw-r--.  1 2002 2002      33 Mar  6  2019 zookeeper-3.4.14.jar.md5
-rw-rw-r--.  1 2002 2002      41 Mar  6  2019 zookeeper-3.4.14.jar.sha1
drwxr-xr-x.  3 2002 2002      47 Mar  6  2019 zookeeper-client
drwxr-xr-x. 12 2002 2002    4096 Mar  6  2019 zookeeper-contrib
drwxr-xr-x.  7 2002 2002    4096 Mar  6  2019 zookeeper-docs
drwxr-xr-x.  3 2002 2002      35 Mar  6  2019 zookeeper-it
drwxr-xr-x.  4 2002 2002      46 Mar  6  2019 zookeeper-jute
drwxr-xr-x.  5 2002 2002     176 Mar  6  2019 zookeeper-recipes
drwxr-xr-x.  3 2002 2002      32 Mar  6  2019 zookeeper-server
[root@clickhouse01 zookeeper-3.4.14]# cd conf/
[root@clickhouse01 conf]# ls -l
total 12
-rw-rw-r--. 1 2002 2002  535 Mar  6  2019 configuration.xsl
-rw-rw-r--. 1 2002 2002 2161 Mar  6  2019 log4j.properties
-rw-rw-r--. 1 2002 2002  922 Mar  6  2019 zoo_sample.cfg
[root@clickhouse01 conf]# cp zoo_sample.cfg zoo.cfg
[root@clickhouse01 conf]# vim zoo.cfg

--修改相关配置


# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/apps/zookeeper-3.4.14/data/zookeeper
dataLogDir= /apps/zookeeper-3.4.14/log/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
autopurge.purgeInterval=0
globalOutstandingLimit=200
server.1=clickhouse01:2888:3888
server.2=clickhouse02:2888:3888
server.3=clickhouse03:2888:3888
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

2.4.创建目录

[root@clickhouse01 conf]# mkdir -p /apps/zookeeper-3.4.14/data/zookeeper
[root@clickhouse01 conf]# mkdir -p /apps/zookeeper-3.4.14/log/zookeeper
2.5.设置myid
[root@clickhouse01 zookeeper-3.4.14]# vim ./data/zookeeper/myid
1
[root@clickhouse01 zookeeper-3.4.14]# vim ./data/zookeeper/myid
2
[root@clickhouse01 zookeeper-3.4.14]# vim ./data/zookeeper/myid
3


2.6.启动zookeeper

[root@clickhouse01 zookeeper-3.4.14]# cd bin/
[root@clickhouse01 bin]# ls -l
total 44
-rwxr-xr-x. 1 2002 2002  232 Mar  6  2019 README.txt
-rwxr-xr-x. 1 2002 2002 1937 Mar  6  2019 zkCleanup.sh
-rwxr-xr-x. 1 2002 2002 1056 Mar  6  2019 zkCli.cmd
-rwxr-xr-x. 1 2002 2002 1534 Mar  6  2019 zkCli.sh
-rwxr-xr-x. 1 2002 2002 1759 Mar  6  2019 zkEnv.cmd
-rwxr-xr-x. 1 2002 2002 2919 Mar  6  2019 zkEnv.sh
-rwxr-xr-x. 1 2002 2002 1089 Mar  6  2019 zkServer.cmd
-rwxr-xr-x. 1 2002 2002 6773 Mar  6  2019 zkServer.sh
-rwxr-xr-x. 1 2002 2002  996 Mar  6  2019 zkTxnLogToolkit.cmd
-rwxr-xr-x. 1 2002 2002 1385 Mar  6  2019 zkTxnLogToolkit.sh
[root@clickhouse01 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /apps/zookeeper-3.4.14/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED


2.7.测试连接

[root@clickhouse01 bin]# ./zkCli.sh -server clickhouse02:2181
Connecting to clickhouse02:2181
2020-09-17 02:53:23,554 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
2020-09-17 02:53:23,558 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=clickhouse01
2020-09-17 02:53:23,558 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_161
2020-09-17 02:53:23,566 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2020-09-17 02:53:23,566 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre
2020-09-17 02:53:23,567 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/apps/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/apps/zookeeper-3.4.14/bin/../build/classes:/apps/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/apps/zookeeper-3.4.14/bin/../build/lib/*.jar:/apps/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/apps/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/apps/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/apps/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/apps/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/apps/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/apps/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/apps/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/apps/zookeeper-3.4.14/bin/../conf:
2020-09-17 02:53:23,567 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2020-09-17 02:53:23,567 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2020-09-17 02:53:23,567 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2020-09-17 02:53:23,568 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2020-09-17 02:53:23,568 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2020-09-17 02:53:23,568 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-862.el7.x86_64
2020-09-17 02:53:23,568 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2020-09-17 02:53:23,568 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2020-09-17 02:53:23,569 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/apps/zookeeper-3.4.14/bin
2020-09-17 02:53:23,570 [myid:] - INFO  [main:ZooKeeper@442] - Initiating client connection, connectString=clickhouse02:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@4b85612c
2020-09-17 02:53:23,612 [myid:] - INFO  [main-SendThread(clickhouse02:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server clickhouse02/192.168.137.21:2181. Will not attempt to authenticate using SASL (unknown error)
Welcome to ZooKeeper!
JLine support is enabled
2020-09-17 02:53:23,860 [myid:] - INFO  [main-SendThread(clickhouse02:2181):ClientCnxn$SendThread@879] - Socket connection established to clickhouse02/192.168.137.21:2181, initiating session
2020-09-17 02:53:23,920 [myid:] - INFO  [main-SendThread(clickhouse02:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server clickhouse02/192.168.137.21:2181, sessionid = 0x20004b37d740000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: clickhouse02:2181(CONNECTED) 0]
[zk: clickhouse02:2181(CONNECTED) 0] exit
ZooKeeper -server host:port cmd args
    stat path [watch]
    set path data [version]
    ls path [watch]
    delquota [-n|-b] path
    ls2 path [watch]
    setAcl path acl
    setquota -n|-b val path
    history
    redo cmdno
    printwatches on|off
    delete path [version]
    sync path
    listquota path
    rmr path
    get path [watch]
    create [-s] [-e] path data acl
    addauth scheme auth
    quit
    getAcl path
    close
    connect host:port
[zk: clickhouse02:2181(CONNECTED) 1] quit
Quitting...
2020-09-17 02:53:48,004 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x20004b37d740000 closed
2020-09-17 02:53:48,008 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x20004b37d740000
[root@clickhouse01 bin]# ./zkCli.sh -server clickhouse03:2181
Connecting to clickhouse03:2181
2020-09-17 02:53:52,700 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT
2020-09-17 02:53:52,720 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=clickhouse01
2020-09-17 02:53:52,720 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_161
2020-09-17 02:53:52,724 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2020-09-17 02:53:52,724 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.161-2.b14.el7.x86_64/jre
2020-09-17 02:53:52,724 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/apps/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/apps/zookeeper-3.4.14/bin/../build/classes:/apps/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/apps/zookeeper-3.4.14/bin/../build/lib/*.jar:/apps/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/apps/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/apps/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/apps/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/apps/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/apps/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/apps/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/apps/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/apps/zookeeper-3.4.14/bin/../conf:
2020-09-17 02:53:52,724 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2020-09-17 02:53:52,725 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2020-09-17 02:53:52,725 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2020-09-17 02:53:52,725 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2020-09-17 02:53:52,725 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2020-09-17 02:53:52,729 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=3.10.0-862.el7.x86_64
2020-09-17 02:53:52,730 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2020-09-17 02:53:52,730 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2020-09-17 02:53:52,731 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/apps/zookeeper-3.4.14/bin
2020-09-17 02:53:52,733 [myid:] - INFO  [main:ZooKeeper@442] - Initiating client connection, connectString=clickhouse03:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@4b85612c
2020-09-17 02:53:52,775 [myid:] - INFO  [main-SendThread(clickhouse03:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server clickhouse03/192.168.137.22:2181. Will not attempt to authenticate using SASL (unknown error)
Welcome to ZooKeeper!
JLine support is enabled
2020-09-17 02:53:53,009 [myid:] - INFO  [main-SendThread(clickhouse03:2181):ClientCnxn$SendThread@879] - Socket connection established to clickhouse03/192.168.137.22:2181, initiating session
2020-09-17 02:53:53,028 [myid:] - INFO  [main-SendThread(clickhouse03:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server clickhouse03/192.168.137.22:2181, sessionid = 0x30000dc7cfd0000, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: clickhouse03:2181(CONNECTED) 0] quit
Quitting...
2020-09-17 02:53:55,354 [myid:] - INFO  [main:ZooKeeper@693] - Session: 0x30000dc7cfd0000 closed
2020-09-17 02:53:55,358 [myid:] - INFO  [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x30000dc7cfd0000


 3.Clickhouse Cluster配置

3.1.添加集群配置文件

每个节点都需要配置该文件，不同节点需要修改对应的 <macros>里面的主机名


vim /etc/metrika.xml

--添加如下内容:


<yandex>
    <clickhouse_remote_servers>
        <perftest_3shards_1replicas>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>clickhouse01</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <internal_replication>true</internal_replication>
                    <host>clickhouse02</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <internal_replication>true</internal_replication>
                <replica>
                    <host>clickhouse03</host>
                    <port>9000</port>
                </replica>
            </shard>
        </perftest_3shards_1replicas>
    </clickhouse_remote_servers>
    
    <zookeeper-servers>
        <node index="1">
            <host>clickhouse01</host>
            <port>2181</port>
        </node>
        <node index="2">
            <host>clickhouse02</host>
            <port>2181</port>
        </node>
        <node index="3">
            <host>clickhouse03</host>
            <port>2181</port>
        </node>
    </zookeeper-servers>
    <macros>
        <replica>clickhouse01</replica>
    </macros>
    <networks>
        <ip>::/0</ip>
    </networks>
    <clickhouse_compression>
        <case>
            <min_part_size>10000000000</min_part_size>
            <min_part_size_ratio>0.01</min_part_size_ratio>
            <method>lz4</method>
        </case>
    </clickhouse_compression>
</yandex>


3.2.每个节点启动clickhouse server

[root@clickhouse01 bin]# service clickhouse-server start
Start clickhouse-server service: Path to data directory in /etc/clickhouse-server/config.xml: /var/lib/clickhouse/
DONE


3.3.使用clickhouse-client登录每个节点,查看集群相关信息是否正常


名称为perftest_3shards_1replicas的cluster有三条记录，对应三台主机上的副本。


来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/69982136/viewspace-2722507/ ，如需转载，请注明出处，否则将追究法律责任。

0
0
上一篇： 使用Data Guard Menu工具管理监控dataguard
下一篇： oracle查询blocking session阻塞情况
请登录后发表评论 登录
全部评论
gaia
技術スタック: Linux / Oracle / MySQL / Docker / Kubernetes / Argo CD / GitLab CI/CD / Ansible / Terraform / Nginx / Git / Network

注册时间：2020-08-19

  • 博文量
    10
  • 访问量
    19060

最新文章

支持我们 作者招募 用户协议 FAQ Contact Us

北京盛拓优讯信息技术有限公司. 版权所有  京ICP备16024965号-8 北京市公安局海淀分局网监中心备案编号：11010802021510 niuxiaotong@pcpop.com 17352615567 未成年人举报专区

广播电视节目制作经营许可证(京) 字第1234号 中国互联网协会会员

违法和不良信息举报电话:18101376593