2022-02-11使用freetsdb组建influxdb集群实战

目录

摘要:

环境准备:

freetsdb组建集群官方说明:

本地虚拟机部署freetsdb:

以192.168.58.132为例, 配置文件:

运行:

组建集群:

查看集群组建情况:

摘要:

记录使用freetsdb组建集群.

环境准备:  至少三台独立的虚拟机 编译安装freetsdb

freetsdb组建集群官方说明:

## 部署FreeTSDB原生支持分布式集群能力，FreeTSDB集群的部署主要涉及到meta节点和DATA节点，meta节点存放的是系统运行所必须的元数据，DATA节点存放的是实际的时序数据。本文档将以3 meta节点、2 DATA节点的集群为例演示如何搭建FreeTSDB集群。### 部署meta节点基于可用性和资源成本的考虑，meta节点推荐为3节点。#### 配置meta接地那##### Step 1: 修改/etc/hosts文件，配置主机名字信息。将各meta节点的名字（比如cluster-meta-node-01）和对应的地址信息（比如）配置在各meta节点和DATA节点的/etc/hosts文件中。``` cluster-meta-node-01 cluster-meta-node-02 cluster-meta-node-03```配置完主机名字信息后，在各meta节点和各DATA节点上分别执行如下ping命令，确保配置的正确性和网络的连通性。>> ping -qc 1 cluster-meta-node-01>> ping -qc 1 cluster-meta-node-02>> ping -qc 1 cluster-meta-node-03##### Step 2: 配置和启动meta服务在各meta节点上，分别执行如下命令。###### I、下载安装包比如，在Linux系统上，使用以下命令下载freetsdb-v0.0.2-beta.1的安装包。```wget https://github.com/freetsdb/freetsdb/releases/download/v0.0.2-beta.1/freetsdb-v0.0.2-beta.1_linux_amd64.tar.gztar -zxvf freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz```###### II、修改配置修改配置文件./freetsdb-meta.conf，并将hostname设置为本机的主机名字（比如）```# Hostname advertised by this host for remote addresses、 This must be resolvable by allother nodes in the clusterhostname=""```###### III、启动meta服务在Linux shell命令行中，执行如下命令。```sudo ./freetsd-meta -config ./freetsd-meta.conf```###### Step 3: 将meta节点加入到集群中在其中的一个meta节点上（比如cluster-meta-node-01），执行如下命令，将已配置好的meta节点加入到集群中。```freetsd-ctl add-meta cluster-meta-node-01:8091freetsd-ctl add-meta cluster-meta-node-02:8091freetsd-ctl add-meta cluster-meta-node-03:8091```正确执行时，add-meta命令的执行输出如下：```Added meta node x at cluster-meta-node-0x:8091```将所有meta节点加入到集群后，可以执行“freetsd-ctl show”来查看集群中meta节点的信息。>> ```> freetsd-ctl show> > ```>> The expected output is:>> ```> Data Nodes:>> meta Nodes:> 1 cluster-meta-node-01:8091> 2 cluster-meta-node-02:8091> 3 cluster-meta-node-03:8091>> ```### 部署DATA节点本文档将以2个DATA节点的集群部署为例，演示如何添加DATA节点。#### 配置DATA节点##### Step 1: 修改/etc/hosts文件，配置DATA节点的主机名字信息。将各DATA节点的名字（比如cluster-data-node-01）和对应的地址信息（比如）配置在各meta节点和DATA节点的/etc/hosts文件中。``` cluster-data-node-01 cluster-data-node-02```配置完主机名字信息后，在各meta节点和DATA节点上分别执行如下ping命令，确保配置的正确性和网络的连通性。>> ping -qc 1 cluster-meta-node-01>> ping -qc 1 cluster-meta-node-02>> ping -qc 1 cluster-meta-node-03>> ping -qc 1 cluster-data-node-01>> ping -qc 1 cluster-data-node-02##### Step 2: 配置和启动DATA服务在各DATA节点上，分别执行如下命令。###### I、下载安装包比如，在Linux系统上，使用以下命令下载freetsdb-v0.0.2-beta.1的安装包。```wget https://github.com/freetsdb/freetsdb/releases/download/v0.0.2-beta.1/freetsdb-v0.0.2-beta.1_linux_amd64.tar.gztar -zxvf freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz```###### II、Edit the data node configuration files修改配置文件./freetsdb.conf，并将hostname设置为本机的主机名字（比如）```# Change this option to true to disable reporting.reporting-disabled = falsebind-address = ":8088"hostname="" ```###### III、启动DATA服务在Linux shell命令行中，执行如下命令。```sudo ./freetsd -config ./freetsd.conf```#### 将DATA节点加入到集群中在其中的一个meta节点上（比如cluster-meta-node-01），执行如下命令，将已配置好的DATA节点加入到集群中。```freetsd-ctl add-data cluster-data-node-01:8088freetsd-ctl add-data cluster-data-node-02:8088```正确执行时，add-data命令的执行输出如下：```Added data node y at cluster-data-node-0x:8088```将所有DATA节点加入到集群后，可以执行“freetsd-ctl show”来查看集群中DATA节点的信息。>> ```> freetsd-ctl show> ```>> The expected output is:>> ```> Data Nodes:> 4 cluster-data-node-01:8088> 5 cluster-data-node-02:8088>> meta Nodes:> 1 cluster-meta-node-01:8091> 2 cluster-meta-node-02:8091> 3 cluster-meta-node-03:8091> ```恭喜你，FreeTSDB集群已搭建完成，欢迎进入FreeTSDB的精彩世界。```# 关闭防火墙systemctl stop firewalld.servicesystemctl disable firewalld.service``````# 添加新的机器./freetsd-ctl add-meta 192.168.58.132:8091./freetsd-ctl add-data 192.168.58.131:8088```

本地虚拟机部署freetsdb:

至少准备三台虚拟机, 为了增加对比, 使用四台虚拟机, ip地址分别为:

192.168.58.128192.168.58.131192.168.58.132192.168.58.133

以192.168.58.132为例, 配置文件:

完整的58.132上的配置的文件夹: freetsdb-v0.0.2-58.132-其它文档类资源-CSDN下载

meta配置:

freetsd-meta.conf

bind-address = ":8089"hostname = "192.168.58.132"[meta] enabled = true dir = "/var/lib/freetsdb/meta" bind-address = ":8089" http-bind-address = ":8091" https-enabled = false https-certificate = "" retention-autocreate = true election-timeout = "1s" heartbeat-timeout = "1s" leader-lease-timeout = "500ms" commit-timeout = "50ms" cluster-tracing = false raft-promotion-enabled = true logging-enabled = true pprof-enabled = false lease-duration = "1m0s"

data配置:

freetsd.conf

reporting-disabled = falsebind-address = ":8088"hostname = "192.168.58.132"[data] enabled = true dir = "/var/lib/freetsdb/data" engine = "tsm1" wal-dir = "/var/lib/freetsdb/wal" wal-logging-enabled = true query-log-enabled = true cache-max-memory-size = 1073741824 cache-snapshot-memory-size = 26214400 cache-snapshot-write-cold-duration = "10m0s" compact-full-write-cold-duration = "24h0m0s" max-points-per-block = 0 data-logging-enabled = true[cluster] force-remote-mapping = false write-timeout = "5s" shard-writer-timeout = "5s" max-remote-write-connections = 3 shard-mapper-timeout = "5s"[retention] enabled = true check-interval = "30m0s"[shard-precreation] enabled = true check-interval = "10m0s" advance-period = "30m0s"[admin] enabled = true bind-address = ":8083" https-enabled = false https-certificate = "/etc/ssl/freetsdb.pem" Version = ""[monitor] store-enabled = true store-database = "_internal" store-interval = "10s"[subscriber] enabled = true[http] enabled = true bind-address = ":8086" auth-enabled = false log-enabled = true write-tracing = false pprof-enabled = false https-enabled = false https-certificate = "/etc/ssl/freetsdb.pem" json-write-enabled = false[[graphite]] enabled = false bind-address = ":2003" database = "graphite" protocol = "tcp" batch-size = 5000 batch-pending = 10 batch-timeout = "1s" consistency-level = "one" separator = "." udp-read-buffer = 0[collectd] enabled = false bind-address = ":25826" database = "collectd" retention-policy = "" batch-size = 5000 batch-pending = 10 batch-timeout = "10s" read-buffer = 0 typesdb = "/usr/share/collectd/types.db"[opentsdb] enabled = false bind-address = ":4242" database = "opentsdb" retention-policy = "" consistency-level = "one" tls-enabled = false certificate = "/etc/ssl/freetsdb.pem" batch-size = 1000 batch-pending = 5 batch-timeout = "1s" log-point-errors = true[[udp]] enabled = false bind-address = ":8089" database = "udp" retention-policy = "" batch-size = 5000 batch-pending = 10 read-buffer = 0 batch-timeout = "1s" precision = "" udp-payload-size = 0[continuous_queries] log-enabled = true enabled = true run-interval = "1s"[hinted-handoff] enabled = true dir = "/var/lib/freetsdb/hh" max-size = 1073741824 max-age = "168h0m0s" retry-rate-limit = 0 retry-interval = "1s" retry-max-interval = "1m0s" purge-interval = "1h0m0s"

其他虚拟机需要修改对应的hostname为本机的ip.

运行:

启动meta

nohup ./freetsd-meta -config ./freetsd-meta.conf > freetsd-meta.log &

启动data:

nohup ./freetsd -config ./freetsd.conf > freetsd.log &

组建集群:

添加meta:

在任意部署meta的机器上, 执行:

./freetsd-ctl add-meta 192.168.58.128:8091./freetsd-ctl add-meta 192.168.58.131:8091./freetsd-ctl add-meta 192.168.58.132:8091./freetsd-ctl add-meta 192.168.58.133:8091

./freetsd-ctl add-data 192.168.58.128:8088./freetsd-ctl add-data 192.168.58.131:8088./freetsd-ctl add-data 192.168.58.132:8088./freetsd-ctl add-data 192.168.58.133:8088

查看集群组建情况:

./freetsd-ctl showmeta Nodes: {1 192.168.58.128:8091 192.168.58.128:8089} {5 192.168.58.132:8091 192.168.58.132:8089} {6 192.168.58.133:8091 192.168.58.133:8089} {8 192.168.58.131:8091 192.168.58.131:8089}Data Nodes: {2 192.168.58.128:8086 192.168.58.128:8088} {7 192.168.58.133:8086 192.168.58.133:8088} {10 192.168.58.132:8086 192.168.58.132:8088} {11 192.168.58.131:8086 192.168.58.131:8088}