disconf分布式配置管理平台部署

发表于 | 分类于 ，

1. 前言

Disconf是由百度开源的一套分布式配置管理平台（Distributed Configuration Management Platform），专注于各种「分布式系统配置管理」的「通用组件」和「通用平台」, 提供统一的「配置管理服务」。
首先，实现了同构系统的配置发布统一化，提供了配置服务server，该服务可以对配置进行持久化管理并对外提供restful接口，在此基础上，基于zookeeper实现对配置更改的实时推送，并且，提供了稳定有效的容灾方案，以及用户体验良好的编程模型和WEB用户管理界面。其次，实现了异构系统的配置包管理，提出基于zookeeper的全局分布式一致性锁来实现主备统一部署、系统异常时的主备自主切换。

2. 配置前准备工作

部署规划

Disconf分为web端与client端，web端统一管理各个环境的配置，在此我们只需要部署web端即可。
Disconf的部署非常简单，它是java语言开发的程序，部署时只需将war包部署到相应位置即可运行，因此只需要在JDK+Tomcat环境中将disconf的war包部署即可。
为保证高可用，我们将disconf分别部署到三台服务器上，并在前端采用nginx实现动静分离+负载均衡。
disconf依赖tomcat环境部署路径：
/usr/local/disconf-tomcat/
disconf部署路径：
/home/disconf/war
disconf打包前配置文件存放路径：
/home/disconf/conf/
disconf日志文件存放路径：
/home/disconf/log/
disconf前端静态文件存放路径：
/home/disconf/war/html/
disconf-web监听端口：8085
disconf前端nginx监听端口：8888

准备安装程序

1
2
3
4
5
6
# ls /home/soft/disconf/
apache-maven-3.3.9-bin.tar.gz  apache-tomcat-8.0.39.tar.gz
# tar zxf apache-maven-3.3.9-bin.tar.gz
# tar zxf apache-tomcat-8.0.39.tar.gz
# ls
apache-maven-3.3.9  apache-maven-3.3.9-bin.tar.gz  apache-tomcat-8.0.39  apache-tomcat-8.0.39.tar.gz

创建相关安装目录

1
2
3
4
5
# mkdir -pv /home/disconf/{conf,war,log}
mkdir: created directory `/home/disconf'
mkdir: created directory `/home/disconf/conf'
mkdir: created directory `/home/disconf/war'
mkdir: created directory `/home/disconf/log'

3. 部署配置Disconf

3.1 安装配置JDK环境

disconf是采用JAVA语言开发的程序，所以需要先安装配置JDK环境，本次因为我们部署disconf的服务器已配置过JDK环境，故在此略过配置详细步骤。

3.2. 部署配置MAVEN

因为最新版本的disconf仅提供源码，因此需要我们将源码下载到服务器，然后通过maven来打包，所以我们要先部署maven环境，只需在其中一台服务器上部署maven即可。

1
2
3
4
5
6
7
8
9
10
11
12
13
# mv apache-maven-3.3.9 /usr/local/maven
# vim /etc/profile.d/maven.sh
export MAVEN_HOME=/usr/local/maven
export PATH=$MAVEN_HOME/bin:$PATH

# source /etc/profile.d/maven.sh 
# mvn -v
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-11T00:41:47+08:00)
Maven home: /usr/local/maven
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_111/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.32-642.4.2.el6.x86_64", arch: "amd64", family: "unix"

maven默认中央仓库为apache官方，在国内使用速度是相当之慢，因此，建议将中央仓库修改为阿里云仓库。

1
2
3
4
5
6
7
8
9
10
# vim /usr/local/maven/conf/settings.xml
添加如下配置：
  <mirrors>
    <mirror>
      <id>alimaven</id>
      <name>aliyun maven</name>
      <url>http://maven.aliyun.com/nexus/content/groups/public/</url>
      <mirrorOf>central</mirrorOf>        
    </mirror>
  </mirrors>

maven配置成功。

3.3 为Disconf准备MySQL数据库环境

在MySQL服务器上创建数据库disconf并设置相应的用户。

1
2
3
4
5
6
7
8
mysql> create database disconf;
Query OK, 1 row affected (0.00 sec)

mysql> grant all privileges on disconf.* to 'disconf'@'192.168.1.%' identified by 'bh7F3d0djPhkTcp9D';
Query OK, 0 rows affected, 1 warning (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

将disconf源码目录下的sql文件上传到MySQL服务器上，并按照readme.md文件中的说明，依此导入sql文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
# pwd
/home/soft/disconf/disconf-master/disconf-web/sql
# ls
0-init_table.sql  1-init_data.sql  201512  20160701  deprecated  readme.md
# more readme.md 
为了方便大家开发，统一了所有SQL，请先后执行：

- 0-init_table.sql        create db,tables
- 1-init_data.sql         create data
- 201512/20151225.sql     patch
- 20160701/20160701.sql   patch

mysql> source /home/soft/sql/0-init_table.sql
Query OK, 1 row affected, 1 warning (0.00 sec)

Database changed
Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

mysql> source /home/soft/sql/1-init_data.sql
Query OK, 1 row affected (0.00 sec)

Query OK, 16 rows affected (0.00 sec)
Records: 16  Duplicates: 0  Warnings: 0

Query OK, 4 rows affected (0.00 sec)
Records: 4  Duplicates: 0  Warnings: 0

Query OK, 3 rows affected (0.00 sec)
Records: 3  Duplicates: 0  Warnings: 0

Query OK, 72 rows affected (0.00 sec)
Records: 72  Duplicates: 0  Warnings: 0

Query OK, 8 rows affected (0.00 sec)
Records: 8  Duplicates: 0  Warnings: 0

mysql> source /home/soft/sql/201512/20151225.sql
Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

mysql> source /home/soft/sql/20160701/20160701.sql
Query OK, 0 rows affected (0.02 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 3 rows affected (0.00 sec)
Records: 3  Duplicates: 0  Warnings: 0

mysql> show tables;
+-------------------+
| Tables_in_disconf |
+-------------------+
| app               |
| config            |
| config_history    |
| env               |
| role              |
| role_resource     |
| user              |
+-------------------+
7 rows in set (0.00 sec)
出现上面7个表，就说明数据库初始化成功。

3.4 下载并打包disconf-web

1
2
3
4
5
6
7
# ls /home/soft/disconf/
disconf-master.zip
# unzip disconf-master.zip
# ls
disconf-master  disconf-master.zip
# pwd
/home/soft/disconf/disconf-master

将mvn编译打包需要的文件拷贝到/home/disconf/conf目录下。

1
2
3
4
5
6
7
8
# cp disconf-web/profile/rd/* /home/disconf/conf/
# cd /home/disconf/conf/
# ls
application-demo.properties  jdbc-mysql.properties  log4j.properties  logback.xml  redis-config.properties  zoo.properties
# mv application-demo.properties application.properties 
# ls
application.properties  jdbc-mysql.properties  log4j.properties  logback.xml  redis-config.properties  zoo.properties
切记，一定要将配置文件application-demo.properties修改为application.properties

修改配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
# vim jdbc-mysql.properties
jdbc.driverClassName=com.mysql.jdbc.Driver

jdbc.db_0.url=jdbc:mysql://192.168.1.110:3306/disconf?useUnicode=true&characterEncoding=utf8&zeroDateTimeBehavior=convertToNull&rewriteBatchedStatements=false
jdbc.db_0.username=disconf
jdbc.db_0.password=bh7F3d0djPhkTcp9D

jdbc.maxPoolSize=20
jdbc.minPoolSize=10
jdbc.initialPoolSize=10
jdbc.idleConnectionTestPeriod=1200
jdbc.maxIdleTime=3600

# vim log4j.properties
log4j.rootLogger=INFO,dailyRolling,CONSOLE

log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.springframework=INFO
log4j.logger.org.springframework.aop.framework.Cglib2AopProxy = INFO

log4j.appender.dailyRolling=org.apache.log4j.DailyRollingFileAppender
log4j.appender.dailyRolling.File=/home/disconf/log/disconf-log4j.log
log4j.appender.dailyRolling.layout=org.apache.log4j.PatternLayout
log4j.appender.dailyRolling.layout.ConversionPattern=%d [%t] %-5p %-17c{2} (%13F:%L) %3x - %m%n

log4j.appender.Threshold=WARN
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.Target=System.out
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d [%t] %-5p %-17c{2} (%13F:%L) %3x - %m%n

# vim redis-config.properties
redis.group1.retry.times=2

redis.group1.client1.name=BeidouRedis1
redis.group1.client1.host=192.168.1.5
redis.group1.client1.port=6379
redis.group1.client1.timeout=5000
redis.group1.client1.password=foobared

redis.group1.client2.name=BeidouRedis2
redis.group1.client2.host=192.168.1.6
redis.group1.client2.port=6379
redis.group1.client2.timeout=5000
redis.group1.client2.password=foobared

redis.group1.client3.name=BeidouRedis3
redis.group1.client3.host=192.168.1.7
redis.group1.client3.port=6379
redis.group1.client3.timeout=5000
redis.group1.client3.password=foobared

redis.evictor.delayCheckSeconds=300
redis.evictor.checkPeriodSeconds=30
redis.evictor.failedTimesToBeTickOut=6

# vim zoo.properties
hosts=192.168.1.104:2181,192.168.1.105:2181,192.168.1.106:2181

# zookeeper\u7684\u524D\u7F00\u8DEF\u5F84\u540D
zookeeper_url_prefix=/disconf

编译disconf源文件，生成war包

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# pwd
/home/soft/disconf/disconf-master
# mvn clean install
这一步如果报错，可以忽略。

设置环境变量
# vim /etc/profile.d/war.sh
export ONLINE_CONFIG_PATH=/home/disconf/conf
export WAR_ROOT_PATH=/home/disconf/war
# source /etc/profile.d/war.sh

执行编译脚本
# cd disconf-web/
# pwd
/home/soft/disconf/disconf-master/disconf-web
# sh deploy/deploy.sh

编译结束后，会在$WAR_ROOT_PATH位置下生成如下文件：
# ls /home/disconf/war/
application.properties  disconf-web.war  html  jdbc-mysql.properties  jpaas_control  log4j.properties  logback.xml  META-INF  redis-config.properties  Release  WEB-INF  zoo.properties

3.5 部署配置nginx+tomcat

nginx与tomcat的部署，在之前的文档有过详细说明，在此不赘述。

修改tomcat配置文件

1
2
3
4
5
6
7
8
9
# vim /usr/local/disconf-tomcat/conf/server.xml
    <Connector port="8085" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />

      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">

        <Context path="" docBase="/home/disconf/war"></Context>

修改nginx配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# vim /usr/local/openresty/nginx/conf/nginx.conf
    upstream disconf {
    server 192.168.1.104:8085 weight=10 max_fails=2 fail_timeout=30s;
    }

    server {
        listen       8888;
        server_name  localhost;

        #charset koi8-r;
        charset utf-8;

        #access_log  logs/host.access.log  main;
        access_log  logs/disconf.access.log  main;
        error_log   logs/disconf.error.log;

        #location / {
        #    root   html;
        #    index  index.html index.htm;
        #}

        location / {
            root /home/disconf/war/html;
            if ($query_string) {
                expires max;
            }
        }
        location ~ ^/(api|export) {
            proxy_pass_header Server;
            proxy_set_header Host $http_host;
            proxy_redirect off;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Scheme $scheme;
            proxy_pass http://disconf;
        }

分别启动nginx、tomcat服务。

3.6 访问Disconf Web页面测试

1
2
3
http://192.168.1.104:8888/

使用用户名密码admin/admin正常登陆，disconf配置完成。