- 1.前期回顾
- 第一章：配置中心的概述
- 第二章：实战K8S配置中心-ConfigMap
- 1.使用ConfigMap管理应用配置
- 1.拆分环境
- 2.重配zookeeper
- 3.准备资源配置清单(dubbo-monitor)
- 4.应用资源配置清单
- 5.重新发版，修改dubbo项目的配置文件
- 6.configmap 复杂配置文件处理方法
- 第三章：apollo配置中心介绍
- 第四章：实战交付apollo配置中心组件–configservice到kubernetes集群
- 1.准备软件包
- 2.安装数据库
- 3.执行数据库脚本
- 4.数据库用户授权
- 5.修改初始数据
- 6.制作docker镜像
- 7.解析域名
- 8.准备资源配置清单
- 9.应用资源配置清单
- 10.浏览器访问
- 11.分析mysql连接ip
- 第五章：实战交付apollo配置中心组件–adminservice到kubernetes集群
- 1.准备软件包
- 2.制作Docker镜像
- 3.准备资源配置清单
- 4.应用资源配置清单
- 5.浏览器访问
- 6.集群维护
- 第六章：实战交付apollo配置中心组件–portal到kubernetes集群
- 1.准备软件包
- 2.执行数据库脚本
- 3.数据库用户授权
- 4.制作docker镜像
- 5.解析域名
- 6.准备资源配置清单
- 7.应用资源配置清单
- 8.浏览器访问
- 第七章：实战配置dubbo微服务接入apollo配置中心管理
- 1.改造dubbo-demo-service项目说明
- 1.创建项目
- 2.进入配置页面
- 3.发布配置
- 4.使用jenkins进行CI
- 5.上线新构建的项目
- 2.改造dubbo-demo-web项目
- 1.创建项目
- 2.进入配置页面
- 3.发布配置
- 4.使用jenkins进行CI
- 5.上线新构建的项目
- 6.浏览器访问
- 第八章：实战使用apollo配置中心管理测试环境和生产环境
- 1.实战维护多套dubbo微服务环境
- 1.生产实践
- 2.系统架构
- 3.修改/添加域名解析
- 4.Apollo的K8S应用配置
- 5.启动Apollo，并查看portal配置
- 1.系统参数
- 2.实战发布dubbo连接apollo配置中心到不同环境
- 1.apollo配置中心创建dubbo-demo-service服务项目
- 2.apollo配置中心创建dubbo-demo-web服务项目
- 6.发布dubbo微服务到k8s集群
- 1.测试环境-dubbo-demo-service资源配置清单
- 2.测试环境-dubbo-demo-web资源配置清单
- 3.测试环境-应用资源配置清单
- 4.测试环境-浏览器访问
- 5.正式环境-dubbo-demo-service资源配置清单
- 6.正式环境-应用资源配置清单
- 7.正式环境-dubbo-demo-web资源配置清单
- 8.正式环境-应用资源配置清单
- 9.正式环境-浏览器访问
- 7.知识点
- 8.实战演示项目提测-发版流程
- 2.互联网公司技术部的日常
1.前期回顾
- Dubbo微服务
- 注册中心zookeeper(集群)
- 提供者(集群)
- 消费者(集群)
- 监控(dubbo-monitor/dubbo-admin)
- 在K8S内交付dubbo微服务的步骤：
- step0:有可用的集群
- step1:部署zk集群(通常放在K8S集群外，有状态)
- step2:部署jenkins(以容器的形式交付在K8S集群内)
- root、时区、ssh-key、docker客户端、harbor连接配置
- step3:部署maven软件
- step4:制作dubbo微服务底包
- step5:配置jenkins持续构建(CI)流水线
- step6:使用流水线构建项目，查看harbor仓库
- step7:使用资源配置清单，交付项目到K8S集群
- 交付dubbo-monitor
- 运维八荣八耻
- 以可配置为荣，以硬编码为耻
- 以互备为荣，以单点为耻
- 以随时重启为荣，以不能迁移为耻
- 以整体交付为荣，以部分交付为耻
- 以无状态为荣，以有状态为耻
- 以标准化为荣，以特殊化为耻
- 以自动化为荣，以手工+人肉为耻
- 以无人值守为荣，以人工介入为耻
- 考虑我们交付近K8S集群的两个dubbo微服务和一个monitor，最大的问题是什么？
- 他们的配置写死在容器里了！
第一章：配置中心的概述
- 配置其实是独立于程序的可配置变量，同一份程序在不同配置下会有不同的行为，常见的配置有连接字符串，应用配置和业务配置等。
- 配置有多种形态，下面是一些常见的：
- 程序内部hardcode，这种做法是反模式，一般我们不建议!
- 配置文件，比如spring应用程序的配置一般放在application.properties文件中。
- 环境变量，配置可以预配置在操作系统的环境变量里头，程序运行时读取。
- 启动参数，可以在程序启动时一次性提供参数，例如java程序启动时可以通过java -D 方式配置启动参数。
- 基于数据库(配置中心)，有经验的开发人员把易变配置放在数据库中，这样可以在运行期间灵活调整配置。
- 配置管理的现状
- 配置散乱。格式不标准(xml、ini、conf、yaml…..)
- 主要采用本地静态配置，应用多副本集下配置修改麻烦
- 易引发生产事故(测试环境、生产环境配置混用)
- 配置缺乏安全审计和版本控制功能
- 不同环境的应用，配置不同，造成多次打包，测试失效
- 配置中心是什么？
- 顾名思义就是集中管理应用程序配置的”中心”。
- 常见的配置中心有：
- XDiamond：全局配置中心，存储应用的配置项，解决配置混乱分散的问题。名字来源于淘宝的开源项目diamond，前面加上一个字母X以示区别。
- Qconf: QConf是一个分布式配置管理工具。用来替代传统的配置文件，使得配置信息和程序分离，同时配置变化能够实时同步到客户端，而且保证用户高效读取配置，这使得工程师从琐碎的配置修改、代码提交、配置上线流程中解放出来，极大地简化了配置管理工作。
- Disconf：专注于各种分布式系统配置管理的通用组件和通用平台，提供统一的配置管理服务。
- SpringCloudConfig：Spring Cloud Config为分布式系统中的外部配置提供服务器和客户端支持。
- K8S ConfigMap：K8S的一种标准资源，专门用来集中管理应用的配置
- Apollo：携程框架部门开源的，分布式配置中心
第二章：实战K8S配置中心-ConfigMap
1.使用ConfigMap管理应用配置
1.拆分环境
| 主机名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk1.od.com(Test环境) | 10.4.7.11 |
| HDSS7-12.host.com | zk2.od.com(Prod环境) | 10.4.7.12 |
2.重配zookeeper
停止所有的zookeeper服务
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh stop
[root@hdss7-12 ~]# /opt/zookeeper/bin/zkServer.sh stop重新配置zookeeper的配置文件
在HDSS7-11.host.com主机上：
[root@hdss7-11 ~]# more /opt/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
clientPort=2181
[root@hdss7-11 ~]# 在HDSS7-12.host.com主机上：
[root@hdss7-12 ~]# more /opt/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/data
dataLogDir=/data/zookeeper/logs
clientPort=2181
[root@hdss7-12 ~]# 重启zk(删除数据文件)
[root@hdss7-11 ~]# rm -rf /data/zookeeper/data/*
[root@hdss7-11 ~]# rm -rf /data/zookeeper/logs/*
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@hdss7-11 ~]# /opt/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Mode: standalone
[root@hdss7-11 ~]#
[root@hdss7-12 ~]# rm -rf /data/zookeeper/data/*
[root@hdss7-12 ~]# rm -rf /data/zookeeper/logs/*
[root@hdss7-12 ~]# /opt/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@hdss7-12 ~]# /opt/zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Mode: standalone
[root@hdss7-12 ~]# 3.准备资源配置清单(dubbo-monitor)
在运维主机HDSS7-200.host.com上：
- configmap
[root@hdss7-200 ~]# vim /data/k8s-yaml/dubbo-monitor/cm.yaml
[root@hdss7-200 ~]# more /data/k8s-yaml/dubbo-monitor/cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: dubbo-monitor-cm
namespace: infra
data:
dubbo.properties: |
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=OldboyEdu
dubbo.registry.address=zookeeper://zk1.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=/dubbo-monitor-simple/logs/dubbo-monitor.log
dubbo.log4j.level=WARN
[root@hdss7-200 ~]# - dp.yaml
[root@hdss7-200 ~]# more /data/k8s-yaml/dubbo-monitor/dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-monitor
namespace: infra
labels:
name: dubbo-monitor
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-monitor
template:
metadata:
labels:
app: dubbo-monitor
name: dubbo-monitor
spec:
containers:
- name: dubbo-monitor
image: harbor.od.com/infra/dubbo-monitor:latest
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
imagePullPolicy: IfNotPresent
volumeMounts:
- name: configmap-volume
mountPath: /dubbo-monitor-simple/conf
volumes:
- name: configmap-volume
configMap:
name: dubbo-monitor-cm
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 ~]#
[root@hdss7-200 ~]# 4.应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/cm.yaml
configmap/dubbo-monitor-cm created
[root@hdss7-21 ~]#
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-monitor/dp.yaml
deployment.extensions/dubbo-monitor configured
[root@hdss7-21 ~]# - 查看生成的configmap
- 查看pods
5.重新发版，修改dubbo项目的配置文件
修改项目源代码，提交至仓库
dubbo-demo-service
dubbo-server/src/main/java/config.properties #代码中的路径
#原来的配置 dubbo.registry=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181 dubbo.port=20880 # 修改为 dubbo.registry=zookeeper://zk1.od.com:2181 dubbo.port=28080dubbo-demo-web
dubbo-client/src/main/java/config.properties #代码中的路径
# 原来的配置 dubbo.registry=zookeeper://zk1.od.com:2181?backup=zk2.od.com:2181,zk3.od.com:2181 # 修改为 dubbo.registry=zookeeper://zk1.od.com:2181
使用Jenkins进行CI
略
修改/应用资源配置清单CD
修改dp.yaml或者k8s的dashboard上，修改deployment使用的容器版本，提交应用
6.configmap 复杂配置文件处理方法
- 查看configmap
[root@hdss7-21 ~]# kubectl get configmap -n infra
NAME DATA AGE
dubbo-monitor-cm 1 17h
[root@hdss7-21 ~]# kubectl get cm -n infra
NAME DATA AGE
dubbo-monitor-cm 1 17h
[root@hdss7-21 ~]# kubectl describe cm dubbo-monitor-cm -n infa
Error from server (NotFound): namespaces "infa" not found
[root@hdss7-21 ~]# kubectl describe cm dubbo-monitor-cm -n infra
Name: dubbo-monitor-cm
Namespace: infra
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"v1","data":{"dubbo.properties":"dubbo.container=log4j,spring,registry,jetty\ndubbo.application.name=simple-monitor\ndubbo.a...
Data
====
dubbo.properties:
----
dubbo.container=log4j,spring,registry,jetty
dubbo.application.name=simple-monitor
dubbo.application.owner=OldboyEdu
dubbo.registry.address=zookeeper://zk1.od.com:2181
dubbo.protocol.port=20880
dubbo.jetty.port=8080
dubbo.jetty.directory=/dubbo-monitor-simple/monitor
dubbo.charts.directory=/dubbo-monitor-simple/charts
dubbo.statistics.directory=/dubbo-monitor-simple/statistics
dubbo.log4j.file=/dubbo-monitor-simple/logs/dubbo-monitor.log
dubbo.log4j.level=WARN
Events: <none>
[root@hdss7-21 ~]# - 根据文件制作configmap
[root@hdss7-21 ~]# cd /opt/kubernetes/server/bin/conf/
[root@hdss7-21 conf]# ls kubelet.kubeconfig
kubelet.kubeconfig
[root@hdss7-21 conf]# kubectl create cm kubelet-configmap --from-file=./kubelet.kubeconfig
configmap/kubelet-configmap created
[root@hdss7-21 conf]# kubectl get cm kubelet-configmap -o yaml第三章：apollo配置中心介绍
Apollo(阿波罗)是携程框架部门研发的分布式配置中心，能够集中化管理应用不同环境，不同集群的配置，配置修改后能够实时推送到应用端，并且具备规范的权限、流程治理等特性，适用于微服务配置管理场景。
官方Github地址：
Apollo官方地址
官方release包
- 基础架构
- Config Service提供配置的读取、推送等功能，服务对象是Apollo客户端
- Admin Service提供配置的修改、发布等功能，服务对象是Apollo Portal（管理界面）
- Config Service和Admin Server都是多实例、无状态部署，所以需要将自己注册到Eureka中并保持心跳
- 在Eureka之上我们架了一层Meta Server用于封装Eureka的服务发现接口
- Client通过域名访问Meta Server获取Config Server服务列表（IP+Port），而后直接通过IP+Port访问服务，同时在Client测绘做load balance、错误重试
- Portal通过域名访问Meta Server获取Admin Service服务列表（IP + port），而后直接通过IP+Port访问服务，同时在Portal测绘做load balance、错误重试
- 简化模型
第四章：实战交付apollo配置中心组件–configservice到kubernetes集群
1.准备软件包
在运维主机HDSS7-200.host.com上：
[root@hdss7-200 ~]# wget -O /opt/src/apollo-configservice-1.5.1-github.zip https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-configservice-1.5.1-github.zip
[root@hdss7-200 ~]# mkdir /data/dockerfile/apollo-configservice && unzip -o /opt/src/apollo-configservice-1.5.1-github.zip -d /data/dockerfile/apollo-configservice2.安装数据库
在数据库主机HDSS7-11.host.com上：
注意：MySQL版本应为5.6或以上！
- 更新yum源
[root@hdss7-11 ~]# more /etc/yum.repos.d/MariaDB.repo
[mariadb]
name = MariaDB
baseurl = https://mirrors.ustc.edu.cn/mariadb/yum/10.1/centos7-amd64/
gpgkey=https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB
gpgcheck=1
[root@hdss7-11 ~]# - 导入GPG-KEY
[root@hdss7-11 ~]# rpm --import https://mirrors.ustc.edu.cn/mariadb/yum/RPM-GPG-KEY-MariaDB- 查看安装数据库版本并安装数据库
[root@hdss7-11 ~]# yum list mariadb-server --show-duplicates
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.njupt.edu.cn
* epel: mirror.sjtu.edu.cn
* extras: mirrors.163.com
* updates: mirrors.163.com
Available Packages
MariaDB-server.x86_64 10.1.46-1.el7.centos mariadb
MariaDB-server.x86_64 10.1.47-1.el7.centos mariadb
MariaDB-server.x86_64 10.1.48-1.el7.centos mariadb
mariadb-server.x86_64 1:5.5.68-1.el7 base
[root@hdss7-11 ~]# yum install MariaDB-server -y- 配置数据库字符集
[root@hdss7-11 ~]# vim /etc/my.cnf.d/mysql-clients.cnf
[mysql]
default-character-set = utf8mb4
[root@hdss7-11 ~]# vim /etc/my.cnf.d/server.cnf
[mysqld]
character_set_server = utf8mb4
collation_server = utf8mb4_general_ci
init_connect = "SET NAMES 'utf8mb4'"- 示例：
[mysql]
default-character-set = utf8mb4
[mysqld]
character_set_server = utf8mb4
collation_server = utf8mb4_general_ci
init_connect = "SET NAMES 'utf8mb4'"- 启动数据库
[root@hdss7-11 ~]# systemctl start mariadb.service
[root@hdss7-11 ~]# systemctl enable mariadb.service- 设置数据库管理员账号
[root@hdss7-11 ~]# mysqladmin -uroot password
New password: （123456）
Confirm new password: （123456）- 检查数据库配置
[root@hdss7-11 ~]# mysql -uroot -p123456
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 5
Server version: 10.1.48-MariaDB MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> \s
--------------
mysql Ver 15.1 Distrib 10.1.48-MariaDB, for Linux (x86_64) using readline 5.1
Connection id: 5
Current database:
Current user: root@localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server: MariaDB
Server version: 10.1.48-MariaDB MariaDB Server
Protocol version: 10
Connection: Localhost via UNIX socket
Server characterset: utf8mb4
Db characterset: utf8mb4
Client characterset: utf8mb4
Conn. characterset: utf8mb4
UNIX socket: /var/lib/mysql/mysql.sock
Uptime: 4 min 16 sec
Threads: 1 Questions: 12 Slow queries: 0 Opens: 17 Flush tables: 1 Open tables: 11 Queries per second avg: 0.046
--------------
MariaDB [(none)]> 3.执行数据库脚本
# 下载数据库初始化脚本
[root@hdss7-11 ~]# wget https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql
[root@hdss7-11 ~]# mysql -uroot -p123456 < V1.0.0__initialization.sql4.数据库用户授权
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| ApolloConfigDB |
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.00 sec)
MariaDB [(none)]> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> 5.修改初始数据
MariaDB [(none)]> update ApolloConfigDB.ServerConfig set ServerConfig.Value="http://config.od.com/eureka" where ServerConfig.Key="eureka.service.url";
MariaDB [ApolloConfigDB]> select * from ServerConfig\G
*************************** 1. row ***************************
Id: 1
Key: eureka.service.url
Cluster: default
Value: http://config.od.com/eureka
Comment: Eureka服务Url，多个service以英文逗号分隔6.制作docker镜像
在运维主机HDSS7-200.host.com上：
- 删除无用的数据
[root@hdss7-200 ~]# cd /data/dockerfile/apollo-configservice/
[root@hdss7-200 apollo-configservice]# rm -f apollo-configservice-1.5.1-sources.jar
[root@hdss7-200 apollo-configservice]# rm -f scripts/shutdown.sh - 配置数据库连接串（这里可以不改，可以用cm挂载）
[root@hdss7-200 apollo-configservice]# cat config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
#apollo.eureka.server.enabled=true
#apollo.eureka.client.enabled=true
[root@hdss7-200 apollo-configservice]# - 更新startup.sh
官方提供的k8s启动脚本
[root@hdss7-200 apollo-configservice]# cat scripts/startup.sh
#!/bin/bash
SERVICE_NAME=apollo-configservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-config-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_CONFIG_SERVICE_NAME=$(hostname -i)
SERVER_URL="http://${APOLLO_CONFIG_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms6144m -Xmx6144m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=4096m -XX:MaxNewSize=4096m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null
[root@hdss7-200 apollo-configservice]# - 写Dockerfile
[root@hdss7-200 apollo-configservice]# cat Dockerfile
FROM hebye/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-configservice-${VERSION}.jar /apollo-configservice/apollo-configservice.jar
ADD config/ /apollo-configservice/config
ADD scripts/ /apollo-configservice/scripts
CMD ["/apollo-configservice/scripts/startup.sh"]
[root@hdss7-200 apollo-configservice]# - 制作镜像并推送
[root@hdss7-200 apollo-configservice]# docker build . -t harbor.od.com/infra/apollo-configservice:v1.5.1
[root@hdss7-200 apollo-configservice]# docker push harbor.od.com/infra/apollo-configservice:v1.5.17.解析域名
DNS主机HDSS7-11.host.com上：
[root@hdss7-11 ~]# tail -2 /var/named/od.com.zone
mysql A 10.4.7.11
config A 10.4.7.10
[root@hdss7-11 ~]#
[root@hdss7-11 ~]# named-checkconf
[root@hdss7-11 ~]# systemctl restart named
[root@hdss7-11 ~]# dig -t A mysql.od.com @10.4.7.11 +short
10.4.7.11
[root@hdss7-11 ~]# dig -t A config.od.com @10.4.7.11 +short
10.4.7.10
[root@hdss7-11 ~]# 8.准备资源配置清单
在运维主机HDSS7-200.host.com上
[root@hdss7-200 ~]# mkdir /data/k8s-yaml/apollo-configservice && cd /data/k8s-yaml/apollo-configservice- configmap
[root@hdss7-200 apollo-configservice]# more cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-configservice-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.od.com/eureka
app.properties: |
appId=100003171
[root@hdss7-200 apollo-configservice]# - dp.yaml
[root@hdss7-200 apollo-configservice]# more dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
labels:
name: apollo-configservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-configservice
template:
metadata:
labels:
app: apollo-configservice
name: apollo-configservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-configservice-cm
containers:
- name: apollo-configservice
image: harbor.od.com/infra/apollo-configservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-configservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 apollo-configservice]# - svc.yaml
[root@hdss7-200 apollo-configservice]# more svc.yaml
kind: Service
apiVersion: v1
metadata:
name: apollo-configservice
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-configservice
[root@hdss7-200 apollo-configservice]# - ingress.yaml
[root@hdss7-200 apollo-configservice]# more ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-configservice
namespace: infra
spec:
rules:
- host: config.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-configservice
servicePort: 8080
[root@hdss7-200 apollo-configservice]# 9.应用资源配置清单
在任意一台k8s运算节点执行：
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/cm.yaml
configmap/apollo-configservice-cm created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/dp.yaml
deployment.extensions/apollo-configservice created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/svc.yaml
service/apollo-configservice created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-configservice/ingress.yaml
ingress.extensions/apollo-configservice created
[root@hdss7-21 ~]# 10.浏览器访问
11.分析mysql连接ip
MariaDB [(none)]> show full processlist;
+----+--------------+-----------------+----------------+---------+------+-------+-----------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+----+--------------+-----------------+----------------+---------+------+-------+-----------------------+----------+
| 6 | apolloconfig | 10.4.7.21:42424 | ApolloConfigDB | Sleep | 0 | | NULL | 0.000 |
| 7 | apolloconfig | 10.4.7.21:42426 | ApolloConfigDB | Sleep | 0 | | NULL | 0.000 |
| 8 | apolloconfig | 10.4.7.21:42428 | ApolloConfigDB | Sleep | 146 | | NULL | 0.000 |
| 9 | apolloconfig | 10.4.7.21:42430 | ApolloConfigDB | Sleep | 146 | | NULL | 0.000 |
| 10 | apolloconfig | 10.4.7.21:42432 | ApolloConfigDB | Sleep | 146 | | NULL | 0.000 |
| 11 | apolloconfig | 10.4.7.21:42434 | ApolloConfigDB | Sleep | 146 | | NULL | 0.000 |
| 12 | apolloconfig | 10.4.7.21:42436 | ApolloConfigDB | Sleep | 145 | | NULL | 0.000 |
| 13 | apolloconfig | 10.4.7.21:42438 | ApolloConfigDB | Sleep | 145 | | NULL | 0.000 |
| 14 | apolloconfig | 10.4.7.21:42442 | ApolloConfigDB | Sleep | 145 | | NULL | 0.000 |
| 15 | apolloconfig | 10.4.7.21:42446 | ApolloConfigDB | Sleep | 145 | | NULL | 0.000 |
| 16 | root | localhost | NULL | Query | 0 | init | show full processlist | 0.000 |
+----+--------------+-----------------+----------------+---------+------+-------+-----------------------+----------+
11 rows in set (0.00 sec)
MariaDB [(none)]> 第五章：实战交付apollo配置中心组件–adminservice到kubernetes集群
1.准备软件包
在运维主机HDSS7-200.host.com上：
下载官方release包
- 下载并解压
[root@hdss7-200 ~]# wget -O /opt/src/apollo-adminservice-1.5.1-github.zip https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-adminservice-1.5.1-github.zip
[root@hdss7-200 ~]# mkdir /data/dockerfile/apollo-adminservice
[root@hdss7-200 ~]# unzip -o /opt/src/apollo-adminservice-1.5.1-github.zip -d /data/dockerfile/apollo-adminservice/- 删除无用的软件
[root@hdss7-200 ~]# cd /data/dockerfile/apollo-adminservice/
[root@hdss7-200 apollo-adminservice]# rm -f apollo-adminservice-1.5.1-sources.jar
[root@hdss7-200 apollo-adminservice]# rm -f apollo-adminservice.conf
[root@hdss7-200 apollo-adminservice]# rm -f scripts/shutdown.sh - 配置数据库连接串（这里可以不改，可以用cm挂载）
[root@hdss7-200 apollo-adminservice]# more config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
[root@hdss7-200 apollo-adminservice]#
[root@hdss7-200 apollo-adminservice]# more config/app.properties
appId=100003172
jdkVersion=1.8
[root@hdss7-200 apollo-adminservice]#
#其中appId=100003172 要和congfig-service的不一致2.制作Docker镜像
- 修改启动脚本
[root@hdss7-200 apollo-adminservice]# more scripts/startup.sh
#!/bin/bash
SERVICE_NAME=apollo-adminservice
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-adminservice
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_ADMIN_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:${SERVER_PORT}"
SERVER_URL="http://${APOLLO_ADMIN_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null在官网的基础上修改了这两个参数
SERVER_PORT=8080
APOLLO_ADMIN_SERVICE_NAME=$(hostname -i)
- 编写Dockerfile
[root@hdss7-200 ~]# cd /data/dockerfile/apollo-adminservice/
[root@hdss7-200 apollo-adminservice]# more Dockerfile
FROM hebye/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-adminservice-${VERSION}.jar /apollo-adminservice/apollo-adminservice.jar
ADD config/ /apollo-adminservice/config
ADD scripts/ /apollo-adminservice/scripts
CMD ["/apollo-adminservice/scripts/startup.sh"]
[root@hdss7-200 apollo-adminservice]# - 制作镜像并推送
[root@hdss7-200 apollo-adminservice]# docker build . -t harbor.od.com/infra/apollo-adminservice:v1.5.1
[root@hdss7-200 apollo-adminservice]# docker push !$
#!? 代表上一个命令的最后参数 ，或者输入 esc .3.准备资源配置清单
在运维主机HDSS7-200.host.com上
[root@hdss7-200 ~]# mkdir /data/k8s-yaml/apollo-adminservice
[root@hdss7-200 ~]# cd /data/k8s-yaml/apollo-adminservice/
[root@hdss7-200 apollo-adminservice]# - configmap
[root@hdss7-200 apollo-adminservice]# more cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-adminservice-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigDB?characterEncoding=utf8
spring.datasource.username = apolloconfig
spring.datasource.password = 123456
eureka.service.url = http://config.od.com/eureka
app.properties: |
appId=100003172
[root@hdss7-200 apollo-adminservice]#- dp.yaml
[root@hdss7-200 apollo-adminservice]# more dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-adminservice
namespace: infra
labels:
name: apollo-adminservice
spec:
replicas: 1
selector:
matchLabels:
name: apollo-adminservice
template:
metadata:
labels:
app: apollo-adminservice
name: apollo-adminservice
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-adminservice-cm
containers:
- name: apollo-adminservice
image: harbor.od.com/infra/apollo-adminservice:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-adminservice/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 apollo-adminservice]# 4.应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/cm.yaml
configmap/apollo-adminservice-cm created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-adminservice/dp.yaml
deployment.extensions/apollo-adminservice created
[root@hdss7-21 ~]# 5.浏览器访问
[root@hdss7-21 ~]# kubectl get pods -n infra -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
apollo-adminservice-5cccf97c64-m9wdj 1/1 Running 0 57s 172.7.21.9 hdss7-21.host.com <none> <none>
apollo-configservice-5f6555448-qxgv9 1/1 Running 0 47m 172.7.21.8 hdss7-21.host.com <none> <none>
dubbo-monitor-6676dd74cc-h5qdx 1/1 Running 0 19h 172.7.21.5 hdss7-21.host.com <none> <none>
jenkins-b69779cdc-h8fw2 1/1 Running 0 26h 172.7.22.11 hdss7-22.host.com <none> <none>
[root@hdss7-21 ~]# curl 172.7.21.9:8080/info
{"git":{"commit":{"time":{"seconds":1573275854,"nanos":0},"id":"c9eae54"},"branch":"1.5.1"}6.集群维护
apollo-configservice和apollo-adminservice是无状态的服务，可以随便扩容服务器，内部会自动建立集群。
第六章：实战交付apollo配置中心组件–portal到kubernetes集群
1.准备软件包
在运维主机HDSS7-200.host.com上：
[root@hdss7-200 ~]# wget -O /opt/src/apollo-portal-1.5.1-github.zip https://github.com/ctripcorp/apollo/releases/download/v1.5.1/apollo-portal-1.5.1-github.zip
[root@hdss7-200 ~]# mkdir /data/dockerfile/apollo-portal
[root@hdss7-200 ~]# unzip -o /opt/src/apollo-portal-1.5.1-github.zip -d /data/dockerfile/apollo-portal/- 清理不用的文件
[root@hdss7-200 ~]# cd /data/dockerfile/apollo-portal/
[root@hdss7-200 apollo-portal]# rm -f apollo-portal-1.5.1-sources.jar
[root@hdss7-200 apollo-portal]# rm -f apollo-portal.conf
[root@hdss7-200 apollo-portal]# rm -f scripts/shutdown.sh
[root@hdss7-200 apollo-portal]# 2.执行数据库脚本
在数据库主机HDSS7-11.host.com上：
数据库脚本地址
[root@hdss7-11 ~]# wget -O apolloportal.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/portaldb/V1.0.0__initialization.sql
[root@hdss7-11 ~]# mysql -uroot -p123456 <apolloportal.sql 3.数据库用户授权
MariaDB [ApolloPortalDB]> grant INSERT,DELETE,UPDATE,SELECT on ApolloPortalDB.* to "apolloportal"@"10.4.7.%" identified by "123456";
MariaDB [ApolloPortalDB]> use ApolloPortalDB;
Database changed
MariaDB [ApolloPortalDB]> update ServerConfig set Value='[{"orgId":"od01","orgName":"Linux学院"},{"orgId":"od02","orgName":"云计算学院"},{"orgId":"od03","orgName":"Python学院"}]' where Id=2;
Query OK, 1 row affected (0.01 sec)
Rows matched: 1 Changed: 1 Warnings: 0
MariaDB [ApolloPortalDB]> 4.制作docker镜像
在运维主机HDSS7-200.host.com上
- 配置数据库连接串（用cm的话这里可以不用修改）
[root@hdss7-200 ~]# cd /data/dockerfile/apollo-portal
[root@hdss7-200 apollo-portal]# cat config/application-github.properties
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
[root@hdss7-200 apollo-portal]# - 配置Portal的meta service（用cm的话这里可以不用修改）
[root@hdss7-200 apollo-portal]# cat config/apollo-env.properties
dev.meta=http://config.od.com
[root@hdss7-200 apollo-portal]# - 更新startup.sh
[root@hdss7-200 apollo-portal]# cat scripts/startup.sh
#!/bin/bash
SERVICE_NAME=apollo-portal
## Adjust log dir if necessary
LOG_DIR=/opt/logs/apollo-portal-server
## Adjust server port if necessary
SERVER_PORT=8080
APOLLO_PORTAL_SERVICE_NAME=$(hostname -i)
# SERVER_URL="http://localhost:$SERVER_PORT"
SERVER_URL="http://${APOLLO_PORTAL_SERVICE_NAME}:${SERVER_PORT}"
## Adjust memory settings if necessary
#export JAVA_OPTS="-Xms2560m -Xmx2560m -Xss256k -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=384m -XX:NewSize=1536m -XX:MaxNewSize=1536m -XX:SurvivorRatio=8"
## Only uncomment the following when you are using server jvm
#export JAVA_OPTS="$JAVA_OPTS -server -XX:-ReduceInitialCardMarks"
########### The following is the same for configservice, adminservice, portal ###########
export JAVA_OPTS="$JAVA_OPTS -XX:ParallelGCThreads=4 -XX:MaxTenuringThreshold=9 -XX:+DisableExplicitGC -XX:+ScavengeBeforeFullGC -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCDetails -XX:+HeapDumpOnOutOfMemoryError -XX:-OmitStackTraceInFastThrow -Duser.timezone=Asia/Shanghai -Dclient.encoding.override=UTF-8 -Dfile.encoding=UTF-8 -Djava.security.egd=file:/dev/./urandom"
export JAVA_OPTS="$JAVA_OPTS -Dserver.port=$SERVER_PORT -Dlogging.file=$LOG_DIR/$SERVICE_NAME.log -XX:HeapDumpPath=$LOG_DIR/HeapDumpOnOutOfMemoryError/"
# Find Java
if [[ -n "$JAVA_HOME" ]] && [[ -x "$JAVA_HOME/bin/java" ]]; then
javaexe="$JAVA_HOME/bin/java"
elif type -p java > /dev/null 2>&1; then
javaexe=$(type -p java)
elif [[ -x "/usr/bin/java" ]]; then
javaexe="/usr/bin/java"
else
echo "Unable to find Java"
exit 1
fi
if [[ "$javaexe" ]]; then
version=$("$javaexe" -version 2>&1 | awk -F '"' '/version/ {print $2}')
version=$(echo "$version" | awk -F. '{printf("%03d%03d",$1,$2);}')
# now version is of format 009003 (9.3.x)
if [ $version -ge 011000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 010000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
elif [ $version -ge 009000 ]; then
JAVA_OPTS="$JAVA_OPTS -Xlog:gc*:$LOG_DIR/gc.log:time,level,tags -Xlog:safepoint -Xlog:gc+heap=trace"
else
JAVA_OPTS="$JAVA_OPTS -XX:+UseParNewGC"
JAVA_OPTS="$JAVA_OPTS -Xloggc:$LOG_DIR/gc.log -XX:+PrintGCDetails"
JAVA_OPTS="$JAVA_OPTS -XX:+UseConcMarkSweepGC -XX:+UseCMSCompactAtFullCollection -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:CMSFullGCsBeforeCompaction=9 -XX:+CMSClassUnloadingEnabled -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintHeapAtGC -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=5M"
fi
fi
printf "$(date) ==== Starting ==== \n"
cd `dirname $0`/..
chmod 755 $SERVICE_NAME".jar"
./$SERVICE_NAME".jar" start
rc=$?;
if [[ $rc != 0 ]];
then
echo "$(date) Failed to start $SERVICE_NAME.jar, return code: $rc"
exit $rc;
fi
tail -f /dev/null- 编写Dockerfile
[root@hdss7-200 apollo-portal]# cat Dockerfile
FROM hebye/jre8:8u112
ENV VERSION 1.5.1
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&\
echo "Asia/Shanghai" > /etc/timezone
ADD apollo-portal-${VERSION}.jar /apollo-portal/apollo-portal.jar
ADD config/ /apollo-portal/config
ADD scripts/ /apollo-portal/scripts
CMD ["/apollo-portal/scripts/startup.sh"]
[root@hdss7-200 apollo-portal]# - 制作镜像并推送
[root@hdss7-200 apollo-portal]# docker build . -t harbor.od.com/infra/apollo-portal:v1.5.1
[root@hdss7-200 apollo-portal]# docker push harbor.od.com/infra/apollo-portal:v1.5.15.解析域名
在DNS主机HDSS7-11.host.com上
[root@hdss7-11 ~]# tail -1 /var/named/od.com.zone
portal A 10.4.7.10
[root@hdss7-11 ~]# named-checkconf
[root@hdss7-11 ~]# systemctl restart named
[root@hdss7-11 ~]# dig -t A portal.od.com @10.4.7.11 +short
10.4.7.106.准备资源配置清单
在运维主机HDSS7-200.host.com上
- configmap
[root@hdss7-200 apollo-portal]# cat cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-portal-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
app.properties: |
appId=100003173
apollo-env.properties: |
dev.meta=http://config.od.com
[root@hdss7-200 apollo-portal]# - dp.yaml
[root@hdss7-200 apollo-portal]# cat dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
labels:
name: apollo-portal
spec:
replicas: 1
selector:
matchLabels:
name: apollo-portal
template:
metadata:
labels:
app: apollo-portal
name: apollo-portal
spec:
volumes:
- name: configmap-volume
configMap:
name: apollo-portal-cm
containers:
- name: apollo-portal
image: harbor.od.com/infra/apollo-portal:v1.5.1
ports:
- containerPort: 8080
protocol: TCP
volumeMounts:
- name: configmap-volume
mountPath: /apollo-portal/config
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 apollo-portal]# - svc.yaml
[root@hdss7-200 apollo-portal]# cat svc.yaml
kind: Service
apiVersion: v1
metadata:
name: apollo-portal
namespace: infra
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: apollo-portal
[root@hdss7-200 apollo-portal]# - ingress.yaml
[root@hdss7-200 apollo-portal]# cat ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: apollo-portal
namespace: infra
spec:
rules:
- host: portal.od.com
http:
paths:
- path: /
backend:
serviceName: apollo-portal
servicePort: 8080
[root@hdss7-200 apollo-portal]# 7.应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/cm.yaml
configmap/apollo-portal-cm created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/dp.yaml
deployment.extensions/apollo-portal created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/svc.yaml
service/apollo-portal created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/ingress.yaml
ingress.extensions/apollo-portal created
[root@hdss7-21 ~]# 8.浏览器访问
- 用户名：apollo
- 密码： admin
- 修改密码
- 可以查询相关的配置
organizations
- 可以修改organizations的相关信息，并且保存到数据库中
[{“orgId”:”od01”,”orgName”:”Linux学院”},{“orgId”:”od02”,”orgName”:”云计算学院”},{“orgId”:”od03”,”orgName”:”Python学院”},{“orgld”:”od04”,”orgName”:”大数据学院”}]
- 查看生成的数据：
第七章：实战配置dubbo微服务接入apollo配置中心管理
1.改造dubbo-demo-service项目说明
代码：https://gitee.com/yodo1/dubbo-demo-service.git
分支：apollo
源代码要连接的zk的配置：
dubbo-server/src/main/resources/config.properties
dubbo.registry=${dubbo.registry}
dubbo.port=${dubbo.port}1.创建项目
- 部门
样例部门1（老男孩linux学院001）
- Appid
dubbo-demo-service
- 应用名称
dubbo服务提供者
- 应用负责人
apollo|apollo
- 项目管理员
apollo|apollo
2.进入配置页面
新增配置项1
- Key
dubbo.registry- Value
zookeeper://zk1.od.com:2181- Comment
dubbo服务注册中心- 选择集群
DEV
提交
新增配置项2
- Key
dubbo.port
- Value
20880
- Comment
dubbo服务提供者的监听端口- 选择集群
DEV
3.发布配置
点击发布，配置生效
4.使用jenkins进行CI
(注意记录镜像的tag harbor.od.com/app/dubbo-demo-service:apollo_210330_2120)
略…..
5.上线新构建的项目
修改资源配置清单(添加相关apollo的配置)
运维主机HDSS7-200.host.com上：[root@hdss7-200 ~]# cat /data/k8s-yaml/dubbo-demo-service/dp.yaml kind: Deployment apiVersion: extensions/v1beta1 metadata: name: dubbo-demo-service namespace: app labels: name: dubbo-demo-service spec: replicas: 1 selector: matchLabels: name: dubbo-demo-service template: metadata: labels: app: dubbo-demo-service name: dubbo-demo-service spec: containers: - name: dubbo-demo-service image: harbor.od.com/app/dubbo-demo-service:apollo_210330_2120 ports: - containerPort: 20880 protocol: TCP env: - name: C_OPTS value: -Denv=dev -Dapollo.meta=http://config.od.com - name: JAR_BALL value: dubbo-server.jar imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 [root@hdss7-200 ~]#注意：增加了env段配置
注意：docker镜像新版的tag应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-service/dp.yaml deployment.extensions/dubbo-demo-service configured [root@hdss7-21 ~]#portal查看注册情况
扩容dubbo-demo-service集群为2
会自动从apollo拉取相关的配置
观察项目运行情况
http://dubbo-monitor.od.com
2.改造dubbo-demo-web项目
代码：git@gitee.com:yodo1/dubbo-demo-web.git
分支：apollo
源代码要连接的zk的配置：dubbo-client/src/main/resources/config.properties
dubbo.registry=${dubbo.registry}1.创建项目
- 部门
样例部门1（老男孩linux学院001）
- Appid
dubbo-demo-web
注意 此Appid一定要和代码中对应，代码：dubbo-client/src/main/resources/META-INF/app.properties 分支：apollo
配置：app.id=dubbo-demo-web
- 应用名称
dubbo服务消费者
- 应用负责人
apollo|apollo
- 项目管理员
apollo|apollo
2.进入配置页面
新增配置项1
- Key
dubbo.registry
注意：此配置是代码在配置文件dubbo-client/src/main/resources/config.properties中
分支：apollo
# 文件内容
dubbo.registry=${dubbo.registry}- Value
zookeeper://zk1.od.com:2181
- 选择集群
DEV
提交
3.发布配置
点击发布，配置生效
4.使用jenkins进行CI
(注意记录镜像的tag harbor.od.com/app/dubbo-demo-consumer:apollo_210330_2150)
略…..
5.上线新构建的项目
- 修改资源配置清单
[root@hdss7-200 ~]# cat /data/k8s-yaml/dubbo-demo-consumer/dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: app
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-consumer:apollo_210330_2150
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=dev -Dapollo.meta=http://config.od.com
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 ~]# 注意：增加了env段配置
注意：docker镜像新版的tag
- 应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/dubbo-demo-consumer/dp.yaml
deployment.extensions/dubbo-demo-consumer configured
[root@hdss7-21 ~]# - 查看apollo配置中心
6.浏览器访问
http://demo.od.com/hello?name=apollo
修改代码，提交gitee，重新构建，重新发版
dubbo-client/src/main/java/com/od/dubbotest/action/HelloAction.java
第八章：实战使用apollo配置中心管理测试环境和生产环境
1.实战维护多套dubbo微服务环境
1.生产实践
- 迭代新需求/修复BUG(编码->提交GIT)
- 测试环境发版，测试(应用通过编译打包发布至TEST命名空间)
- 测试通过，上线(应用镜像直接发布在PROD命名空间)
2.系统架构
- 物理架构
| 主机名 | 角色 | ip |
|---|---|---|
| HDSS7-11.host.com | zk-test(测试环境Test) | 10.4.7.11 |
| HDSS7-12.host.com | zk-prod(生产环境Prod) | 10.4.7.12 |
| HDSS7-21.host.com | kubernetes运算节点 | 10.4.7.21 |
| HDSS7-22.host.com | kubernetes运算节点 | 10.4.7.22 |
| HDSS7-200.host.com | 运维主机，harbor仓库 | 10.4.7.200 |
- K8S内系统架构
| 环境 | 命名空间 | 应用 |
|---|---|---|
| 测试环境(TEST) | test | apollo-config,apollo-admin |
| 测试环境(TEST) | test | dubbo-demo-service,dubbo-demo-web |
| 生产环境(PROD) | prod | apollo-config,apollo-admin |
| 生产环境(PROD) | prod | dubbo-demo-service,dubbo-demo-web |
| ops环境(infra) | infra | jenkins,dubbo-monitor,apollo-portal |
3.修改/添加域名解析
DNS主机HDSS7-11.host.com上：
[root@hdss7-11 ~]# tail -6 /var/named/od.com.zone
zk-test A 10.4.7.11
zk-prod A 10.4.7.12
config-test A 10.4.7.10
config-prod A 10.4.7.10
demo-test A 10.4.7.10
demo-prod A 10.4.7.10
[root@hdss7-11 ~]# named-checkconf
[root@hdss7-11 ~]# systemctl restart named
[root@hdss7-11 ~]# 4.Apollo的K8S应用配置
- 创建test和prod名称空间(app名称空间可以删除)
[root@hdss7-21 ~]# kubectl create ns test
[root@hdss7-21 ~]# kubectl create ns prod- 配置连接docker仓库的认证
[root@hdss7-21 ~]# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=Harbor12345 -n test
[root@hdss7-21 ~]# kubectl create secret docker-registry harbor --docker-server=harbor.od.com --docker-username=admin --docker-password=Harbor12345 -n prod- 删除infra名称空间内的apollo-configservice/apollo-adminservice应用
K8S的dashboard界面，进行删除(容器组缩容为0) - 数据库内删除ApollConigDB,创建ApolloConfigTestDB/ApolloConfigProdDB
MariaDB [(none)]> drop database ApolloConfigDB;#下载数据库初始化脚本
[root@hdss7-11 ~]# wget -O apolloconf.sql https://raw.githubusercontent.com/ctripcorp/apollo/1.5.1/scripts/db/migration/configdb/V1.0.0__initialization.sql
#修改创建的数据库语句，区分test环境， 创建test目录
[root@hdss7-11 ~]# mkdir test
[root@hdss7-11 ~]# cp apolloconfig.sql test/
[root@hdss7-11 ~]# sed -i 's#ApolloConfigDB#ApolloConfigTestDB#g' test/apolloconfig.sql
#修改创建的数据库语句，区分prod环境， 创建prod目录
[root@hdss7-11 ~]# mkdir prod
[root@hdss7-11 ~]# cp apolloconfig.sql prod/
[root@hdss7-11 ~]# sed -i 's#ApolloConfigDB#ApolloConfigProdDB#g' prod/apolloconfig.sql- 测试环境
[root@hdss7-11 ~]# mysql -uroot -p123456 <test/apolloconfig.sql
[root@hdss7-11 ~]# mysql -uroot -p123456
MariaDB [(none)]> update ApolloConfigTestDB.ServerConfig set ServerConfig.Value="http://config-test.od.com/eureka" where ServerConfig.Key="eureka.service.url";
MariaDB [(none)]> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigTestDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";- 正式环境
[root@hdss7-11 ~]# mysql -uroot -p123456 < prod/apolloconfig.sql
MariaDB [(none)]> update ApolloConfigProdDB.ServerConfig set ServerConfig.Value="http://config-prod.od.com/eureka" where ServerConfig.Key="eureka.service.url";
MariaDB [(none)]> grant INSERT,DELETE,UPDATE,SELECT on ApolloConfigProdDB.* to "apolloconfig"@"10.4.7.%" identified by "123456";- 更改portal数据库-分环境
MariaDB [(none)]> use ApolloPortalDB;
MariaDB [ApolloPortalDB]> update ApolloPortalDB.ServerConfig set Value='fat,pro' where Id=1;
# 查看并清除配置中心的历史记录，如果是新环境的话，可以忽略这一步
MariaDB [ApolloPortalDB]> truncate table ApolloPortalDB.App;
Query OK, 0 rows affected (0.02 sec)
MariaDB [ApolloPortalDB]> truncate table ApolloPortalDB.AppNamespace;
Query OK, 0 rows affected (0.01 sec)
MariaDB [ApolloPortalDB]> - 修改portal的configmap文件-分环境
[root@hdss7-200 ~]# cat /data/k8s-yaml/apollo-portal/cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: apollo-portal-cm
namespace: infra
data:
application-github.properties: |
# DataSource
spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloPortalDB?characterEncoding=utf8
spring.datasource.username = apolloportal
spring.datasource.password = 123456
app.properties: |
appId=100003173
apollo-env.properties: |
fat.meta=http://config-test.od.com
pro.meta=http://config-prod.od.com- 应用portal的configmap文件
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/apollo-portal/cm.yaml
configmap/apollo-portal-cm configured
[root@hdss7-21 ~]# - 查看生成的portal的configmap文件
准备apollo-config，apollo-admin的资源配置清单（测试环境和生成环境）
测试环境：
测试环境-创建目录
[root@hdss7-200 ~]# mkdir /data/k8s-yaml/test/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer} -pv测试环境-apollo-configservice-资源配置清单
#拷贝文件 [root@hdss7-200 ~]# cp /data/k8s-yaml/apollo-configservice/* /data/k8s-yaml/test/apollo-configservice/ [root@hdss7-200 ~]# cd /data/k8s-yaml/test/apollo-configservice/修改资源配置清单
[root@hdss7-200 apollo-configservice]# cat dp.yaml kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: test labels: name: apollo-configservice spec: replicas: 1 selector: matchLabels: name: apollo-configservice template: metadata: labels: app: apollo-configservice name: apollo-configservice spec: volumes: - name: configmap-volume configMap: name: apollo-configservice-cm containers: - name: apollo-configservice image: harbor.od.com/infra/apollo-configservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-configservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600[root@hdss7-200 apollo-configservice]# cat svc.yaml kind: Service apiVersion: v1 metadata: name: apollo-configservice namespace: test spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-configservice [root@hdss7-200 apollo-configservice]# cat ingress.yaml kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: test spec: rules: - host: config-test.od.com http: paths: - path: / backend: serviceName: apollo-configservice servicePort: 8080 [root@hdss7-200 apollo-configservice]#应用测试环境-apollo-configservice-资源配置清单
[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/cm.yaml configmap/apollo-configservice-cm created [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/dp.yaml deployment.extensions/apollo-configservice created [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/svc.yaml service/apollo-configservice created [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-configservice/ingress.yaml ingress.extensions/apollo-configservice created [root@hdss7-22 ~]#
生产环境
生产环境-创建目录
[root@hdss7-200 ~]# mkdir /data/k8s-yaml/prod/{apollo-configservice,apollo-adminservice,dubbo-demo-service,dubbo-demo-consumer} -pv生产环境-apollo-configservice-资源配置清单
#拷贝test环境的配置文件，再进行修改 [root@hdss7-200 ~]# cp /data/k8s-yaml/test/apollo-configservice/* /data/k8s-yaml/prod/apollo-configservice/ [root@hdss7-200 ~]# cd /data/k8s-yaml/prod/apollo-configservice/ [root@hdss7-200 apollo-configservice]# cat dp.yaml kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: prod labels: name: apollo-configservice spec: replicas: 1 selector: matchLabels: name: apollo-configservice template: metadata: labels: app: apollo-configservice name: apollo-configservice spec: volumes: - name: configmap-volume configMap: name: apollo-configservice-cm containers: - name: apollo-configservice image: harbor.od.com/infra/apollo-configservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-configservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 [root@hdss7-200 apollo-configservice]# [root@hdss7-200 apollo-configservice]# cat svc.yaml kind: Service apiVersion: v1 metadata: name: apollo-configservice namespace: prod spec: ports: - protocol: TCP port: 8080 targetPort: 8080 selector: app: apollo-configservice [root@hdss7-200 apollo-configservice]# cat ingress.yaml kind: Ingress apiVersion: extensions/v1beta1 metadata: name: apollo-configservice namespace: prod spec: rules: - host: config-prod.od.com http: paths: - path: / backend: serviceName: apollo-configservice servicePort: 8080 [root@hdss7-200 apollo-configservice]#
应用正式环境-apollo-configservice-资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/cm.yaml configmap/apollo-configservice-cm created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/dp.yaml deployment.extensions/apollo-configservice created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/svc.yaml service/apollo-configservice created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-configservice/ingress.yaml ingress.extensions/apollo-configservice created [root@hdss7-21 ~]#测试环境-apollo-adminservice-资源配置清单
#拷贝文件，进行修改 [root@hdss7-200 ~]# cd /data/k8s-yaml/test/apollo-adminservice/ [root@hdss7-200 apollo-adminservice]# cp /data/k8s-yaml/apollo-adminservice/* [root@hdss7-200 apollo-adminservice]# pwd /data/k8s-yaml/test/apollo-adminservice [root@hdss7-200 apollo-adminservice]# cat cm.yaml apiVersion: v1 kind: ConfigMap metadata: name: apollo-adminservice-cm namespace: test data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigTestDB?characterEncoding=utf8 spring.datasource.username = apolloconfig spring.datasource.password = 123456 eureka.service.url = http://config-test.od.com/eureka app.properties: | appId=100003172 [root@hdss7-200 apollo-adminservice]# cat dp.yaml kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-adminservice namespace: test labels: name: apollo-adminservice spec: replicas: 1 selector: matchLabels: name: apollo-adminservice template: metadata: labels: app: apollo-adminservice name: apollo-adminservice spec: volumes: - name: configmap-volume configMap: name: apollo-adminservice-cm containers: - name: apollo-adminservice image: harbor.od.com/infra/apollo-adminservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-adminservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 [root@hdss7-200 apollo-adminservice]#应用测试环境-apollo-adminservice-资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/cm.yaml configmap/apollo-adminservice-cm created [root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/apollo-adminservice/dp.yaml deployment.extensions/apollo-adminservice created [root@hdss7-21 ~]#正式环境-apollo-adminservice-资源配置清单
#拷贝文件，进行修改 [root@hdss7-200 ~]# cd /data/k8s-yaml/prod/apollo-adminservice/ [root@hdss7-200 apollo-adminservice]# ls [root@hdss7-200 apollo-adminservice]# cp /data/k8s-yaml/test/apollo-adminservice/* . [root@hdss7-200 apollo-adminservice]# cat cm.yaml apiVersion: v1 kind: ConfigMap metadata: name: apollo-adminservice-cm namespace: prod data: application-github.properties: | # DataSource spring.datasource.url = jdbc:mysql://mysql.od.com:3306/ApolloConfigProdDB?characterEncoding=utf8 spring.datasource.username = apolloconfig spring.datasource.password = 123456 eureka.service.url = http://config-prod.od.com/eureka app.properties: | appId=100003172 [root@hdss7-200 apollo-adminservice]# [root@hdss7-200 apollo-adminservice]# cat dp.yaml kind: Deployment apiVersion: extensions/v1beta1 metadata: name: apollo-adminservice namespace: prod labels: name: apollo-adminservice spec: replicas: 1 selector: matchLabels: name: apollo-adminservice template: metadata: labels: app: apollo-adminservice name: apollo-adminservice spec: volumes: - name: configmap-volume configMap: name: apollo-adminservice-cm containers: - name: apollo-adminservice image: harbor.od.com/infra/apollo-adminservice:v1.5.1 ports: - containerPort: 8080 protocol: TCP volumeMounts: - name: configmap-volume mountPath: /apollo-adminservice/config terminationMessagePath: /dev/termination-log terminationMessagePolicy: File imagePullPolicy: IfNotPresent imagePullSecrets: - name: harbor restartPolicy: Always terminationGracePeriodSeconds: 30 securityContext: runAsUser: 0 schedulerName: default-scheduler strategy: type: RollingUpdate rollingUpdate: maxUnavailable: 1 maxSurge: 1 revisionHistoryLimit: 7 progressDeadlineSeconds: 600 [root@hdss7-200 apollo-adminservice]#应用正式环境-apollo-adminservice-资源配置清单
[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/cm.yaml [root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/apollo-adminservice/dp.yaml
5.启动Apollo，并查看portal配置
启动k8s-portal项目，浏览器访问http://portal.od.com/
1.系统参数
- Key
apollo.portal.envs
- Value
fat,pro
查询
- Value
fat,pro
保存
2.实战发布dubbo连接apollo配置中心到不同环境
1.apollo配置中心创建dubbo-demo-service服务项目
- 在TEST/PROD环境分别增加配置项并发布
进入配置页面
新增配置1-测试环境
然后发布即可。新增配置2-正式环境
发布即可。
2.apollo配置中心创建dubbo-demo-web服务项目
- 新建dubbo消费者项目
- 新增配置-测试环境
发布即可！
- 新增配置-正式环境
发布即可！
6.发布dubbo微服务到k8s集群
- 准备dubbo-demo-service和dubbo-demo-web的资源配置清单（各2套）
- 依次应用，分别发布至test和prod命名空间
- 使用dubbo-monitor查验
1.测试环境-dubbo-demo-service资源配置清单
[root@hdss7-200 ~]# cd /data/k8s-yaml/test/dubbo-demo-service/
[root@hdss7-200 dubbo-demo-service]# cat dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: test
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_210330_2120
ports:
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=fat -Dapollo.meta=http://config-test.od.com
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 dubbo-demo-service]# - 测试环境-应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-service/dp.yaml
deployment.extensions/dubbo-demo-service created
[root@hdss7-21 ~]# 2.测试环境-dubbo-demo-web资源配置清单
[root@hdss7-200 dubbo-demo-consumer]# pwd
/data/k8s-yaml/test/dubbo-demo-consumer
[root@hdss7-200 dubbo-demo-consumer]# cat dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: test
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-consumer:apollo_210330_2150
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=fat -Dapollo.meta=http://config-test.od.com
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 dubbo-demo-consumer]#
[root@hdss7-200 dubbo-demo-consumer]# cat svc.yaml
kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: test
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer
clusterIP: None
type: ClusterIP
sessionAffinity: None
[root@hdss7-200 dubbo-demo-consumer]# cat ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: test
spec:
rules:
- host: demo-test.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-demo-consumer
servicePort: 8080
[root@hdss7-200 dubbo-demo-consumer]# 3.测试环境-应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/dp.yaml
deployment.extensions/dubbo-demo-consumer created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/svc.yaml
service/dubbo-demo-consumer created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/test/dubbo-demo-consumer/ingress.yaml
ingress.extensions/dubbo-demo-consumer created
[root@hdss7-21 ~]# 4.测试环境-浏览器访问
http://demo-test.od.com/hello?name=test
5.正式环境-dubbo-demo-service资源配置清单
[root@hdss7-200 ~]# cat /data/k8s-yaml/prod/dubbo-demo-service/dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-service
namespace: prod
labels:
name: dubbo-demo-service
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-service
template:
metadata:
labels:
app: dubbo-demo-service
name: dubbo-demo-service
spec:
containers:
- name: dubbo-demo-service
image: harbor.od.com/app/dubbo-demo-service:apollo_210330_2120
ports:
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://config-prod.od.com
- name: JAR_BALL
value: dubbo-server.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 ~]# 6.正式环境-应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-service/dp.yaml
deployment.extensions/dubbo-demo-service created
[root@hdss7-21 ~]# 7.正式环境-dubbo-demo-web资源配置清单
[root@hdss7-200 dubbo-demo-consumer]# pwd
/data/k8s-yaml/prod/dubbo-demo-consumer
[root@hdss7-200 dubbo-demo-consumer]# cat dp.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: prod
labels:
name: dubbo-demo-consumer
spec:
replicas: 1
selector:
matchLabels:
name: dubbo-demo-consumer
template:
metadata:
labels:
app: dubbo-demo-consumer
name: dubbo-demo-consumer
spec:
containers:
- name: dubbo-demo-consumer
image: harbor.od.com/app/dubbo-demo-consumer:apollo_210330_2150
ports:
- containerPort: 8080
protocol: TCP
- containerPort: 20880
protocol: TCP
env:
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://config-prod.od.com
- name: JAR_BALL
value: dubbo-client.jar
imagePullPolicy: IfNotPresent
imagePullSecrets:
- name: harbor
restartPolicy: Always
terminationGracePeriodSeconds: 30
securityContext:
runAsUser: 0
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
revisionHistoryLimit: 7
progressDeadlineSeconds: 600
[root@hdss7-200 dubbo-demo-consumer]#
[root@hdss7-200 dubbo-demo-consumer]# cat svc.yaml
kind: Service
apiVersion: v1
metadata:
name: dubbo-demo-consumer
namespace: prod
spec:
ports:
- protocol: TCP
port: 8080
targetPort: 8080
selector:
app: dubbo-demo-consumer
clusterIP: None
type: ClusterIP
sessionAffinity: None
[root@hdss7-200 dubbo-demo-consumer]# cat ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: dubbo-demo-consumer
namespace: prod
spec:
rules:
- host: demo-prod.od.com
http:
paths:
- path: /
backend:
serviceName: dubbo-demo-consumer
servicePort: 8080
[root@hdss7-200 dubbo-demo-consumer]# 8.正式环境-应用资源配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/dp.yaml
deployment.extensions/dubbo-demo-consumer created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/svc.yaml
service/dubbo-demo-consumer created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/prod/dubbo-demo-consumer/ingress.yaml
ingress.extensions/dubbo-demo-consumer created
[root@hdss7-21 ~]# 9.正式环境-浏览器访问
http://demo-prod.od.com/hello?name=prod
7.知识点
env:
- name: JAR_BALL
value: dubbo-server.jar
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://config-test.od.com #这里用的是apollo-configservice的ingress地址
env:
- name: JAR_BALL
value: dubbo-server.jar
- name: C_OPTS
value: -Denv=pro -Dapollo.meta=http://apollo-configservice:8080 #这里用的是apollo-configservice的svc地址
[root@hdss7-22 ~]# kubectl get svc -n prod
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
apollo-configservice ClusterIP 192.168.225.202 <none> 8080/TCP 35h
[root@hdss7-22 ~]#
#要区分好环境8.实战演示项目提测-发版流程
修改dubbo-demo-web的源代码，提交至gitee,用commit id d153aae149af 进行jenkins发版，用相同的镜像发布到不同的环境。
代码：git@gitee.com:yodo1/dubbo-demo-web.git
分支：apollo
源代码：dubbo-client/src/main/java/com/od/dubbotest/action/HelloAction.java构建jenkins项目：dubbo-demo
记录镜像：harbor.od.com/app/dubbo-demo-consumer:d153aae149af_210331_2030
发布到K8S集群的测试环境：
- 1.修改dp.yaml或者修改k8s的dashbord的yaml文件
- 2.访问测试环境：http://demo-test.od.com/hello?name=test
发布到K8S集群的正式环境：
- 1.修改dp.yaml或者修改k8s的dashbord的yaml文件
- 2.访问正式环境：http://demo-prod.od.com/hello?name=prod
2.互联网公司技术部的日常
- 产品经理整理需求，需求评审，出产品原型
- 开发同学夜以继日的开发，提测
- 测试同学使用Jenkins持续集成，并发布至测试环境
- 验证功能，通过->待上线or打回->修改代码
- 提交发版申请，运维同学将测试后的包发往生产环境
- 无尽的BUG修复（笑cry）
文档更新时间: 2021-03-31 20:44 作者：xtyang
