One

主题

One2024-12-24
7940
41.5m
Get...

灰度发布 ​

将发行版发布到一部分用户或服务器的一种模式。这个想法是首先将更改部署到一小部分服务器，进行测试，然后将更改推广到其余服务器。一旦通过所有运行状况检查，当没有问题时，所有的客户将被路由到该应用程序的新版本，而旧版本将被删除。

img

img

nginx 权重模拟:

bash
upstreamwebservers{server192.168.1.223:8099weight=100;server192.168.1.222:8099weight=100;server192.168.1.221:8099weight=100;}server{listen8017;location/{proxy_passhttp:}}nginx-sreload

版本回滚 ​

  • 版本一直升级，则无需回滚。
  • 选择旧版本文件，进行发布。

前端后端项目发布 ​

1、前端项目

复制静态文件到nginx站点目录，nginx -s reload

bash
## 进入Web服务器的站点目录下## 下载包[root@master html]# curl -u admin:admin123 http:%Total%Received%XferdAverageSpeedTimeTimeTimeCurrentDloadUploadTotalSpentLeftSpeed100196k100196k0024.0M0--:--:----:--:----:--:--24.0M## 解压包[root@master html]# tar zxf anyops-devops-ui-1.1.1.tar.gz [root@master html]# lsanyops-devops-ui-1.1.1.tar.gzindex.htmlstatic## 触发nginx重载[root@master html]# nginx -s reload

tstmp_20230702072623

2、后端项目

  • 复制jar包到目标目录， 使用nohup java -jar 启动服务。
  • nohup java -jar app.jar >output 2>&1 &

tstmp_20230702072810

1、CI ​

拷贝Jenkins流水线 ​

  • 拷贝Jenkins作业devops6-maven-service到devops6-maven-service_CI

image-20230630124853726

保存后，点击参数化构建，会发现branchName的页面参数为空，我们先直接运行一次流水线看看效果：

运行一次流水线后，再次运行时，就会看到branchName正常了。

接下来我们就用devops6-maven-service_CI来测试。

image-20230630125131260

  • 我们再次运行下，看下效果

image-20230630125359978

可以看到，流水线运行成功。

image-20230630125435262

image-20230630125458519

可以看到nexus仓库里制品被上传成功了。

优化pipeline代码，去除制品库里CI字样 ​

image-20230630125913459

groovy
appName ="${JOB_NAME}".split('_')[0] repoName =appName.split('-')[0] appVersion ="${env.branchName}".split("-")[-1] targetDir="${appName}/${appVersion}"

再次运行测试效果：

image-20230630130045837

image-20230630130137478

符合预期。

新建Jenkins CD流水线 ​

  • 创建一个devops6-maven-service_CD作业，然后添加一些页面参数

image-20230630212933661

image-20230630212942420

image-20230630212950308

image-20230630212959078

点击参数化构建：

image-20230630212847814

创建一个devops6的视图 ​

image-20230630214923752

image-20230630214939599

image-20230630214955849

优化pipeline代码，nexus仓库的版本里要带上commitID ​

  • 之前仓库是这样的

image-20230630213211572

  • 先来手动获取下项目仓库的commitID

image-20230701062133819

bash
[root@Devops6 ~]#cd /opt/jenkinsagent/workspace/[root@Devops6 workspace]#lsday2-pipeline-demodevops6-gradle-servicedevops6-maven-servicedevops6-maven-service_CI@tmpdevops6-maven-testdevops6-npm-servicetest-mavenday2-pipeline-demo@tmpdevops6-gradle-service@tmpdevops6-maven-service_CIdevops6-maven-service@tmpdevops6-maven-test@tmpdevops6-npm-service@tmptest-maven@tmp[root@Devops6 workspace]#cd devops6-maven-service_CI[root@Devops6 devops6-maven-service_CI]#lsmvnwmvnw.cmdpom.xmlsonar-project.propertiessrctarget[root@Devops6 devops6-maven-service_CI]#git rev-parse HEAD #通过这个命令之可以获取仓库comitID的。b5cfb8eeee597edd752cb11f5daa9ac843fb9f97[root@Devops6 devops6-maven-service_CI]#

然后利用片段生成器生成代码：

image-20230701062623617

bash
shreturnStdout:true,script:'git rev-parse HEAD'

然后集成到piepeline代码里。

  • 我们想让这里的版本号也带上commitID

这里直接写代码：

groovy
appVersion ="${appVersion}-${env.commitID}"env.commitID =gitlab.GetCommitID()println("commitID:${env.commitID}")packageorg.devopsdefGetCommitID(){ID=sh returnStdout:true,script:"git rev-parse HEAD"ID=ID-"\n"returnID[0..7] }

image-20230630214612592

image-20230630214641609

image-20230630214733321

  • 在gitlab的devops6-maven-service里以main分支创建RELEASE-9.9.9分支

image-20230630214020268

  • 运行devops6-maven-service_CI流水线

image-20230630214129250

image-20230630214320025

image-20230630214335518

image-20230630214418763

image-20230630214452618

测试成功。

2、CD ​

下载制品

cd部分就不用再下载代码获取commitID了。

我们来使用gitlab api获取分支commit。

Step1:获取GitLab 分支CommitID ​

  • 打开gitlab api官方文档

https:--header 'PRIVATE-TOKEN:TLDgj3sz-cioyk6AfxZi'

创建Gitlab.groovy文件

groovy
packageorg.devopsdefHttpReq(method,apiUrl){response =sh returnStdout:true,script:"""curl --location --request ${method} \http:172.29.9.101:8076/api/v4/${apiUrl} \--header "PRIVATE-TOKEN:TLDgj3sz-cioyk6AfxZi""""response =readJSON text:response -"\n"returnresponse}

image-20230701070729445

但是，这里的gitlab token是明文的，因此需要在jenkins里配置个凭据。

image-20230701070836183

image-20230701070942488

image-20230701070956541

然后利用片段生成器来利用次token，生成代码：

image-20230701071159724

bash
withCredentials([string(credentialsId:'5782c77d-ce9d-44e5-b9ba-1ba2097fc31d',variable:'gitlabtoken')]) {}
groovy
packageorg.devopsdefHttpReq(method,apiUrl){withCredentials([string(credentialsId:'5782c77d-ce9d-44e5-b9ba-1ba2097fc31d',variable:'gitlabtoken')]) {response =sh returnStdout:true,script:"""curl --location --request ${method} \http:172.29.9.101:8076/api/v4/${apiUrl} \--header "PRIVATE-TOKEN:${gitlabtoken}""""}response =readJSON text:response -"\n"returnresponse}

image-20230701071409361

但是，存在一个问题，apiUrl里我们还需要知道ProjectID才行，这里继续查找gitlab api。

image-20230701073750499

image-20230701074248170

bash
http:curl--location'http:--header 'PRIVATE-TOKEN:TLDgj3sz-cioyk6AfxZi'

image-20230701074437455

groovy
packageorg.devopsdefHttpReq(method,apiUrl){withCredentials([string(credentialsId:'5782c77d-ce9d-44e5-b9ba-1ba2097fc31d',variable:'gitlabtoken')]) {response =sh returnStdout:true,script:"""curl --location --request ${method} \http:172.29.9.101:8076/api/v4/${apiUrl} \--header "PRIVATE-TOKEN:${gitlabtoken}""""}response =readJSON text:response -"\n"returnresponse}defGetProjectIDByName(projectName,groupName){apiUrl ="projects?search=${projectName}"response =HttpReq("GET",apiUrl)if(response !=[]){for(p inresponse) {if(p["namespace"]["name"] ==groupName){returnresponse[0]["id"]}}}}defGetBranchCommitID(projectID,branchName){apiUrl ="projects/${projectID}/repository/branches/${branchName}"response =HttpReq("GET",apiUrl)returnresponse.commit.short_id}

image-20230701082636305

groovy
@Library("devops06@main") _ defmygit =neworg.devops.Gitlab()pipeline{agent {label "build"}options {skipDefaultCheckout true}stages{stage("GetArtifact"){steps{script{env.projectName ="${JOB_NAME}".split('_')[0] env.groupName ="${env.projectName}".split('-')[0] projectID =mygit.GetProjectIDByName(env.projectName,env.groupName)commitID =mygit.GetBranchCommitID("${projectID}","${env.branchName}")println(commitID)currentBuild.displayName ="第${BUILD_NUMBER}次构建-${commitID}"currentBuild.description ="构建分支名称：${env.branchName}"}}}}}

Gitlab.groovy代码

groovy
packageorg.devopsdefHttpReq(method,apiUrl){withCredentials([string(credentialsId:'5782c77d-ce9d-44e5-b9ba-1ba2097fc31d',variable:'gitlabtoken')]) {response =sh returnStdout:true,script:"""curl --location --request ${method} \http:--header "PRIVATE-TOKEN:${gitlabtoken}""""}response =readJSON text:response -"\n"returnresponse}defGetProjectIDByName(projectName,groupName){apiUrl ="projects?search=${projectName}"response =HttpReq("GET",apiUrl)if(response !=[]){for(p inresponse) {if(p["namespace"]["name"] ==groupName){returnresponse[0]["id"]}}}}defGetBranchCommitID(projectID,branchName){apiUrl ="projects/${projectID}/repository/branches/${branchName}"response =HttpReq("GET",apiUrl)returnresponse.commit.short_id}

image-20230701082755595

image-20230701082806637

运行流水线：

image-20230701082824215

image-20230701082839197

测试成功。😘

Step2:下载制品 ​

  • nexus仓库制品地址如下
bash
http:defmygit =neworg.devops.Gitlab()pipeline{agent {label "build"}options {skipDefaultCheckout true}stages{stage("GetArtifact"){steps{script{env.projectName ="${JOB_NAME}".split('_')[0] env.groupName ="${env.projectName}".split('-')[0] projectID =mygit.GetProjectIDByName(env.projectName,env.groupName)commitID =mygit.GetBranchCommitID("${projectID}","${env.branchName}")println(commitID)appVersion ="${env.branchName}".split("-")[-1] println(appVersion)currentBuild.displayName ="第${BUILD_NUMBER}次构建-${commitID}"currentBuild.description ="构建分支名称：${env.branchName}"repoUrl ="http:artifactName ="${env.projectName}-${appVersion}-${commitID}.jar"artifactUrl ="${repoUrl}/${env.projectName}/${appVersion}-${commitID}/${artifactName}"sh "wget --no-verbose ${artifactUrl} &&ls -l"}}}}}

image-20230701090006744

  • 运行观察效果

image-20230701085928107

image-20230701085915156

下载制品成功。

我们再运行一次流水线：

image-20230701090110353

会看到多了一个包，

最后我们发布完，会把它清掉的：

这里先手动给清掉。

image-20230701090216170

Step3:发布 ​

准备2台linux机器 ​

bash
devops-deploy1-172.29.9.110devops-deploy2-172.29.9.111
  • 给这2台机器装好java-11
bash
yuminstall-yjava-11-openjdk.x86_64

devops机器安装ansible环境 ​

bash
yuminstallepel-release-yyuminstallansible-y
  • 编辑下ansible的主机管理文件：
bash
[root@Devops6 ~]#vim /etc/ansible/hosts172.29.9.110172.29.9.111
  • 给ansible机器到2个节点做个免密
bash
ssh-keygenssh-copy-id-i~/.ssh/id_rsa.pubroot@172.29.9.110ssh-copy-id-i~/.ssh/id_rsa.pubroot@172.29.9.111
  • 查看当前主机是否在线：
bash
[root@Devops6 ~]#ansible all -m ping -u root

image-20230701191542756

构建一次devops6-maven-service_CD，下载制品 ​

image-20230701191814077

image-20230701191928480

我们先来手动发布一次，再集成到CD流水线里 ​

  • 拷贝制品到deploy1
bash
[root@Devops6 devops6-maven-service_CD]#ansible 172.29.9.110 -m copy -a "src=devops6-maven-service-9.9.9-b5cfb8ee.jar dest=/opt/devops6-maven-service-9.9.9-b5cfb8ee.jar"

image-20230701201005371

image-20230701201043602

  • 启动服务：

image-20230701201734093

image-20230701201803075

  • 用准备好的服务启动脚本来启动/停止java服务

服务启动脚本：service.sh（原始脚本如下）

bash
#!/bin/bash# sh service.sh anyops-devops-service 1.1.1 8091 startAPPNAME=NULLVERSION=NULLPORT=NULLstart(){port_result=`netstat -anlpt | grep "${PORT}" || echo false`if[[$port_result =="false"]];thennohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&elsestopsleep5nohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&fi}stop(){pid=`netstat -anlpt | grep "${PORT}" | awk '{print $NF}' | awk -F '/' '{print $1}' | head -1`kill-15$pid}check(){proc_result=`ps aux | grep java | grep "${APPNAME}" | grep -v grep || echo false`port_result=`netstat -anlpt | grep "${PORT}" || echo false`url_result=`curl -s http://localhost:${PORT} || echo false `if[[$proc_result =="false"||$port_result =="false"||$url_result =="false"]];thenecho"server not running"elseecho"ok"fi}case$1instart)startsleep5check;;stop)stopsleep5check;;restart)stopsleep5startsleep5check;;check)check;;*)echo"sh service.sh {start|stop|restart|check}";;esac

参数写入后脚本如下

bash
#!/bin/bash# sh service.sh anyops-devops-service 1.1.1 8091 startAPPNAME=devops6-maven-serviceVERSION=9.9.9-b5cfb8eePORT=8080start(){port_result=`netstat -anlpt | grep "${PORT}" || echo false`if[[$port_result =="false"]];thennohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&elsestopsleep5nohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&fi}stop(){pid=`netstat -anlpt | grep "${PORT}" | awk '{print $NF}' | awk -F '/' '{print $1}' | head -1`kill-15$pid}check(){proc_result=`ps aux | grep java | grep "${APPNAME}" | grep -v grep || echo false`port_result=`netstat -anlpt | grep "${PORT}" || echo false`url_result=`curl -s http://localhost:${PORT} || echo false `if[[$proc_result =="false"||$port_result =="false"||$url_result =="false"]];thenecho"server not running"elseecho"ok"fi}case$1instart)startsleep5check;;stop)stopsleep5check;;restart)stopsleep5startsleep5check;;check)check;;*)echo"sh service.sh {start|stop|restart|check}";;esac
  • 将service.sh脚本拷贝到测试节点：
bash
[root@Devops6 devops6-maven-service_CD]#ansible 172.29.9.110 -m copy -a "src=service.sh dest=/opt/service.sh"172.29.9.110|CHANGED=>{"ansible_facts":{"discovered_interpreter_python":"/usr/bin/python"},"changed":true,"checksum":"666b4746afbb9fa684f79a89102715906417c848","dest":"/opt/service.sh","gid":0,"group":"root","md5sum":"22868400cb2784f7c7bcf63f38a977fe","mode":"0644","owner":"root","size":1367,"src":"/root/.ansible/tmp/ansible-tmp-1688219597.85-41901-255991227715874/source","state":"file","uid":0}

然后启动程序：

给予脚本执行权限：

bash
[root@devops-deploy1 opt]#lltotal17284-rw-r--r--1rootroot17690913Jul120:12devops6-maven-service-9.9.9-b5cfb8ee.jar-rw-r--r--1rootroot1367Jul121:53service.sh[root@devops-deploy1 opt]#chmod +x service.sh

启动程序：

bash
[root@devops-deploy1 opt]#sh service.sh startok[root@devops-deploy1 opt]#ps -aux|grepjavaroot762637.48.73202716163300pts/0Sl21:550:04java-jar-Dserver.port=8080devops6-maven-service-9.9.9-b5cfb8ee.jarroot76740.00.0112708972pts/0R+21:550:00grep--color=autojava[root@devops-deploy1 opt]#

image-20230701215558442

开始集成 ​

  • 最终代码如下

image-20230702075024487

Deploy.groovy文件

groovy
packageorg.devopsdefAnsibleRollBack(){sh """# 停止服务ansible "${env.deployHosts}"-m shell -a "cd ${env.targetDir}/${env.projectName} ;source /etc/profile &&sh service.sh stop"-u rootsleep 300# 清理和创建发布目录ansible "${env.deployHosts}"-m shell -a "rm -fr ${env.targetDir}/${env.projectName}packagecom.example.demo;importorg.springframework.stereotype.Controller;importorg.springframework.web.bind.annotation.ModelAttribute;importorg.springframework.web.bind.annotation.RequestMapping;importorg.springframework.web.bind.annotation.RequestParam;importorg.springframework.web.bind.annotation.ResponseBody;@ControllerpublicclassBasicController{@RequestMapping("/hello")@ResponseBodypublicString hello(@RequestParam(name="name",defaultValue="xyy") String name) {return"Hello RELEASE-10.1.0 "+name;}}

然后打包，运行，观察效果：

image-20230702110042099

image-20230702110104946

  • 最后，将devops6-maven-service的RELEASE-9.9.9/代码合并到main分支。

  • 打上tag

image-20230702112212540

image-20230702112219976

Step4:回滚 ​

image-20230701215632309

推荐第一种。

第二种方法会存在很多逻辑问题的。

  • 回滚代码见上述文件，这里测试下效果

1、直接发布版本方式来回滚

先运行CI流水线

image-20230702075835437

CI pipeline运行成功：

image-20230702075903928

image-20230702075919711

再运行CD：

image-20230702075951461

观察效果：

image-20230702080049756

image-20230702080104522

可以看到发布老版本程序成功。

2、使用回滚代码

注意：如果要回滚时，就需要跳过发布阶段，否则会有问题的，因此这里我给发布阶段加了一个判断选项。

发布1.1.1

image-20230702092448462

发布9.9.9

image-20230702092637660

回滚到1.1.1：

image-20230702092740901

image-20230702092800153

image-20230702092900471

符合预期。😘

扩展：参数动态获取实践 ​

  • 需要安装active choices插件重启Jenkins服务器后再操作。

tstmp_20230702100125

根据不同的环境带出不同的机器 ​

  • 效果

image-20230702101549409

image-20230702101941216

  • envName参数设置

image-20230702101647504

groovy
return["dev","uat","stag","prod"]
  • deployHosts参数设置

image-20230702101716888

groovy
if(envName.equals("dev")){return["172.29.9.110,172.29.9.111"]} elseif(envName.equals("uat")){return["172.29.9.120,172.29.9.121"]}

image-20230702101741319

⚠️ 注意：记得删除前面定义好的envName和deployHosts选项参数。

  • 运行测试

image-20230702101549409

image-20230702101525323

image-20230702101609022

符合预期。😘

根据不同发布工具，动态展示主机参数 ​

这个就不做演示了，和上面这个实践有冲突。

tstmp_20230702102119

  • 定义发布工具参数

tstmp_20230702102143

groovy
return["ansible","saltstack"]

单选类型

tstmp_20230702102211

  • 定义发布主机

tstmp_20230702102226

选择关联的参数，多个参数用逗号分割

tstmp_20230702102300

3、代码汇总 ​

  • 本次实验代码

链接：https:jenkins:2.346.3-2-lts-jdk11sonarqube:9.9.0-communitynexus3:3.53.0

image-20230702194106832

这2条流水线都是测试ok的。(以后就一直用这2条流水线来测试devops了)

image-20230702194138050

image-20230702194213612

image-20230702194227921

gitlab仓库devops6-maven-service：RELEASE-9.9.9和main分支都是一样的代码。

image-20230702194255650

jenkins共享库代码：

image-20230702194401287

image-20230702194944945

service.sh

bash
#!/bin/bash# sh service.sh anyops-devops-service 1.1.1 8091 startAPPNAME=NULLVERSION=NULLPORT=NULLstart(){port_result=`netstat -anlpt | grep "${PORT}" || echo false`if[[$port_result =="false"]];thennohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&elsestopsleep5nohupjava-jar-Dserver.port=${PORT}${APPNAME}-${VERSION}.jar>${APPNAME}.log.txt2>&1&fi}stop(){pid=`netstat -anlpt | grep "${PORT}" | awk '{print $NF}' | awk -F '/' '{print $1}' | head -1`kill-15$pid}check(){proc_result=`ps aux | grep java | grep "${APPNAME}" | grep -v grep || echo false`port_result=`netstat -anlpt | grep "${PORT}" || echo false`url_result=`curl -s http://localhost:${PORT} || echo false `if[[$proc_result =="false"||$port_result =="false"||$url_result =="false"]];thenecho"server not running"elseecho"ok"fi}case$1instart)startsleep5check;;stop)stopsleep5check;;restart)stopsleep5startsleep5check;;check)check;;*)echo"sh service.sh {start|stop|restart|check}";;esac

Jenkinsfile

groovy
@Library("devops06@main") _defcheckout =neworg.devops.CheckOut()defbuild =neworg.devops.Build()defsonar =neworg.devops.Sonar()defartifact =neworg.devops.Artifact()env.branchName ="${env.branchName}"-"origin/"println(env.branchName)pipeline {agent {label "build"}options {skipDefaultCheckout true} stages{stage("CheckOut"){steps{script{checkout.CheckOut()env.commitID =checkout.GetCommitID()println("commitID:${env.commitID}")currentBuild.displayName ="第${BUILD_NUMBER}次构建-${env.commitID}"currentBuild.description ="构建分支名称：${env.branchName}"}}}stage("Build"){steps{script{build.Build()}}} stage("CodeScan"){when {environment name:'skipSonar',value:'false'}steps{script{sonar.SonarScannerByPlugin() }}}stage("PushArtifact"){steps{script{appName ="${JOB_NAME}".split('_')[0] repoName =appName.split('-')[0] appVersion ="${env.branchName}".split("-")[-1] appVersion ="${appVersion}-${env.commitID}"targetDir="${appName}/${appVersion}"POM=readMavenPom file:'pom.xml'env.artifactId ="${POM.artifactId}"env.packaging ="${POM.packaging}"env.groupId ="${POM.groupId}"env.art_version ="${POM.version}"sourcePkgName ="${env.artifactId}-${env.art_version}.${env.packaging}"pkgPath ="target"targetPkgName ="${appName}-${appVersion}.${env.packaging}"artifact.PushNexusArtifact(repoName,targetDir,pkgPath,sourcePkgName,targetPkgName)}}} }}

cd.jenkinsfile

groovy
@Library("devops06@main") _ defmygit =neworg.devops.Gitlab()defmydeploy =neworg.devops.Deploy()pipeline{agent {label "build"}options {skipDefaultCheckout true}stages{stage("GetArtifact"){steps{script{env.projectName ="${JOB_NAME}".split('_')[0] env.groupName ="${env.projectName}".split('-')[0] projectID =mygit.GetProjectIDByName(env.projectName,env.groupName)commitID =mygit.GetBranchCommitID("${projectID}","${env.branchName}")println(commitID)appVersion ="${env.branchName}".split("-")[-1] println(appVersion)currentBuild.displayName ="第${BUILD_NUMBER}次构建-${commitID}"currentBuild.description ="构建分支名称：${env.branchName}"repoUrl ="http:env.artifactName ="${env.projectName}-${appVersion}-${commitID}.jar"artifactUrl ="${repoUrl}/${env.projectName}/${appVersion}-${commitID}/${env.artifactName}"sh "wget --no-verbose ${artifactUrl} &&ls -l"env.releaseVersion ="${appVersion}-${commitID}"}}}stage("Deploy"){when {environment name:'skipDeploy',value:'false'}steps{script{mydeploy.AnsibleDeploy()}}}stage("RollBack"){input {message "是否进行回滚？"ok "Yes"submitter ""parameters {choice choices:['NO','YES'],name:'OPS'}}steps {echo "OPS ${OPS},doing......."script{if("${OPS}"=="YES"){mydeploy.AnsibleRollBack()}}}} }}

Deploy.groovy

groovy
packageorg.devopsdefAnsibleRollBack(){sh """# 停止服务ansible "${env.deployHosts}"-m shell -a "cd ${env.targetDir}/${env.projectName} ;source /etc/profile &&sh service.sh stop"-u rootsleep 20# 清理和创建发布目录ansible "${env.deployHosts}"-m shell -a "rm -fr ${env.targetDir}/${env.projectName}