spring cloud+maven+rancher2+Kubernetes(k8s)+git+jenkins+pipeline+docker实现打包自动部署

会有一股神秘感。 提交于 2020-01-07 02:06:12

【推荐】2019 Java 开发者跳槽指南.pdf(吐血整理) >>>

环境准备:

rancher2.3管理的k8s环境(自己搭)

jenkins(可使用docker搭建,也可直接安装,如果k8s的不是特别熟悉不建议使用helm安装jenkins,很多配置不方便改。我是用的docker搭建的,但要记录挂载docker命令。建议直接在主机上安装,可避免发布是的ssh免key的问题

阿里镜像仓库(自己可注册阿里云账号就有了免费的)

maven的本地nexus代理服务器(方便公司内部的jar管理)

 

1.maven打包并docker编译发布我使用的是:

<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>1.2.0</version>

此插件已有代替:dockerfile-maven-plugin,不过我没用这个,因为我本机开发机并没有装docker客户端之类的软件,而这个好像是要求本机至少得有docker客户端环境,还得配置docker环境变量。所以我没有使用。这样本机也可以直接使用maven打包docker build:push代码到阿里仓库。方便开发环境应用升级。

使用此docker-maven-plugin发现一个bug:

由于我拿的是docker安装的jenkins,启动容器前没有挂载docker命令,而后我又依赖此容器,重新生成了镜像,后来也把阿里镜像仓库的登陆认证信息给加入到用户根目录下(就是docker的config.json放到/root/.docker/下,不知道百度)后,导致docker-maven-plugin插件里的serverId与此config.json冲突,无法把镜像推送到阿里镜像仓库。

也调了我2天,逼的我拿源码来研究了都。最后发现问题所在。

解决方法:

如何要直接使用docker命令时,直接使用pipeline指定用户也密码。然后我是用pipeline部署的。

 

2.首先在项目根目录(注意不是maven的模块目录)下放置jenkinsfile如图:

jenkinsfile内容参照:

properties([
	parameters([
		string(name: 'PORT', defaultValue: '1111', description: '程序运行端口'),
		choice(name: 'ACTIVE_TYPE', choices: ['local', 'dev', 'test', 'prod'], description: '程序打包环境'),
		choice(name: 'ENV_TYPE', choices: ['offline', 'online'], description: '还是线下、线上环境'),
		booleanParam(name: 'All_COMPILE', defaultValue: false, description: '是否需要重新编译全模块'),
		booleanParam(name: 'DEPLOYMENT', defaultValue: true, description: '是否部署'),
		booleanParam(name: 'ON_PINPOINT', defaultValue: false, description: '是否添加Pinpoint监控'),
		booleanParam(name: 'ON_PROMETHEUS', defaultValue: false, description: '是否添加Prometheus监控'),
		string(name: 'EMAIL', defaultValue: '******@tintinhealth.com', description: '打包结果通知')

	])
])
node {

	stage('Prepare') {


		echo "1.Prepare Stage"

		def MVNHOME = tool 'maven-3.6.3'
		// 再把变量加入到环境变量中
		//env.PATH = "${jdk77}/bin:${MVNHOME}/bin:${env.PATH}"
		env.PATH = "${MVNHOME}/bin:${env.PATH}"
		MVNCONFIG= '/var/jenkins_home/maven/settings.xml'


		//echo "UUID=${UUID.randomUUID().toString()}"

		checkout scm

		//需要处理的项目多项目时先进入子项目

		projectwk = "."

		mainpom = readMavenPom file: 'pom.xml'

		repostory = "${mainpom.properties['docker.repostory']}"

		//存在多个模块时,选择其中一个进行编译

		//        if(mainpom.modules.size() > 0 ) {

		//          echo "项目拥有模块==${mainpom.modules}"

		//         timeout(time: 10, unit: 'MINUTES') {

		//            def selproj = input message: '请选择需要处理的项目', parameters: [choice(choices: mainpom.modules, description: '请选择需要处理的项目', name: 'selproj')] //, submitterParameter: 'project'

		//            projectwk = selproj

		//           echo "选择项目=${projectwk}"

		//          }



		//        }
		projectwk="${JOB_NAME}"

		dir("${projectwk}") {

			pom = readMavenPom file: 'pom.xml'

			echo "group: ${pom.groupId}, artifactId: ${pom.artifactId}, version: ${pom.version} ,description: ${pom.description}"

			artifactId = "${pom.artifactId}"

			version = "${pom.version}"

			description = "${pom.description}"

		}
		if(version == 'null'){
			version = "${mainpom.version}"
			echo "使用父version:${version}"
		}

		script {

			GIT_TAG = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()

			echo "GIT_TAG== ${GIT_TAG}"

		}

		image = "registry.cn-hangzhou.aliyuncs.com/ddyk/${artifactId}:${version}"

		//    if (params.ENV_TYPE == 'offline' || params.ENV_TYPE == null) {

		//sh "sed -i 's#39.95.40.97:5000#10.3.80.50:5000#g' pom.xml"

		//      image = "registry.cn-hangzhou.aliyuncs.com/ddyk/${artifactId}:${version}"

		//     }

	}

	if(params.All_COMPILE){

		if(mainpom.modules.size() > 0 ) {

			stage('编译总项目') {

				sh "mvn -s ${MVNCONFIG} -DskipTests clean install"

			}

		}
	}

	dir("${projectwk}") {

		stage('编译模块') {

			echo "2.编译模块 ${artifactId}"

			def jarparam=''

			def pinname = artifactId

			if( pinname.length() > 23) {

				pinname = artifactId.substring(0,23)

			}

			//添加pinpoint

			if(params.ON_PINPOINT) {

				jarparam = '"-javaagent:/app/pinpoint-agent/pinpoint-bootstrap-1.8.0.jar","-Dpinpoint.agentId={pinname}", "-Dpinpoint.applicationName={pinname}",'

			}

			//添加prometheus

			if(params.ON_PROMETHEUS) {

				jarparam = jarparam + '"-javaagent:/app/prometheus/jmx_prometheus_javaagent-0.11.0.jar=1234:/app/prometheus/jmx.yaml",'

			}

			sh "sed -i 's#{jarparam}#${jarparam}#g;s#{port}#${params.PORT}#g' Dockerfile"

			sh "sed -i 's#{description}#${description}#g;s#{artifactId}#${artifactId}#g;s#{version}#${version}#g;s#{active}#${params.ACTIVE_TYPE}#g;s#{pinname}#${pinname}#g' Dockerfile"

			sh "sed -i 's#{artifactId}#${artifactId}#g;s#{version}#${version}#g;s#{port}#${params.PORT}#g;s#{image}#${image}#g' Deployment.yaml"

			sh "mvn -s ${MVNCONFIG} -DskipTests clean package"

			stash includes: 'target/*.jar', name: 'app'

		}

		stage('Docker打包') {

			echo "3.Docker打包"

			unstash 'app'

			sh "mvn -s ${MVNCONFIG}  docker:build"

		}

		stage('推送镜像') {

			echo "4.Push Docker Image Stage"

			sh "mvn -s ${MVNCONFIG}  docker:push"

		}

		//        timeout(time: 10, unit: 'MINUTES') {

		//            input '确认要部署吗?'

		//         }

		if(params.DEPLOYMENT){
			stage('发布') {


				if (params.ENV_TYPE == 'offline' || params.ENV_TYPE == null) {

					sshagent(credentials: ['deploy_ssh_key_dev']) {

						sh "scp -r Deployment.yaml root@192.168.1.108:/home/sam/devops/kubernetes/manifests/workTools/ddyk/Deployment-${artifactId}.yaml"

						sh "ssh root@192.168.1.108 'kubectl apply -f /home/sam/devops/kubernetes/manifests/workTools/ddyk/Deployment-${artifactId}.yaml && kubectl set env deploy/${artifactId} DEPLOY_DATE=${env.BUILD_ID}'"

					}

				} else {

					sshagent(credentials: ['deploy_ssh_key_238']) {

						sh "scp -P 22 -r Deployment.yaml root@192.168.1.108:/home/sam/devops/kubernetes/manifests/workTools/ddyk/Deployment-${artifactId}.yaml"

						sh "ssh -p 22 root@192.168.1.108 'kubectl apply -f /home/sam/devops/kubernetes/manifests/workTools/ddyk/Deployment-${artifactId}.yaml && kubectl set env deploy/${artifactId} DEPLOY_DATE=${env.BUILD_ID}'"

					}

				}

				echo "发布完成"

			}
		}

	}

	stage('通知负责人'){

		//    emailext body: "构建项目:${artifactId}\r\n构建完成", subject: '构建结果通知【成功】', to: "${EMAIL}"

		//        echo "构建项目:${description}\r\n构建完成"
		echo "构建项目:${artifactId}\r\n构建完成"

	}

}

以上内容可自己根据需要更改删除。

3.maven模块下放如下文件:

Dockerfile内容参照说明:

FROM openjdk:8-alpine

MAINTAINER ddyk gsj "*****@tintinhealth.com"
ENV WORK_PATH /app
#ENV APP_NAME @project.build.finalName@.@project.packaging@

EXPOSE 9108
#统一时区
RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai  /etc/localtime

COPY /app/message-service*.jar $WORK_PATH/app.jar

WORKDIR $WORK_PATH

ENTRYPOINT ["java", "-Xmx512m","-Dspring.profiles.active={active}","-Dserver.port={port}",{jarparam} "-jar", "/app/app.jar"]

Deployment.yaml内容参照:

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {artifactId}
  namespace: default
  labels:
    app: {artifactId}
    version: {version}
spec:
  selector:
    matchLabels:
      app: {artifactId}
  replicas: 1
  template:
    metadata:
      labels:
        app: {artifactId}
      annotations:
        prometheus.io.jmx: "true"
        prometheus.io.jmx.port: "1234"
    spec:
      hostAliases:
        - ip: "172.20.246.2"
          hostnames:
            - "node-1"
        - ip: "172.20.246.3"
          hostnames:
            - "node-2"
      containers:
        - name: {artifactId}
          image: {image}
          # IfNotPresent\Always
          imagePullPolicy: Always
          ports:
            - name: prometheusjmx
              containerPort: 1234
          livenessProbe: #kubernetes认为该pod是存活的,不存活则需要重启
            httpGet:
              path: /actuator/health
              port: {port}
              scheme: HTTP
            initialDelaySeconds: 60 ## 设置为系统完全启动起来所需的最大时间+若干秒
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 5
          readinessProbe: #kubernetes认为该pod是启动成功的
            httpGet:
              path: /actuator/health
              port: {port}
              scheme: HTTP
            initialDelaySeconds: 40 ## 设置为系统完全启动起来所需的最少时间
            timeoutSeconds: 5
            successThreshold: 1
            failureThreshold: 5
          env:
            - name: register-eureka
              value: "register-eureka.default.svc.cluster.local"
            - name: register-eureka-replica
              value: "register-eureka-replica.default.svc.cluster.local"
          resources:
            # 5%的CPU时间和700MiB的内存
            requests:
              #            cpu: 50m
              memory: 250Mi
            # 最多允许它使用
            limits:
              #            cpu: 100m
              memory: 1000Mi
          # 指定在容器中挂载路径
          volumeMounts:
            - name: logs-volume
              mountPath: /logs
            - name: host-time
              mountPath: /etc/localtime
              readOnly: true
            - name: host-timezone
              mountPath: /etc/timezone
              readOnly: true
#            - name: pinpoint-config
#              mountPath: /app/pinpoint-agent/pinpoint.config
      imagePullSecrets:
      - name: registrykey-m2-1
      volumes:
        - name: logs-volume
          hostPath:
            # 宿主机上的目录
            path: /logs
        - name: host-time
          hostPath:
            path: /etc/localtime
        - name: host-timezone
          hostPath:
            path: /usr/share/zoneinfo/Asia/Shanghai
#
        # 运行在指定标签的节点,前提是先给节点打标  kubectl label nodes 192.168.0.113 edgenode=flow
#      nodeSelector:
#        edgenode: flow
---
apiVersion: v1
kind: Service
metadata:
  name: {artifactId}
  namespace: default
  labels:
    app: {artifactId}
    version: {version}
spec:
  selector:
    app: {artifactId}
#  type: NodePort
  ports:
    - name: tcp-{port}-{port}
      protocol: TCP
      port: {port}
      targetPort: {port}
#      nodePort: 31111

注意:prometheus我还没用上,你们可以自行修改部署文件调试。

关于register-eureka的部署:服务名与register-eureka一样。

 

注意事项:jenkins中建的项目名要和maven的模块名一样,不然需要修改jenkinsfile文件的部署相关的脚本 。

有不懂的欢迎咨询,其实这套部署主要就是这些个脚本文件。

对了,自动部署(git提交代码后自动触发)只能实现在开发环境,因为毕竟程序没有能通知什么时候发送正式环境(当前也可以根据版本判断加release与开发版区别),可自己看我前一篇文件。

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!