环境：

Jenkins v2.516.2 插件Kubernetes plugin 、GitLab Plugin

Harbor  v2.13.2-436918f8

gitlab v18.3

k8s AWS eks V1.33

Jenkins agent pod 容器镜像：jenkins/inbound-agent:3327.v868139a_d00e0-7、docker:24-dind、gcr.io/cloud-builders/kubectl

部署略

一、创建harbor仓库

二、Jenkins配置

1、创建访问harbor和gitlab的凭证

Manage Jenkins-----Credentials中配置访问harbor和Gitlab的Credentials

需要三个凭证、harbor拉取上传镜像的用户密码、Gitlab提交代码的用户密码和项目访问的API Token

配置harbor的凭证 ID要与流水线脚本中credentialsId:的值一致

配置Gitlab API Token

2、创建流水线中调用执行任务的Pod

Manage Jenkins-----Clouds中配置kubernets

3、Manage Jenkins-----Clouds中配置Pod Templates 创建jnlp、dind、kubectl三个容器

jenkins agent容器jnlp

点击高级勾选以最高权限运行，三个容器都需要勾选分配伪终端和以最高权限运行

点击添加容器

dind容器负责docker镜像构建、上传

kubectl容器负责运行从git仓库拉取的yaml构建pod

4、配置Gitlab Connection

Manage Jenkins  System中配置Jenkins Location和Gitlab connections，URL均为登录Web UI的URL

Credentials选择第一步创建的API Token

5、创建流水线

点选左上角Jenkins图标点击创建item,输入流水线名称后，点选Pipeline

三、Gitlab配置

在局域网内使用jenkins,也就是jenkins域名解析为私有IP,gitlab配置webhook指向jenkins url需要做的额外配置

如果在jenkins url解析成是私有ip，也就是局域网内使用，需要做如下配置，否则Gitlab配置webhook会报错

以管理员用户登录后点击左下角管理员按钮

点击网络

出站点请求中勾选这几项

配置Gitlab项目

在项目中点选设置--访问令牌 为jenkins创建访问令牌

点选webhook，添加创建jenkins流水线时页面显示的webhook和生成的token，注意URI中如果有特殊字符必须转换为%编码，触发来源只勾选推送事件

jenkinsfile文件内容，脚本上传至git仓库根目录

    node('jenkins-jenkins-agent') {
        stage('Clone') {container('jnlp'){
            echo "1.Clone Stage"
            git url: "https://gitlab-test.xxxx.com/root/pipeline.git",
                branch: "main"
            script {
                build_tag = sh(returnStdout: true, script: 'git rev-parse --short HEAD').trim()
            }
        }}
        stage('Test') {
            echo "2.Test Stage"
        }
        stage('Build') {container('dind'){
            echo "3.Build Docker Image Stage"
            sh "docker build -t harbor-test.xxxx.com/jenkins-demo/jenkins-demo:${build_tag} ."
        }}
        stage('Push')  {container('dind'){
            echo "4.Push Docker Image Stage"
            withCredentials([usernamePassword(credentialsId: 'harbor', passwordVariable: 'dockerHubPassword', usernameVariable: 'dockerHubUser')]) {
                sh "docker login harbor-test.xxxx.com -u ${dockerHubUser} -p ${dockerHubPassword}"
                sh "docker push harbor-test.xxxx.com/jenkins-demo/jenkins-demo:${build_tag}"
            }
        }}
        stage('Deploy to dev') {container('kubectl'){
            echo "5. Deploy DEV"
            sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s-dev-harbor.yaml"
            sh "sed -i 's/<BRANCH_NAME>/${env.BRANCH_NAME}/' k8s-dev-harbor.yaml"
            sh "kubectl apply -f k8s-dev-harbor.yaml --validate=false"
        }}
        stage('Promote to qa') {container('kubectl'){
            def userInput = input(
                id: 'userInput',
                message: 'Promote to qa?',
                parameters: [
                    [$class: 'ChoiceParameterDefinition', choices: "YES\nNO", name: 'Env']
                ]
            )
            echo "This is a deploy step to ${userInput}"
            if (userInput == "YES") {
                sh "sed -i 's/<BUILD_TAG>/${build_tag}/' k8s-qa-harbor.yaml"
                sh "sed -i 's/<BRANCH_NAME>/${env.BRANCH_NAME}/' k8s-qa-harbor.yaml"
                sh "kubectl apply -f k8s-qa-harbor.yaml --validate=false"
            }
        }}
        stage('Promote to pro') {
            def userInput = input(
                id: 'userInput',
                message: 'Promote to pro?',
                parameters: [
                    [$class: 'ChoiceParameterDefinition', choices: "YES\nNO", name: 'Env']
                ]
            )
            echo "This is a deploy step to ${userInput}"
        }
    }

Git仓库代码下载地址 https://pan.baidu.com/s/1xbW-zUf23tu0YSV-5c1dew?pwd=lmzf

k8s中需要创建devlopment qatest命名空间

kubectl create ns devlopment
kubectl create ns qatest

还需要为jenkins命名空间的jenkins账户角色创建对devlopment和qatest命名空间的deploy和svc的访问权限

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: devlopment
  name: jenkins-role
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins-rolebinding
  namespace: devlopment
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins
roleRef:
  kind: Role
  name: jenkins-role
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: qatest
  name: jenkins-role
rules:
- apiGroups: ["apps"]
  resources: ["deployments"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: [""]
  resources: ["services"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: jenkins-rolebinding
  namespace: qatest
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: jenkins
roleRef:
  kind: Role
  name: jenkins-role
  apiGroup: rbac.authorization.k8s.io

最终效果是用户向仓库提交推送代码后，自动完成开发环境创建pod，该pod是基于仓库内容通过dockerfile打包生成并上传到harbor，kubectl容器再通过仓库中k8s的yaml文件从harbor下载并部署在k8s中，测试环境需要在jenkins流水线中promoto点击yes继续