본문 바로가기
DevOps

Istio

by 이강복 2023. 11. 13.

Microservice Architecture

Service Mesh

 
서비스매쉬는 보통 sidecar 형태의 container로 구성됩니다.
ex) 하나의 Pod에는 nginx 서버 + istio_proxy(init container) 구성됨

 

실무자를 위한 서비스 메시 - 지금 서비스 메시가 의미 있는 이유 | 인사이트리포트 | 삼성SDS

 

기존 쿠버네티스와 서비스 메시가 적용된 쿠버네티스의 트래픽 관리

 

Prod Deployment Pod 가 2개이고 Canary Deployment Pod 가 하나일 경우 (Label은 같음) 서비스에서 파드로 가는 트래픽은 동일하다. (prod = 66%, canary = 33%)

하지만 서비스 매쉬를 사용할 경우 그 트래픽을 조절할 수 있다.

 

Canary 배포

Kubernetes의 다양한 배포방식 (3) Canary 배포

Canary 배포는 새 버전이 모든 사용자에게 릴리스되기 전에 초기 테스트로 일부 사용자에게 점진적으로 롤아웃하는 방법이다. 이 방법의 목적은 성능 메트릭을 수집하고 전체 배포에 대한 사용자의 영향을 예측하기 위해 소수의 사용자를 대상으로 하는 것입니다. 또한 Istio 등을 사용하면 특정 조건 ( 예를 들어, IE 사용자는 신규앱, 나머지는 기존앱, 특정 IP는 신규앱, 나머지는 기존앱 )으로도 사용자 요청을 분배해서 처리할 수 있다.

기존 쿠버네티스와 서비스 메시가 적용된 쿠버네티스의 트래픽 관리

Prod Deployment Pod 가 2개이고 Canary Deployment Pod 가 하나일 경우 (Label은 같음) 서비스에서 파드로 가는 트래픽은 동일하다. (prod = 66%, canary = 33%)

하지만 서비스 매쉬를 사용할 경우 그 트래픽을 조절할 수 있다.

 

 

Istio


Istioctl Download

curl -L https://istio.io/downloadIstio | sh -
cd istio-1.19.3
export PATH=$PWD/bin:$PATH

 
Istio Install

istioctl install --set profile=default -y
istioctl verify-install

 
Dashboard

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.19/samples/addons/grafana.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.19/samples/addons/prometheus.yaml

 
dashboard.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: dashboard-gateway
  namespace: istio-system
  annotations:
    kubernetes.io/ingress.class: istio
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - dashboard.k-tech.cloud
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - dashboard.k-tech.cloud
    tls:
      mode: SIMPLE
      credentialName: dashboard-secret
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: dashboard-virtualservice
  namespace: istio-system
spec:
  gateways:
  - dashboard-gateway
  hosts:
  - dashboard.k-tech.cloud
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: grafana
        port:
          number: 3000

 
dashboard-certificate.yaml

# nginx-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: dashboard-secret
  namespace: istio-system
spec:
  secretName: dashboard-secret
  privateKey:
    rotationPolicy: Always
  commonName: dashboard.k-tech.cloud
  dnsNames:
    - dashboard.k-tech.cloud
  usages:
    - digital signature
    - key encipherment
    - server auth
  issuerRef:
    name: letsencrypt2
    kind: ClusterIssuer

 

kubectl apply -f dashboard.yaml
kubectl apply -f dashboard-certificate.yaml

 

kiali

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.20/samples/addons/kiali.yaml

 

kiali-gw.yaml

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: dashboard-gateway
  namespace: istio-system
  annotations:
    kubernetes.io/ingress.class: istio
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - dashboard.k-tech.cloud
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - dashboard.k-tech.cloud
    tls:
      mode: SIMPLE
      credentialName: dashboard-secret
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: dashboard-virtualservice
  namespace: istio-system
spec:
  gateways:
  - dashboard-gateway
  hosts:
  - dashboard.k-tech.cloud
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: kiali
        port:
          number: 443

 

auto sidecar injection

kubectl label namespace default istio-injection=enabled --overwrite
kubectl get namespace -L istio-injection

 

sidecar check

istioctl experimental check-inject -n istio-system <pod-name>

 

Jenkins

Pipeline 생성
 
💡 check : Do not allow the pipeline to resume if the controller restarts
💡 check : GitHub hook trigger for GITScm polling
💡 check : GitHub project
 
nginx

podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: alpine
        image: alpine/git:latest
        command:
        - sleep
        args:
        - 99d
      - name: kaniko
        image: gcr.io/kaniko-project/executor:debug
        command:
        - sleep
        args:
        - 99d
        volumeMounts:
        - name: kaniko-secret
          mountPath: /kaniko/.docker
      restartPolicy: Never
      volumes:
      - name: kaniko-secret
        secret:
          secretName: docker-config-secret
          items:
            - key: .dockerconfigjson
              path: config.json
''') {

    node(POD_LABEL) {

        stage('Kaniko Build and Push') {
            git url: 'https://github.com/kangbock/msa_nginx.git', branch: 'main'
            script(){
                GIT_TAG = sh (
                    script: 'git describe --always',
                    returnStdout: true
                ).trim()
            }
            
            container('kaniko') {
                stage('Build with kaniko') {
                    sh '/kaniko/executor -f `pwd`/Dockerfile -c `pwd` --insecure --cache=true --destination=harbor.k-tech.cloud/msa/nginx:${BUILD_NUMBER}'
                }
            }
            
            git url: 'https://github.com/kangbock/msa_deploy.git', branch: 'main'
            container('alpine') {
                stage('GitHub Push') {
                    sh 'sed -i s/nginx:.*/nginx:${BUILD_NUMBER}/g ./nginx.yaml'
                    sh 'git config --global --add safe.directory /home/jenkins/agent/workspace/nginx'
                    sh 'git config --global user.email \'kangbock@naver.com\''
                    sh 'git config --global user.name \'kangbock\''
                    sh 'git add ./'
                    sh 'git commit -a -m "updated the image tag to ${BUILD_NUMBER}" || true'
                    sh 'git remote add kb97 https://[GITHUB_TOKEN]@github.com/kangbock/msa_deploy.git'
                    sh 'git push -u kb97 main'
                }
            }
        }
    }
}

 
login_js

podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: alpine
        image: alpine/git:latest
        command:
        - sleep
        args:
        - 99d
      - name: kaniko
        image: gcr.io/kaniko-project/executor:debug
        command:
        - sleep
        args:
        - 99d
        volumeMounts:
        - name: kaniko-secret
          mountPath: /kaniko/.docker
      restartPolicy: Never
      volumes:
      - name: kaniko-secret
        secret:
          secretName: docker-config-secret
          items:
            - key: .dockerconfigjson
              path: config.json
''') {

    node(POD_LABEL) {

        stage('Kaniko Build and Push') {
            git url: 'https://github.com/kangbock/msa_loginjs.git', branch: 'main'
            script(){
                GIT_TAG = sh (
                    script: 'git describe --always',
                    returnStdout: true
                ).trim()
            }
            
            container('kaniko') {
                stage('Build with kaniko') {
                    sh '/kaniko/executor -f `pwd`/Dockerfile -c `pwd` --insecure --cache=true --destination=harbor.k-tech.cloud/msa/login_js:${BUILD_NUMBER}'
                }
            }
            
            git url: 'https://github.com/kangbock/msa_deploy.git', branch: 'main'
            container('alpine') {
                stage('GitHub Push') {
                    sh 'sed -i s/login_js:.*/login_js:${BUILD_NUMBER}/g ./login_js.yaml'
                    sh 'git config --global --add safe.directory /home/jenkins/agent/workspace/login_js'
                    sh 'git config --global user.email \'kangbock@naver.com\''
                    sh 'git config --global user.name \'kangbock\''
                    sh 'git add ./'
                    sh 'git commit -a -m "updated the image tag to ${BUILD_NUMBER}" || true'
                    sh 'git remote add kb97 https://[GITHUB_TOKEN]@github.com/kangbock/msa_deploy.git'
                    sh 'git push -u kb97 main'
                }
            }
        }
    }
}

 
register_js

podTemplate(yaml: '''
    apiVersion: v1
    kind: Pod
    spec:
      containers:
      - name: alpine
        image: alpine/git:latest
        command:
        - sleep
        args:
        - 99d
      - name: kaniko
        image: gcr.io/kaniko-project/executor:debug
        command:
        - sleep
        args:
        - 99d
        volumeMounts:
        - name: kaniko-secret
          mountPath: /kaniko/.docker
      restartPolicy: Never
      volumes:
      - name: kaniko-secret
        secret:
          secretName: docker-config-secret
          items:
            - key: .dockerconfigjson
              path: config.json
''') {

    node(POD_LABEL) {

        stage('Kaniko Build and Push') {
            git url: 'https://github.com/kangbock/msa_registerjs.git', branch: 'main'
            script(){
                GIT_TAG = sh (
                    script: 'git describe --always',
                    returnStdout: true
                ).trim()
            }
            
            container('kaniko') {
                stage('Build with kaniko') {
                    sh '/kaniko/executor -f `pwd`/Dockerfile -c `pwd` --insecure --cache=true --destination=harbor.k-tech.cloud/msa/register_js:${BUILD_NUMBER}'
                }
            }
            
            git url: 'https://github.com/kangbock/msa_deploy.git', branch: 'main'
            container('alpine') {
                stage('GitHub Push') {
                    sh 'sed -i s/register_js:.*/register_js:${BUILD_NUMBER}/g ./register_js.yaml'
                    sh 'git config --global --add safe.directory /home/jenkins/agent/workspace/register_js'
                    sh 'git config --global user.email \'kangbock@naver.com\''
                    sh 'git config --global user.name \'kangbock\''
                    sh 'git add ./'
                    sh 'git commit -a -m "updated the image tag to ${BUILD_NUMBER}" || true'
                    sh 'git remote add kb97 https://[GITHUB_TOKEN]@github.com/kangbock/msa_deploy.git'
                    sh 'git push -u kb97 main'
                }
            }
        }
    }
}

 

ArgoCD

kubectl create namespace argocd
kubectl apply -n argocd -f <https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml>

 
argocd-certificate.yaml

# jenkins-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: argocd-server-tls
  namespace: argocd
spec:
  secretName: argocd-server-tls
  privateKey:
    rotationPolicy: Always
  commonName: argocd.k-tech.cloud
  dnsNames:
    - argocd.k-tech.cloud
  usages:
    - digital signature
    - key encipherment
    - server auth
  issuerRef:
    name: letsencrypt
    kind: ClusterIssuer

 
argocd-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd
  namespace: argocd
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
    # If you encounter a redirect loop or are getting a 307 response code
    # then you need to force the nginx ingress to connect to the backend using HTTPS.
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  tls:
  - hosts:
      - argocd.k-tech.cloud
    secretName: argocd-server-tls
  ingressClassName: nginx
  rules:
  - host: argocd.k-tech.cloud
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: argocd-server
            port:
              name: https

# 암호 찾기
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d; echo

 
nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
#  replicas: 1
  revisionHistoryLimit: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "true"
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: harbor.k-tech.cloud/msa/nginx:6
        ports:
        - containerPort: 80
      imagePullSecrets:
      - name: docker-config-secret
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  selector:
    app: nginx
  ports:
    - port: 80
      name: http-nginx
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gateway
  annotations:
    kubernetes.io/ingress.class: istio
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - www.k-tech.cloud
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - www.k-tech.cloud
    tls:
      mode: SIMPLE
      credentialName: nginx-secret
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx-virtualservice
spec:
  gateways:
  - nginx-gateway
  hosts:
  - www.k-tech.cloud
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: nginx-service
        port:
          number: 80

 
nginx-certificate.yaml

# nginx-certificate.yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: nginx-secret
  namespace: istio-system
spec:
  secretName: nginx-secret
  privateKey: 
    rotationPolicy: Always
  commonName: www.k-tech.cloud
  dnsNames:
    - www.k-tech.cloud
  usages:
    - digital signature
    - key encipherment
    - server auth
  issuerRef:
    name: letsencrypt2
    kind: ClusterIssuer

 

docker login https://harbor.k-tech.cloud
cat ~/.docker/config.json
cat ~/.docker/config.json | base64

 
regcred2.yaml

apiVersion: v1
kind: Secret
metadata:
  name: docker-config-secret
  namespace: istio-system
data:
  .dockerconfigjson: 인코딩한 데이터
type: kubernetes.io/dockerconfigjson

 

nginx-regcred.yaml

apiVersion: v1
kind: Secret
metadata:
  name: docker-config-secret
  namespace: default
data:
  .dockerconfigjson: 인코딩한 데이터
type: kubernetes.io/dockerconfigjson

 

kubectl apply -f regcred2.yaml
kubectl apply -f nginx-regcred.yaml

 
cluster-issuer2.yaml

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt2
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: kangbock0827@naver.com
    privateKeySecretRef:
      name: letsencrypt2
    solvers:
    - http01:
        ingress:
          class: istio
          podTemplate:
            spec:
              nodeSelector:
                "kubernetes.io/os": linux
kubectl apply -f cluster-issuer2.yaml

 
mysql

apt install -y mysql-client=8.*
sh kaniko/nginx/mysql.sh

 
Ingress

 
Gateway

 

Grafana Dashboard

 

Kiali Dashboard

Harbor

 

Jenkins

 

ArgoCD

 
Path from Nginx to Nodejs

'DevOps' 카테고리의 다른 글

Slack Notification  (1) 2023.10.31
Harbor (cert-manager)  (0) 2023.10.13
FortiGate 방화벽의 SNMP 를 이용한 Grafana Dashboard  (0) 2023.10.10
Jenkins + Argo CD (kaniko, harbor, cert-manager)  (0) 2023.08.11
Dapr with AKS  (0) 2023.06.09

관련글

티스토리툴바