Kubernetes Binary 설치 - 2

https://mgujob.tistory.com/11 이전 내용에서 계속

Kubernetes Binary 설치 - 1

Kubeadm을 통한 클러스터 구성

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

Installing kubeadm

 

selinux 설정 

# Set SELinux in permissive mode (effectively disabling it)
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

 

최신 버전 (2024.02.20기준) 1.29.2 버전 설치를 위한 repo 설정

# This overwrites any existing configuration in /etc/yum.repos.d/kubernetes.repo
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.29/rpm/repodata/repomd.xml.key
exclude=kubelet kubeadm kubectl cri-tools kubernetes-cni
EOF

 

커널 파라메터 수정

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# Apply sysctl params without reboot
sudo sysctl --system

패키지 설치

sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet

 

kubeadm 다중 마스터 클러스터 생성

sudo kubeadm init --control-plane-endpoint (마스터IP or LB-IP):6443 --upload-certs

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

  export KUBECONFIG=/etc/kubernetes/admin.conf

마스터 조인 구문
  kubeadm join (마스터IP or LB-IP):6443 --token ubefqk.io01ioi0vqg1uf4g \
        --discovery-token-ca-cert-hash sha256:5755dcb3d42baf664afc166fb7da811118a4feb218a393d4882203eafeccf1ae \
        --control-plane --certificate-key 42118b3346ec1c7632383aad4efbca22a2200ca15691054580dc014d435848c1

워커 조인 구문
kubeadm join (마스터IP or LB-IP):6443 --token ubefqk.io01ioi0vqg1uf4g \
        --discovery-token-ca-cert-hash sha256:5755dcb3d42baf664afc166fb7da811118a4feb218a393d4882203eafeccf1ae

 

마스터로 지정한 노드에서

 kubeadm join (마스터IP or LB-IP):6443 --token ubefqk.io01ioi0vqg1uf4g \
        --discovery-token-ca-cert-hash sha256:5755dcb3d42baf664afc166fb7da811118a4feb218a393d4882203eafeccf1ae \
        --control-plane --certificate-key 42118b3346ec1c7632383aad4efbca22a2200ca15691054580dc014d435848c1

 

워커로 지정한 노드에서

kubeadm join (마스터IP or LB-IP):6443 --token ubefqk.io01ioi0vqg1uf4g \
        --discovery-token-ca-cert-hash sha256:5755dcb3d42baf664afc166fb7da811118a4feb218a393d4882203eafeccf1ae

 

노드 확인

kubectl get node

NAME                   STATUS   ROLES           AGE     VERSION
node-1 				   Ready    control-plane   5m1s    v1.29.2
node-2  			   Ready    control-plane   4m38s   v1.29.2
node-3   			   Ready    control-plane   3m36s   v1.29.2
node-4   			   Ready    <none>          3m25s   v1.29.2
node-5   			   Ready    <none>          3m24s   v1.29.2
node-6   			   Ready    <none>          3m23s   v1.29.2

 

Pod Network를 위한 Calico apply

(이전 글에서 wget으로 받아온 yaml 적용)

kubectl apply -f calico.yaml

 

POD 상태 확인

kubectl get pod -A
NAMESPACE     NAME                                           READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-7ddc4f45bc-cvpfh       1/1     Running   0          3m24s
kube-system   calico-node-brll6                              1/1     Running   0          3m24s
kube-system   calico-node-lsstj                              1/1     Running   0          3m24s
kube-system   calico-node-p97b5                              1/1     Running   0          3m24s
kube-system   calico-node-wgv5g                              1/1     Running   0          3m24s
kube-system   calico-node-x9mtr                              1/1     Running   0          3m24s
kube-system   calico-node-znswp                              1/1     Running   0          3m24s
kube-system   coredns-76f75df574-msjhk                       1/1     Running   0          7m26s
kube-system   coredns-76f75df574-q2wpz                       1/1     Running   0          7m26s
kube-system   etcd-node-1                                    1/1     Running   4          7m32s
kube-system   etcd-node-2                                    1/1     Running   1          7m11s
kube-system   etcd-node-3                                    1/1     Running   0          6m8s
kube-system   kube-apiserver-lim-geon-1.novalocal            1/1     Running   4          7m32s
kube-system   kube-apiserver-lim-geon-2.novalocal            1/1     Running   1          7m10s
kube-system   kube-apiserver-lim-geon-3.novalocal            1/1     Running   1          6m8s
kube-system   kube-controller-manager-lim-geon-1.novalocal   1/1     Running   4          7m32s
kube-system   kube-controller-manager-lim-geon-2.novalocal   1/1     Running   1          7m10s
kube-system   kube-controller-manager-lim-geon-3.novalocal   1/1     Running   1          6m1s
kube-system   kube-proxy-hjpxj                               1/1     Running   0          5m59s
kube-system   kube-proxy-kq5bp                               1/1     Running   0          7m26s
kube-system   kube-proxy-m5f7h                               1/1     Running   0          5m58s
kube-system   kube-proxy-qjzj7                               1/1     Running   0          7m12s
kube-system   kube-proxy-rdrsf                               1/1     Running   0          5m57s
kube-system   kube-proxy-rqbwl                               1/1     Running   0          6m10s
kube-system   kube-scheduler-node-1                          1/1     Running   4          7m33s
kube-system   kube-scheduler-node-2                          1/1     Running   1          7m1s
kube-system   kube-scheduler-node-3                          1/1     Running   1          6m1s

 

+ 선택사항 메트릭 서버 추가

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

 

메트릭 서버 deployment 수정

kubectl edit deployments.apps -n kube-system metrics-server

...

    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=10250
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --metric-resolution=15s
        - --kubelet-insecure-tls <- 추가

 

POD 확인

kubectl get po -A
NAMESPACE     NAME                                           READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-7ddc4f45bc-cvpfh       1/1     Running   0          168m
...
kube-system   metrics-server-98bc7f888-jn5pv                 1/1     Running   0          90s