RKE 部署rancher k8s ha
节点规划
| 节点名称 | 节点最低配置 | 集群角色 | IP地址 | docker版本 | rancher版本 | 操作系统版本 |
|---|---|---|---|---|---|---|
| node01 | 2c8g | master | 192.168.189.20 | 18.09.3 | rancher2.3 | Ubuntu18 |
| node02 | 2c8g | node | 192.168.189.21 | 18.09.3 | rancher2.3 | Ubuntu18 |
| nginx | 2c1g | nginx | 192.168.188.100 | nginx1.15.1 | Ubuntu18 |
一、节点基础配置
注意：本章节操作需要在每个节点执行，用非root账户 使用sudo 加以操作
1.配置apt源
sudo cp /etc/apt/sources.list /etc/apt/sources.list.bak
sudo sed -i 's/http\:\/\/cn.archive.ubuntu.com/https\:\/\/mirrors.aliyun.com/g' /etc/apt/sources.list
sudo apt update注：http://cn.archive.ubuntu.com 为原sources.list文件的apt源码仓库地址
2.关闭ufw(防火墙),配置hosts解析
sudo ufw disable
sudo vim /etc/hosts
192.168.189.20 node1
192.168.189.21 node2
#nginx服务代理ingrees服务
192.168.188.100 nginx abc.yefxx.cn3.开启内核转发
sudo -i
cat >> /etc/sysctl.conf<<EOF
net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1
net.ipv4.ip_forward=1
net.ipv4.conf.all.forwarding=1
net.ipv4.neigh.default.gc_thresh1=4096
net.ipv4.neigh.default.gc_thresh2=6144
net.ipv4.neigh.default.gc_thresh3=8192
net.ipv4.neigh.default.gc_interval=60
net.ipv4.neigh.default.gc_stale_time=120
EOF
exit4.时间同步
sudo ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
sudo apt install ntpdate
sudo crontab -e
#添加定时任务
*/2 * * * * ntpdate ntp1.aliyun.com > /dev/null二、安装rancher
安装docker(公网环境下执行)
curl -fsSL https://get.docker.com |sudo bash -s docker --mirror Aliyun
#镜像加速
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"oom-score-adjust": -1000,
"log-driver": "json-file",
"log-opts": {
"max-size": "100m",
"max-file": "3"
},
"max-concurrent-downloads": 10,
"max-concurrent-uploads": 10,
"bip": "169.254.123.1/24",
"registry-mirrors": ["https://7bezldxe.mirror.aliyuncs.com","https://ip3m2hdn.mirror.aliyuncs.com"],
"storage-driver": "overlay2",
"storage-opts": [
"overlay2.override_kernel_check=true"
]
}
EOF
#修改docker.service
sudo vim /lib/systemd/system/docker.service
#防止docker服务OOM：
OOMScoreAdjust=-1000
ExecStartPost=/sbin/iptables -P FORWARD ACCEPT
sudo systemctl daemon-reload && sudo systemctl restart docker
#以下为可执行优化项 请谨慎执行
#解决docker info提示WARNING: No swap limit support
#sudo sed -i 's/GRUB_CMDLINE_LINUX="\(.*\)"/GRUB_CMDLINE_LINUX=" #cgroup_enable=memory swapaccount=1 \1"/g' /etc/default/grub
#sudo update-grub
#sudo reboot普通用户执行docker
sudo usermod -aG docker spms配置spms用户节点互相互信
ssh-keygen -f /home/spms/.ssh/id_rsa -N ''
ssh-copy-id -i /home/spms/.ssh/id_rsa.pub node1
ssh-copy-id -i /home/spms/.ssh/id_rsa.pub node2
scp /home/spms/.ssh/id_rsa node01:/home/spms/.ssh/
scp /home/spms/.ssh/id_rsa node02:/home/spms/.ssh/rke部署k8s集群
创建集群配置文件
cat >> rancher-cluster.yml <<EOF
nodes:
- address: 168.168.189.20
user: spms
role: [controlplane,worker,etcd]
- address: 168.168.189.21
user: spms
role: [controlplane,worker,etcd]
services:
etcd:
snapshot: true
creation: 6h
retention: 24h
EOF创建k8s集群
wget https://www.rancher.cn/download/rke/v0.3.2-rke_linux-amd64
sudo chmod 777 v0.3.2-rke_linux-amd64 && sudo mv v0.3.2-rke_linux-amd64 /usr/local/bin/rke
rke up --config rancher-cluster.yml下载并配置kubectl
wget https://www.rancher.cn/download/kubernetes/linux-amd64-v1.16.2-kubectl
sudo chmod 777 kubectl_amd64-linux
sudo mv kubectl_amd64-linux /usr/local/bin/kubectl
echo "source <(kubectl completion bash)" >> ~/.bashrc
source ~/.bashrc
mkdir ~/.kube
cp /home/spms/kube_config_rancher-cluster.yml /spms/.kube/config检查k8s集群pod状态
kubectl get pods --all-namespaces[spms@node1 ~]$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx default-http-backend-7f8fbb85db-rxs9r 1/1 Running 0 106s
ingress-nginx nginx-ingress-controller-9vhbj 1/1 Running 0 10m
ingress-nginx nginx-ingress-controller-lhvk4 1/1 Running 0 10m
kube-system canal-9lhlr 2/2 Running 0 10m
kube-system canal-xxz5p 2/2 Running 0 10m
kube-system kube-dns-5fd74c7488-54dgp 3/3 Running 0 10m
kube-system kube-dns-autoscaler-c89df977f-fb42z 1/1 Running 0 10m
kube-system metrics-server-7fbd549b78-8hftl 1/1 Running 0 10m
kube-system rke-ingress-controller-deploy-job-8c9c2 0/1 Completed 0 10m
kube-system rke-kubedns-addon-deploy-job-lp5tc 0/1 Completed 0 10m
kube-system rke-metrics-addon-deploy-job-j585d 0/1 Completed 0 10m
kube-system rke-network-plugin-deploy-job-xssrc 0/1 Completed 0 10mpod的状态只有以上两种状态为正常状态，若有其他状态则需要查看pod日志
kubectl describe pod pod-xxx -n namespacehelm将rancher部署在k8s集群
安装并配置helm客户端
wget https://www.rancher.cn/download/helm/helm-v2.15.2-linux-amd64.tar.gz
tar xf helm-v2.15.2-linux-amd64.tar.gz
sudo cp linux-amd64/helm /usr/local/bin/
sudo cp linux-amd64/tiller /usr/local/bin/配置helm客户端具有访问k8s集群的权限
kubectl -n kube-system create serviceaccount tiller
kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller将helm server（titler）部署到k8s集群
helm init --service-account tiller --tiller-image sapcc/tiller:v2.15.2 --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
helm init --service-account tiller --canary-image
检查tiller pod运行正常,若tiller pod不正常需要修复。
kubectl get pods --all-namespaces为helm客户端配置chart仓库
helm repo add rancher-stable https://releases.rancher.com/server-charts/stable检查rancher chart仓库可用
helm search rancherNAME CHART VERSION APP VERSION DESCRIPTION
rancher-stable/rancher 2.3.2 v2.3.2 Install Rancher Server to manage Kubernetes clusters acro...安装证书管理器
helm install stable/cert-manager --name cert-manager --namespace kube-system --set rancherImage=rancher/rancher:stable检查证书管理器pod是否正常
kubectl get pods --all-namespaces|grep cert-manager安装rancher
helm install rancher-stable/rancher --name rancher --namespace cattle-system --set hostname=abc.yefxx.cn配置nginx
cat /etc/nginx/conf/rancher.conf
upstream rancher {
server 192.168.189.20:443 max_fails=3 fail_timeout=5s;
server 192.168.189.21:443 max_fails=3 fail_timeout=5s;
}
map $http_upgrade $connection_upgrade {
default Upgrade;
'' close;
}
server {
listen 443 ssl http2; # 如果是升级或者全新安装v2.2.2,需要禁止http2，其他版本不需修改。
server_name abc.yefxx.cn;
ssl_certificate /root/.acme.sh/yefxx.cn/fullchain.cer;
ssl_certificate_key /root/.acme.sh/yefxx.cn/yefxx.cn.key;
location / {
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://rancher;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
# This allows the ability for the execute shell window to remain open for up to 15 minutes.
## Without this parameter, the default is 1 minute and will automatically close.
proxy_read_timeout 900s;
proxy_buffering off;
}
}
server {
listen 80;
server_name abc.yefxx.cn;
return 301 https://$server_name$request_uri;
}由于rancher HA需要通过域名访问，所有在本地添加hosts解析后。浏览器访问nginx
https://abc.yefxx.cn本作品采用 知识共享署名-相同方式共享 4.0 国际许可协议 进行许可。