网站建设基础课程,汕尾北京网站建设,龙岗区网站建设哪个公司好,wordpress 国际支付宝前言#xff1a;如无特殊说明#xff0c;所有操作都用root账号在所有节点执行。 说明#xff1a;kubeasz是一款国产开源的k8s部署软件#xff0c;采用ansible role的部署方式#xff0c;部署k8s二进制集群。熟悉ansible role的用该软件部署k8s方便快捷。
一、机器
deplo…前言如无特殊说明所有操作都用root账号在所有节点执行。 说明kubeasz是一款国产开源的k8s部署软件采用ansible role的部署方式部署k8s二进制集群。熟悉ansible role的用该软件部署k8s方便快捷。
一、机器
deploy #部署机器 master01 master02 node01
二、环境
2.1 部署机器环境
[rootlocalhost opt]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[rootlocalhost opt]# uname -a
Linux localhost.localdomain 3.10.0-229.el7.x86_64 #1 SMP Fri Mar 6 11:36:42 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[rootlocalhost opt]# python -V
Python 3.9.22.2 master和node节点机器环境
[rootlocalhost ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
[rootlocalhost ~]# uname -a
Linux mater01 3.10.0-862.el7.x86_64 #1 SMP Fri Apr 20 16:44:24 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
[rootlocalhost ~]# python -V
Python 3.9.22.3 ip和主机名
deploy 192.168.30.13
master01 192.168.30.10
master02 192.168.30.11
node01 192.168.10.12三、配置yum源
说明CentOS 7 系统的yum源不再可用需要重新配置。
3.1 备份旧yum源文件
cd /etc/yum.repos.d/
mkdir bak/
mv * ./bak/
3.2 配置Base源
cat CentOS-Base.repo EOF
# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client. You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist does not work for you, as a fall back you can try the
# remarked out baseurl line instead.
#
#[base]
nameCentOS-$releasever - Base
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepoosinfra$infra
#baseurlhttp://mirror.centos.org/centos/$releasever/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#released updates
[updates]
nameCentOS-$releasever - Updates
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepoupdatesinfra$infra
#baseurlhttp://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras]
nameCentOS-$releasever - Extras
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepoextrasinfra$infra
#baseurlhttp://mirror.centos.org/centos/$releasever/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that extend functionality of existing packages
[centosplus]
nameCentOS-$releasever - Plus
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepocentosplusinfra$infra
#baseurlhttp://mirror.centos.org/centos/$releasever/centosplus/$basearch/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF3.3 配置CR源
cat CentOS-CR.repo EOF
# CentOS-CR.repo
#
# The Continuous Release ( CR ) repository contains rpms that are due in the next
# release for a specific CentOS Version ( eg. next release in CentOS-7 ); these rpms
# are far less tested, with no integration checking or update path testing having
# taken place. They are still built from the upstream sources, but might not map
# to an exact upstream distro release.
#
# These packages are made available soon after they are built, for people willing
# to test their environments, provide feedback on content for the next release, and
# for people looking for early-access to next release content.
#
# The CR repo is shipped in a disabled state by default; its important that users
# understand the implications of turning this on.
#
# NOTE: We do not use a mirrorlist for the CR repos, to ensure content is available
# to everyone as soon as possible, and not need to wait for the external
# mirror network to seed first. However, many local mirrors will carry CR repos
# and if desired you can use one of these local mirrors by editing the baseurl
# line in the repo config below.
#[cr]
nameCentOS-$releasever - cr
baseurlhttp://mirror.centos.org/centos/$releasever/cr/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0
EOF3.4 配置Debuginfo源
cat CentOS-Debuginfo.repo EOF
# CentOS-Debug.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client. You should use this for CentOS updates
# unless you are manually picking other mirrors.
## All debug packages from all the various CentOS-7 releases
# are merged into a single repo, split by BaseArch
#
# Note: packages in the debuginfo repo are currently not signed
#[base-debuginfo]
nameCentOS-7 - Debuginfo
baseurlhttp://debuginfo.centos.org/7/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Debug-7
enabled0
#
EOF3.5 配置fasttrack源
cat CentOS-fasttrack.repo EOF
#CentOS-fasttrack.repo[fasttrack]
nameCentOS-7 - fasttrack
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepofasttrackinfra$infra
#baseurlhttp://mirror.centos.org/centos/$releasever/fasttrack/$basearch/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF3.6 配置Media源
cat CentOS-Media.repo EOF
# CentOS-Media.repo
#
# This repo can be used with mounted DVD media, verify the mount point for
# CentOS-7. You can use this repo and yum to install items directly off the
# DVD ISO that we release.
#
# To use this repo, put in your DVD and use it with the other repos too:
# yum --enablerepoc7-media [command]
#
# or for ONLY the media repo, do this:
#
# yum --disablerepo\* --enablerepoc7-media [command][c7-media]
nameCentOS-$releasever - Media
baseurlfile:///media/CentOS/file:///media/cdrom/file:///media/cdrecorder/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF3.7 配置Sources源
cat CentOS-Sources.repo EOF
# CentOS-Sources.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client. You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist does not work for you, as a fall back you can try the
# remarked out baseurl line instead.
#
#[base-source]
nameCentOS-$releasever - Base Sources
baseurlhttp://vault.centos.org/centos/$releasever/os/Source/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#released updates
[updates-source]
nameCentOS-$releasever - Updates Sources
baseurlhttp://vault.centos.org/centos/$releasever/updates/Source/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that may be useful
[extras-source]
nameCentOS-$releasever - Extras Sources
baseurlhttp://vault.centos.org/centos/$releasever/extras/Source/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7#additional packages that extend functionality of existing packages
[centosplus-source]
nameCentOS-$releasever - Plus Sources
baseurlhttp://vault.centos.org/centos/$releasever/centosplus/Source/
gpgcheck1
enabled0
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF3.8 配置Vault源
cat CentOS-Vault.repo EOF
# CentOS Vault contains rpms from older releases in the CentOS-7
# tree.#c7.0.1406
[C7.0.1406-base]
nameCentOS-7.0.1406 - Base
baseurlhttp://vault.centos.org/7.0.1406/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.0.1406-updates]
nameCentOS-7.0.1406 - Updates
baseurlhttp://vault.centos.org/7.0.1406/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.0.1406-extras]
nameCentOS-7.0.1406 - Extras
baseurlhttp://vault.centos.org/7.0.1406/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.0.1406-centosplus]
nameCentOS-7.0.1406 - CentOSPlus
baseurlhttp://vault.centos.org/7.0.1406/centosplus/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.0.1406-fasttrack]
nameCentOS-7.0.1406 - CentOSPlus
baseurlhttp://vault.centos.org/7.0.1406/fasttrack/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0# C7.1.1503
[C7.1.1503-base]
nameCentOS-7.1.1503 - Base
baseurlhttp://vault.centos.org/7.1.1503/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.1.1503-updates]
nameCentOS-7.1.1503 - Updates
baseurlhttp://vault.centos.org/7.1.1503/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.1.1503-extras]
nameCentOS-7.1.1503 - Extras
baseurlhttp://vault.centos.org/7.1.1503/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.1.1503-centosplus]
nameCentOS-7.1.1503 - CentOSPlus
baseurlhttp://vault.centos.org/7.1.1503/centosplus/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.1.1503-fasttrack]
nameCentOS-7.1.1503 - CentOSPlus
baseurlhttp://vault.centos.org/7.1.1503/fasttrack/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0# C7.2.1511
[C7.2.1511-base]
nameCentOS-7.2.1511 - Base
baseurlhttp://vault.centos.org/7.2.1511/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.2.1511-updates]
nameCentOS-7.2.1511 - Updates
baseurlhttp://vault.centos.org/7.2.1511/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.2.1511-extras]
nameCentOS-7.2.1511 - Extras
baseurlhttp://vault.centos.org/7.2.1511/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.2.1511-centosplus]
nameCentOS-7.2.1511 - CentOSPlus
baseurlhttp://vault.centos.org/7.2.1511/centosplus/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.2.1511-fasttrack]
nameCentOS-7.2.1511 - CentOSPlus
baseurlhttp://vault.centos.org/7.2.1511/fasttrack/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0# C7.3.1611
[C7.3.1611-base]
nameCentOS-7.3.1611 - Base
baseurlhttp://vault.centos.org/7.3.1611/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.3.1611-updates]
nameCentOS-7.3.1611 - Updates
baseurlhttp://vault.centos.org/7.3.1611/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.3.1611-extras]
nameCentOS-7.3.1611 - Extras
baseurlhttp://vault.centos.org/7.3.1611/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.3.1611-centosplus]
nameCentOS-7.3.1611 - CentOSPlus
baseurlhttp://vault.centos.org/7.3.1611/centosplus/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.3.1611-fasttrack]
nameCentOS-7.3.1611 - CentOSPlus
baseurlhttp://vault.centos.org/7.3.1611/fasttrack/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0# C7.4.1708
[C7.4.1708-base]
nameCentOS-7.4.1708 - Base
baseurlhttp://vault.centos.org/7.4.1708/os/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.4.1708-updates]
nameCentOS-7.4.1708 - Updates
baseurlhttp://vault.centos.org/7.4.1708/updates/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.4.1708-extras]
nameCentOS-7.4.1708 - Extras
baseurlhttp://vault.centos.org/7.4.1708/extras/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.4.1708-centosplus]
nameCentOS-7.4.1708 - CentOSPlus
baseurlhttp://vault.centos.org/7.4.1708/centosplus/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0[C7.4.1708-fasttrack]
nameCentOS-7.4.1708 - CentOSPlus
baseurlhttp://vault.centos.org/7.4.1708/fasttrack/$basearch/
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
enabled0
EOF3.9 配置x86_64-kernel源
cat CentOS-x86_64-kernel.repo EOF
[centos-kernel]
nameCentOS LTS Kernels for $basearch
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepokernelinfra$infra
#baseurlhttp://mirror.centos.org/altarch/7/kernel/$basearch/
enabled0
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7[centos-kernel-experimental]
nameCentOS Experimental Kernels for $basearch
mirrorlisthttp://mirrorlist.centos.org/?release$releaseverarch$basearchrepoexperimentalinfra$infra
#baseurlhttp://mirror.centos.org/altarch/7/experimental/$basearch/
enabled0
gpgcheck1
gpgkeyfile:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
EOF3.10 配置docker-ce源
cat docker-ce.repo EOF
[docker-ce-stable]
nameDocker CE Stable - $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stable
enabled1
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-stable-debuginfo]
nameDocker CE Stable - Debuginfo $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/stable
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-stable-source]
nameDocker CE Stable - Sources
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/stable
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test]
nameDocker CE Test - $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/test
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test-debuginfo]
nameDocker CE Test - Debuginfo $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/test
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test-source]
nameDocker CE Test - Sources
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/test
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly]
nameDocker CE Nightly - $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/nightly
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly-debuginfo]
nameDocker CE Nightly - Debuginfo $basearch
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/nightly
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly-source]
nameDocker CE Nightly - Sources
baseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/nightly
enabled0
gpgcheck1
gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg
EOF3.11 配置nginx源
cat nginx.repo EOF
[nginx-stable]
namenginx stable repo
baseurlhttp://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck1
enabled1
gpgkeyhttps://nginx.org/keys/nginx_signing.key
module_hotfixestrue[nginx-mainline]
namenginx mainline repo
baseurlhttp://nginx.org/packages/mainline/centos/$releasever/$basearch/
gpgcheck1
enabled0
gpgkeyhttps://nginx.org/keys/nginx_signing.key
module_hotfixestrue
EOF四、关闭selinux和防火墙
4.1 关闭防火墙
systemctl disable firewalld
systemctl stop firewalld
4.2 关闭selinux
setenforce 0
sed -i s#SELINUX.*#SELINUXdisabled#g /etc/selinux/config警告关闭selinux后一定要重启机器否则会被部署代码识别为未关闭selinux而报错。
五、修改主机名
说明分别在maser01 master02 node节点执行。
5.1 master01执行
hostnamectl set-hostname master01
5.2 master02执行
hostnamectl set-hostname master025.3 node01执行
hostnamectl set-hostname node01六、部署步骤
说明仅在deploy节点执行。
6.1 准备ssh免密登陆
配置从部署节点能够ssh免密登陆所有节点并且设置python软连接
$IP为所有节点地址包括自身按照提示输入yes 和root密码
ssh-copy-id $IP 6.2 为每个节点设置python软链接
ssh $IP ln -s /usr/bin/python3 /usr/bin/python6.3 在部署节点编排k8s安装
6.3.1 下载项目源码、二进制及离线镜像
下载工具脚本ezdown举例使用kubeasz版本3.5.0
export release3.5.0
wget https://github.com/easzlab/kubeasz/releases/download/${release}/ezdown
chmod x ./ezdown下载kubeasz代码、二进制、默认容器镜像更多关于ezdown的参数运行./ezdown 查看
#国内环境
./ezdown -D#海外环境
./ezdown -D -m standard【可选】下载额外容器镜像cilium,flannel,prometheus等
./ezdown -X【可选】下载离线系统包 (适用于无法使用yum/apt仓库情形)
./ezdown -P上述脚本运行成功后所有文件kubeasz代码、二进制、离线镜像均已整理好放入目录/etc/kubeasz
6.3.2 创建集群配置实例
#容器化运行kubeasz
./ezdown -S#创建新集群 k8s-01
docker exec -it kubeasz ezctl new k8s-012021-01-19 10:48:23 DEBUG generate custom cluster files in /etc/kubeasz/clusters/k8s-01
2021-01-19 10:48:23 DEBUG set version of common plugins
2021-01-19 10:48:23 DEBUG cluster k8s-01: files successfully created.
2021-01-19 10:48:23 INFO next steps 1: to config /etc/kubeasz/clusters/k8s-01/hosts
2021-01-19 10:48:23 INFO next steps 2: to config /etc/kubeasz/clusters/k8s-01/config.yml然后根据提示配置’/etc/kubeasz/clusters/k8s-01/hosts’ 和 ‘/etc/kubeasz/clusters/k8s-01/config.yml’根据前面节点规划修改hosts 文件和其他集群层面的主要配置选项其他集群组件等配置项可以在config.yml 文件中修改。 修改后的配置文件如下
cat /etc/kubeasz/clusters/k8s-01/hosts
# etcd cluster should have odd member(s) (1,3,5,...)
[etcd]
192.168.30.16
192.168.30.17
192.168.30.18# master node(s)
[kube_master]
192.168.30.16
192.168.30.17# work node(s)
[kube_node]
192.168.30.18# [optional] harbor server, a private docker registry
# NEW_INSTALL: true to install a harbor server; false to integrate with existed one
[harbor]
#192.168.1.8 NEW_INSTALLfalse
192.168.30.16 NEW_INSTALLtrue# [optional] loadbalance for accessing k8s from outside
[ex_lb]
#192.168.1.6 LB_ROLEbackup EX_APISERVER_VIP192.168.30.1750 EX_APISERVER_PORT8443
#192.168.1.7 LB_ROLEmaster EX_APISERVER_VIP192.168.30.1750 EX_APISERVER_PORT8443
192.168.30.16 LB_ROLEbackup EX_APISERVER_VIP192.168.30.200 EX_APISERVER_PORT8443
192.168.30.17 LB_ROLEmaster EX_APISERVER_VIP192.168.30.200 EX_APISERVER_PORT8443# [optional] ntp server for the cluster
[chrony]
#192.168.30.16
192.168.30.16[all:vars]
# --------- Main Variables ---------------
# Secure port for apiservers
SECURE_PORT6443# Cluster container-runtime supported: docker, containerd
# if k8s version 1.24, docker is not supported
CONTAINER_RUNTIMEcontainerd# Network plugins supported: calico, flannel, kube-router, cilium, kube-ovn
CLUSTER_NETWORKcalico# Service proxy mode of kube-proxy: iptables or ipvs
PROXY_MODEipvs# K8S Service CIDR, not overlap with node(host) networking
SERVICE_CIDR10.68.0.0/16# Cluster CIDR (Pod CIDR), not overlap with node(host) networking
CLUSTER_CIDR172.20.0.0/16# NodePort Range
NODE_PORT_RANGE30000-32767# Cluster DNS Domain
CLUSTER_DNS_DOMAINcluster.local# -------- Additional Variables (dont change the default value right now) ---
# Binaries Directory
bin_dir/opt/kube/bin# Deploy Directory (kubeasz workspace)
base_dir/etc/kubeasz# Directory for a specific cluster
cluster_dir{{ base_dir }}/clusters/k8s-01# CA and other components cert/key Directory
ca_dir/etc/kubernetes/sslcat /etc/kubeasz/clusters/k8s-01/config.yml
############################
# prepare
############################
# 可选离线安装系统软件包 (offline|online)
INSTALL_SOURCE: online# 可选进行系统安全加固 github.com/dev-sec/ansible-collection-hardening
OS_HARDEN: false############################
# role:deploy
############################
# default: ca will expire in 100 years
# default: certs issued by the ca will expire in 50 years
CA_EXPIRY: 876000h
CERT_EXPIRY: 438000h# force to recreate CA and other certs, not suggested to set true
CHANGE_CA: false# kubeconfig 配置参数
CLUSTER_NAME: cluster1
CONTEXT_NAME: context-{{ CLUSTER_NAME }}# k8s version
K8S_VER: 1.26.0############################
# role:etcd
############################
# 设置不同的wal目录可以避免磁盘io竞争提高性能
ETCD_DATA_DIR: /var/lib/etcd
ETCD_WAL_DIR: ############################
# role:runtime [containerd,docker]
############################
# ------------------------------------------- containerd
# [.]启用容器仓库镜像
ENABLE_MIRROR_REGISTRY: true# [containerd]基础容器镜像
SANDBOX_IMAGE: easzlab.io.local:5000/easzlab/pause:3.9# [containerd]容器持久化存储目录
CONTAINERD_STORAGE_DIR: /var/lib/containerd# ------------------------------------------- docker
# [docker]容器存储目录
DOCKER_STORAGE_DIR: /var/lib/docker# [docker]开启Restful API
ENABLE_REMOTE_API: false# [docker]信任的HTTP仓库
INSECURE_REG: [http://easzlab.io.local:5000]############################
# role:kube-master
############################
# k8s 集群 master 节点证书配置可以添加多个ip和域名比如增加公网ip和域名
MASTER_CERT_HOSTS:- 192.168.30.16- 192.168.30.17- 192.168.30.18- k8s.easzlab.io- easzlab.io.local- harbor.easzlab.io.local#- www.test.com# node 节点上 pod 网段掩码长度决定每个节点最多能分配的pod ip地址
# 如果flannel 使用 --kube-subnet-mgr 参数那么它将读取该设置为每个节点分配pod网段
# https://github.com/coreos/flannel/issues/847
NODE_CIDR_LEN: 24############################
# role:kube-node
############################
# Kubelet 根目录
KUBELET_ROOT_DIR: /var/lib/kubelet# node节点最大pod 数
MAX_PODS: 110# 配置为kube组件kubelet,kube-proxy,dockerd等预留的资源量
# 数值设置详见templates/kubelet-config.yaml.j2
KUBE_RESERVED_ENABLED: no# k8s 官方不建议草率开启 system-reserved, 除非你基于长期监控了解系统的资源占用状况
# 并且随着系统运行时间需要适当增加资源预留数值设置详见templates/kubelet-config.yaml.j2
# 系统预留设置基于 4c/8g 虚机最小化安装系统服务如果使用高性能物理机可以适当增加预留
# 另外集群安装时候apiserver等资源占用会短时较大建议至少预留1g内存
SYS_RESERVED_ENABLED: no############################
# role:network [flannel,calico,cilium,kube-ovn,kube-router]
############################
# ------------------------------------------- flannel
# [flannel]设置flannel 后端host-gw,vxlan等
FLANNEL_BACKEND: vxlan
DIRECT_ROUTING: false# [flannel]
flannel_ver: v0.19.2# ------------------------------------------- calico
# [calico] IPIP隧道模式可选项有: [Always, CrossSubnet, Never],跨子网可以配置为Always与CrossSubnet(公有云建议使用always比较省事其他的话需要修改各自公有云的网络配置具体可以参考各个公有云说明)
# 其次CrossSubnet为隧道BGP路由混合模式可以提升网络性能同子网配置为Never即可.
CALICO_IPV4POOL_IPIP: Always# [calico]设置 calico-node使用的host IPbgp邻居通过该地址建立可手工指定也可以自动发现
IP_AUTODETECTION_METHOD: can-reach{{ groups[kube_master][0] }}# [calico]设置calico 网络 backend: brid, vxlan, none
CALICO_NETWORKING_BACKEND: brid# [calico]设置calico 是否使用route reflectors
# 如果集群规模超过50个节点建议启用该特性
CALICO_RR_ENABLED: false# CALICO_RR_NODES 配置route reflectors的节点如果未设置默认使用集群master节点
# CALICO_RR_NODES: [192.168.1.1, 192.168.1.2]
CALICO_RR_NODES: []# [calico]更新支持calico 版本: [3.19, 3.23]
calico_ver: v3.23.5# [calico]calico 主版本
calico_ver_main: {{ calico_ver.split(.)[0] }}.{{ calico_ver.split(.)[1] }}# ------------------------------------------- cilium
# [cilium]镜像版本
cilium_ver: 1.12.4
cilium_connectivity_check: true
cilium_hubble_enabled: false
cilium_hubble_ui_enabled: false# ------------------------------------------- kube-ovn
# [kube-ovn]选择 OVN DB and OVN Control Plane 节点默认为第一个master节点
OVN_DB_NODE: {{ groups[kube_master][0] }}# [kube-ovn]离线镜像tar包
kube_ovn_ver: v1.5.3# ------------------------------------------- kube-router
# [kube-router]公有云上存在限制一般需要始终开启 ipinip自有环境可以设置为 subnet
OVERLAY_TYPE: full# [kube-router]NetworkPolicy 支持开关
FIREWALL_ENABLE: true# [kube-router]kube-router 镜像版本
kube_router_ver: v0.3.1
busybox_ver: 1.28.4############################
# role:cluster-addon
############################
# coredns 自动安装
dns_install: yes
corednsVer: 1.9.3
ENABLE_LOCAL_DNS_CACHE: true
dnsNodeCacheVer: 1.22.13
# 设置 local dns cache 地址
LOCAL_DNS_CACHE: 169.254.20.10# metric server 自动安装
metricsserver_install: yes
metricsVer: v0.5.2# dashboard 自动安装
dashboard_install: yes
dashboardVer: v2.7.0
dashboardMetricsScraperVer: v1.0.8# prometheus 自动安装
prom_install: no
prom_namespace: monitor
prom_chart_ver: 39.11.0# nfs-provisioner 自动安装
nfs_provisioner_install: no
nfs_provisioner_namespace: kube-system
nfs_provisioner_ver: v4.0.2
nfs_storage_class: managed-nfs-storage
nfs_server: 192.168.30.16
nfs_path: /data/nfs# network-check 自动安装
network_check_enabled: false
network_check_schedule: */5 * * * *############################
# role:harbor
############################
# harbor version完整版本号
HARBOR_VER: v2.1.5
HARBOR_DOMAIN: harbor.easzlab.io.local
HARBOR_PATH: /var/data
HARBOR_TLS_PORT: 8443
HARBOR_REGISTRY: {{ HARBOR_DOMAIN }}:{{ HARBOR_TLS_PORT }}# if set false, you need to put certs named harbor.pem and harbor-key.pem in directory down
HARBOR_SELF_SIGNED_CERT: true# install extra component
HARBOR_WITH_NOTARY: false
HARBOR_WITH_TRIVY: false
HARBOR_WITH_CLAIR: false
HARBOR_WITH_CHARTMUSEUM: true
6.3.3 开始安装 如果你对集群安装流程不熟悉请阅读项目首页 安装步骤 讲解后分步安装并对 每步都进行验证
#建议使用alias命令查看~/.bashrc 文件应该包含alias dk‘docker exec -it kubeasz’
source ~/.bashrc#一键安装等价于执行docker exec -it kubeasz ezctl setup k8s-01 all
dk ezctl setup k8s-01 all#或者分步安装具体使用 dk ezctl help setup 查看分步安装帮助信息
dk ezctl setup k8s-01 01
dk ezctl setup k8s-01 02
dk ezctl setup k8s-01 03
......七、坑
7.1 操作系统的坑
不要使用CentOS 7.9 系统作为master和node节点该系统ssh有bug。 bug为宿主机设置了免密登录宿主机可以免密登录到其他机器但是kubeasz容器不可以而kubeasz部署k8s是容器内的ansible需要有免密登录到其他机器的权限。
