Comments on How to Install a Kubernetes Docker Cluster on CentOS 7

Thank you very much for the article!

You can follow up the repository made by one of our developer with an additional thing of Horizontal Pod autoscaling of stateless application.

https://github.com/vevsatechnologies/Install-Kubernetes-on-CentOs

Was doing well, until I find I cant add nodes to the master, I get "no route to host" which makes no sense.   :(

Re: my comment on not working,  I traced this down to a firewall on the master which is installed by default on centos 7.5.   Odd thing is the port is 6443 but looks like ipv6 only according to netstat -tunlp  

So, firewall-cmd --add-port=6443/tcp --permanent ; firewall-cmd --relaod

However this rule needs tighetning to the kubernetes nodes only.

There is some explanation/command(s) missing to run 2 instances, one on each node. ie  I have 1 nginx instance running fine on one node which is what I expected to happen.  From the "kubectl deployment nginx" it say for replicas 1 desired etc 

Reading your screen shot I see "kubectl edit deployment nginx" and changing replicas from 1 to 2 and saving and I now get 2 nginx instances as shown.

rather neat.

 

 

I followed your tutorial without problems till cluster configuration on Centos7 (step 2). I'm connecting to internet over authenticated proxy (no other option) connection where I get error:

 

[root@docker4uat log]# kubeadm init --apiserver-advertise-address=10.x.x.x --pod-network-cidr=10.244.0.0/16

unable to get URL "https://dl.k8s.io/release/stable-1.11.txt": Get https://dl.k8s.io/release/stable-1.11.txt: x509: certificate signed by unknown authority

[root@docker4uat log]#

 

I see Kubelet is not running:

 

[root@docker4uat log]# systemctl status kubelet

? kubelet.service - kubelet: The Kubernetes Node Agent

   Loaded: loaded (/etc/systemd/system/kubelet.service; enabled; vendor preset: disabled)

  Drop-In: /etc/systemd/system/kubelet.service.d

           ??10-kubeadm.conf

   Active: activating (auto-restart) (Result: exit-code) since Tue 2018-09-11 09:12:32 CEST; 3s ago

     Docs: https://kubernetes.io/docs/

  Process: 62472 ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS (code=exited, status=255)

 Main PID: 62472 (code=exited, status=255)

Sep 11 09:12:32 docker4uat systemd[1]: kubelet.service: main process exited, code=exited, status=255/n/a

Sep 11 09:12:32 docker4uat systemd[1]: Unit kubelet.service entered failed state.

Sep 11 09:12:32 docker4uat systemd[1]: kubelet.service failed.

[root@docker4uat log]#

 

Logs from messages:

 

Sep 11 09:15:46 docker4uat systemd: Started kubelet: The Kubernetes Node Agent.

Sep 11 09:15:46 docker4uat systemd: Starting kubelet: The Kubernetes Node Agent...

Sep 11 09:15:46 docker4uat kubelet: F0911 09:15:46.503054   62789 server.go:190] failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory

Sep 11 09:15:46 docker4uat systemd: kubelet.service: main process exited, code=exited, status=255/n/a

Sep 11 09:15:46 docker4uat systemd: Unit kubelet.service entered failed state.

Sep 11 09:15:46 docker4uat systemd: kubelet.service failed.

 

Thanks for help!

Nice guide, thanks much!  I stumbled a bit upon one issue when trying to get node01/02 to join:

[root@node01 ~]# kubeadm join <myip>:6443 --token <mytoken> --discovery-toke n-ca-cert-hash sha256:<myhash>[preflight] running pre-flight checks        [WARNING RequiredIPVSKernelModulesAvailable]: the IPVS proxier will not be used, because the following required kernel modules are not loaded: [ip_vs_rr ip_vs_wrr ip_vs_sh ip_vs] or no  builtin kernel ipvs support: map[ip_vs:{} ip_vs_rr:{} ip_vs_wrr:{} ip_vs_sh:{} nf_conntrack_ipv4:{}]

My resolution was:

yum install -y ipvsadmmodprobe ip_vs_rrmodprobe ip_vs_wrrmodprobe ip_vs_shmodprobe ip_vs

Background:

[root@node02 ~]# cat /etc/redhat-releaseCentOS Linux release 7.4.1708 (Core)

Note also: [WARNING SystemVerification]: docker version is greater than the most recently validated version. Docker version: 18.06.1-ce. Max validated version: 17.03

Thanks again!

After reboot step vm is not connection. Brlow messege is coming ... in google cloud.

Transferring SSH keys to the VM.

I followed the article and the steps mentioned here to install Kubernetes cluster, but when tried "kubectl apply -f kube-flannel.yml

", getting the error as "http://<kubernetes-master-machine-ip>/api?timeout=32s: net/http: TLS handshake timeout", firewall, selinux in this machine is disabled, also from this machine wget can be done for any url. Could you please let me know, is it the our network issue or some kind of bug in kubernetes(docker version:1.13.1 and kubectl and kubeadm version is 1.12.1) 

Good Articles but it is not working. When i am trying to deploy a service on pods getting error Error Image pull. Kubernetes is not able to pull docker image from docker registry. can you resolve it

Just tried, wendorful instruction.

One thing to be mentioned. The latest version of k8s doesn't support the latest docker at this moment. (today: 2018-11-20 17:41:45)

The latest version of k8s is "v1.12.2" and it supports Docker-CE "18.06".

While the latest version of Docker-CE is "18.09" which has not been verified by k8s team. So it leads error in "kubeadm init".

Ref:

https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/util/system/docker_validator.go#L41 

 

The good way is to install docker-ce-18.06.1.ce at beginning. But if you have got into the trouble like me, you need to `yum remove docker-ce docker-ce-cli` and then `yum install docker-ce-18.06.1.ce`. Don't forget to `systemctl restart docker && systemctl enable docker` after reinstallation.

 

Cheers and thanks!

Charles - https://www.linkedin.com/in/iamchen/

 

Thanks, i followed this and was able to create two node K8s cluster for my POC.

Thank you for the article. It was short, to the point, easy to follow, gets the job done. Good job!

helped me a lot..

but unable to list kubectl get nodes

The connection to the server :8080 was refused - did you specify the right host or port?

Hi,

 

I have same problem

Nice starter article!

Just a tip that should be added:

* Get Token

# kubeadm token list # Check the expiration, it might need to be created a new token

* Get Token ca cert hash

# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \

openssl dgst -sha256 -hex | sed 's/^.* //'

 

 Great article so far.  Thank you for this.  I'm having an issue though. When I run

sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

I get this:  sed: can't read /etc/systemd/system/kubelet.service.d/10-kubeadm.conf: No such file or directory

Can anybody help?

 

Many thanks

Hi,

       Although it is a very good article yet I don't understand how external IP address appears with Nginx service? You didn't mention it. Can you help me with this issue?  Please don't share any link. I have checked almost every one of it. My Output with your solution is this:

NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE

kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        22h

nginx        NodePort    10.104.138.19   <none>        80:30901/TCP   21h

 

you can see external Ip address is missing.

 

 

Solution for:

sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

I get this:  sed: can't read /etc/systemd/system/kubelet.service.d/10-kubeadm.conf: No such file or directory

 

look for the file "10-kubeadm.conf" in another location

find / -name "10-kubeadm.conf"

then excute the sed on the aother location. for me it was:

sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

Thank you, that works!

This guide should be updated to include this

Hi 

I was able to setup cluster ,can you please help me in setting up HA master in kubernates that will be really help full

Thanks! Same here!

Post reboot worker node and running below command:

sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

-getting below error. can someone please help on this.

sed: can't read /etc/systemd/system/kubelet.service.d/10-kubeadm.conf: No such file or directory

Bhai location of the file has been changed in new version. Better use locate command to get the exact location and change there.

Thank you bro. You saved my time. 

check your internet connection and if you can reach to the URL from your browser.

Hi, Please could you help me.

have made it to step 2 and command

sudo minikube start --vm-driver=none --extra-config=kubelet.cgroup-driver=systemd

Then I have these errors: Could anyone help me get around this please?

[root@localhost ~]# kubeadm init --apiserver-advertise-address=192.168.1.50 --pod-network-cidr=10.244.0.0/16

[init] Using Kubernetes version: v1.16.3

[preflight] Running pre-flight checks

        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.5. Latest validated version: 18.09

error execution phase preflight: [preflight] Some fatal errors occurred:

        [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1

[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

To see the stack trace of this error execute with --v=5 or higher

 

Would anyone be kind enough to help me with these new errors please?

I have followed the instructions to the T in my Centos 7.4 build and cannot get the node01 to talk to the master, here are the errors below, please help if you can..

 

Many thanks

 

[root@localhost ~]# kubeadm join 192.168.1.50:6443 --token ur6qar.924b9jbm6nnrr5hb     --discovery-token-ca-cert-hash sha256:1822caf7b3120ad3682e2a476ebe91050bfc02d22953c051ad0871afce9d25d7

[preflight] Running pre-flight checks

        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/

[preflight] The system verification failed. Printing the output from the verification:

KERNEL_VERSION: 3.10.0-693.el7.x86_64

DOCKER_VERSION: 19.03.5

OS: Linux

CGROUPS_CPU: enabled

CGROUPS_CPUACCT: enabled

CGROUPS_CPUSET: enabled

CGROUPS_DEVICES: enabled

CGROUPS_FREEZER: enabled

CGROUPS_MEMORY: enabled

        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.5. Latest validated version: 18.09

error execution phase preflight: [preflight] Some fatal errors occurred:

        [ERROR SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found.\n", err: exit status 1

[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

To see the stack trace of this error execute with --v=5 or higher

[root@localhost ~]#

 

Getting below error after running kubeadm join 192.168.33.223:6443 --token ykbgtz.e9xulvbgos1ghtfv --discovery-token-ca-cert-hash sha256:3dd47b2b6e706e0076723b7c2723c8169c59da60d2cbfd449 

 

error execution phase preflight: couldn't validate the identity of the API Server: abort connecting to API servers after timeout of 5m0s

To see the stack trace of this error execute with --v=5 or higher

 

Solution:firewall-cmd --permanent --add-port=6443/tcp on master and worker node

I have already configured this cluster but I want to configure master and slave Kubernetes servers with 2 workers node. I also want to access the graphical mode of Kubernetes. Could you please write an article on that.

2 Kubernetes with 2 worker nodes Graphical GUI of Kubernetes based on IP address not localhost IP.  Create Apache cluster service and how to access them.

 

You may also visit us: curiousviral.com

I got error - 

[root@localhost ~]# sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

sed: can't read /etc/systemd/system/kubelet.service.d/10-kubeadm.conf: No such file or directory

However, 'kubeadm init' got success by giving token in tha last. But when I try to install in my office environment (where there is no direct internet access; internet access is over proxy server) 'kubeadm init' command didn't got sucsess. [Note that I set http proxy and https proxy in /etc/envirnoment file and curl gets success] 

My question how can I set proxy IP in host machine to get kubeadm success? 

 

kubeadm init command unsuccessful in Office environment-

[root@k8s-master ~]# kubeadm init

W0202 05:02:21.434263   27744 validation.go:28] Cannot validate kube-proxy config - no validator is available

W0202 05:02:21.434429   27744 validation.go:28] Cannot validate kubelet config - no validator is available

[init] Using Kubernetes version: v1.17.2

[preflight] Running pre-flight checks

        [WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly

        [WARNING Hostname]: hostname "k8s-master" could not be reached

        [WARNING Hostname]: hostname "k8s-master": lookup k8s-master on [::1]:53: read udp [::1]:59607->[::1]:53: read: connection refused

[preflight] Pulling images required for setting up a Kubernetes cluster

[preflight] This might take a minute or two, depending on the speed of your internet connection

[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'

error execution phase preflight: [preflight] Some fatal errors occurred:

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.17.2: output: Trying to pull repository k8s.gcr.io/kube-apiserver ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:35312->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.17.2: output: Trying to pull repository k8s.gcr.io/kube-controller-manager ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:54559->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.17.2: output: Trying to pull repository k8s.gcr.io/kube-scheduler ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:35964->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.17.2: output: Trying to pull repository k8s.gcr.io/kube-proxy ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:43769->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: Trying to pull repository k8s.gcr.io/pause ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:53031->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.4.3-0: output: Trying to pull repository k8s.gcr.io/etcd ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:47086->[::1]:53: read: connection refused

, error: exit status 1

        [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.6.5: output: Trying to pull repository k8s.gcr.io/coredns ...

Get https://k8s.gcr.io/v1/_ping: dial tcp: lookup k8s.gcr.io on [::1]:53: read udp [::1]:50361->[::1]:53: read: connection refused

, error: exit status 1

[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

To see the stack trace of this error execute with --v=5 or higher

 

 

hello, thank you for this articlebut I have a problem when I use this command I get error

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

error: unable to read URL "https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml", server reported 500 Internal Server Error, status code=500

would anyone can help?

I have problem, after instalation kubernetes version 1.18.8

systemctl status kubelet.service

Unit kubelet.service entered failed state

kubelet.service failed

 

tailf /var/log/messages

 

failed to load Kubelet config file /var/lib/kubelet/config.yaml, error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory

 

Kubernetes 1.11 (or higher) packages are not supported by the moment. The support for newer Kubernetes packages is planned in the future.

Maybe this is the problem, when i got 

failed to load Kubelet config file /var/lib/kubelet/config.yaml

error failed to read kubelet config file "/var/lib/kubelet/config.yaml", error: open /var/lib/kubelet/config.yaml: no such file or directory

I don't have directory ll /var/lib/ku

 

service didn't create the localization /var/lib/kubelet