Question: What are the benefits of Volume Snapshot Classes?

Answer: Volume Snapshot ClassesThis document describes the concept of VolumeSnapshotClass in Kubernetes. Familiarity with volume snapshots and storage classes is suggested.IntroductionJust like StorageClass provides a way for administrators to describe the "classes" of storage they offer when provisioning a volume, VolumeSnapshotClass provides a way to describe the "classes" of storage when provisioning a volume snapshot.The VolumeSnapshotClass ResourceEach VolumeSnapshotClass contains the fields driver, deletionPolicy, and parameters, which are used when a VolumeSnapshot belonging to the class needs to be dynamically provisioned.The name of a VolumeSnapshotClass object is significant, and is how users can request a particular class. Administrators set the name and other parameters of a class when first creating VolumeSnapshotClass objects, and the objects cannot be updated once they are created.Note:Installation of the CRDs is the responsibility of the Kubernetes distribution. Without the required CRDs present, the creation of a VolumeSnapshotClass fails.apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-hostpath-snapclass driver: hostpath.csi.k8s.io deletionPolicy: Delete parameters: Administrators can specify a default VolumeSnapshotClass for VolumeSnapshots that don't request any particular class to bind to by adding the snapshot.storage.kubernetes.io/is-default-class: "true" annotation:apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: csi-hostpath-snapclass annotations: snapshot.storage.kubernetes.io/is-default-class: "true" driver: hostpath.csi.k8s.io deletionPolicy: Delete parameters: If multiple CSI drivers exist, a default VolumeSnapshotClass can be specified for each of them.VolumeSnapshotClass dependenciesWhen you create a VolumeSnapshot without specifying a VolumeSnapshotClass, Kubernetes automatically selects a default VolumeSnapshotClass that has a CSI driver matching the CSI driver of the PVC’s StorageClass.This behavior allows multiple default VolumeSnapshotClass objects to coexist in a cluster, as long as each one is associated with a unique CSI driver.Always ensure that there is only one default VolumeSnapshotClass for each CSI driver. If multiple default VolumeSnapshotClass objects are created using the same CSI driver, a VolumeSnapshot creation will fail because Kubernetes cannot determine which one to use.DriverVolume snapshot classes have a driver that determines what CSI volume plugin is used for provisioning VolumeSnapshots. This field must be specified.DeletionPolicyVolume snapshot classes have a deletionPolicy. It enables you to configure what happens to a VolumeSnapshotContent when the VolumeSnapshot object it is bound to is to be deleted. The deletionPolicy of a volume snapshot class can either be Retain or Delete. This field must be specified.If the deletionPolicy is Delete, then the underlying storage snapshot will be deleted along with the VolumeSnapshotConten


Question: What is Volume Snapshot Classes?

Answer: T h i s   d o c u m e n t   d e s c r i b e s   t h e   c o n c e p t   o f   V o l u m e S n a p s h o t C l a s s   i n   K u b e r n e t e s .   F a m i l i a r i t y   w i t h   v o l u m e   s n a p s h o t s   a n d   s t o r a g e   c l a s s e s   i s   s u g g e s t e d .


Question: What is Introduction?

Answer: J u s t   l i k e   S t o r a g e C l a s s   p r o v i d e s   a   w a y   f o r   a d m i n i s t r a t o r s   t o   d e s c r i b e   t h e   " c l a s s e s "   o f   s t o r a g e   t h e y   o f f e r   w h e n   p r o v i s i o n i n g   a   v o l u m e ,   V o l u m e S n a p s h o t C l a s s   p r o v i d e s   a   w a y   t o   d e s c r i b e   t h e   " c l a s s e s "   o f   s t o r a g e   w h e n   p r o v i s i o n i n g   a   v o l u m e   s n a p s h o t .


Question: What is The VolumeSnapshotClass Resource?

Answer: E a c h   V o l u m e S n a p s h o t C l a s s   c o n t a i n s   t h e   f i e l d s   d r i v e r ,   d e l e t i o n P o l i c y ,   a n d   p a r a m e t e r s ,   w h i c h   a r e   u s e d   w h e n   a   V o l u m e S n a p s h o t   b e l o n g i n g   t o   t h e   c l a s s   n e e d s   t o   b e   d y n a m i c a l l y   p r o v i s i o n e d .   T h e   n a m e   o f   a   V o l u m e S n a p s h o t C l a s s   o b j e c t   i s   s i g n i f i c a n t ,   a n d   i s   h o w   u s e r s   c a n   r e q u e s t   a   p a r t i c u l a r   c l a s s .   A d m i n i s t r a t o r s   s e t   t h e   n a m e   a n d   o t h e r   p a r a m e t e r s   o f   a   c l a s s   w h e n   f i r s t   c r e a t i n g   V o l u m e S n a p s h o t C l a s s   o b j e c t s ,   a n d   t h e   o b j e c t s   c a n n o t   b e   u p d a t e d   o n c e   t h e y   a r e   c r e a t e d .


Question: What is Note:?

Answer: a p i V e r s i o n :   s n a p s h o t . s t o r a g e . k 8 s . i o / v 1   k i n d :   V o l u m e S n a p s h o t C l a s s   m e t a d a t a :   n a m e :   c s i - h o s t p a t h - s n a p c l a s s   d r i v e r :   h o s t p a t h . c s i . k 8 s . i o   d e l e t i o n P o l i c y :   D e l e t e   p a r a m e t e r s :   A d m i n i s t r a t o r s   c a n   s p e c i f y   a   d e f a u l t   V o l u m e S n a p s h o t C l a s s   f o r   V o l u m e S n a p s h o t s   t h a t   d o n ' t   r e q u e s t   a n y   p a r t i c u l a r   c l a s s   t o   b i n d   t o   b y   a d d i n g   t h e   s n a p s h o t . s t o r a g e . k u b e r n e t e s . i o / i s - d e f a u l t - c l a s s :   " t r u e "   a n n o t a t i o n :   a p i V e r s i o n :   s n a p s h o t . s t o r a g e . k 8 s . i o / v 1   k i n d :   V o l u m e S n a p s h o t C l a s s   m e t a d a t a :   n a m e :   c s i - h o s t p a t h - s n a p c l a s s   a n n o t a t i o n s :   s n a p s h o t . s t o r a g e . k u b e r n e t e s . i o / i s - d e f a u l t - c l a s s :   " t r u e "   d r i v e r :   h o s t p a t h . c s i . k 8 s . i o   d e l e t i o n P o l i c y :   D e l e t e   p a r a m e t e r s :   I f   m u l t i p l e   C S I   d r i v e r s   e x i s t ,   a   d e f a u l t   V o l u m e S n a p s h o t C l a s s   c a n   b e   s p e c i f i e d   f o r   e a c h   o f   t h e m .


Question: What is VolumeSnapshotClass dependencies?

Answer: W h e n   y o u   c r e a t e   a   V o l u m e S n a p s h o t   w i t h o u t   s p e c i f y i n g   a   V o l u m e S n a p s h o t C l a s s ,   K u b e r n e t e s   a u t o m a t i c a l l y   s e l e c t s   a   d e f a u l t   V o l u m e S n a p s h o t C l a s s   t h a t   h a s   a   C S I   d r i v e r   m a t c h i n g   t h e   C S I   d r i v e r   o f   t h e   P V C ’ s   S t o r a g e C l a s s .   T h i s   b e h a v i o r   a l l o w s   m u l t i p l e   d e f a u l t   V o l u m e S n a p s h o t C l a s s   o b j e c t s   t o   c o e x i s t   i n   a   c l u s t e r ,   a s   l o n g   a s   e a c h   o n e   i s   a s s o c i a t e d   w i t h   a   u n i q u e   C S I   d r i v e r .   A l w a y s   e n s u r e   t h a t   t h e r e   i s   o n l y   o n e   d e f a u l t   V o l u m e S n a p s h o t C l a s s   f o r   e a c h   C S I   d r i v e r .   I f   m u l t i p l e   d e f a u l t   V o l u m e S n a p s h o t C l a s s   o b j e c t s   a r e   c r e a t e d   u s i n g   t h e   s a m e   C S I   d r i v e r ,   a   V o l u m e S n a p s h o t   c r e a t i o n   w i l l   f a i l   b e c a u s e   K u b e r n e t e s   c a n n o t   d e t e r m i n e   w h i c h   o n e   t o   u s e .


Question: What is Driver?

Answer: V o l u m e   s n a p s h o t   c l a s s e s   h a v e   a   d r i v e r   t h a t   d e t e r m i n e s   w h a t   C S I   v o l u m e   p l u g i n   i s   u s e d   f o r   p r o v i s i o n i n g   V o l u m e S n a p s h o t s .   T h i s   f i e l d   m u s t   b e   s p e c i f i e d .


Question: What is DeletionPolicy?

Answer: V o l u m e   s n a p s h o t   c l a s s e s   h a v e   a   d e l e t i o n P o l i c y .   I t   e n a b l e s   y o u   t o   c o n f i g u r e   w h a t   h a p p e n s   t o   a   V o l u m e S n a p s h o t C o n t e n t   w h e n   t h e   V o l u m e S n a p s h o t   o b j e c t   i t   i s   b o u n d   t o   i s   t o   b e   d e l e t e d .   T h e   d e l e t i o n P o l i c y   o f   a   v o l u m e   s n a p s h o t   c l a s s   c a n   e i t h e r   b e   R e t a i n   o r   D e l e t e .   T h i s   f i e l d   m u s t   b e   s p e c i f i e d .   I f   t h e   d e l e t i o n P o l i c y   i s   D e l e t e ,   t h e n   t h e   u n d e r l y i n g   s t o r a g e   s n a p s h o t   w i l l   b e   d e l e t e d   a l o n g   w i t h   t h e   V o l u m e S n a p s h o t C o n t e n t   o b j e c t .   I f   t h e   d e l e t i o n P o l i c y   i s   R e t a i n ,   t h e n   b o t h   t h e   u n d e r l y i n g   s n a p s h o t   a n d   V o l u m e S n a p s h o t C o n t e n t   r e m a i n .


Question: What is Parameters?

Answer: V o l u m e   s n a p s h o t   c l a s s e s   h a v e   p a r a m e t e r s   t h a t   d e s c r i b e   v o l u m e   s n a p s h o t s   b e l o n g i n g   t o   t h e   v o l u m e   s n a p s h o t   c l a s s .   D i f f e r e n t   p a r a m e t e r s   m a y   b e   a c c e p t e d   d e p e n d i n g   o n   t h e   d r i v e r .


Question: What is Securing a Cluster?

Answer: Securing a ClusterThis document covers topics related to protecting a cluster from accidental or malicious access and provides recommendations on overall security.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Controlling access to the Kubernetes APIAs Kubernetes is entirely API-driven, controlling and limiting who can access the cluster and what actions they are allowed to perform is the first line of defense.Use Transport Layer Security (TLS) for all API trafficKubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components. Note that some components and installation methods may enable local ports over HTTP and administrators should familiarize themselves with the settings of each component to identify potentially unsecured traffic.API AuthenticationChoose an authentication mechanism for the API servers to use that matches the common access patterns when you install a cluster. For instance, small, single-user clusters may wish to use a simple certificate or static Bearer token approach. Larger clusters may wish to integrate an existing OIDC or LDAP server that allow users to be subdivided into groups.All API clients must be authenticated, even those that are part of the infrastructure like nodes, proxies, the scheduler, and volume plugins. These clients are typically service accounts or use x509 client certificates, and they are created automatically at cluster startup or are setup as part of the cluster installation.Consult the authentication reference document for more information.API AuthorizationOnce authenticated, every API call is also expected to pass an authorization check. Kubernetes ships an integrated Role-Based Access Control (RBAC) component that matches an incoming user or group to a set of permissions bundled into roles. These permissions combine verbs (get, create, delete) with resources (pods, services, nodes) and can be namespace-scoped or cluster-scoped. A set of out-of-the-box roles are provided that offer reasonable default separation of responsibility depending on what actions a client might want to perform. It is recommended that you use the Node and RBAC authorizers together, in combination with the NodeRestriction admission plugin.As with authentication, simple and broad roles may be appropriate for smaller clusters, but as more users interact with the cluster, it may become necessary to separate teams into separate namespaces wi


Question: How does Securing a Cluster work?

Answer: Securing a ClusterThis document covers topics related to protecting a cluster from accidental or malicious access and provides recommendations on overall security.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Controlling access to the Kubernetes APIAs Kubernetes is entirely API-driven, controlling and limiting who can access the cluster and what actions they are allowed to perform is the first line of defense.Use Transport Layer Security (TLS) for all API trafficKubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components. Note that some components and installation methods may enable local ports over HTTP and administrators should familiarize themselves with the settings of each component to identify potentially unsecured traffic.API AuthenticationChoose an authentication mechanism for the API servers to use that matches the common access patterns when you install a cluster. For instance, small, single-user clusters may wish to use a simple certificate or static Bearer token approach. Larger clusters may wish to integrate an existing OIDC or LDAP server that allow users to be subdivided into groups.All API clients must be authenticated, even those that are part of the infrastructure like nodes, proxies, the scheduler, and volume plugins. These clients are typically service accounts or use x509 client certificates, and they are created automatically at cluster startup or are setup as part of the cluster installation.Consult the authentication reference document for more information.API AuthorizationOnce authenticated, every API call is also expected to pass an authorization check. Kubernetes ships an integrated Role-Based Access Control (RBAC) component that matches an incoming user or group to a set of permissions bundled into roles. These permissions combine verbs (get, create, delete) with resources (pods, services, nodes) and can be namespace-scoped or cluster-scoped. A set of out-of-the-box roles are provided that offer reasonable default separation of responsibility depending on what actions a client might want to perform. It is recommended that you use the Node and RBAC authorizers together, in combination with the NodeRestriction admission plugin.As with authentication, simple and broad roles may be appropriate for smaller clusters, but as more users interact with the cluster, it may become necessary to separate teams into separate namespaces wi


Question: When would you use Securing a Cluster?

Answer: Securing a ClusterThis document covers topics related to protecting a cluster from accidental or malicious access and provides recommendations on overall security.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Controlling access to the Kubernetes APIAs Kubernetes is entirely API-driven, controlling and limiting who can access the cluster and what actions they are allowed to perform is the first line of defense.Use Transport Layer Security (TLS) for all API trafficKubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components. Note that some components and installation methods may enable local ports over HTTP and administrators should familiarize themselves with the settings of each component to identify potentially unsecured traffic.API AuthenticationChoose an authentication mechanism for the API servers to use that matches the common access patterns when you install a cluster. For instance, small, single-user clusters may wish to use a simple certificate or static Bearer token approach. Larger clusters may wish to integrate an existing OIDC or LDAP server that allow users to be subdivided into groups.All API clients must be authenticated, even those that are part of the infrastructure like nodes, proxies, the scheduler, and volume plugins. These clients are typically service accounts or use x509 client certificates, and they are created automatically at cluster startup or are setup as part of the cluster installation.Consult the authentication reference document for more information.API AuthorizationOnce authenticated, every API call is also expected to pass an authorization check. Kubernetes ships an integrated Role-Based Access Control (RBAC) component that matches an incoming user or group to a set of permissions bundled into roles. These permissions combine verbs (get, create, delete) with resources (pods, services, nodes) and can be namespace-scoped or cluster-scoped. A set of out-of-the-box roles are provided that offer reasonable default separation of responsibility depending on what actions a client might want to perform. It is recommended that you use the Node and RBAC authorizers together, in combination with the NodeRestriction admission plugin.As with authentication, simple and broad roles may be appropriate for smaller clusters, but as more users interact with the cluster, it may become necessary to separate teams into separate namespaces wi


Question: What are the benefits of Securing a Cluster?

Answer: Securing a ClusterThis document covers topics related to protecting a cluster from accidental or malicious access and provides recommendations on overall security.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesTo check the version, enter kubectl version.Controlling access to the Kubernetes APIAs Kubernetes is entirely API-driven, controlling and limiting who can access the cluster and what actions they are allowed to perform is the first line of defense.Use Transport Layer Security (TLS) for all API trafficKubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components. Note that some components and installation methods may enable local ports over HTTP and administrators should familiarize themselves with the settings of each component to identify potentially unsecured traffic.API AuthenticationChoose an authentication mechanism for the API servers to use that matches the common access patterns when you install a cluster. For instance, small, single-user clusters may wish to use a simple certificate or static Bearer token approach. Larger clusters may wish to integrate an existing OIDC or LDAP server that allow users to be subdivided into groups.All API clients must be authenticated, even those that are part of the infrastructure like nodes, proxies, the scheduler, and volume plugins. These clients are typically service accounts or use x509 client certificates, and they are created automatically at cluster startup or are setup as part of the cluster installation.Consult the authentication reference document for more information.API AuthorizationOnce authenticated, every API call is also expected to pass an authorization check. Kubernetes ships an integrated Role-Based Access Control (RBAC) component that matches an incoming user or group to a set of permissions bundled into roles. These permissions combine verbs (get, create, delete) with resources (pods, services, nodes) and can be namespace-scoped or cluster-scoped. A set of out-of-the-box roles are provided that offer reasonable default separation of responsibility depending on what actions a client might want to perform. It is recommended that you use the Node and RBAC authorizers together, in combination with the NodeRestriction admission plugin.As with authentication, simple and broad roles may be appropriate for smaller clusters, but as more users interact with the cluster, it may become necessary to separate teams into separate namespaces wi


Question: What is Securing a Cluster?

Answer: T h i s   d o c u m e n t   c o v e r s   t o p i c s   r e l a t e d   t o   p r o t e c t i n g   a   c l u s t e r   f r o m   a c c i d e n t a l   o r   m a l i c i o u s   a c c e s s   a n d   p r o v i d e s   r e c o m m e n d a t i o n s   o n   o v e r a l l   s e c u r i t y .


Question: What is Before you begin?

Answer: Y o u   n e e d   t o   h a v e   a   K u b e r n e t e s   c l u s t e r ,   a n d   t h e   k u b e c t l   c o m m a n d - l i n e   t o o l   m u s t   b e   c o n f i g u r e d   t o   c o m m u n i c a t e   w i t h   y o u r   c l u s t e r .   I t   i s   r e c o m m e n d e d   t o   r u n   t h i s   t u t o r i a l   o n   a   c l u s t e r   w i t h   a t   l e a s t   t w o   n o d e s   t h a t   a r e   n o t   a c t i n g   a s   c o n t r o l   p l a n e   h o s t s .   I f   y o u   d o   n o t   a l r e a d y   h a v e   a   c l u s t e r ,   y o u   c a n   c r e a t e   o n e   b y   u s i n g   m i n i k u b e   o r   y o u   c a n   u s e   o n e   o f   t h e s e   K u b e r n e t e s   p l a y g r o u n d s : i x i m i u z   L a b s K i l l e r c o d a K o d e K l o u d P l a y   w i t h   K u b e r n e t e s T o   c h e c k   t h e   v e r s i o n ,   e n t e r   k u b e c t l   v e r s i o n .   Y o u   n e e d   t o   h a v e   a   K u b e r n e t e s   c l u s t e r ,   a n d   t h e   k u b e c t l   c o m m a n d - l i n e   t o o l   m u s t   b e   c o n f i g u r e d   t o   c o m m u n i c a t e   w i t h   y o u r   c l u s t e r .   I t   i s   r e c o m m e n d e d   t o   r u n   t h i s   t u t o r i a l   o n   a   c l u s t e r   w i t h   a t   l e a s t   t w o   n o d e s   t h a t   a r e   n o t   a c t i n g   a s   c o n t r o l   p l a n e   h o s t s .   I f   y o u   d o   n o t   a l r e a d y   h a v e   a   c l u s t e r ,   y o u   c a n   c r e a t e   o n e   b y   u s i n g   m i n i k u b e   o r   y o u   c a n   u s e   o n e   o f   t h e s e   K u b e r n e t e s   p l a y g r o u n d s :   i x i m i u z   L a b s K i l l e r c o d a K o d e K l o u d P l a y   w i t h   K u b e r n e t e s   T o   c h e c k   t h e   v e r s i o n ,   e n t e r   k u b e c t l   v e r s i o n .


Question: What is Controlling access to the Kubernetes API?

Answer: A s   K u b e r n e t e s   i s   e n t i r e l y   A P I - d r i v e n ,   c o n t r o l l i n g   a n d   l i m i t i n g   w h o   c a n   a c c e s s   t h e   c l u s t e r   a n d   w h a t   a c t i o n s   t h e y   a r e   a l l o w e d   t o   p e r f o r m   i s   t h e   f i r s t   l i n e   o f   d e f e n s e .


Question: What is Use Transport Layer Security (TLS) for all API traffic?

Answer: K u b e r n e t e s   e x p e c t s   t h a t   a l l   A P I   c o m m u n i c a t i o n   i n   t h e   c l u s t e r   i s   e n c r y p t e d   b y   d e f a u l t   w i t h   T L S ,   a n d   t h e   m a j o r i t y   o f   i n s t a l l a t i o n   m e t h o d s   w i l l   a l l o w   t h e   n e c e s s a r y   c e r t i f i c a t e s   t o   b e   c r e a t e d   a n d   d i s t r i b u t e d   t o   t h e   c l u s t e r   c o m p o n e n t s .   N o t e   t h a t   s o m e   c o m p o n e n t s   a n d   i n s t a l l a t i o n   m e t h o d s   m a y   e n a b l e   l o c a l   p o r t s   o v e r   H T T P   a n d   a d m i n i s t r a t o r s   s h o u l d   f a m i l i a r i z e   t h e m s e l v e s   w i t h   t h e   s e t t i n g s   o f   e a c h   c o m p o n e n t   t o   i d e n t i f y   p o t e n t i a l l y   u n s e c u r e d   t r a f f i c .


Question: What is API Authentication?

Answer: C h o o s e   a n   a u t h e n t i c a t i o n   m e c h a n i s m   f o r   t h e   A P I   s e r v e r s   t o   u s e   t h a t   m a t c h e s   t h e   c o m m o n   a c c e s s   p a t t e r n s   w h e n   y o u   i n s t a l l   a   c l u s t e r .   F o r   i n s t a n c e ,   s m a l l ,   s i n g l e - u s e r   c l u s t e r s   m a y   w i s h   t o   u s e   a   s i m p l e   c e r t i f i c a t e   o r   s t a t i c   B e a r e r   t o k e n   a p p r o a c h .   L a r g e r   c l u s t e r s   m a y   w i s h   t o   i n t e g r a t e   a n   e x i s t i n g   O I D C   o r   L D A P   s e r v e r   t h a t   a l l o w   u s e r s   t o   b e   s u b d i v i d e d   i n t o   g r o u p s .   A l l   A P I   c l i e n t s   m u s t   b e   a u t h e n t i c a t e d ,   e v e n   t h o s e   t h a t   a r e   p a r t   o f   t h e   i n f r a s t r u c t u r e   l i k e   n o d e s ,   p r o x i e s ,   t h e   s c h e d u l e r ,   a n d   v o l u m e   p l u g i n s .   T h e s e   c l i e n t s   a r e   t y p i c a l l y   s e r v i c e   a c c o u n t s   o r   u s e   x 5 0 9   c l i e n t   c e r t i f i c a t e s ,   a n d   t h e y   a r e   c r e a t e d   a u t o m a t i c a l l y   a t   c l u s t e r   s t a r t u p   o r   a r e   s e t u p   a s   p a r t   o f   t h e   c l u s t e r   i n s t a l l a t i o n .   C o n s u l t   t h e   a u t h e n t i c a t i o n   r e f e r e n c e   d o c u m e n t   f o r   m o r e   i n f o r m a t i o n .


Question: What is API Authorization?

Answer: O n c e   a u t h e n t i c a t e d ,   e v e r y   A P I   c a l l   i s   a l s o   e x p e c t e d   t o   p a s s   a n   a u t h o r i z a t i o n   c h e c k .   K u b e r n e t e s   s h i p s   a n   i n t e g r a t e d   R o l e - B a s e d   A c c e s s   C o n t r o l   ( R B A C )   c o m p o n e n t   t h a t   m a t c h e s   a n   i n c o m i n g   u s e r   o r   g r o u p   t o   a   s e t   o f   p e r m i s s i o n s   b u n d l e d   i n t o   r o l e s .   T h e s e   p e r m i s s i o n s   c o m b i n e   v e r b s   ( g e t ,   c r e a t e ,   d e l e t e )   w i t h   r e s o u r c e s   ( p o d s ,   s e r v i c e s ,   n o d e s )   a n d   c a n   b e   n a m e s p a c e - s c o p e d   o r   c l u s t e r - s c o p e d .   A   s e t   o f   o u t - o f - t h e - b o x   r o l e s   a r e   p r o v i d e d   t h a t   o f f e r   r e a s o n a b l e   d e f a u l t   s e p a r a t i o n   o f   r e s p o n s i b i l i t y   d e p e n d i n g   o n   w h a t   a c t i o n s   a   c l i e n t   m i g h t   w a n t   t o   p e r f o r m .   I t   i s   r e c o m m e n d e d   t h a t   y o u   u s e   t h e   N o d e   a n d   R B A C   a u t h o r i z e r s   t o g e t h e r ,   i n   c o m b i n a t i o n   w i t h   t h e   N o d e R e s t r i c t i o n   a d m i s s i o n   p l u g i n .   A s   w i t h   a u t h e n t i c a t i o n ,   s i m p l e   a n d   b r o a d   r o l e s   m a y   b e   a p p r o p r i a t e   f o r   s m a l l e r   c l u s t e r s ,   b u t   a s   m o r e   u s e r s   i n t e r a c t   w i t h   t h e   c l u s t e r ,   i t   m a y   b e c o m e   n e c e s s a r y   t o   s e p a r a t e   t e a m s   i n t o   s e p a r a t e   n a m e s p a c e s   w i t h   m o r e   l i m i t e d   r o l e s .   W i t h   a u t h o r i z a t i o n ,   i t   i s   i m p o r t a n t   t o   u n d e r s t a n d   h o w   u p d a t e s   o n   o n e   o b j e c t   m a y   c a u s e   a c t i o n s   i n   o t h e r   p l


Question: What is Controlling access to the Kubelet?

Answer: K u b e l e t s   e x p o s e   H T T P S   e n d p o i n t s   w h i c h   g r a n t   p o w e r f u l   c o n t r o l   o v e r   t h e   n o d e   a n d   c o n t a i n e r s .   B y   d e f a u l t   K u b e l e t s   a l l o w   u n a u t h e n t i c a t e d   a c c e s s   t o   t h i s   A P I .   P r o d u c t i o n   c l u s t e r s   s h o u l d   e n a b l e   K u b e l e t   a u t h e n t i c a t i o n   a n d   a u t h o r i z a t i o n .   C o n s u l t   t h e   K u b e l e t   a u t h e n t i c a t i o n / a u t h o r i z a t i o n   r e f e r e n c e   f o r   m o r e   i n f o r m a t i o n .


Question: What is Controlling the capabilities of a workload or user at runtime?

Answer: A u t h o r i z a t i o n   i n   K u b e r n e t e s   i s   i n t e n t i o n a l l y   h i g h   l e v e l ,   f o c u s e d   o n   c o a r s e   a c t i o n s   o n   r e s o u r c e s .   M o r e   p o w e r f u l   c o n t r o l s   e x i s t   a s   p o l i c i e s   t o   l i m i t   b y   u s e   c a s e   h o w   t h o s e   o b j e c t s   a c t   o n   t h e   c l u s t e r ,   t h e m s e l v e s ,   a n d   o t h e r   r e s o u r c e s .


Question: What is Limiting resource usage on a cluster?

Answer: R e s o u r c e   q u o t a   l i m i t s   t h e   n u m b e r   o r   c a p a c i t y   o f   r e s o u r c e s   g r a n t e d   t o   a   n a m e s p a c e .   T h i s   i s   m o s t   o f t e n   u s e d   t o   l i m i t   t h e   a m o u n t   o f   C P U ,   m e m o r y ,   o r   p e r s i s t e n t   d i s k   a   n a m e s p a c e   c a n   a l l o c a t e ,   b u t   c a n   a l s o   c o n t r o l   h o w   m a n y   p o d s ,   s e r v i c e s ,   o r   v o l u m e s   e x i s t   i n   e a c h   n a m e s p a c e .   L i m i t   r a n g e s   r e s t r i c t   t h e   m a x i m u m   o r   m i n i m u m   s i z e   o f   s o m e   o f   t h e   r e s o u r c e s   a b o v e ,   t o   p r e v e n t   u s e r s   f r o m   r e q u e s t i n g   u n r e a s o n a b l y   h i g h   o r   l o w   v a l u e s   f o r   c o m m o n l y   r e s e r v e d   r e s o u r c e s   l i k e   m e m o r y ,   o r   t o   p r o v i d e   d e f a u l t   l i m i t s   w h e n   n o n e   a r e   s p e c i f i e d .


Question: What is Controlling what privileges containers run with?

Answer: A   p o d   d e f i n i t i o n   c o n t a i n s   a   s e c u r i t y   c o n t e x t   t h a t   a l l o w s   i t   t o   r e q u e s t   a c c e s s   t o   r u n   a s   a   s p e c i f i c   L i n u x   u s e r   o n   a   n o d e   ( l i k e   r o o t ) ,   a c c e s s   t o   r u n   p r i v i l e g e d   o r   a c c e s s   t h e   h o s t   n e t w o r k ,   a n d   o t h e r   c o n t r o l s   t h a t   w o u l d   o t h e r w i s e   a l l o w   i t   t o   r u n   u n f e t t e r e d   o n   a   h o s t i n g   n o d e .   Y o u   c a n   c o n f i g u r e   P o d   s e c u r i t y   a d m i s s i o n   t o   e n f o r c e   u s e   o f   a   p a r t i c u l a r   P o d   S e c u r i t y   S t a n d a r d   i n   a   n a m e s p a c e ,   o r   t o   d e t e c t   b r e a c h e s .   G e n e r a l l y ,   m o s t   a p p l i c a t i o n   w o r k l o a d s   n e e d   l i m i t e d   a c c e s s   t o   h o s t   r e s o u r c e s   s o   t h e y   c a n   s u c c e s s f u l l y   r u n   a s   a   r o o t   p r o c e s s   ( u i d   0 )   w i t h o u t   a c c e s s   t o   h o s t   i n f o r m a t i o n .   H o w e v e r ,   c o n s i d e r i n g   t h e   p r i v i l e g e s   a s s o c i a t e d   w i t h   t h e   r o o t   u s e r ,   y o u   s h o u l d   w r i t e   a p p l i c a t i o n   c o n t a i n e r s   t o   r u n   a s   a   n o n - r o o t   u s e r .   S i m i l a r l y ,   a d m i n i s t r a t o r s   w h o   w i s h   t o   p r e v e n t   c l i e n t   a p p l i c a t i o n s   f r o m   e s c a p i n g   t h e i r   c o n t a i n e r s   s h o u l d   a p p l y   t h e   B a s e l i n e   o r   R e s t r i c t e d   P o d   S e c u r i t y   S t a n d a r d .


Question: What is Preventing containers from loading unwanted kernel modules?

Answer: T h e   L i n u x   k e r n e l   a u t o m a t i c a l l y   l o a d s   k e r n e l   m o d u l e s   f r o m   d i s k   i f   n e e d e d   i n   c e r t a i n   c i r c u m s t a n c e s ,   s u c h   a s   w h e n   a   p i e c e   o f   h a r d w a r e   i s   a t t a c h e d   o r   a   f i l e s y s t e m   i s   m o u n t e d .   O f   p a r t i c u l a r   r e l e v a n c e   t o   K u b e r n e t e s ,   e v e n   u n p r i v i l e g e d   p r o c e s s e s   c a n   c a u s e   c e r t a i n   n e t w o r k - p r o t o c o l - r e l a t e d   k e r n e l   m o d u l e s   t o   b e   l o a d e d ,   j u s t   b y   c r e a t i n g   a   s o c k e t   o f   t h e   a p p r o p r i a t e   t y p e .   T h i s   m a y   a l l o w   a n   a t t a c k e r   t o   e x p l o i t   a   s e c u r i t y   h o l e   i n   a   k e r n e l   m o d u l e   t h a t   t h e   a d m i n i s t r a t o r   a s s u m e d   w a s   n o t   i n   u s e .   T o   p r e v e n t   s p e c i f i c   m o d u l e s   f r o m   b e i n g   a u t o m a t i c a l l y   l o a d e d ,   y o u   c a n   u n i n s t a l l   t h e m   f r o m   t h e   n o d e ,   o r   a d d   r u l e s   t o   b l o c k   t h e m .   O n   m o s t   L i n u x   d i s t r i b u t i o n s ,   y o u   c a n   d o   t h a t   b y   c r e a t i n g   a   f i l e   s u c h   a s   / e t c / m o d p r o b e . d / k u b e r n e t e s - b l a c k l i s t . c o n f   w i t h   c o n t e n t s   l i k e :   #   D C C P   i s   u n l i k e l y   t o   b e   n e e d e d ,   h a s   h a d   m u l t i p l e   s e r i o u s   #   v u l n e r a b i l i t i e s ,   a n d   i s   n o t   w e l l - m a i n t a i n e d .   b l a c k l i s t   d c c p   #   S C T P   i s   n o t   u s e d   i n   m o s t   K u b e r n e t e s   c l u s t e r s ,   a n d   h a s   a l s o   h a d   #   v u l n e r a b i l i t i e s   i n   t h e   p a s t .   b l a c k l i s t   s c t p   T o   b l o c k   m o d u l e   l o a d i n g   m o r


Question: What is Restricting network access?

Answer: T h e   n e t w o r k   p o l i c i e s   f o r   a   n a m e s p a c e   a l l o w s   a p p l i c a t i o n   a u t h o r s   t o   r e s t r i c t   w h i c h   p o d s   i n   o t h e r   n a m e s p a c e s   m a y   a c c e s s   p o d s   a n d   p o r t s   w i t h i n   t h e i r   n a m e s p a c e s .   M a n y   o f   t h e   s u p p o r t e d   K u b e r n e t e s   n e t w o r k i n g   p r o v i d e r s   n o w   r e s p e c t   n e t w o r k   p o l i c y .   Q u o t a   a n d   l i m i t   r a n g e s   c a n   a l s o   b e   u s e d   t o   c o n t r o l   w h e t h e r   u s e r s   m a y   r e q u e s t   n o d e   p o r t s   o r   l o a d - b a l a n c e d   s e r v i c e s ,   w h i c h   o n   m a n y   c l u s t e r s   c a n   c o n t r o l   w h e t h e r   t h o s e   u s e r s   a p p l i c a t i o n s   a r e   v i s i b l e   o u t s i d e   o f   t h e   c l u s t e r .   A d d i t i o n a l   p r o t e c t i o n s   m a y   b e   a v a i l a b l e   t h a t   c o n t r o l   n e t w o r k   r u l e s   o n   a   p e r - p l u g i n   o r   p e r -   e n v i r o n m e n t   b a s i s ,   s u c h   a s   p e r - n o d e   f i r e w a l l s ,   p h y s i c a l l y   s e p a r a t i n g   c l u s t e r   n o d e s   t o   p r e v e n t   c r o s s   t a l k ,   o r   a d v a n c e d   n e t w o r k i n g   p o l i c y .


Question: What is Restricting cloud metadata API access?

Answer: C l o u d   p l a t f o r m s   ( A W S ,   A z u r e ,   G C E ,   e t c . )   o f t e n   e x p o s e   m e t a d a t a   s e r v i c e s   l o c a l l y   t o   i n s t a n c e s .   B y   d e f a u l t   t h e s e   A P I s   a r e   a c c e s s i b l e   b y   p o d s   r u n n i n g   o n   a n   i n s t a n c e   a n d   c a n   c o n t a i n   c l o u d   c r e d e n t i a l s   f o r   t h a t   n o d e ,   o r   p r o v i s i o n i n g   d a t a   s u c h   a s   k u b e l e t   c r e d e n t i a l s .   T h e s e   c r e d e n t i a l s   c a n   b e   u s e d   t o   e s c a l a t e   w i t h i n   t h e   c l u s t e r   o r   t o   o t h e r   c l o u d   s e r v i c e s   u n d e r   t h e   s a m e   a c c o u n t .   W h e n   r u n n i n g   K u b e r n e t e s   o n   a   c l o u d   p l a t f o r m ,   l i m i t   p e r m i s s i o n s   g i v e n   t o   i n s t a n c e   c r e d e n t i a l s ,   u s e   n e t w o r k   p o l i c i e s   t o   r e s t r i c t   p o d   a c c e s s   t o   t h e   m e t a d a t a   A P I ,   a n d   a v o i d   u s i n g   p r o v i s i o n i n g   d a t a   t o   d e l i v e r   s e c r e t s .


Question: What is Controlling which nodes pods may access?

Answer: B y   d e f a u l t ,   t h e r e   a r e   n o   r e s t r i c t i o n s   o n   w h i c h   n o d e s   m a y   r u n   a   p o d .   K u b e r n e t e s   o f f e r s   a   r i c h   s e t   o f   p o l i c i e s   f o r   c o n t r o l l i n g   p l a c e m e n t   o f   p o d s   o n t o   n o d e s   a n d   t h e   t a i n t - b a s e d   p o d   p l a c e m e n t   a n d   e v i c t i o n   t h a t   a r e   a v a i l a b l e   t o   e n d   u s e r s .   F o r   m a n y   c l u s t e r s   u s e   o f   t h e s e   p o l i c i e s   t o   s e p a r a t e   w o r k l o a d s   c a n   b e   a   c o n v e n t i o n   t h a t   a u t h o r s   a d o p t   o r   e n f o r c e   v i a   t o o l i n g .   A s   a n   a d m i n i s t r a t o r ,   a   b e t a   a d m i s s i o n   p l u g i n   P o d N o d e S e l e c t o r   c a n   b e   u s e d   t o   f o r c e   p o d s   w i t h i n   a   n a m e s p a c e   t o   d e f a u l t   o r   r e q u i r e   a   s p e c i f i c   n o d e   s e l e c t o r ,   a n d   i f   e n d   u s e r s   c a n n o t   a l t e r   n a m e s p a c e s ,   t h i s   c a n   s t r o n g l y   l i m i t   t h e   p l a c e m e n t   o f   a l l   o f   t h e   p o d s   i n   a   s p e c i f i c   w o r k l o a d .


Question: What is Protecting cluster components from compromise?

Answer: T h i s   s e c t i o n   d e s c r i b e s   s o m e   c o m m o n   p a t t e r n s   f o r   p r o t e c t i n g   c l u s t e r s   f r o m   c o m p r o m i s e .


Question: What is Restrict access to etcd?

Answer: W r i t e   a c c e s s   t o   t h e   e t c d   b a c k e n d   f o r   t h e   A P I   i s   e q u i v a l e n t   t o   g a i n i n g   r o o t   o n   t h e   e n t i r e   c l u s t e r ,   a n d   r e a d   a c c e s s   c a n   b e   u s e d   t o   e s c a l a t e   f a i r l y   q u i c k l y .   A d m i n i s t r a t o r s   s h o u l d   a l w a y s   u s e   s t r o n g   c r e d e n t i a l s   f r o m   t h e   A P I   s e r v e r s   t o   t h e i r   e t c d   s e r v e r ,   s u c h   a s   m u t u a l   a u t h   v i a   T L S   c l i e n t   c e r t i f i c a t e s ,   a n d   i t   i s   o f t e n   r e c o m m e n d e d   t o   i s o l a t e   t h e   e t c d   s e r v e r s   b e h i n d   a   f i r e w a l l   t h a t   o n l y   t h e   A P I   s e r v e r s   m a y   a c c e s s .


Question: What is Enable audit logging?

Answer: T h e   a u d i t   l o g g e r   i s   a   b e t a   f e a t u r e   t h a t   r e c o r d s   a c t i o n s   t a k e n   b y   t h e   A P I   f o r   l a t e r   a n a l y s i s   i n   t h e   e v e n t   o f   a   c o m p r o m i s e .   I t   i s   r e c o m m e n d e d   t o   e n a b l e   a u d i t   l o g g i n g   a n d   a r c h i v e   t h e   a u d i t   f i l e   o n   a   s e c u r e   s e r v e r .


Question: What is Restrict access to alpha or beta features?

Answer: A l p h a   a n d   b e t a   K u b e r n e t e s   f e a t u r e s   a r e   i n   a c t i v e   d e v e l o p m e n t   a n d   m a y   h a v e   l i m i t a t i o n s   o r   b u g s   t h a t   r e s u l t   i n   s e c u r i t y   v u l n e r a b i l i t i e s .   A l w a y s   a s s e s s   t h e   v a l u e   a n   a l p h a   o r   b e t a   f e a t u r e   m a y   p r o v i d e   a g a i n s t   t h e   p o s s i b l e   r i s k   t o   y o u r   s e c u r i t y   p o s t u r e .   W h e n   i n   d o u b t ,   d i s a b l e   f e a t u r e s   y o u   d o   n o t   u s e .


Question: What is Rotate infrastructure credentials frequently?

Answer: T h e   s h o r t e r   t h e   l i f e t i m e   o f   a   s e c r e t   o r   c r e d e n t i a l   t h e   h a r d e r   i t   i s   f o r   a n   a t t a c k e r   t o   m a k e   u s e   o f   t h a t   c r e d e n t i a l .   S e t   s h o r t   l i f e t i m e s   o n   c e r t i f i c a t e s   a n d   a u t o m a t e   t h e i r   r o t a t i o n .   U s e   a n   a u t h e n t i c a t i o n   p r o v i d e r   t h a t   c a n   c o n t r o l   h o w   l o n g   i s s u e d   t o k e n s   a r e   a v a i l a b l e   a n d   u s e   s h o r t   l i f e t i m e s   w h e r e   p o s s i b l e .   I f   y o u   u s e   s e r v i c e - a c c o u n t   t o k e n s   i n   e x t e r n a l   i n t e g r a t i o n s ,   p l a n   t o   r o t a t e   t h o s e   t o k e n s   f r e q u e n t l y .   F o r   e x a m p l e ,   o n c e   t h e   b o o t s t r a p   p h a s e   i s   c o m p l e t e ,   a   b o o t s t r a p   t o k e n   u s e d   f o r   s e t t i n g   u p   n o d e s   s h o u l d   b e   r e v o k e d   o r   i t s   a u t h o r i z a t i o n   r e m o v e d .


Question: What is Review third party integrations before enabling them?

Answer: M a n y   t h i r d   p a r t y   i n t e g r a t i o n s   t o   K u b e r n e t e s   m a y   a l t e r   t h e   s e c u r i t y   p r o f i l e   o f   y o u r   c l u s t e r .   W h e n   e n a b l i n g   a n   i n t e g r a t i o n ,   a l w a y s   r e v i e w   t h e   p e r m i s s i o n s   t h a t   a n   e x t e n s i o n   r e q u e s t s   b e f o r e   g r a n t i n g   i t   a c c e s s .   F o r   e x a m p l e ,   m a n y   s e c u r i t y   i n t e g r a t i o n s   m a y   r e q u e s t   a c c e s s   t o   v i e w   a l l   s e c r e t s   o n   y o u r   c l u s t e r   w h i c h   i s   e f f e c t i v e l y   m a k i n g   t h a t   c o m p o n e n t   a   c l u s t e r   a d m i n .   W h e n   i n   d o u b t ,   r e s t r i c t   t h e   i n t e g r a t i o n   t o   f u n c t i o n i n g   i n   a   s i n g l e   n a m e s p a c e   i f   p o s s i b l e .   C o m p o n e n t s   t h a t   c r e a t e   p o d s   m a y   a l s o   b e   u n e x p e c t e d l y   p o w e r f u l   i f   t h e y   c a n   d o   s o   i n s i d e   n a m e s p a c e s   l i k e   t h e   k u b e - s y s t e m   n a m e s p a c e ,   b e c a u s e   t h o s e   p o d s   c a n   g a i n   a c c e s s   t o   s e r v i c e   a c c o u n t   s e c r e t s   o r   r u n   w i t h   e l e v a t e d   p e r m i s s i o n s   i f   t h o s e   s e r v i c e   a c c o u n t s   a r e   g r a n t e d   a c c e s s   t o   p e r m i s s i v e   P o d S e c u r i t y P o l i c i e s .   I f   y o u   u s e   P o d   S e c u r i t y   a d m i s s i o n   a n d   a l l o w   a n y   c o m p o n e n t   t o   c r e a t e   P o d s   w i t h i n   a   n a m e s p a c e   t h a t   p e r m i t s   p r i v i l e g e d   P o d s ,   t h o s e   P o d s   m a y   b e   a b l e   t o   e s c a p e   t h e i r   c o n t a i n e r s   a n d   u s e   t h i s   w i d e n e d   a c c e s s   t o   e l e v a t e   t h e i r   p r i v i l e g e s .   Y o u   s h o u l d   n o t


Question: What is Encrypt secrets at rest?

Answer: I n   g e n e r a l ,   t h e   e t c d   d a t a b a s e   w i l l   c o n t a i n   a n y   i n f o r m a t i o n   a c c e s s i b l e   v i a   t h e   K u b e r n e t e s   A P I   a n d   m a y   g r a n t   a n   a t t a c k e r   s i g n i f i c a n t   v i s i b i l i t y   i n t o   t h e   s t a t e   o f   y o u r   c l u s t e r .   A l w a y s   e n c r y p t   y o u r   b a c k u p s   u s i n g   a   w e l l   r e v i e w e d   b a c k u p   a n d   e n c r y p t i o n   s o l u t i o n ,   a n d   c o n s i d e r   u s i n g   f u l l   d i s k   e n c r y p t i o n   w h e r e   p o s s i b l e .   K u b e r n e t e s   s u p p o r t s   o p t i o n a l   e n c r y p t i o n   a t   r e s t   f o r   i n f o r m a t i o n   i n   t h e   K u b e r n e t e s   A P I .   T h i s   l e t s   y o u   e n s u r e   t h a t   w h e n   K u b e r n e t e s   s t o r e s   d a t a   f o r   o b j e c t s   ( f o r   e x a m p l e ,   S e c r e t   o r   C o n f i g M a p   o b j e c t s ) ,   t h e   A P I   s e r v e r   w r i t e s   a n   e n c r y p t e d   r e p r e s e n t a t i o n   o f   t h e   o b j e c t .   T h a t   e n c r y p t i o n   m e a n s   t h a t   e v e n   s o m e o n e   w h o   h a s   a c c e s s   t o   e t c d   b a c k u p   d a t a   i s   u n a b l e   t o   v i e w   t h e   c o n t e n t   o f   t h o s e   o b j e c t s .   I n   K u b e r n e t e s   1 . 3 4   y o u   c a n   a l s o   e n c r y p t   c u s t o m   r e s o u r c e s ;   e n c r y p t i o n - a t - r e s t   f o r   e x t e n s i o n   A P I s   d e f i n e d   i n   C u s t o m R e s o u r c e D e f i n i t i o n s   w a s   a d d e d   t o   K u b e r n e t e s   a s   p a r t   o f   t h e   v 1 . 2 6   r e l e a s e .


Question: What is Receiving alerts for security updates and reporting vulnerabilities?

Answer: J o i n   t h e   k u b e r n e t e s - a n n o u n c e   g r o u p   f o r   e m a i l s   a b o u t   s e c u r i t y   a n n o u n c e m e n t s .   S e e   t h e   s e c u r i t y   r e p o r t i n g   p a g e   f o r   m o r e   o n   h o w   t o   r e p o r t   v u l n e r a b i l i t i e s .


Question: What is What's next?

Answer: S e c u r i t y   C h e c k l i s t   f o r   a d d i t i o n a l   i n f o r m a t i o n   o n   K u b e r n e t e s   s e c u r i t y   g u i d a n c e . S e c c o m p   N o d e   R e f e r e n c e


Question: What is Troubleshooting kubeadm?

Answer: Troubleshooting kubeadmAs with any program, you might run into an error installing or running kubeadm. This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.If your problem is not listed below, please follow the following steps:If you think your problem is a bug with kubeadm:Go to github.com/kubernetes/kubeadm and search for existing issues.If no issue exists, please open one and follow the issue template.If you are unsure about how kubeadm works, you can ask on Slack in #kubeadm, or open a question on StackOverflow. Please include relevant tags like #kubernetes and #kubeadm so folks can help you.Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBACIn v1.18 kubeadm added prevention for joining a Node in the cluster if a Node with the same name already exists. This required adding RBAC for the bootstrap-token user to be able to GET a Node object.However this causes an issue where kubeadm join from v1.18 cannot join a cluster created by kubeadm v1.17.To workaround the issue you have two options:Execute kubeadm init phase bootstrap-token on a control-plane node using kubeadm v1.18. Note that this enables the rest of the bootstrap-token permissions as well.orApply the following RBAC manually using kubectl apply -f ...:apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kubeadm:get-nodes rules: - apiGroups: - "" resources: - nodes verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubeadm:get-nodes roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubeadm:get-nodes subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: system:bootstrappers:kubeadm:default-node-token ebtables or some similar executable not found during installationIf you see the following warnings while running kubeadm init[preflight] WARNING: ebtables not found in system path [preflight] WARNING: ethtool not found in system path Then you may be missing ebtables, ethtool or a similar executable on your node. You can install them with the following commands:For Ubuntu/Debian users, run apt install ebtables ethtool.For CentOS/Fedora users, run yum install ebtables ethtool.kubeadm blocks waiting for control plane during installationIf you notice that kubeadm init hangs after printing out the following line:[apiclient] Created API client, waiting for the control plane to become ready This may be caused by a number of problems. The most common are:network connection problems. Check that your machine has full network connectivity before continuing.the cgroup driver of the container runtime differs from that of the kubelet. To understand how to configure it properly, see Configuring a cgroup driver.control plane containers are crashlooping or hanging. You can check this by running docker ps and investigating each container by running docker logs. For other container runtime, see Debugging Kubernetes nodes with


Question: How does Troubleshooting kubeadm work?

Answer: Troubleshooting kubeadmAs with any program, you might run into an error installing or running kubeadm. This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.If your problem is not listed below, please follow the following steps:If you think your problem is a bug with kubeadm:Go to github.com/kubernetes/kubeadm and search for existing issues.If no issue exists, please open one and follow the issue template.If you are unsure about how kubeadm works, you can ask on Slack in #kubeadm, or open a question on StackOverflow. Please include relevant tags like #kubernetes and #kubeadm so folks can help you.Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBACIn v1.18 kubeadm added prevention for joining a Node in the cluster if a Node with the same name already exists. This required adding RBAC for the bootstrap-token user to be able to GET a Node object.However this causes an issue where kubeadm join from v1.18 cannot join a cluster created by kubeadm v1.17.To workaround the issue you have two options:Execute kubeadm init phase bootstrap-token on a control-plane node using kubeadm v1.18. Note that this enables the rest of the bootstrap-token permissions as well.orApply the following RBAC manually using kubectl apply -f ...:apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kubeadm:get-nodes rules: - apiGroups: - "" resources: - nodes verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubeadm:get-nodes roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubeadm:get-nodes subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: system:bootstrappers:kubeadm:default-node-token ebtables or some similar executable not found during installationIf you see the following warnings while running kubeadm init[preflight] WARNING: ebtables not found in system path [preflight] WARNING: ethtool not found in system path Then you may be missing ebtables, ethtool or a similar executable on your node. You can install them with the following commands:For Ubuntu/Debian users, run apt install ebtables ethtool.For CentOS/Fedora users, run yum install ebtables ethtool.kubeadm blocks waiting for control plane during installationIf you notice that kubeadm init hangs after printing out the following line:[apiclient] Created API client, waiting for the control plane to become ready This may be caused by a number of problems. The most common are:network connection problems. Check that your machine has full network connectivity before continuing.the cgroup driver of the container runtime differs from that of the kubelet. To understand how to configure it properly, see Configuring a cgroup driver.control plane containers are crashlooping or hanging. You can check this by running docker ps and investigating each container by running docker logs. For other container runtime, see Debugging Kubernetes nodes with


Question: When would you use Troubleshooting kubeadm?

Answer: Troubleshooting kubeadmAs with any program, you might run into an error installing or running kubeadm. This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.If your problem is not listed below, please follow the following steps:If you think your problem is a bug with kubeadm:Go to github.com/kubernetes/kubeadm and search for existing issues.If no issue exists, please open one and follow the issue template.If you are unsure about how kubeadm works, you can ask on Slack in #kubeadm, or open a question on StackOverflow. Please include relevant tags like #kubernetes and #kubeadm so folks can help you.Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBACIn v1.18 kubeadm added prevention for joining a Node in the cluster if a Node with the same name already exists. This required adding RBAC for the bootstrap-token user to be able to GET a Node object.However this causes an issue where kubeadm join from v1.18 cannot join a cluster created by kubeadm v1.17.To workaround the issue you have two options:Execute kubeadm init phase bootstrap-token on a control-plane node using kubeadm v1.18. Note that this enables the rest of the bootstrap-token permissions as well.orApply the following RBAC manually using kubectl apply -f ...:apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kubeadm:get-nodes rules: - apiGroups: - "" resources: - nodes verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubeadm:get-nodes roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubeadm:get-nodes subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: system:bootstrappers:kubeadm:default-node-token ebtables or some similar executable not found during installationIf you see the following warnings while running kubeadm init[preflight] WARNING: ebtables not found in system path [preflight] WARNING: ethtool not found in system path Then you may be missing ebtables, ethtool or a similar executable on your node. You can install them with the following commands:For Ubuntu/Debian users, run apt install ebtables ethtool.For CentOS/Fedora users, run yum install ebtables ethtool.kubeadm blocks waiting for control plane during installationIf you notice that kubeadm init hangs after printing out the following line:[apiclient] Created API client, waiting for the control plane to become ready This may be caused by a number of problems. The most common are:network connection problems. Check that your machine has full network connectivity before continuing.the cgroup driver of the container runtime differs from that of the kubelet. To understand how to configure it properly, see Configuring a cgroup driver.control plane containers are crashlooping or hanging. You can check this by running docker ps and investigating each container by running docker logs. For other container runtime, see Debugging Kubernetes nodes with


Question: What are the benefits of Troubleshooting kubeadm?

Answer: Troubleshooting kubeadmAs with any program, you might run into an error installing or running kubeadm. This page lists some common failure scenarios and have provided steps that can help you understand and fix the problem.If your problem is not listed below, please follow the following steps:If you think your problem is a bug with kubeadm:Go to github.com/kubernetes/kubeadm and search for existing issues.If no issue exists, please open one and follow the issue template.If you are unsure about how kubeadm works, you can ask on Slack in #kubeadm, or open a question on StackOverflow. Please include relevant tags like #kubernetes and #kubeadm so folks can help you.Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBACIn v1.18 kubeadm added prevention for joining a Node in the cluster if a Node with the same name already exists. This required adding RBAC for the bootstrap-token user to be able to GET a Node object.However this causes an issue where kubeadm join from v1.18 cannot join a cluster created by kubeadm v1.17.To workaround the issue you have two options:Execute kubeadm init phase bootstrap-token on a control-plane node using kubeadm v1.18. Note that this enables the rest of the bootstrap-token permissions as well.orApply the following RBAC manually using kubectl apply -f ...:apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: kubeadm:get-nodes rules: - apiGroups: - "" resources: - nodes verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: kubeadm:get-nodes roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubeadm:get-nodes subjects: - apiGroup: rbac.authorization.k8s.io kind: Group name: system:bootstrappers:kubeadm:default-node-token ebtables or some similar executable not found during installationIf you see the following warnings while running kubeadm init[preflight] WARNING: ebtables not found in system path [preflight] WARNING: ethtool not found in system path Then you may be missing ebtables, ethtool or a similar executable on your node. You can install them with the following commands:For Ubuntu/Debian users, run apt install ebtables ethtool.For CentOS/Fedora users, run yum install ebtables ethtool.kubeadm blocks waiting for control plane during installationIf you notice that kubeadm init hangs after printing out the following line:[apiclient] Created API client, waiting for the control plane to become ready This may be caused by a number of problems. The most common are:network connection problems. Check that your machine has full network connectivity before continuing.the cgroup driver of the container runtime differs from that of the kubelet. To understand how to configure it properly, see Configuring a cgroup driver.control plane containers are crashlooping or hanging. You can check this by running docker ps and investigating each container by running docker logs. For other container runtime, see Debugging Kubernetes nodes with


Question: What is Troubleshooting kubeadm?

Answer: A s   w i t h   a n y   p r o g r a m ,   y o u   m i g h t   r u n   i n t o   a n   e r r o r   i n s t a l l i n g   o r   r u n n i n g   k u b e a d m .   T h i s   p a g e   l i s t s   s o m e   c o m m o n   f a i l u r e   s c e n a r i o s   a n d   h a v e   p r o v i d e d   s t e p s   t h a t   c a n   h e l p   y o u   u n d e r s t a n d   a n d   f i x   t h e   p r o b l e m .   I f   y o u r   p r o b l e m   i s   n o t   l i s t e d   b e l o w ,   p l e a s e   f o l l o w   t h e   f o l l o w i n g   s t e p s :   I f   y o u   t h i n k   y o u r   p r o b l e m   i s   a   b u g   w i t h   k u b e a d m : G o   t o   g i t h u b . c o m / k u b e r n e t e s / k u b e a d m   a n d   s e a r c h   f o r   e x i s t i n g   i s s u e s . I f   n o   i s s u e   e x i s t s ,   p l e a s e   o p e n   o n e   a n d   f o l l o w   t h e   i s s u e   t e m p l a t e . I f   y o u   a r e   u n s u r e   a b o u t   h o w   k u b e a d m   w o r k s ,   y o u   c a n   a s k   o n   S l a c k   i n   # k u b e a d m ,   o r   o p e n   a   q u e s t i o n   o n   S t a c k O v e r f l o w .   P l e a s e   i n c l u d e   r e l e v a n t   t a g s   l i k e   # k u b e r n e t e s   a n d   # k u b e a d m   s o   f o l k s   c a n   h e l p   y o u .   I f   y o u   t h i n k   y o u r   p r o b l e m   i s   a   b u g   w i t h   k u b e a d m :   G o   t o   g i t h u b . c o m / k u b e r n e t e s / k u b e a d m   a n d   s e a r c h   f o r   e x i s t i n g   i s s u e s . I f   n o   i s s u e   e x i s t s ,   p l e a s e   o p e n   o n e   a n d   f o l l o w   t h e   i s s u e   t e m p l a t e .   I f   y o u   a r e   u n s u r e   a b o u t   h o w   k u b e a d m   w o r k s ,   y o u   c a n   a s k   o n   S l a c k   i n   # k u b e a d m ,   o r   o p e n   a   q u e s t i o n   o n   S t a c k O v e r f l o w .   P l e a s e   i n c l u d e   r e l e v a n t   t a g s   l i k e   # k u b e r n e t e s   a n d   # k u b


Question: What is Not possible to join a v1.18 Node to a v1.17 cluster due to missing RBAC?

Answer: I n   v 1 . 1 8   k u b e a d m   a d d e d   p r e v e n t i o n   f o r   j o i n i n g   a   N o d e   i n   t h e   c l u s t e r   i f   a   N o d e   w i t h   t h e   s a m e   n a m e   a l r e a d y   e x i s t s .   T h i s   r e q u i r e d   a d d i n g   R B A C   f o r   t h e   b o o t s t r a p - t o k e n   u s e r   t o   b e   a b l e   t o   G E T   a   N o d e   o b j e c t .   H o w e v e r   t h i s   c a u s e s   a n   i s s u e   w h e r e   k u b e a d m   j o i n   f r o m   v 1 . 1 8   c a n n o t   j o i n   a   c l u s t e r   c r e a t e d   b y   k u b e a d m   v 1 . 1 7 .   T o   w o r k a r o u n d   t h e   i s s u e   y o u   h a v e   t w o   o p t i o n s :   E x e c u t e   k u b e a d m   i n i t   p h a s e   b o o t s t r a p - t o k e n   o n   a   c o n t r o l - p l a n e   n o d e   u s i n g   k u b e a d m   v 1 . 1 8 .   N o t e   t h a t   t h i s   e n a b l e s   t h e   r e s t   o f   t h e   b o o t s t r a p - t o k e n   p e r m i s s i o n s   a s   w e l l .   o r   A p p l y   t h e   f o l l o w i n g   R B A C   m a n u a l l y   u s i n g   k u b e c t l   a p p l y   - f   . . . :   a p i V e r s i o n :   r b a c . a u t h o r i z a t i o n . k 8 s . i o / v 1   k i n d :   C l u s t e r R o l e   m e t a d a t a :   n a m e :   k u b e a d m : g e t - n o d e s   r u l e s :   -   a p i G r o u p s :   -   " "   r e s o u r c e s :   -   n o d e s   v e r b s :   -   g e t   - - -   a p i V e r s i o n :   r b a c . a u t h o r i z a t i o n . k 8 s . i o / v 1   k i n d :   C l u s t e r R o l e B i n d i n g   m e t a d a t a :   n a m e :   k u b e a d m : g e t - n o d e s   r o l e R e f :   a p i G r o u p :   r b a c . a u t h o r i z a t i o n . k 8 s . i o   k i n d :   C l u s t e r R o l e   n a m e :   k u b e a d m : g e t - n o d e s   s u b j e c t s :   -   a p i G r o u p :   r b a c . a u t h o r i z a t i o n . k 8 s . i o   k i n d :   G r o u p   n a m e :   s y s t e m : b o o


Question: What is ebtables or some similar executable not found during installation?

Answer: I f   y o u   s e e   t h e   f o l l o w i n g   w a r n i n g s   w h i l e   r u n n i n g   k u b e a d m   i n i t   [ p r e f l i g h t ]   W A R N I N G :   e b t a b l e s   n o t   f o u n d   i n   s y s t e m   p a t h   [ p r e f l i g h t ]   W A R N I N G :   e t h t o o l   n o t   f o u n d   i n   s y s t e m   p a t h   T h e n   y o u   m a y   b e   m i s s i n g   e b t a b l e s ,   e t h t o o l   o r   a   s i m i l a r   e x e c u t a b l e   o n   y o u r   n o d e .   Y o u   c a n   i n s t a l l   t h e m   w i t h   t h e   f o l l o w i n g   c o m m a n d s :   F o r   U b u n t u / D e b i a n   u s e r s ,   r u n   a p t   i n s t a l l   e b t a b l e s   e t h t o o l . F o r   C e n t O S / F e d o r a   u s e r s ,   r u n   y u m   i n s t a l l   e b t a b l e s   e t h t o o l .


Question: What is kubeadm blocks waiting for control plane during installation?

Answer: I f   y o u   n o t i c e   t h a t   k u b e a d m   i n i t   h a n g s   a f t e r   p r i n t i n g   o u t   t h e   f o l l o w i n g   l i n e :   [ a p i c l i e n t ]   C r e a t e d   A P I   c l i e n t ,   w a i t i n g   f o r   t h e   c o n t r o l   p l a n e   t o   b e c o m e   r e a d y   T h i s   m a y   b e   c a u s e d   b y   a   n u m b e r   o f   p r o b l e m s .   T h e   m o s t   c o m m o n   a r e :   n e t w o r k   c o n n e c t i o n   p r o b l e m s .   C h e c k   t h a t   y o u r   m a c h i n e   h a s   f u l l   n e t w o r k   c o n n e c t i v i t y   b e f o r e   c o n t i n u i n g . t h e   c g r o u p   d r i v e r   o f   t h e   c o n t a i n e r   r u n t i m e   d i f f e r s   f r o m   t h a t   o f   t h e   k u b e l e t .   T o   u n d e r s t a n d   h o w   t o   c o n f i g u r e   i t   p r o p e r l y ,   s e e   C o n f i g u r i n g   a   c g r o u p   d r i v e r . c o n t r o l   p l a n e   c o n t a i n e r s   a r e   c r a s h l o o p i n g   o r   h a n g i n g .   Y o u   c a n   c h e c k   t h i s   b y   r u n n i n g   d o c k e r   p s   a n d   i n v e s t i g a t i n g   e a c h   c o n t a i n e r   b y   r u n n i n g   d o c k e r   l o g s .   F o r   o t h e r   c o n t a i n e r   r u n t i m e ,   s e e   D e b u g g i n g   K u b e r n e t e s   n o d e s   w i t h   c r i c t l .


Question: What is kubeadm blocks when removing managed containers?

Answer: T h e   f o l l o w i n g   c o u l d   h a p p e n   i f   t h e   c o n t a i n e r   r u n t i m e   h a l t s   a n d   d o e s   n o t   r e m o v e   a n y   K u b e r n e t e s - m a n a g e d   c o n t a i n e r s :   s u d o   k u b e a d m   r e s e t   [ p r e f l i g h t ]   R u n n i n g   p r e - f l i g h t   c h e c k s   [ r e s e t ]   S t o p p i n g   t h e   k u b e l e t   s e r v i c e   [ r e s e t ]   U n m o u n t i n g   m o u n t e d   d i r e c t o r i e s   i n   " / v a r / l i b / k u b e l e t "   [ r e s e t ]   R e m o v i n g   k u b e r n e t e s - m a n a g e d   c o n t a i n e r s   ( b l o c k )   A   p o s s i b l e   s o l u t i o n   i s   t o   r e s t a r t   t h e   c o n t a i n e r   r u n t i m e   a n d   t h e n   r e - r u n   k u b e a d m   r e s e t .   Y o u   c a n   a l s o   u s e   c r i c t l   t o   d e b u g   t h e   s t a t e   o f   t h e   c o n t a i n e r   r u n t i m e .   S e e   D e b u g g i n g   K u b e r n e t e s   n o d e s   w i t h   c r i c t l .


Question: What is Pods in RunContainerError, CrashLoopBackOff or Error state?

Answer: R i g h t   a f t e r   k u b e a d m   i n i t   t h e r e   s h o u l d   n o t   b e   a n y   p o d s   i n   t h e s e   s t a t e s .   I f   t h e r e   a r e   p o d s   i n   o n e   o f   t h e s e   s t a t e s   r i g h t   a f t e r   k u b e a d m   i n i t ,   p l e a s e   o p e n   a n   i s s u e   i n   t h e   k u b e a d m   r e p o .   c o r e d n s   ( o r   k u b e - d n s )   s h o u l d   b e   i n   t h e   P e n d i n g   s t a t e   u n t i l   y o u   h a v e   d e p l o y e d   t h e   n e t w o r k   a d d - o n . I f   y o u   s e e   P o d s   i n   t h e   R u n C o n t a i n e r E r r o r ,   C r a s h L o o p B a c k O f f   o r   E r r o r   s t a t e   a f t e r   d e p l o y i n g   t h e   n e t w o r k   a d d - o n   a n d   n o t h i n g   h a p p e n s   t o   c o r e d n s   ( o r   k u b e - d n s ) ,   i t ' s   v e r y   l i k e l y   t h a t   t h e   P o d   N e t w o r k   a d d - o n   t h a t   y o u   i n s t a l l e d   i s   s o m e h o w   b r o k e n .   Y o u   m i g h t   h a v e   t o   g r a n t   i t   m o r e   R B A C   p r i v i l e g e s   o r   u s e   a   n e w e r   v e r s i o n .   P l e a s e   f i l e   a n   i s s u e   i n   t h e   P o d   N e t w o r k   p r o v i d e r s '   i s s u e   t r a c k e r   a n d   g e t   t h e   i s s u e   t r i a g e d   t h e r e .


Question: What is coredns is stuck in the Pending state?

Answer: T h i s   i s   e x p e c t e d   a n d   p a r t   o f   t h e   d e s i g n .   k u b e a d m   i s   n e t w o r k   p r o v i d e r - a g n o s t i c ,   s o   t h e   a d m i n   s h o u l d   i n s t a l l   t h e   p o d   n e t w o r k   a d d - o n   o f   c h o i c e .   Y o u   h a v e   t o   i n s t a l l   a   P o d   N e t w o r k   b e f o r e   C o r e D N S   m a y   b e   d e p l o y e d   f u l l y .   H e n c e   t h e   P e n d i n g   s t a t e   b e f o r e   t h e   n e t w o r k   i s   s e t   u p .


Question: What is HostPort services do not work?

Answer: T h e   H o s t P o r t   a n d   H o s t I P   f u n c t i o n a l i t y   i s   a v a i l a b l e   d e p e n d i n g   o n   y o u r   P o d   N e t w o r k   p r o v i d e r .   P l e a s e   c o n t a c t   t h e   a u t h o r   o f   t h e   P o d   N e t w o r k   a d d - o n   t o   f i n d   o u t   w h e t h e r   H o s t P o r t   a n d   H o s t I P   f u n c t i o n a l i t y   a r e   a v a i l a b l e .   C a l i c o ,   C a n a l ,   a n d   F l a n n e l   C N I   p r o v i d e r s   a r e   v e r i f i e d   t o   s u p p o r t   H o s t P o r t .   F o r   m o r e   i n f o r m a t i o n ,   s e e   t h e   C N I   p o r t m a p   d o c u m e n t a t i o n .   I f   y o u r   n e t w o r k   p r o v i d e r   d o e s   n o t   s u p p o r t   t h e   p o r t m a p   C N I   p l u g i n ,   y o u   m a y   n e e d   t o   u s e   t h e   N o d e P o r t   f e a t u r e   o f   s e r v i c e s   o r   u s e   H o s t N e t w o r k = t r u e .


Question: What is Pods are not accessible via their Service IP?

Answer: M a n y   n e t w o r k   a d d - o n s   d o   n o t   y e t   e n a b l e   h a i r p i n   m o d e   w h i c h   a l l o w s   p o d s   t o   a c c e s s   t h e m s e l v e s   v i a   t h e i r   S e r v i c e   I P .   T h i s   i s   a n   i s s u e   r e l a t e d   t o   C N I .   P l e a s e   c o n t a c t   t h e   n e t w o r k   a d d - o n   p r o v i d e r   t o   g e t   t h e   l a t e s t   s t a t u s   o f   t h e i r   s u p p o r t   f o r   h a i r p i n   m o d e . I f   y o u   a r e   u s i n g   V i r t u a l B o x   ( d i r e c t l y   o r   v i a   V a g r a n t ) ,   y o u   w i l l   n e e d   t o   e n s u r e   t h a t   h o s t n a m e   - i   r e t u r n s   a   r o u t a b l e   I P   a d d r e s s .   B y   d e f a u l t ,   t h e   f i r s t   i n t e r f a c e   i s   c o n n e c t e d   t o   a   n o n - r o u t a b l e   h o s t - o n l y   n e t w o r k .   A   w o r k   a r o u n d   i s   t o   m o d i f y   / e t c / h o s t s ,   s e e   t h i s   V a g r a n t f i l e   f o r   a n   e x a m p l e .   M a n y   n e t w o r k   a d d - o n s   d o   n o t   y e t   e n a b l e   h a i r p i n   m o d e   w h i c h   a l l o w s   p o d s   t o   a c c e s s   t h e m s e l v e s   v i a   t h e i r   S e r v i c e   I P .   T h i s   i s   a n   i s s u e   r e l a t e d   t o   C N I .   P l e a s e   c o n t a c t   t h e   n e t w o r k   a d d - o n   p r o v i d e r   t o   g e t   t h e   l a t e s t   s t a t u s   o f   t h e i r   s u p p o r t   f o r   h a i r p i n   m o d e .   I f   y o u   a r e   u s i n g   V i r t u a l B o x   ( d i r e c t l y   o r   v i a   V a g r a n t ) ,   y o u   w i l l   n e e d   t o   e n s u r e   t h a t   h o s t n a m e   - i   r e t u r n s   a   r o u t a b l e   I P   a d d r e s s .   B y   d e f a u l t ,   t h e   f i r s t   i n t e r f a c e   i s   c o n n e c t e d   t o   a   n o n - r o u t a b l e   h o s t - o n l y   n e t w o r k .   A   w o r k   a r


Question: What is TLS certificate errors?

Answer: T h e   f o l l o w i n g   e r r o r   i n d i c a t e s   a   p o s s i b l e   c e r t i f i c a t e   m i s m a t c h .   #   k u b e c t l   g e t   p o d s   U n a b l e   t o   c o n n e c t   t o   t h e   s e r v e r :   x 5 0 9 :   c e r t i f i c a t e   s i g n e d   b y   u n k n o w n   a u t h o r i t y   ( p o s s i b l y   b e c a u s e   o f   " c r y p t o / r s a :   v e r i f i c a t i o n   e r r o r "   w h i l e   t r y i n g   t o   v e r i f y   c a n d i d a t e   a u t h o r i t y   c e r t i f i c a t e   " k u b e r n e t e s " )   V e r i f y   t h a t   t h e   $ H O M E / . k u b e / c o n f i g   f i l e   c o n t a i n s   a   v a l i d   c e r t i f i c a t e ,   a n d   r e g e n e r a t e   a   c e r t i f i c a t e   i f   n e c e s s a r y .   T h e   c e r t i f i c a t e s   i n   a   k u b e c o n f i g   f i l e   a r e   b a s e 6 4   e n c o d e d .   T h e   b a s e 6 4   - - d e c o d e   c o m m a n d   c a n   b e   u s e d   t o   d e c o d e   t h e   c e r t i f i c a t e   a n d   o p e n s s l   x 5 0 9   - t e x t   - n o o u t   c a n   b e   u s e d   f o r   v i e w i n g   t h e   c e r t i f i c a t e   i n f o r m a t i o n . U n s e t   t h e   K U B E C O N F I G   e n v i r o n m e n t   v a r i a b l e   u s i n g : u n s e t   K U B E C O N F I G   O r   s e t   i t   t o   t h e   d e f a u l t   K U B E C O N F I G   l o c a t i o n : e x p o r t   K U B E C O N F I G = / e t c / k u b e r n e t e s / a d m i n . c o n f   A n o t h e r   w o r k a r o u n d   i s   t o   o v e r w r i t e   t h e   e x i s t i n g   k u b e c o n f i g   f o r   t h e   " a d m i n "   u s e r : m v   $ H O M E / . k u b e   $ H O M E / . k u b e . b a k   m k d i r   $ H O M E / . k u b e   s u d o   c p   - i   / e t c / k u b e r n e t e s / a d m i n . c o n f   $ H O M E / . k u b e / c o n f i g   s u d o   c h o w n   $ ( i d   - u ) : $ ( i d   - g )   $ H O M E / . k u b e / c o n f i g   V e r i f y   t


Question: What is Kubelet client certificate rotation fails?

Answer: B y   d e f a u l t ,   k u b e a d m   c o n f i g u r e s   a   k u b e l e t   w i t h   a u t o m a t i c   r o t a t i o n   o f   c l i e n t   c e r t i f i c a t e s   b y   u s i n g   t h e   / v a r / l i b / k u b e l e t / p k i / k u b e l e t - c l i e n t - c u r r e n t . p e m   s y m l i n k   s p e c i f i e d   i n   / e t c / k u b e r n e t e s / k u b e l e t . c o n f .   I f   t h i s   r o t a t i o n   p r o c e s s   f a i l s   y o u   m i g h t   s e e   e r r o r s   s u c h   a s   x 5 0 9 :   c e r t i f i c a t e   h a s   e x p i r e d   o r   i s   n o t   y e t   v a l i d   i n   k u b e - a p i s e r v e r   l o g s .   T o   f i x   t h e   i s s u e   y o u   m u s t   f o l l o w   t h e s e   s t e p s :   B a c k u p   a n d   d e l e t e   / e t c / k u b e r n e t e s / k u b e l e t . c o n f   a n d   / v a r / l i b / k u b e l e t / p k i / k u b e l e t - c l i e n t *   f r o m   t h e   f a i l e d   n o d e . F r o m   a   w o r k i n g   c o n t r o l   p l a n e   n o d e   i n   t h e   c l u s t e r   t h a t   h a s   / e t c / k u b e r n e t e s / p k i / c a . k e y   e x e c u t e   k u b e a d m   k u b e c o n f i g   u s e r   - - o r g   s y s t e m : n o d e s   - - c l i e n t - n a m e   s y s t e m : n o d e : $ N O D E   >   k u b e l e t . c o n f .   $ N O D E   m u s t   b e   s e t   t o   t h e   n a m e   o f   t h e   e x i s t i n g   f a i l e d   n o d e   i n   t h e   c l u s t e r .   M o d i f y   t h e   r e s u l t e d   k u b e l e t . c o n f   m a n u a l l y   t o   a d j u s t   t h e   c l u s t e r   n a m e   a n d   s e r v e r   e n d p o i n t ,   o r   p a s s   k u b e c o n f i g   u s e r   - - c o n f i g   ( s e e   G e n e r a t i n g   k u b e c o n f i g   f i l e s   f o r   a d d i t i o n a l   u s e r s ) .   I f   y o u r   c l u s t e r   d o e s   n o t   h a v e   t h e   c a . k e y   y o u   m u s t   s i g n   t h e   e m b e d d e d   c e r t i f


Question: What is Default NIC When using flannel as the pod network in Vagrant?

Answer: T h e   f o l l o w i n g   e r r o r   m i g h t   i n d i c a t e   t h a t   s o m e t h i n g   w a s   w r o n g   i n   t h e   p o d   n e t w o r k :   E r r o r   f r o m   s e r v e r   ( N o t F o u n d ) :   t h e   s e r v e r   c o u l d   n o t   f i n d   t h e   r e q u e s t e d   r e s o u r c e   I f   y o u ' r e   u s i n g   f l a n n e l   a s   t h e   p o d   n e t w o r k   i n s i d e   V a g r a n t ,   t h e n   y o u   w i l l   h a v e   t o   s p e c i f y   t h e   d e f a u l t   i n t e r f a c e   n a m e   f o r   f l a n n e l . V a g r a n t   t y p i c a l l y   a s s i g n s   t w o   i n t e r f a c e s   t o   a l l   V M s .   T h e   f i r s t ,   f o r   w h i c h   a l l   h o s t s   a r e   a s s i g n e d   t h e   I P   a d d r e s s   1 0 . 0 . 2 . 1 5 ,   i s   f o r   e x t e r n a l   t r a f f i c   t h a t   g e t s   N A T e d . T h i s   m a y   l e a d   t o   p r o b l e m s   w i t h   f l a n n e l ,   w h i c h   d e f a u l t s   t o   t h e   f i r s t   i n t e r f a c e   o n   a   h o s t .   T h i s   l e a d s   t o   a l l   h o s t s   t h i n k i n g   t h e y   h a v e   t h e   s a m e   p u b l i c   I P   a d d r e s s .   T o   p r e v e n t   t h i s ,   p a s s   t h e   - - i f a c e   e t h 1   f l a g   t o   f l a n n e l   s o   t h a t   t h e   s e c o n d   i n t e r f a c e   i s   c h o s e n .   I f   y o u ' r e   u s i n g   f l a n n e l   a s   t h e   p o d   n e t w o r k   i n s i d e   V a g r a n t ,   t h e n   y o u   w i l l   h a v e   t o   s p e c i f y   t h e   d e f a u l t   i n t e r f a c e   n a m e   f o r   f l a n n e l .   V a g r a n t   t y p i c a l l y   a s s i g n s   t w o   i n t e r f a c e s   t o   a l l   V M s .   T h e   f i r s t ,   f o r   w h i c h   a l l   h o s t s   a r e   a s s i g n e d   t h e   I P   a d d r e s s   1 0 . 0 . 2 . 1 5 ,   i s   f o r   e x t e r n a l   t r a f f i c   t h a t   g e t s   N A T e d .   T h


Question: What is Non-public IP used for containers?

Answer: I n   s o m e   s i t u a t i o n s   k u b e c t l   l o g s   a n d   k u b e c t l   r u n   c o m m a n d s   m a y   r e t u r n   w i t h   t h e   f o l l o w i n g   e r r o r s   i n   a n   o t h e r w i s e   f u n c t i o n a l   c l u s t e r :   E r r o r   f r o m   s e r v e r :   G e t   h t t p s : / / 1 0 . 1 9 . 0 . 4 1 : 1 0 2 5 0 / c o n t a i n e r L o g s / d e f a u l t / m y s q l - d d c 6 5 b 8 6 8 - g l c 5 m / m y s q l :   d i a l   t c p   1 0 . 1 9 . 0 . 4 1 : 1 0 2 5 0 :   g e t s o c k o p t :   n o   r o u t e   t o   h o s t   T h i s   m a y   b e   d u e   t o   K u b e r n e t e s   u s i n g   a n   I P   t h a t   c a n   n o t   c o m m u n i c a t e   w i t h   o t h e r   I P s   o n   t h e   s e e m i n g l y   s a m e   s u b n e t ,   p o s s i b l y   b y   p o l i c y   o f   t h e   m a c h i n e   p r o v i d e r . D i g i t a l O c e a n   a s s i g n s   a   p u b l i c   I P   t o   e t h 0   a s   w e l l   a s   a   p r i v a t e   o n e   t o   b e   u s e d   i n t e r n a l l y   a s   a n c h o r   f o r   t h e i r   f l o a t i n g   I P   f e a t u r e ,   y e t   k u b e l e t   w i l l   p i c k   t h e   l a t t e r   a s   t h e   n o d e ' s   I n t e r n a l I P   i n s t e a d   o f   t h e   p u b l i c   o n e . U s e   i p   a d d r   s h o w   t o   c h e c k   f o r   t h i s   s c e n a r i o   i n s t e a d   o f   i f c o n f i g   b e c a u s e   i f c o n f i g   w i l l   n o t   d i s p l a y   t h e   o f f e n d i n g   a l i a s   I P   a d d r e s s .   A l t e r n a t i v e l y   a n   A P I   e n d p o i n t   s p e c i f i c   t o   D i g i t a l O c e a n   a l l o w s   t o   q u e r y   f o r   t h e   a n c h o r   I P   f r o m   t h e   d r o p l e t : c u r l   h t t p : / / 1 6 9 . 2 5 4 . 1 6 9 . 2 5 4 / m e t a d a t a / v 1 / i n t e r f a c e s / p u b l i c / 0 / a n c h o r _ i p v 4 / a d d r e s s   T h e   w o r k a r o u n d   i s   t o   t e l l   k u b e l


Question: What is coredns pods have CrashLoopBackOff or Error state?

Answer: I f   y o u   h a v e   n o d e s   t h a t   a r e   r u n n i n g   S E L i n u x   w i t h   a n   o l d e r   v e r s i o n   o f   D o c k e r ,   y o u   m i g h t   e x p e r i e n c e   a   s c e n a r i o   w h e r e   t h e   c o r e d n s   p o d s   a r e   n o t   s t a r t i n g .   T o   s o l v e   t h a t ,   y o u   c a n   t r y   o n e   o f   t h e   f o l l o w i n g   o p t i o n s :   U p g r a d e   t o   a   n e w e r   v e r s i o n   o f   D o c k e r . D i s a b l e   S E L i n u x . M o d i f y   t h e   c o r e d n s   d e p l o y m e n t   t o   s e t   a l l o w P r i v i l e g e E s c a l a t i o n   t o   t r u e :   U p g r a d e   t o   a   n e w e r   v e r s i o n   o f   D o c k e r .   D i s a b l e   S E L i n u x .   M o d i f y   t h e   c o r e d n s   d e p l o y m e n t   t o   s e t   a l l o w P r i v i l e g e E s c a l a t i o n   t o   t r u e :   k u b e c t l   - n   k u b e - s y s t e m   g e t   d e p l o y m e n t   c o r e d n s   - o   y a m l   |   \   s e d   ' s / a l l o w P r i v i l e g e E s c a l a t i o n :   f a l s e / a l l o w P r i v i l e g e E s c a l a t i o n :   t r u e / g '   |   \   k u b e c t l   a p p l y   - f   -   A n o t h e r   c a u s e   f o r   C o r e D N S   t o   h a v e   C r a s h L o o p B a c k O f f   i s   w h e n   a   C o r e D N S   P o d   d e p l o y e d   i n   K u b e r n e t e s   d e t e c t s   a   l o o p .   A   n u m b e r   o f   w o r k a r o u n d s   a r e   a v a i l a b l e   t o   a v o i d   K u b e r n e t e s   t r y i n g   t o   r e s t a r t   t h e   C o r e D N S   P o d   e v e r y   t i m e   C o r e D N S   d e t e c t s   t h e   l o o p   a n d   e x i t s .


Question: What is etcd pods restart continually?

Answer: I f   y o u   e n c o u n t e r   t h e   f o l l o w i n g   e r r o r :   r p c   e r r o r :   c o d e   =   2   d e s c   =   o c i   r u n t i m e   e r r o r :   e x e c   f a i l e d :   c o n t a i n e r _ l i n u x . g o : 2 4 7 :   s t a r t i n g   c o n t a i n e r   p r o c e s s   c a u s e d   " p r o c e s s _ l i n u x . g o : 1 1 0 :   d e c o d i n g   i n i t   e r r o r   f r o m   p i p e   c a u s e d   \ " r e a d   p a r e n t :   c o n n e c t i o n   r e s e t   b y   p e e r \ " "   T h i s   i s s u e   a p p e a r s   i f   y o u   r u n   C e n t O S   7   w i t h   D o c k e r   1 . 1 3 . 1 . 8 4 .   T h i s   v e r s i o n   o f   D o c k e r   c a n   p r e v e n t   t h e   k u b e l e t   f r o m   e x e c u t i n g   i n t o   t h e   e t c d   c o n t a i n e r .   T o   w o r k   a r o u n d   t h e   i s s u e ,   c h o o s e   o n e   o f   t h e s e   o p t i o n s :   R o l l   b a c k   t o   a n   e a r l i e r   v e r s i o n   o f   D o c k e r ,   s u c h   a s   1 . 1 3 . 1 - 7 5 y u m   d o w n g r a d e   d o c k e r - 1 . 1 3 . 1 - 7 5 . g i t 8 6 3 3 8 7 0 . e l 7 . c e n t o s . x 8 6 _ 6 4   d o c k e r - c l i e n t - 1 . 1 3 . 1 - 7 5 . g i t 8 6 3 3 8 7 0 . e l 7 . c e n t o s . x 8 6 _ 6 4   d o c k e r - c o m m o n - 1 . 1 3 . 1 - 7 5 . g i t 8 6 3 3 8 7 0 . e l 7 . c e n t o s . x 8 6 _ 6 4   I n s t a l l   o n e   o f   t h e   m o r e   r e c e n t   r e c o m m e n d e d   v e r s i o n s ,   s u c h   a s   1 8 . 0 6 : s u d o   y u m - c o n f i g - m a n a g e r   - - a d d - r e p o   h t t p s : / / d o w n l o a d . d o c k e r . c o m / l i n u x / c e n t o s / d o c k e r - c e . r e p o   y u m   i n s t a l l   d o c k e r - c e - 1 8 . 0 6 . 1 . c e - 3 . e l 7 . x 8 6 _ 6 4   R o l l   b a c k   t o   a n   e a r l i e r   v e r s i o n   o f   D o c k e r ,   s u c h   a s   1 . 1 3 . 1 - 7 5   y u m   d o w n g r a d e   d o c k e r - 1 . 1 3 . 1 - 7 5 . g i t 8 6 3 3 8 7 0 . e l 7 . c


Question: What is Not possible to pass a comma separated list of values to arguments inside a --component-extra-args flag?

Answer: k u b e a d m   i n i t   f l a g s   s u c h   a s   - - c o m p o n e n t - e x t r a - a r g s   a l l o w   y o u   t o   p a s s   c u s t o m   a r g u m e n t s   t o   a   c o n t r o l - p l a n e   c o m p o n e n t   l i k e   t h e   k u b e - a p i s e r v e r .   H o w e v e r ,   t h i s   m e c h a n i s m   i s   l i m i t e d   d u e   t o   t h e   u n d e r l y i n g   t y p e   u s e d   f o r   p a r s i n g   t h e   v a l u e s   ( m a p S t r i n g S t r i n g ) .   I f   y o u   d e c i d e   t o   p a s s   a n   a r g u m e n t   t h a t   s u p p o r t s   m u l t i p l e ,   c o m m a - s e p a r a t e d   v a l u e s   s u c h   a s   - - a p i s e r v e r - e x t r a - a r g s   " e n a b l e - a d m i s s i o n - p l u g i n s = L i m i t R a n g e r , N a m e s p a c e E x i s t s "   t h i s   f l a g   w i l l   f a i l   w i t h   f l a g :   m a l f o r m e d   p a i r ,   e x p e c t   s t r i n g = s t r i n g .   T h i s   h a p p e n s   b e c a u s e   t h e   l i s t   o f   a r g u m e n t s   f o r   - - a p i s e r v e r - e x t r a - a r g s   e x p e c t s   k e y = v a l u e   p a i r s   a n d   i n   t h i s   c a s e   N a m e s p a c e s E x i s t s   i s   c o n s i d e r e d   a s   a   k e y   t h a t   i s   m i s s i n g   a   v a l u e .   A l t e r n a t i v e l y ,   y o u   c a n   t r y   s e p a r a t i n g   t h e   k e y = v a l u e   p a i r s   l i k e   s o :   - - a p i s e r v e r - e x t r a - a r g s   " e n a b l e - a d m i s s i o n - p l u g i n s = L i m i t R a n g e r , e n a b l e - a d m i s s i o n - p l u g i n s = N a m e s p a c e E x i s t s "   b u t   t h i s   w i l l   r e s u l t   i n   t h e   k e y   e n a b l e - a d m i s s i o n - p l u g i n s   o n l y   h a v i n g   t h e   v a l u e   o f   N a m e s p a c e E x i s t s .   A   k n o w n   w o r k a r o u n d   i s   t o   u s e   t h e   k u b e a d m   c o n f i g u r a t i o n   f i l e .


Question: What is kube-proxy scheduled before node is initialized by cloud-controller-manager?

Answer: I n   c l o u d   p r o v i d e r   s c e n a r i o s ,   k u b e - p r o x y   c a n   e n d   u p   b e i n g   s c h e d u l e d   o n   n e w   w o r k e r   n o d e s   b e f o r e   t h e   c l o u d - c o n t r o l l e r - m a n a g e r   h a s   i n i t i a l i z e d   t h e   n o d e   a d d r e s s e s .   T h i s   c a u s e s   k u b e - p r o x y   t o   f a i l   t o   p i c k   u p   t h e   n o d e ' s   I P   a d d r e s s   p r o p e r l y   a n d   h a s   k n o c k - o n   e f f e c t s   t o   t h e   p r o x y   f u n c t i o n   m a n a g i n g   l o a d   b a l a n c e r s .   T h e   f o l l o w i n g   e r r o r   c a n   b e   s e e n   i n   k u b e - p r o x y   P o d s :   s e r v e r . g o : 6 1 0 ]   F a i l e d   t o   r e t r i e v e   n o d e   I P :   h o s t   I P   u n k n o w n ;   k n o w n   a d d r e s s e s :   [ ]   p r o x i e r . g o : 3 4 0 ]   i n v a l i d   n o d e I P ,   i n i t i a l i z i n g   k u b e - p r o x y   w i t h   1 2 7 . 0 . 0 . 1   a s   n o d e I P   A   k n o w n   s o l u t i o n   i s   t o   p a t c h   t h e   k u b e - p r o x y   D a e m o n S e t   t o   a l l o w   s c h e d u l i n g   i t   o n   c o n t r o l - p l a n e   n o d e s   r e g a r d l e s s   o f   t h e i r   c o n d i t i o n s ,   k e e p i n g   i t   o f f   o f   o t h e r   n o d e s   u n t i l   t h e i r   i n i t i a l   g u a r d i n g   c o n d i t i o n s   a b a t e :   k u b e c t l   - n   k u b e - s y s t e m   p a t c h   d s   k u b e - p r o x y   - p = ' {   " s p e c " :   {   " t e m p l a t e " :   {   " s p e c " :   {   " t o l e r a t i o n s " :   [   {   " k e y " :   " C r i t i c a l A d d o n s O n l y " ,   " o p e r a t o r " :   " E x i s t s "   } ,   {   " e f f e c t " :   " N o S c h e d u l e " ,   " k e y " :   " n o d e - r o l e . k u b e r n e t e s . i o / c o n t r o l - p l a n e "   }   ]   }   }   }   } '   T h e   t r a c k i n g   i s s u e   f o r   t h i s   p r o b l


Question: What is /usr is mounted read-only on nodes?

Answer: O n   L i n u x   d i s t r i b u t i o n s   s u c h   a s   F e d o r a   C o r e O S   o r   F l a t c a r   C o n t a i n e r   L i n u x ,   t h e   d i r e c t o r y   / u s r   i s   m o u n t e d   a s   a   r e a d - o n l y   f i l e s y s t e m .   F o r   f l e x - v o l u m e   s u p p o r t ,   K u b e r n e t e s   c o m p o n e n t s   l i k e   t h e   k u b e l e t   a n d   k u b e - c o n t r o l l e r - m a n a g e r   u s e   t h e   d e f a u l t   p a t h   o f   / u s r / l i b e x e c / k u b e r n e t e s / k u b e l e t - p l u g i n s / v o l u m e / e x e c / ,   y e t   t h e   f l e x - v o l u m e   d i r e c t o r y   m u s t   b e   w r i t e a b l e   f o r   t h e   f e a t u r e   t o   w o r k .


Question: What is Note:?

Answer: T o   w o r k a r o u n d   t h i s   i s s u e ,   y o u   c a n   c o n f i g u r e   t h e   f l e x - v o l u m e   d i r e c t o r y   u s i n g   t h e   k u b e a d m   c o n f i g u r a t i o n   f i l e .   O n   t h e   p r i m a r y   c o n t r o l - p l a n e   N o d e   ( c r e a t e d   u s i n g   k u b e a d m   i n i t ) ,   p a s s   t h e   f o l l o w i n g   f i l e   u s i n g   - - c o n f i g :   a p i V e r s i o n :   k u b e a d m . k 8 s . i o / v 1 b e t a 4   k i n d :   I n i t C o n f i g u r a t i o n   n o d e R e g i s t r a t i o n :   k u b e l e t E x t r a A r g s :   -   n a m e :   " v o l u m e - p l u g i n - d i r "   v a l u e :   " / o p t / l i b e x e c / k u b e r n e t e s / k u b e l e t - p l u g i n s / v o l u m e / e x e c / "   - - -   a p i V e r s i o n :   k u b e a d m . k 8 s . i o / v 1 b e t a 4   k i n d :   C l u s t e r C o n f i g u r a t i o n   c o n t r o l l e r M a n a g e r :   e x t r a A r g s :   -   n a m e :   " f l e x - v o l u m e - p l u g i n - d i r "   v a l u e :   " / o p t / l i b e x e c / k u b e r n e t e s / k u b e l e t - p l u g i n s / v o l u m e / e x e c / "   O n   j o i n i n g   N o d e s :   a p i V e r s i o n :   k u b e a d m . k 8 s . i o / v 1 b e t a 4   k i n d :   J o i n C o n f i g u r a t i o n   n o d e R e g i s t r a t i o n :   k u b e l e t E x t r a A r g s :   -   n a m e :   " v o l u m e - p l u g i n - d i r "   v a l u e :   " / o p t / l i b e x e c / k u b e r n e t e s / k u b e l e t - p l u g i n s / v o l u m e / e x e c / "   A l t e r n a t i v e l y ,   y o u   c a n   m o d i f y   / e t c / f s t a b   t o   m a k e   t h e   / u s r   m o u n t   w r i t e a b l e ,   b u t   p l e a s e   b e   a d v i s e d   t h a t   t h i s   i s   m o d i f y i n g   a   d e s i g n   p r i n c i p l e   o f   t h e   L i n u x   d i s t r i b u t i o n .


Question: What is kubeadm upgrade plan prints out context deadline exceeded error message?

Answer: T h i s   e r r o r   m e s s a g e   i s   s h o w n   w h e n   u p g r a d i n g   a   K u b e r n e t e s   c l u s t e r   w i t h   k u b e a d m   i n   t h e   c a s e   o f   r u n n i n g   a n   e x t e r n a l   e t c d .   T h i s   i s   n o t   a   c r i t i c a l   b u g   a n d   h a p p e n s   b e c a u s e   o l d e r   v e r s i o n s   o f   k u b e a d m   p e r f o r m   a   v e r s i o n   c h e c k   o n   t h e   e x t e r n a l   e t c d   c l u s t e r .   Y o u   c a n   p r o c e e d   w i t h   k u b e a d m   u p g r a d e   a p p l y   . . . .   T h i s   i s s u e   i s   f i x e d   a s   o f   v e r s i o n   1 . 1 9 .


Question: What is kubeadm reset unmounts /var/lib/kubelet?

Answer: I f   / v a r / l i b / k u b e l e t   i s   b e i n g   m o u n t e d ,   p e r f o r m i n g   a   k u b e a d m   r e s e t   w i l l   e f f e c t i v e l y   u n m o u n t   i t .   T o   w o r k a r o u n d   t h e   i s s u e ,   r e - m o u n t   t h e   / v a r / l i b / k u b e l e t   d i r e c t o r y   a f t e r   p e r f o r m i n g   t h e   k u b e a d m   r e s e t   o p e r a t i o n .   T h i s   i s   a   r e g r e s s i o n   i n t r o d u c e d   i n   k u b e a d m   1 . 1 5 .   T h e   i s s u e   i s   f i x e d   i n   1 . 2 0 .


Question: What is Cannot use the metrics-server securely in a kubeadm cluster?

Answer: I n   a   k u b e a d m   c l u s t e r ,   t h e   m e t r i c s - s e r v e r   c a n   b e   u s e d   i n s e c u r e l y   b y   p a s s i n g   t h e   - - k u b e l e t - i n s e c u r e - t l s   t o   i t .   T h i s   i s   n o t   r e c o m m e n d e d   f o r   p r o d u c t i o n   c l u s t e r s .   I f   y o u   w a n t   t o   u s e   T L S   b e t w e e n   t h e   m e t r i c s - s e r v e r   a n d   t h e   k u b e l e t   t h e r e   i s   a   p r o b l e m ,   s i n c e   k u b e a d m   d e p l o y s   a   s e l f - s i g n e d   s e r v i n g   c e r t i f i c a t e   f o r   t h e   k u b e l e t .   T h i s   c a n   c a u s e   t h e   f o l l o w i n g   e r r o r s   o n   t h e   s i d e   o f   t h e   m e t r i c s - s e r v e r :   x 5 0 9 :   c e r t i f i c a t e   s i g n e d   b y   u n k n o w n   a u t h o r i t y   x 5 0 9 :   c e r t i f i c a t e   i s   v a l i d   f o r   I P - f o o   n o t   I P - b a r   S e e   E n a b l i n g   s i g n e d   k u b e l e t   s e r v i n g   c e r t i f i c a t e s   t o   u n d e r s t a n d   h o w   t o   c o n f i g u r e   t h e   k u b e l e t s   i n   a   k u b e a d m   c l u s t e r   t o   h a v e   p r o p e r l y   s i g n e d   s e r v i n g   c e r t i f i c a t e s .   A l s o   s e e   H o w   t o   r u n   t h e   m e t r i c s - s e r v e r   s e c u r e l y .


Question: What is Upgrade fails due to etcd hash not changing?

Answer: O n l y   a p p l i c a b l e   t o   u p g r a d i n g   a   c o n t r o l   p l a n e   n o d e   w i t h   a   k u b e a d m   b i n a r y   v 1 . 2 8 . 3   o r   l a t e r ,   w h e r e   t h e   n o d e   i s   c u r r e n t l y   m a n a g e d   b y   k u b e a d m   v e r s i o n s   v 1 . 2 8 . 0 ,   v 1 . 2 8 . 1   o r   v 1 . 2 8 . 2 .   H e r e   i s   t h e   e r r o r   m e s s a g e   y o u   m a y   e n c o u n t e r :   [ u p g r a d e / e t c d ]   F a i l e d   t o   u p g r a d e   e t c d :   c o u l d n ' t   u p g r a d e   c o n t r o l   p l a n e .   k u b e a d m   h a s   t r i e d   t o   r e c o v e r   e v e r y t h i n g   i n t o   t h e   e a r l i e r   s t a t e .   E r r o r s   f a c e d :   s t a t i c   P o d   h a s h   f o r   c o m p o n e n t   e t c d   o n   N o d e   k i n d e r - u p g r a d e - c o n t r o l - p l a n e - 1   d i d   n o t   c h a n g e   a f t e r   5 m 0 s :   t i m e d   o u t   w a i t i n g   f o r   t h e   c o n d i t i o n   [ u p g r a d e / e t c d ]   W a i t i n g   f o r   p r e v i o u s   e t c d   t o   b e c o m e   a v a i l a b l e   I 0 9 0 7   1 0 : 1 0 : 0 9 . 1 0 9 1 0 4   3 7 0 4   e t c d . g o : 5 8 8 ]   [ e t c d ]   a t t e m p t i n g   t o   s e e   i f   a l l   c l u s t e r   e n d p o i n t s   ( [ h t t p s : / / 1 7 2 . 1 7 . 0 . 6 : 2 3 7 9 /   h t t p s : / / 1 7 2 . 1 7 . 0 . 4 : 2 3 7 9 /   h t t p s : / / 1 7 2 . 1 7 . 0 . 3 : 2 3 7 9 / ] )   a r e   a v a i l a b l e   1 / 1 0   [ u p g r a d e / e t c d ]   E t c d   w a s   r o l l e d   b a c k   a n d   i s   n o w   a v a i l a b l e   s t a t i c   P o d   h a s h   f o r   c o m p o n e n t   e t c d   o n   N o d e   k i n d e r - u p g r a d e - c o n t r o l - p l a n e - 1   d i d   n o t   c h a n g e   a f t e r   5 m 0 s :   t i m e d   o u t   w a i t i n g   f o r   t h e   c o n d i t i o n   c o u l d n ' t   u p g r a d e   c o n t r o l   p l a n e .   k u b e a d m   h a s   t r i e d   t


Question: What is Manual Rotation of CA Certificates?

Answer: Manual Rotation of CA CertificatesThis page shows how to manually rotate the certificate authority (CA) certificates.Before you beginYou need to have a Kubernetes cluster, and the kubectl command-line tool must be configured to communicate with your cluster. It is recommended to run this tutorial on a cluster with at least two nodes that are not acting as control plane hosts. If you do not already have a cluster, you can create one by using minikube or you can use one of these Kubernetes playgrounds:iximiuz LabsKillercodaKodeKloudPlay with KubernetesFor more information about authentication in Kubernetes, see Authenticating.For more information about best practices for CA certificates, see Single root CA.Rotate the CA certificates manuallyCaution:Make sure to back up your certificate directory along with configuration files and any other necessary files.This approach assumes operation of the Kubernetes control plane in a HA configuration with multiple API servers. Graceful termination of the API server is also assumed so clients can cleanly disconnect from one API server and reconnect to another.Configurations with a single API server will experience unavailability while the API server is being restarted.Distribute the new CA certificates and private keys (for example: ca.crt, ca.key, front-proxy-ca.crt, and front-proxy-ca.key) to all your control plane nodes in the Kubernetes certificates directory.Update the --root-ca-file flag for the kube-controller-manager to include both old and new CA, then restart the kube-controller-manager.Any ServiceAccount created after this point will get Secrets that include both old and new CAs.Note:The files specified by the kube-controller-manager flags --client-ca-file and --cluster-signing-cert-file cannot be CA bundles. If these flags and --root-ca-file point to the same ca.crt file which is now a bundle (includes both old and new CA) you will face an error. To workaround this problem you can copy the new CA to a separate file and make the flags --client-ca-file and --cluster-signing-cert-file point to the copy. Once ca.crt is no longer a bundle you can restore the problem flags to point to ca.crt and delete the copy.Issue 1350 for kubeadm tracks an bug with the kube-controller-manager being unable to accept a CA bundle.Wait for the controller manager to update ca.crt in the service account Secrets to include both old and new CA certificates.If any Pods are started before new CA is used by API servers, the new Pods get this update and will trust both old and new CAs.Restart all pods using in-cluster configurations (for example: kube-proxy, CoreDNS, etc) so they can use the updated certificate authority data from Secrets that link to ServiceAccounts.Make sure CoreDNS, kube-proxy and other Pods using in-cluster configurations are working as expected.Append the both old and new CA to the file against --client-ca-file and --kubelet-certificate-authority flag in the kube-apiserver configuration.Append the both old