https://kb.novaordis.com/api.php?action=feedcontributions&user=Ovidiu&feedformat=atomNovaOrdis Knowledge Base - User contributions [en]2024-03-28T22:24:57ZUser contributionsMediaWiki 1.39.6https://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108984Kubernetes Security Concepts2024-03-27T22:37:27Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and the cluster's CA certificate:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
Note that the token is base64-encoded in the descriptor and it will need decoding to be used as credentials.<br />
<br />
The content of the token, which represents the credentials of the service account, are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the CA certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as shown in the example below. Note that the context of the token must be base64-decoded with:<br />
<syntaxhighlight lang='bash'> <br />
kubectl get secret some-sa-token-hqmmc -o=jsonpath='{.data.token}' | base64 -d <br />
</syntaxhighlight> <br />
The CA certificate does not require decoding.<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <base64-decoded_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server: https://...example.com:443<br />
certificate-authority-data: <the_content_of_the_service_account_secret_ca_certficate_as_is><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108983Kubernetes Security Concepts2024-03-27T22:36:35Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and the cluster's CA certificate:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
Note that the token is base64-encoded in the descriptor and it will need decoding to be used as credentials.<br />
<br />
The content of the token, which represents the credentials of the service account, are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the CA certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as shown in the example below. Note that the context of the token must be base64-decoded with:<br />
<syntaxhighlight lang='bash'> <br />
kubectl get secret some-sa-token-hqmmc -o=jsonpath='{.data.token}' | base64 -d <br />
</syntaxhighlight> <br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <base64-decoded_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server: https://...example.com:443<br />
certificate-authority-data: <the_content_of_the_service_account_secret_certficate><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108982Kubernetes Security Concepts2024-03-27T22:32:35Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and the cluster's CA certificate:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as follows <font color=darkkhaki>(needs testing and confirmation it works)</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <the_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server: https://...example.com:443<br />
certificate-authority-data: <the_content_of_the_service_account_secret_certficate><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108981Kubernetes Security Concepts2024-03-27T22:02:59Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as follows <font color=darkkhaki>(needs testing and confirmation it works)</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <the_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server: https://...example.com:443<br />
certificate-authority-data: <the_content_of_the_service_account_secret_certficate><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108980Kubernetes Security Concepts2024-03-27T21:53:05Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as follows:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <the_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server: https://...example.com:443<br />
certificate-authority-data: <the_content_of_the_service_account_secret_certficate><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108979Kubernetes Security Concepts2024-03-27T21:43:40Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]]. For that, the token and the certificate stored in the Secret associated with the Service Account must be configured in a <code>kubectl</code> context as follows:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Config<br />
users:<br />
- name: some-sa-on-some-cluster<br />
user:<br />
token: <the_content_of_the_service_account_secret_token><br />
clusters:<br />
- name: some-cluster<br />
cluster:<br />
server:<br />
certificate-authority-data: <the_content_of_the_service_account_secret_certficate><br />
contexts:<br />
- name: some-context<br />
context:<br />
cluster: some-cluster<br />
namespace: some-namespace<br />
user: some-sa-on-some-cluster<br />
</syntaxhighlight><br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108978Kubernetes Security Concepts2024-03-27T21:35:53Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that [[#Kubectl_System_Account_Identity|allows interaction with the API server under the identity of the System Account]].<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108977Kubernetes Security Concepts2024-03-27T21:35:20Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'></span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that allows interaction with the API server under the identity of the System Account.<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108976Kubernetes Security Concepts2024-03-27T21:35:04Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<span id='Kubectl_System_Account_Identity'</span><code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that allows interaction with the API server under the identity of the System Account.<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108975Kubernetes Security Concepts2024-03-27T21:34:37Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
The system account token can be used to configure a <code>kubectl</code> context that allows interaction with the API server under the identity of the System Account.<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108974Kubernetes Security Concepts2024-03-27T21:33:23Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
A Secret of type <code>kubernetes.io/service-account-token</code>, which contains the '''system account token''' is automatically created when the Service Account is created. A <code>kubectl</code> context must be configured with this system account token to allow interaction under the identity of the System Account. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108973Kubernetes Security Concepts2024-03-27T21:28:10Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
<br />
A Secret of type <code>kubernetes.io/service-account-token</code> is automatically created when the Service Account is created. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
The service account token metadata includes the token itself, and a certificate <font color=darkkhaki>signed by the cluster's CA</font>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
metadata:<br />
name: some-sa-token-hqmmc<br />
[...]<br />
kind: Secret<br />
type: kubernetes.io/service-account-token<br />
data:<br />
ca.crt: LS0tL...Cg==<br />
namespace: ZGlj...w==<br />
token: ZXlK...Zw==<br />
</syntaxhighlight><br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
{{Warn|⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly}}<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108972Kubernetes Security Concepts2024-03-27T21:24:25Z<p>Ovidiu: /* Service Account Token */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
<br />
A Secret of type <code>kubernetes.io/service-account-token</code> is automatically created when the Service Account is created. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
⚠️ The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108971Kubernetes Security Concepts2024-03-27T21:23:37Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
<br />
A Secret of type <code>kubernetes.io/service-account-token</code> is automatically created when the Service Account is created. For more details, see the [[#Service_Account_Token|Service Account Token]] section below.<br />
<br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
<br />
====<span id='Service_Accounts_and_Credentials'></span><span id='Service_Accounts_Credentials_(service-account-token)'></span>Service Account Token==== <br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
When a new Service Account is created, a Secret of type <code>kubernetes.io/service-account-token</code> is automatically created for the Service Account. Its name is listed in the Service Account's descriptor:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: some-sa<br />
[...]<br />
secrets:<br />
- name: some-sa-token-hqmmc<br />
</syntaxhighlight><br />
<br />
<br />
⚠️ A '''service account token''' is created when the Service Account is created. The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
<br />
<br />
<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108970Kubernetes Security Concepts2024-03-27T21:18:15Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
⚠️ A '''service account token''' is created when the Service Account is created. The service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
====<span id='Service_Accounts_and_Credentials'></span>Service Accounts Credentials (<tt>service-account-token</tt>)====<br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108969Kubernetes Security Concepts2024-03-27T21:09:48Z<p>Ovidiu: /* Service Accounts Credentials (service-account-token) */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
⚠️ A service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
====<span id='Service_Accounts_and_Credentials'></span>Service Accounts Credentials (<tt>service-account-token</tt>)====<br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting <code>[https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken]</code> in the service account manifest.<br />
<br />
<font color=darkkhaki><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108968Kubernetes Security Concepts2024-03-27T21:08:14Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes, in that there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
⚠️ A service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
====<span id='Service_Accounts_and_Credentials'></span>Service Accounts Credentials (<tt>service-account-token</tt>)====<br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting [https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken] in the service account manifest.<br />
<br />
<font color=darkgray><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Security_Concepts&diff=108967Kubernetes Security Concepts2024-03-27T21:07:46Z<p>Ovidiu: /* Service Account */</p>
<hr />
<div>=Internal=<br />
<br />
* [[Kubernetes_Concepts#Subjects|Kubernetes Concepts]]<br />
* [[Kubernetes Security Operations]]<br />
* [[Linux Security Concepts]]<br />
=Overview=<br />
Kubernetes security covers several areas:<br />
# The Kubernetes cluster must provide infrastructure and configuration capabilities that ensure the API sever is [[#Authentication|only accessible to authenticated identities]]. Once the identity of the caller is established through authentication, the Kubernetes cluster must ensure that [[#Controlling_Access_to_the_Kubernetes_API|access is limited only to resources that are supposed to be accessible]] to the authenticated identity and the access is denied for all other resources.<br />
# Kubernetes clusters runs arbitrary applications in pods, and they must provide mechanisms that prevent a possibly malicious application from accessing node and network resources it is not supposed to access. This subject is covered by [[#Pod_and_Container_Security|Pod and Container Security]] section.<br />
# Kubernetes expects that all API communication in the cluster is encrypted. This subject is covered in [[#Transport_Security|Transport Security]] section.<br />
<br />
=<span id='Authentication'></span><span id='Identity_while_Accessing_the_Cluster'></span>API Server Authentication - Identity while Accessing the Cluster=<br />
<br />
The [[#Identities|identity]] while accessing the Kubernetes cluster is associated with a (usually human) [[#User|user]], authenticated as part of the request handling, that uses an external CLI tool such as <code>[[kubectl]]</code>, or with a [[#Service_Account|service account]], which provides identity to pods and containers running inside the pods that make API requests as part of their logic. If an API request is not associated with any of these identities, it is treaded as an [[#Anonymous_Request|anonymous request]]. The available [[#API_Authentication_Strategies|API request authentication strategies]] are described below.<br />
<br />
==Identities==<br />
===<span id='User_Account'></span>User===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/}}<br />
Users are sometimes referred to as "users accounts" or "normal users". There is no "User" Kubernetes API resource, and users cannot be added through an API call. It is assumed that a cluster-independent service manages users. That service can be implemented as a username/password file, a user store like Google Accounts, or an administrator that distributes private keys. When the authentication credentials are presented to the API server, the API server extracts the user name from the credentials (e.g. from the common name field in the "subject" of the certificate, "/CN=alice"). <font color=darkgray>Explain how the Kubernetes authentication token is associated with such a request.</font><br />
====User Operations====<br />
* [[Kubernetes_User_Operations#Create_a_Normal_User|Create a Normal User]]<br />
* [[Kubernetes User Operations#Add_a_Normal_User_via_a_Certificate|Add a Normal User via a Certificate]]<br />
* Define a user in EKS:<br />
** [[Amazon_EKS_Operations#Associate_an_IAM_Role_with_a_Kubernetes_User|Associate an IAM Role with the Kubernetes User]]<br />
<br />
===Group===<br />
===Service Account===<br />
{{External|https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/}}<br />
Processes in containers inside [[Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|pods]] can contact the API server, and they need an identity when doing so. A '''service account''' provides the identity for processes that run in a the pod. Processes will authenticate using the identity provided by the service account. By default, in absence of specific configuration, the pods will authenticate as the [[#Default_Service_Account|default service account]] in the namespace they are running in. A specific service account name can be specified in the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. Also see [[#Non-Default_Service_Accounts|Non-Default Service Accounts]] below. <br />
<br />
Unlike [[#User|users]], the service accounts are resources managed by Kubernetes - there is a <code>[[Kubernetes_API_Resources_Concepts#Service_Account|ServiceAccount]]</code> Kubernetes [[Kubernetes_API_Resources_Concepts#API_Resources|API resource]]. Service accounts can be created via API calls, or automatically by the server. The service accounts are bound to specific namespaces. A service account is tied to a set of credentials stored as Secrets, which are mounted into the pod allowing the in-cluster processes to talk to the Kubernetes API. For more details on associating service account with secrets, see [[#Service_Accounts_and_Credentials|Service Accounts and Credentials]] below. Service accounts can be bound to specific roles. For more details on associating service account with roles, see [[#Service_Accounts_and_Roles|Service Accounts and Roles]] below.<br />
<br />
⚠️ A service account token is a long-lived, static credential. If it is compromised, lost, or stolen, an attacker may be able to perform all the actions associated with that token until the service account is deleted, so use them sparingly.<br />
<br />
Service accounts are rendered in logs using their [[#Service_Account_Full_Name|full name]] (e.g. "system:serviceaccount:blue:default)<br />
<br />
<code>kubectl</code> operations can be conducted under the identity of a specific service account using [[Kubectl#--as|kubectl --as option]]:<br />
<syntaxhighlight lang='bash'><br />
kubectl --as=system:serviceaccount:<namespace>:<service-account-name> -n <namespace> ...<br />
</syntaxhighlight><br />
Also see: {{Internal|Kubernetes_Pod_and_Container_Concepts#Pods_and_Service_Accounts|Pod Service Account}}<br />
====<span id='Service_Accounts_and_Credentials'></span>Service Accounts Credentials (<tt>service-account-token</tt>)====<br />
{{External|https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets}}<br />
<br />
The credentials (the token) for a service account are projected into the filesystem of each container of the pod as a <code>[[Kubernetes_Cluster_Configuration_Concepts#kubernetes.io.2Fservice-account-token| kubernetes.io/service-account-token]]</code>-type secret. The secret content is available to the container processes as <code>/var/run/secrets/kubernetes.io/serviceaccount/ca.crt</code> and by default is auto-mounted. In Kubernetes 1.6+ it can be opted out of auto-mounting by setting [https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#use-the-default-service-account-to-access-the-api-server automountServiceAccountToken] in the service account manifest.<br />
<br />
<font color=darkgray><br />
Clarify the following:<br />
* The default namespace to be used for namespaced API operations is placed on the filesystem of each container of the pod at <code>/var/run/secrets/kubernetes.io/serviceaccount/namespace</code>. <br />
* What are the default credentials associated with the default service account.<br />
* How can this be association displayed.<br />
* What happens with a newly created service account - does it need to be explicitly associated with those credentials or the association is done automatically upon creation.<br />
</font><br />
<br />
====Service Account Full Name====<br />
<br />
Service accounts can be referred to using their full name <code>system:serviceaccount:<namespace>:<account-name></code> (e.g. <code>system:serviceaccount:blue:default</code>).<br />
<br />
====Default Service Account====<br />
<br />
Each namespace comes with a default service account:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: ServiceAccount<br />
metadata:<br />
name: default<br />
namespace: default<br />
secrets:<br />
- name: default-token-dddkl<br />
</syntaxhighlight><br />
<br />
A [[Kubernetes_Pod_Manifest#Example|pod]] whose service account was not explicitly configured will run with the default service account for its namespace, its configuration is equivalent with:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: default<br />
[...]<br />
</syntaxhighlight><br />
The a specific service account can be configured as such:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Pod<br />
spec:<br />
containers:<br />
- name: [...]<br />
[...]<br />
serviceAccountName: something<br />
[...]<br />
</syntaxhighlight><br />
<br />
====Non-Default Service Accounts====<br />
<br />
To use a non-default service account, set <code>spec.serviceAccountName</code> field of the [[Kubernetes_Pod_Manifest#serviceAccountName|pod manifest]]. The service account has to exist at the time the pod is created, or it will be rejected. If the pod was already created, the service account cannot be updated.<br />
<br />
====Service Accounts and Roles====<br />
The recommended way to ensure that an application operates within a specific security scope is to bind a role or a set of roles to an application-specific service account. For more details, see: {{Internal|Kubernetes_Role_Based_Access_Control_Concepts|Role Based Access Control Concepts}}<br />
====Service Account Operations====<br />
* [[Kubernetes_Service_Account_Operations#Details_about_the_Namespace.27s_Default_Service_Account|Details about the Namespace's Default Service Account]]<br />
* [[Kubernetes_Service_Account_Operations#Deploy_a_Service_Account.2C_a_Role_and_a_Role_Binding_with_a_Helm_Chart|Deploy a Service Account, a Role and a Role Binding with a Helm Chart]]<br />
* [[Kubernetes_RBAC_Operations#Assigning_a_Cluster_Role_to_a_Service_Account|Assigning a Cluster Role to a Service Account]]<br />
<br />
===Anonymous Request===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#anonymous-requests}}<br />
When the API server handles a request, it first attempts to authenticate the identity making the request with one of the available authentication methods. If all authentication methods fail, and if anonymous request support is enabled, the identity is treated as anonymous requests, and given a username of <code>system:anonymous</code> and a group of <code>system:unauthenticated</code>.<br />
<br />
==API Authentication Strategies==<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authentication-strategies}}<br />
Kubernetes provides various authentication strategies to be used by the clients that send API requests into the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|Kubernetes API server]]. These authentication strategies are implemented by the server's authentication plugins. <br />
===Client X.509 Certificates===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certs}}<br />
===Bearer Tokens===<br />
[[Kubectl#--token|kubectl]] allows specifying a bearer token in-line with <code>--token</code>:<br />
<syntaxhighlight lang='bash'><br />
kubectl --token aHR0c...NiYg get pods<br />
</syntaxhighlight><br />
====Webhook Token Authentication====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#webhook-token-authentication}}<br />
=====EKS Webhook Token Authentication=====<br />
EKS has native support for webhook token authentication. See: {{Internal|EKS Webhook Token Authentication|EKS Webhook Token Authentication}}<br />
====Service Account Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens}}<br />
<font color=darkgray>TODO: reconcile Service Account Tokens and <code>service-account-token</code> [[#Service_Accounts_Credentials_.28service-account-token.29|secrets]].</font><br />
====Static Token File====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#static-token-file}}<br />
====Bootstrap Tokens====<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#bootstrap-tokens}}<br />
===Authenticating Proxy===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy}}<br />
===HTTP Basic Auth===<br />
===OpenID Connect Tokens===<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/authentication/#openid-connect-tokens}}<br />
<br />
=Controlling Access to the Kubernetes API=<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/controlling-access/}}<br />
<br />
==<span id='Cluster_Administrator'></span>Role Based Access Control (RBAC)==<br />
{{Internal|Kubernetes Role Based Access Control Concepts|Kubernetes Role Based Access Control (RBAC) Concepts}}<br />
<br />
=<span id='Pod_Security_Context_and_Container_Security_Context'></span><span id='Pod_Security_Context'></span><span id='Pod_Security'></span><span id='Pod_Security_Policies'></span>Pod and Container Security=<br />
<br />
For more details on pod and container security concepts, including '''pod and container security contexts''' and '''pod security policies''', see:<br />
{{Internal|Kubernetes Pod and Container Security|Pod and Container Security}}<br />
<br />
=Transport Security=<br />
{{External|https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#use-transport-layer-security-tls-for-all-api-traffic}}<br />
{{External|https://kubernetes.io/docs/reference/access-authn-authz/#transport-security}}<br />
Kubernetes expects that all API communication in the cluster is encrypted by default with TLS, and the majority of installation methods will allow the necessary certificates to be created and distributed to the cluster components.<br />
==Certificates Required by a Kubernetes Cluster==<br />
{{External|https://kubernetes.io/docs/setup/best-practices/certificates/}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Ingress_Concepts&diff=108966Kubernetes Ingress Concepts2024-03-22T22:18:52Z<p>Ovidiu: /* OpenShift Ingress Operator */</p>
<hr />
<div>=External=<br />
<br />
* https://medium.com/google-cloud/understanding-kubernetes-networking-ingress-1bc341c84078<br />
<br />
=Internal=<br />
<br />
* [[Kubernetes_Networking_Concepts#Ingress|Networking Concepts]]<br />
* [[Kubernetes_Service_Concepts#Ingress|Service Concepts]]<br />
* [[Amazon_EKS_Concepts#Load_Balancing_and_Ingress|Amazon EKS Concepts - Load Balancing and Ingress]]<br />
=Overview=<br />
<br />
An Ingress is a mechanism that operates at the application layer of the network stack (HTTP) and brings layer 7 features such as host and path-based routing and cookie-based session affinity to [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]]. Ingress cooperates with services to distribute load to pods. It exposes multiple services through a single IP address, and its implementation differs fundamentally from the implementation of [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]], [[Kubernetes_Service_Concepts#NodePort_Service|NodePort]] and [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] services. The reasons to use an Ingress include:<br />
* One Ingress can serve multiple services, behind a single public IP address, while each [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] requires its own native load balancer, each of them requiring their own public IP address.<br />
* Host, paths and cookies can be used to route the request.<br />
<br />
The Ingress mechanism consists of an [[Kubernetes_Ingress_Concepts#Ingress_Controller|Ingress controller]] and an [[Kubernetes_Ingress_Concepts#Ingress_API_Resource|Ingress resource]].<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ingress}}<br />
<br />
=How Ingress Works=<br />
<br />
A client attempting to access http://a.example.com performs a DNS query and resolves the name to the public IP address of the [[#Ingress_Controller|ingress controller]] external load balancer. The client then establishes a connection to the ingress external load balancer, specifying "http://a.example.com" in the "Host" header. The ingress controller load balancer forwards the request, via the ingress controller LoadBalancer/NodePort/ClientIP service, to the ingress controller pod. The ingress controller's state was updated when an ingress resource that defines the host "a.example.com" was deployed, and it knows that all requests for "a.example.com" must be forwarded to the "a" ClusterIP service. <br />
<br />
When a request with an "a.example.com" Host header arrives, the ingress controller locates the corresponding service, gets the associated Endpoints, picks up a pod and forwards the request to the pod. This is how virtual hosts are handled in web servers.<br />
<br />
Note that the ingress controller does not forward the request to the service, but it only uses it to get the associated Endpoints and select a pod. <br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Ingress.png]]<br />
<br><br />
<br />
=Ingress Controller=<br />
<br />
The ingress controller is the process - most likely running as a pod or pods inside the Kubernetes cluster itself - that accepts the HTTP connections, distributes traffic, terminates SSL connections, etc. Some Kubernetes distributions provide an ingress controller as an "add-on". For example, minikube has a <code>minikube addons enable ingress</code> command. If the ingress controller is not provided as add-on, it can be installed. There is a default "ingress-nginx" ingress controller that can be installed in any Kubernetes instance. More details: {{Internal|ingress-nginx|ingress-nginx}}<br />
<br />
The ingress controller deploys as part of its installation a layer 4 service of its own, most likely a [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]. As such, the ingress controller pod(s) is exposed to external requests via its own [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]/[[Kubernetes_Service_Concepts#NodePort_Service|NodePort]]/[[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]] service.<br />
<br />
To access a service through an its external name, the external service DNS name must resolve to the public IP address of the load balancer deployed by the ingress controller's LoadBalancer service.<br />
<br />
=Ingress API Resource=<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/ingress/}}<br />
<br />
An Ingress is the [[Kubernetes API Resources Concepts#Ingress|Kubernetes API resource]] that declares access to level 4 [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]] and allows configuring this access based on host and path. The ingress controller is notified when new ingress objects are created or when the existing ingress objects are modified and updates its internal state accordingly.<br />
<br />
Both "rules" and "paths" elements of an ingress manifest are arrays, so they can contain multiple items. An ingress can map multiple hosts and paths to multiple services. Multiple paths on the same host can be mapped to multiple services.<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: extensions/v1beta1<br />
kind: Ingress<br />
metadata:<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
name: example<br />
spec:<br />
rules:<br />
- host: a.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: a<br />
servicePort: 80<br />
<br />
- host: b.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: b<br />
servicePort: 80<br />
defaultBackend:<br />
service:<br />
name: c<br />
port:<br />
number: 81<br />
# This section is only required if TLS is to be enabled for the Ingress<br />
tls:<br />
- hosts:<br />
- a.local<br />
secretName: example-tls-a<br />
</syntaxhighlight><br />
<br />
Some cloud providers require that the ingress points to a NodePort service, but this is not a Kubernetes requirement.<br />
<br />
The ingress must be deployed in the same namespace as the services it serves.<br />
<br />
Upon deployment, the hosts served by an ingress, as well as its public address and port can be displayed with the <code>kubectl get ingress</code> command. In the example below, the ingress and the ingress controller have been deployed locally on Docker Desktop Kubernetes, hence the "localhost":<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl -n <namespace> get ingress <ingress-name><br />
<br />
NAME HOSTS ADDRESS PORTS AGE<br />
example a.example.com,b.example.com localhost 80 83s<br />
</syntaxhighlight><br />
<br />
==TLS Support==<br />
<br />
<font color=darkkhaki>TODO Kubernetes in Action Section 5.4.4.</font><br />
<br />
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Secret<br />
type: kubernetes.io/tls<br />
metadata:<br />
name: example-tls<br />
namespace: test-namespace<br />
data:<br />
tls.crt: <base64 encoded cert><br />
tls.key: <base64 encoded key><br />
</syntaxhighlight><br />
<br />
==Rewriting Paths==<br />
{{External|https://github.com/kubernetes/ingress-nginx/blob/main/docs/examples/rewrite/README.md}}<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: networking.k8s.io/v1<br />
kind: Ingress<br />
metadata:<br />
name: test<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
nginx.ingress.kubernetes.io/rewrite-target: /$2<br />
spec:<br />
rules:<br />
- host: somehost.example.com<br />
http:<br />
paths:<br />
- path: /prod(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-prod<br />
port:<br />
number: 8080<br />
- path: /stage(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-stage<br />
port:<br />
number: 8080<br />
</syntaxhighlight><br />
<br />
==Different Ingresses with Different Path for the Same Host==<br />
<font color=darkkhaki><br />
If multiple Ingresses define different paths for the same host, the behavior depends on the ingress controller. NGINX merges the definitions.<br />
</font></div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Ingress_Concepts&diff=108965Kubernetes Ingress Concepts2024-03-22T22:18:26Z<p>Ovidiu: /* Ingress API Resource */</p>
<hr />
<div>=External=<br />
<br />
* https://medium.com/google-cloud/understanding-kubernetes-networking-ingress-1bc341c84078<br />
<br />
=Internal=<br />
<br />
* [[Kubernetes_Networking_Concepts#Ingress|Networking Concepts]]<br />
* [[Kubernetes_Service_Concepts#Ingress|Service Concepts]]<br />
* [[Amazon_EKS_Concepts#Load_Balancing_and_Ingress|Amazon EKS Concepts - Load Balancing and Ingress]]<br />
=Overview=<br />
<br />
An Ingress is a mechanism that operates at the application layer of the network stack (HTTP) and brings layer 7 features such as host and path-based routing and cookie-based session affinity to [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]]. Ingress cooperates with services to distribute load to pods. It exposes multiple services through a single IP address, and its implementation differs fundamentally from the implementation of [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]], [[Kubernetes_Service_Concepts#NodePort_Service|NodePort]] and [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] services. The reasons to use an Ingress include:<br />
* One Ingress can serve multiple services, behind a single public IP address, while each [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] requires its own native load balancer, each of them requiring their own public IP address.<br />
* Host, paths and cookies can be used to route the request.<br />
<br />
The Ingress mechanism consists of an [[Kubernetes_Ingress_Concepts#Ingress_Controller|Ingress controller]] and an [[Kubernetes_Ingress_Concepts#Ingress_API_Resource|Ingress resource]].<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ingress}}<br />
<br />
=How Ingress Works=<br />
<br />
A client attempting to access http://a.example.com performs a DNS query and resolves the name to the public IP address of the [[#Ingress_Controller|ingress controller]] external load balancer. The client then establishes a connection to the ingress external load balancer, specifying "http://a.example.com" in the "Host" header. The ingress controller load balancer forwards the request, via the ingress controller LoadBalancer/NodePort/ClientIP service, to the ingress controller pod. The ingress controller's state was updated when an ingress resource that defines the host "a.example.com" was deployed, and it knows that all requests for "a.example.com" must be forwarded to the "a" ClusterIP service. <br />
<br />
When a request with an "a.example.com" Host header arrives, the ingress controller locates the corresponding service, gets the associated Endpoints, picks up a pod and forwards the request to the pod. This is how virtual hosts are handled in web servers.<br />
<br />
Note that the ingress controller does not forward the request to the service, but it only uses it to get the associated Endpoints and select a pod. <br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Ingress.png]]<br />
<br><br />
<br />
=Ingress Controller=<br />
<br />
The ingress controller is the process - most likely running as a pod or pods inside the Kubernetes cluster itself - that accepts the HTTP connections, distributes traffic, terminates SSL connections, etc. Some Kubernetes distributions provide an ingress controller as an "add-on". For example, minikube has a <code>minikube addons enable ingress</code> command. If the ingress controller is not provided as add-on, it can be installed. There is a default "ingress-nginx" ingress controller that can be installed in any Kubernetes instance. More details: {{Internal|ingress-nginx|ingress-nginx}}<br />
<br />
The ingress controller deploys as part of its installation a layer 4 service of its own, most likely a [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]. As such, the ingress controller pod(s) is exposed to external requests via its own [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]/[[Kubernetes_Service_Concepts#NodePort_Service|NodePort]]/[[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]] service.<br />
<br />
To access a service through an its external name, the external service DNS name must resolve to the public IP address of the load balancer deployed by the ingress controller's LoadBalancer service.<br />
<br />
=Ingress API Resource=<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/ingress/}}<br />
<br />
An Ingress is the [[Kubernetes API Resources Concepts#Ingress|Kubernetes API resource]] that declares access to level 4 [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]] and allows configuring this access based on host and path. The ingress controller is notified when new ingress objects are created or when the existing ingress objects are modified and updates its internal state accordingly.<br />
<br />
Both "rules" and "paths" elements of an ingress manifest are arrays, so they can contain multiple items. An ingress can map multiple hosts and paths to multiple services. Multiple paths on the same host can be mapped to multiple services.<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: extensions/v1beta1<br />
kind: Ingress<br />
metadata:<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
name: example<br />
spec:<br />
rules:<br />
- host: a.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: a<br />
servicePort: 80<br />
<br />
- host: b.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: b<br />
servicePort: 80<br />
defaultBackend:<br />
service:<br />
name: c<br />
port:<br />
number: 81<br />
# This section is only required if TLS is to be enabled for the Ingress<br />
tls:<br />
- hosts:<br />
- a.local<br />
secretName: example-tls-a<br />
</syntaxhighlight><br />
<br />
Some cloud providers require that the ingress points to a NodePort service, but this is not a Kubernetes requirement.<br />
<br />
The ingress must be deployed in the same namespace as the services it serves.<br />
<br />
Upon deployment, the hosts served by an ingress, as well as its public address and port can be displayed with the <code>kubectl get ingress</code> command. In the example below, the ingress and the ingress controller have been deployed locally on Docker Desktop Kubernetes, hence the "localhost":<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl -n <namespace> get ingress <ingress-name><br />
<br />
NAME HOSTS ADDRESS PORTS AGE<br />
example a.example.com,b.example.com localhost 80 83s<br />
</syntaxhighlight><br />
<br />
==TLS Support==<br />
<br />
<font color=darkkhaki>TODO Kubernetes in Action Section 5.4.4.</font><br />
<br />
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Secret<br />
type: kubernetes.io/tls<br />
metadata:<br />
name: example-tls<br />
namespace: test-namespace<br />
data:<br />
tls.crt: <base64 encoded cert><br />
tls.key: <base64 encoded key><br />
</syntaxhighlight><br />
<br />
==Rewriting Paths==<br />
{{External|https://github.com/kubernetes/ingress-nginx/blob/main/docs/examples/rewrite/README.md}}<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: networking.k8s.io/v1<br />
kind: Ingress<br />
metadata:<br />
name: test<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
nginx.ingress.kubernetes.io/rewrite-target: /$2<br />
spec:<br />
rules:<br />
- host: somehost.example.com<br />
http:<br />
paths:<br />
- path: /prod(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-prod<br />
port:<br />
number: 8080<br />
- path: /stage(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-stage<br />
port:<br />
number: 8080<br />
</syntaxhighlight><br />
<br />
==Different Ingresses with Different Path for the Same Host==<br />
<font color=darkkhaki><br />
If multiple Ingresses define different paths for the same host, the behavior depends on the ingress controller. NGINX merges the definitions.<br />
</font><br />
<br />
=OpenShift Ingress Operator=<br />
{{Internal|OpenShift Ingress Operator|Openshift Ingress Operator}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Ingress_Concepts&diff=108964Kubernetes Ingress Concepts2024-03-22T22:17:10Z<p>Ovidiu: /* TLS Support */</p>
<hr />
<div>=External=<br />
<br />
* https://medium.com/google-cloud/understanding-kubernetes-networking-ingress-1bc341c84078<br />
<br />
=Internal=<br />
<br />
* [[Kubernetes_Networking_Concepts#Ingress|Networking Concepts]]<br />
* [[Kubernetes_Service_Concepts#Ingress|Service Concepts]]<br />
* [[Amazon_EKS_Concepts#Load_Balancing_and_Ingress|Amazon EKS Concepts - Load Balancing and Ingress]]<br />
=Overview=<br />
<br />
An Ingress is a mechanism that operates at the application layer of the network stack (HTTP) and brings layer 7 features such as host and path-based routing and cookie-based session affinity to [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]]. Ingress cooperates with services to distribute load to pods. It exposes multiple services through a single IP address, and its implementation differs fundamentally from the implementation of [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]], [[Kubernetes_Service_Concepts#NodePort_Service|NodePort]] and [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] services. The reasons to use an Ingress include:<br />
* One Ingress can serve multiple services, behind a single public IP address, while each [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] requires its own native load balancer, each of them requiring their own public IP address.<br />
* Host, paths and cookies can be used to route the request.<br />
<br />
The Ingress mechanism consists of an [[Kubernetes_Ingress_Concepts#Ingress_Controller|Ingress controller]] and an [[Kubernetes_Ingress_Concepts#Ingress_API_Resource|Ingress resource]].<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ingress}}<br />
<br />
=How Ingress Works=<br />
<br />
A client attempting to access http://a.example.com performs a DNS query and resolves the name to the public IP address of the [[#Ingress_Controller|ingress controller]] external load balancer. The client then establishes a connection to the ingress external load balancer, specifying "http://a.example.com" in the "Host" header. The ingress controller load balancer forwards the request, via the ingress controller LoadBalancer/NodePort/ClientIP service, to the ingress controller pod. The ingress controller's state was updated when an ingress resource that defines the host "a.example.com" was deployed, and it knows that all requests for "a.example.com" must be forwarded to the "a" ClusterIP service. <br />
<br />
When a request with an "a.example.com" Host header arrives, the ingress controller locates the corresponding service, gets the associated Endpoints, picks up a pod and forwards the request to the pod. This is how virtual hosts are handled in web servers.<br />
<br />
Note that the ingress controller does not forward the request to the service, but it only uses it to get the associated Endpoints and select a pod. <br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Ingress.png]]<br />
<br><br />
<br />
=Ingress Controller=<br />
<br />
The ingress controller is the process - most likely running as a pod or pods inside the Kubernetes cluster itself - that accepts the HTTP connections, distributes traffic, terminates SSL connections, etc. Some Kubernetes distributions provide an ingress controller as an "add-on". For example, minikube has a <code>minikube addons enable ingress</code> command. If the ingress controller is not provided as add-on, it can be installed. There is a default "ingress-nginx" ingress controller that can be installed in any Kubernetes instance. More details: {{Internal|ingress-nginx|ingress-nginx}}<br />
<br />
The ingress controller deploys as part of its installation a layer 4 service of its own, most likely a [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]. As such, the ingress controller pod(s) is exposed to external requests via its own [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]/[[Kubernetes_Service_Concepts#NodePort_Service|NodePort]]/[[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]] service.<br />
<br />
To access a service through an its external name, the external service DNS name must resolve to the public IP address of the load balancer deployed by the ingress controller's LoadBalancer service.<br />
<br />
=Ingress API Resource=<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/ingress/}}<br />
<br />
An Ingress is the [[Kubernetes API Resources Concepts#Ingress|Kubernetes API resource]] that declares access to level 4 [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]] and allows configuring this access based on host and path. The ingress controller is notified when new ingress objects are created or when the existing ingress objects are modified and updates its internal state accordingly.<br />
<br />
Both "rules" and "paths" elements of an ingress manifest are arrays, so they can contain multiple items. An ingress can map multiple hosts and paths to multiple services. Multiple paths on the same host can be mapped to multiple services.<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: extensions/v1beta1<br />
kind: Ingress<br />
metadata:<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
name: example<br />
spec:<br />
rules:<br />
- host: a.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: a<br />
servicePort: 80<br />
<br />
- host: b.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: b<br />
servicePort: 80<br />
defaultBackend:<br />
service:<br />
name: c<br />
port:<br />
number: 81<br />
# This section is only required if TLS is to be enabled for the Ingress<br />
tls:<br />
- hosts:<br />
- a.local<br />
secretName: example-tls-a<br />
</syntaxhighlight><br />
<br />
Some cloud providers require that the ingress points to a NodePort service, but this is not a Kubernetes requirement.<br />
<br />
The ingress must be deployed in the same namespace as the services it serves.<br />
<br />
Upon deployment, the hosts served by an ingress, as well as its public address and port can be displayed with (in the example below, the ingress and the ingress controller have been deployed locally on Docker Desktop Kubernetes, hence the "localhost"):<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl -n <namespace> get ingress <ingress-name><br />
<br />
NAME HOSTS ADDRESS PORTS AGE<br />
example a.example.com,b.example.com localhost 80 83s<br />
</syntaxhighlight><br />
<br />
==TLS Support==<br />
<br />
<font color=darkkhaki>TODO Kubernetes in Action Section 5.4.4.</font><br />
<br />
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Secret<br />
type: kubernetes.io/tls<br />
metadata:<br />
name: example-tls<br />
namespace: test-namespace<br />
data:<br />
tls.crt: <base64 encoded cert><br />
tls.key: <base64 encoded key><br />
</syntaxhighlight><br />
<br />
==Rewriting Paths==<br />
{{External|https://github.com/kubernetes/ingress-nginx/blob/main/docs/examples/rewrite/README.md}}<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: networking.k8s.io/v1<br />
kind: Ingress<br />
metadata:<br />
name: test<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
nginx.ingress.kubernetes.io/rewrite-target: /$2<br />
spec:<br />
rules:<br />
- host: somehost.example.com<br />
http:<br />
paths:<br />
- path: /prod(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-prod<br />
port:<br />
number: 8080<br />
- path: /stage(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-stage<br />
port:<br />
number: 8080<br />
</syntaxhighlight><br />
<br />
==Different Ingresses with Different Path for the Same Host==<br />
<font color=darkkhaki><br />
If multiple Ingresses define different paths for the same host, the behavior depends on the ingress controller. NGINX merges the definitions.<br />
</font><br />
<br />
=OpenShift Ingress Operator=<br />
{{Internal|OpenShift Ingress Operator|Openshift Ingress Operator}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Ingress_Concepts&diff=108963Kubernetes Ingress Concepts2024-03-22T22:16:29Z<p>Ovidiu: /* Ingress API Resource */</p>
<hr />
<div>=External=<br />
<br />
* https://medium.com/google-cloud/understanding-kubernetes-networking-ingress-1bc341c84078<br />
<br />
=Internal=<br />
<br />
* [[Kubernetes_Networking_Concepts#Ingress|Networking Concepts]]<br />
* [[Kubernetes_Service_Concepts#Ingress|Service Concepts]]<br />
* [[Amazon_EKS_Concepts#Load_Balancing_and_Ingress|Amazon EKS Concepts - Load Balancing and Ingress]]<br />
=Overview=<br />
<br />
An Ingress is a mechanism that operates at the application layer of the network stack (HTTP) and brings layer 7 features such as host and path-based routing and cookie-based session affinity to [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]]. Ingress cooperates with services to distribute load to pods. It exposes multiple services through a single IP address, and its implementation differs fundamentally from the implementation of [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]], [[Kubernetes_Service_Concepts#NodePort_Service|NodePort]] and [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] services. The reasons to use an Ingress include:<br />
* One Ingress can serve multiple services, behind a single public IP address, while each [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] requires its own native load balancer, each of them requiring their own public IP address.<br />
* Host, paths and cookies can be used to route the request.<br />
<br />
The Ingress mechanism consists of an [[Kubernetes_Ingress_Concepts#Ingress_Controller|Ingress controller]] and an [[Kubernetes_Ingress_Concepts#Ingress_API_Resource|Ingress resource]].<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ingress}}<br />
<br />
=How Ingress Works=<br />
<br />
A client attempting to access http://a.example.com performs a DNS query and resolves the name to the public IP address of the [[#Ingress_Controller|ingress controller]] external load balancer. The client then establishes a connection to the ingress external load balancer, specifying "http://a.example.com" in the "Host" header. The ingress controller load balancer forwards the request, via the ingress controller LoadBalancer/NodePort/ClientIP service, to the ingress controller pod. The ingress controller's state was updated when an ingress resource that defines the host "a.example.com" was deployed, and it knows that all requests for "a.example.com" must be forwarded to the "a" ClusterIP service. <br />
<br />
When a request with an "a.example.com" Host header arrives, the ingress controller locates the corresponding service, gets the associated Endpoints, picks up a pod and forwards the request to the pod. This is how virtual hosts are handled in web servers.<br />
<br />
Note that the ingress controller does not forward the request to the service, but it only uses it to get the associated Endpoints and select a pod. <br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Ingress.png]]<br />
<br><br />
<br />
=Ingress Controller=<br />
<br />
The ingress controller is the process - most likely running as a pod or pods inside the Kubernetes cluster itself - that accepts the HTTP connections, distributes traffic, terminates SSL connections, etc. Some Kubernetes distributions provide an ingress controller as an "add-on". For example, minikube has a <code>minikube addons enable ingress</code> command. If the ingress controller is not provided as add-on, it can be installed. There is a default "ingress-nginx" ingress controller that can be installed in any Kubernetes instance. More details: {{Internal|ingress-nginx|ingress-nginx}}<br />
<br />
The ingress controller deploys as part of its installation a layer 4 service of its own, most likely a [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]. As such, the ingress controller pod(s) is exposed to external requests via its own [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]/[[Kubernetes_Service_Concepts#NodePort_Service|NodePort]]/[[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]] service.<br />
<br />
To access a service through an its external name, the external service DNS name must resolve to the public IP address of the load balancer deployed by the ingress controller's LoadBalancer service.<br />
<br />
=Ingress API Resource=<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/ingress/}}<br />
<br />
An Ingress is the [[Kubernetes API Resources Concepts#Ingress|Kubernetes API resource]] that declares access to level 4 [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]] and allows configuring this access based on host and path. The ingress controller is notified when new ingress objects are created or when the existing ingress objects are modified and updates its internal state accordingly.<br />
<br />
Both "rules" and "paths" elements of an ingress manifest are arrays, so they can contain multiple items. An ingress can map multiple hosts and paths to multiple services. Multiple paths on the same host can be mapped to multiple services.<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: extensions/v1beta1<br />
kind: Ingress<br />
metadata:<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
name: example<br />
spec:<br />
rules:<br />
- host: a.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: a<br />
servicePort: 80<br />
<br />
- host: b.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: b<br />
servicePort: 80<br />
defaultBackend:<br />
service:<br />
name: c<br />
port:<br />
number: 81<br />
# This section is only required if TLS is to be enabled for the Ingress<br />
tls:<br />
- hosts:<br />
- a.local<br />
secretName: example-tls-a<br />
</syntaxhighlight><br />
<br />
Some cloud providers require that the ingress points to a NodePort service, but this is not a Kubernetes requirement.<br />
<br />
The ingress must be deployed in the same namespace as the services it serves.<br />
<br />
Upon deployment, the hosts served by an ingress, as well as its public address and port can be displayed with (in the example below, the ingress and the ingress controller have been deployed locally on Docker Desktop Kubernetes, hence the "localhost"):<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl -n <namespace> get ingress <ingress-name><br />
<br />
NAME HOSTS ADDRESS PORTS AGE<br />
example a.example.com,b.example.com localhost 80 83s<br />
</syntaxhighlight><br />
<br />
==TLS Support==<br />
<br />
<font color=darkgray>TODO Kubernetes in Action Section 5.4.4.</font><br />
<br />
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Secret<br />
type: kubernetes.io/tls<br />
metadata:<br />
name: example-tls<br />
namespace: test-namespace<br />
data:<br />
tls.crt: <base64 encoded cert><br />
tls.key: <base64 encoded key><br />
</syntaxhighlight><br />
==Rewriting Paths==<br />
{{External|https://github.com/kubernetes/ingress-nginx/blob/main/docs/examples/rewrite/README.md}}<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: networking.k8s.io/v1<br />
kind: Ingress<br />
metadata:<br />
name: test<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
nginx.ingress.kubernetes.io/rewrite-target: /$2<br />
spec:<br />
rules:<br />
- host: somehost.example.com<br />
http:<br />
paths:<br />
- path: /prod(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-prod<br />
port:<br />
number: 8080<br />
- path: /stage(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-stage<br />
port:<br />
number: 8080<br />
</syntaxhighlight><br />
<br />
==Different Ingresses with Different Path for the Same Host==<br />
<font color=darkkhaki><br />
If multiple Ingresses define different paths for the same host, the behavior depends on the ingress controller. NGINX merges the definitions.<br />
</font><br />
<br />
=OpenShift Ingress Operator=<br />
{{Internal|OpenShift Ingress Operator|Openshift Ingress Operator}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Ingress_Concepts&diff=108962Kubernetes Ingress Concepts2024-03-22T22:14:45Z<p>Ovidiu: /* Ingress Controller */</p>
<hr />
<div>=External=<br />
<br />
* https://medium.com/google-cloud/understanding-kubernetes-networking-ingress-1bc341c84078<br />
<br />
=Internal=<br />
<br />
* [[Kubernetes_Networking_Concepts#Ingress|Networking Concepts]]<br />
* [[Kubernetes_Service_Concepts#Ingress|Service Concepts]]<br />
* [[Amazon_EKS_Concepts#Load_Balancing_and_Ingress|Amazon EKS Concepts - Load Balancing and Ingress]]<br />
=Overview=<br />
<br />
An Ingress is a mechanism that operates at the application layer of the network stack (HTTP) and brings layer 7 features such as host and path-based routing and cookie-based session affinity to [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]]. Ingress cooperates with services to distribute load to pods. It exposes multiple services through a single IP address, and its implementation differs fundamentally from the implementation of [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]], [[Kubernetes_Service_Concepts#NodePort_Service|NodePort]] and [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] services. The reasons to use an Ingress include:<br />
* One Ingress can serve multiple services, behind a single public IP address, while each [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]] requires its own native load balancer, each of them requiring their own public IP address.<br />
* Host, paths and cookies can be used to route the request.<br />
<br />
The Ingress mechanism consists of an [[Kubernetes_Ingress_Concepts#Ingress_Controller|Ingress controller]] and an [[Kubernetes_Ingress_Concepts#Ingress_API_Resource|Ingress resource]].<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ingress}}<br />
<br />
=How Ingress Works=<br />
<br />
A client attempting to access http://a.example.com performs a DNS query and resolves the name to the public IP address of the [[#Ingress_Controller|ingress controller]] external load balancer. The client then establishes a connection to the ingress external load balancer, specifying "http://a.example.com" in the "Host" header. The ingress controller load balancer forwards the request, via the ingress controller LoadBalancer/NodePort/ClientIP service, to the ingress controller pod. The ingress controller's state was updated when an ingress resource that defines the host "a.example.com" was deployed, and it knows that all requests for "a.example.com" must be forwarded to the "a" ClusterIP service. <br />
<br />
When a request with an "a.example.com" Host header arrives, the ingress controller locates the corresponding service, gets the associated Endpoints, picks up a pod and forwards the request to the pod. This is how virtual hosts are handled in web servers.<br />
<br />
Note that the ingress controller does not forward the request to the service, but it only uses it to get the associated Endpoints and select a pod. <br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Ingress.png]]<br />
<br><br />
<br />
=Ingress Controller=<br />
<br />
The ingress controller is the process - most likely running as a pod or pods inside the Kubernetes cluster itself - that accepts the HTTP connections, distributes traffic, terminates SSL connections, etc. Some Kubernetes distributions provide an ingress controller as an "add-on". For example, minikube has a <code>minikube addons enable ingress</code> command. If the ingress controller is not provided as add-on, it can be installed. There is a default "ingress-nginx" ingress controller that can be installed in any Kubernetes instance. More details: {{Internal|ingress-nginx|ingress-nginx}}<br />
<br />
The ingress controller deploys as part of its installation a layer 4 service of its own, most likely a [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]. As such, the ingress controller pod(s) is exposed to external requests via its own [[Kubernetes_Service_Concepts#LoadBalancer_Service|LoadBalancer]]/[[Kubernetes_Service_Concepts#NodePort_Service|NodePort]]/[[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|ClusterIP]] service.<br />
<br />
To access a service through an its external name, the external service DNS name must resolve to the public IP address of the load balancer deployed by the ingress controller's LoadBalancer service.<br />
<br />
=Ingress API Resource=<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/ingress/}}<br />
<br />
An Ingress is the [[Kubernetes API Resources Concepts#Ingress|Kubernetes API resource]] that declares access to level 4 [[Kubernetes_Service_Concepts#Service_.28ClusterIP_Service.29|services]] and allows configuring this access based on host and path. The ingress controller is notified when new ingress objects are created or when the existing ingress objects are modified and updates its internal state accordingly.<br />
<br />
Both "rules" and "paths" elements of an ingress manifest are arrays, so they can contain multiple items. An ingress can map multiple hosts and paths to multiple services. Multiple paths on the same host can be mapped to multiple services.<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: extensions/v1beta1<br />
kind: Ingress<br />
metadata:<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
name: example<br />
spec:<br />
rules:<br />
- host: a.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: a<br />
servicePort: 80<br />
<br />
- host: b.example.com<br />
http:<br />
paths:<br />
- path: /<br />
backend:<br />
serviceName: b<br />
servicePort: 80<br />
# This section is only required if TLS is to be enabled for the Ingress<br />
tls:<br />
- hosts:<br />
- a.local<br />
secretName: example-tls-a<br />
</syntaxhighlight><br />
<br />
Some cloud providers require that the ingress points to a NodePort service, but this is not a Kubernetes requirement.<br />
<br />
The ingress must be deployed in the same namespace as the services it serves.<br />
<br />
Upon deployment, the hosts served by an ingress, as well as its public address and port can be displayed with (in the example below, the ingress and the ingress controller have been deployed locally on Docker Desktop Kubernetes, hence the "localhost"):<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl -n <namespace> get ingress <ingress-name><br />
<br />
NAME HOSTS ADDRESS PORTS AGE<br />
example a.example.com,b.example.com localhost 80 83s<br />
</syntaxhighlight><br />
<br />
==TLS Support==<br />
<br />
<font color=darkgray>TODO Kubernetes in Action Section 5.4.4.</font><br />
<br />
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Secret<br />
type: kubernetes.io/tls<br />
metadata:<br />
name: example-tls<br />
namespace: test-namespace<br />
data:<br />
tls.crt: <base64 encoded cert><br />
tls.key: <base64 encoded key><br />
</syntaxhighlight><br />
==Rewriting Paths==<br />
{{External|https://github.com/kubernetes/ingress-nginx/blob/main/docs/examples/rewrite/README.md}}<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: networking.k8s.io/v1<br />
kind: Ingress<br />
metadata:<br />
name: test<br />
annotations:<br />
kubernetes.io/ingress.class: nginx<br />
nginx.ingress.kubernetes.io/rewrite-target: /$2<br />
spec:<br />
rules:<br />
- host: somehost.example.com<br />
http:<br />
paths:<br />
- path: /prod(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-prod<br />
port:<br />
number: 8080<br />
- path: /stage(/|$)(.*)<br />
pathType: Prefix<br />
backend:<br />
service:<br />
name: service-stage<br />
port:<br />
number: 8080<br />
</syntaxhighlight><br />
<br />
==Different Ingresses with Different Path for the Same Host==<br />
<font color=darkkhaki><br />
If multiple Ingresses define different paths for the same host, the behavior depends on the ingress controller. NGINX merges the definitions.<br />
</font><br />
<br />
=OpenShift Ingress Operator=<br />
{{Internal|OpenShift Ingress Operator|Openshift Ingress Operator}}</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108961Kubernetes Service Concepts2024-03-22T22:10:36Z<p>Ovidiu: /* Ingress */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is set to <code>Local</code>, because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkkhaki><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
====kind====<br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is <code>clusterIP: None</code>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108960Kubernetes Service Concepts2024-03-22T22:09:59Z<p>Ovidiu: /* Headless Service */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is set to <code>Local</code>, because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkkhaki><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
====kind====<br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is <code>clusterIP: None</code>:<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108959Kubernetes Service Concepts2024-03-22T22:09:04Z<p>Ovidiu: /* Ingress */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is set to <code>Local</code>, because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkkhaki><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
====kind====<br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108958Kubernetes Service Concepts2024-03-22T22:08:30Z<p>Ovidiu: /* minikube */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is set to <code>Local</code>, because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkkhaki><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
====kind====<br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108957Kubernetes Service Concepts2024-03-22T22:06:30Z<p>Ovidiu: /* Client IP Preservation */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is set to <code>Local</code>, because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108956Kubernetes Service Concepts2024-03-22T22:05:37Z<p>Ovidiu: /* Node/Pod Relationship */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> to <code>Local</code> in the service specification section. If <code>[[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]]</code> is configured to <code>Local</code> and a connection is established with the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108955Kubernetes Service Concepts2024-03-22T22:04:05Z<p>Ovidiu: /* Node/Pod Relationship */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may not be forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108954Kubernetes Service Concepts2024-03-22T22:02:43Z<p>Ovidiu: /* NodePort Service */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. It is not mandatory to provide a value for <code>nodePort</code>. If not specified, a random port value will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108953Kubernetes Service Concepts2024-03-22T22:01:36Z<p>Ovidiu: /* NodePort Service */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the <code>[[Kubernetes_Service_Manifest#type|type]]</code> (<code>NodePort</code>) and the optional <code>[[Kubernetes_Service_Manifest#nodePort|nodePort]]</code> configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108952Kubernetes Service Concepts2024-03-22T22:00:17Z<p>Ovidiu: /* NodePort Service */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself, hence the name, and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108951Kubernetes Service Concepts2024-03-22T21:58:05Z<p>Ovidiu: /* Readiness Probe */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]. The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108950Kubernetes Service Concepts2024-03-22T21:57:40Z<p>Ovidiu: /* Readiness Probe */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint, which consists of the IP address and port pair, is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe. This behavior can be changed via configuration, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108949Kubernetes Service Concepts2024-03-22T21:55:04Z<p>Ovidiu: /* Endpoints */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with <code>kubectl get endpoints <name></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108948Kubernetes Service Concepts2024-03-22T21:53:42Z<p>Ovidiu: /* Endpoints */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108947Kubernetes Service Concepts2024-03-22T21:52:33Z<p>Ovidiu: /* Associating a Service with Pods */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels declared by the pod, but for a pod to match a service, in must declare all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108946Kubernetes Service Concepts2024-03-22T21:50:25Z<p>Ovidiu: /* Session Affinity */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To enable cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108945Kubernetes Service Concepts2024-03-22T21:49:49Z<p>Ovidiu: /* Session Affinity */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a <code>[[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]]</code> configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108944Kubernetes Service Concepts2024-03-22T21:48:47Z<p>Ovidiu: /* DNS */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike in the case of [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108943Kubernetes Service Concepts2024-03-22T21:48:19Z<p>Ovidiu: /* DNS */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and <code>[[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]]</code> is a configurable suffix used in all local cluster service names. <code>svc.cluster.local</code> can be omitted, because all pods are configured to list <code>svc.cluster.local</code> in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|<code>/etc/resolv.conf</code> search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108942Kubernetes Service Concepts2024-03-22T21:47:00Z<p>Ovidiu: /* DNS */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <code><namespace></code> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108941Kubernetes Service Concepts2024-03-22T21:46:05Z<p>Ovidiu: /* Environment Variables */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <code><SERVICE-NAME>_SERVICE_HOST</code> and the port will be exposed as <code><SERVICE-NAME>_SERVICE_PORT</code>, where all letters in the service name are uppercased and the dashes in the name are converted to underscores ("some-service" → "SOME_SERVICE"). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<code><SERVICE-NAME>_SERVICE_PORT_<PORT-NAME></code>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108940Kubernetes Service Concepts2024-03-22T21:44:08Z<p>Ovidiu: /* Service Target Port */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for <code>[[Kubernetes_Service_Manifest#targetPort|targetPort]]</code> in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108939Kubernetes Service Concepts2024-03-22T21:43:03Z<p>Ovidiu: /* Service Target Port */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the <code>[[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]]</code> array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> in the service manifest.<br />
<br />
In most cases, an integral port value is used as <code>[[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]]</code> value, but it is also it is possible to name ports in the pod's manifest using <code>[[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]]</code> elements, and use those names as value for [[Kubernetes_Service_Manifest#targetPort|targetPort]] in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108938Kubernetes Service Concepts2024-03-22T21:41:44Z<p>Ovidiu: /* Port Name */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with <code>[[Kubernetes_Service_Manifest#name|.spec.ports[*].name]]</code>, as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the [[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]] array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] in the service manifest.<br />
<br />
In most cases, an integral port value is used as [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] value, but it is also it is possible to name ports in the pod's manifest using [[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]] elements, and use those names as value for [[Kubernetes_Service_Manifest#targetPort|targetPort]] in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108937Kubernetes Service Concepts2024-03-22T21:40:47Z<p>Ovidiu: /* Service Port(s) */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with <code>[[Kubernetes_Service_Manifest#port|.spec.ports[*].port]]</code> in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with [[Kubernetes_Service_Manifest#name|.spec.ports[*].name]], as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the [[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]] array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] in the service manifest.<br />
<br />
In most cases, an integral port value is used as [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] value, but it is also it is possible to name ports in the pod's manifest using [[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]] elements, and use those names as value for [[Kubernetes_Service_Manifest#targetPort|targetPort]] in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108936Kubernetes Service Concepts2024-03-22T21:38:07Z<p>Ovidiu: /* Service (ClusterIP Service) */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying <code>[[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP</code> in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with [[Kubernetes_Service_Manifest#port|.spec.ports[*].port]] in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with [[Kubernetes_Service_Manifest#name|.spec.ports[*].name]], as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the [[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]] array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] in the service manifest.<br />
<br />
In most cases, an integral port value is used as [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] value, but it is also it is possible to name ports in the pod's manifest using [[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]] elements, and use those names as value for [[Kubernetes_Service_Manifest#targetPort|targetPort]] in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiuhttps://kb.novaordis.com/index.php?title=Kubernetes_Service_Concepts&diff=108935Kubernetes Service Concepts2024-03-22T21:36:50Z<p>Ovidiu: /* Service (ClusterIP Service) */</p>
<hr />
<div>=External=<br />
<br />
* https://kubernetes.io/docs/concepts/services-networking/service/<br />
* https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/manage_network/kubernetes_types.html<br />
<br />
=Internal=<br />
* [[Kubernetes Concepts#Service|Kubernetes Concepts]]<br />
* [[Kubernetes_Pod_and_Container_Concepts|Pod and Container Concepts]]<br />
* [[Kubernetes Networking Concepts]]<br />
* [[Kubernetes Ingress Concepts|Ingress]]<br />
<br />
=Playground=<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/httpd-pod-and-service}}<br />
<br />
=Overview=<br />
<br />
<br />
A service is a Kubernetes resource that provides stable network access to a set of ephemeral [[Kubernetes_Pod_and_Container_Concepts|pods]]. The need for such a resource is explained in the [[#Need_for_Services|Need for Services]] section. The simplest kind of [[#Service|service]] provides a stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]]. More complex services, such as [[#NodePort_Service|NodePort]] and [[#LoadBalancer_Service|LoadBalancer]] services, use ClusterIP services and expand their functionality. The services and the pods they are exposed [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|form a dynamic association labels and selectors]]. Once the logical association is declared, the Kubernetes cluster monitors the state of the pods and includes or excludes pods from the pod set exposed by the service. Other Kubernetes API resources, such as [[#Endpoints|Endpoints]], are involved in this process.<br />
<br />
=Need for Services=<br />
<br />
Most [[Kubernetes_Pod_and_Container_Concepts|pods]] exist to serve requests sent over the network, so the pods' clients need to know how to open a network connection into the pod: they need an IP address and a port. This is true for clients external to the Kubernetes cluster, and also for pods running inside the cluster, which act as clients for other pods.<br />
<br />
However, as described in the [[Kubernetes_Pod_and_Container_Concepts#Lifecycle|Pod Lifecycle]] section, the pods are '''ephemeral''', they come and go. If a pod fails, it is not resurrected, but its failure is detected as result of lack of response from the liveness probe embedded within it and another pod is usually scheduled as replacement. The pod does not need to fail to be removed, the removal can be the result of a normal scale-down operation. In consequence, the IP address of an individual pod cannot be relied on. <br />
<br />
First off, the IP address is '''dynamically allocated''' to the pod at the time of pod initialization, after the pod has been scheduled to a node and before the pod is started, and secondly, the IP address becomes obsolete the moment the pod disappears. As such, it is practically very difficult, if not impossible for application's clients running outside the Kubernetes cluster, or even for other pods acting as clients to keep track of ephemeral pod IP addresses. <br />
<br />
Multiple pods may be '''providing the same service''', and each of them has its own IP address. The client must not be put in the position to have to chose a specific IP address and pod, they should not even need to know there are multiple pods providing the service.<br />
<br />
The solution to all these problems consists of maintaining an instance of a specialized resource, the [[#Service|service]], for each of such group of pods. The service instance exposes a stable IP address and set of ports for the entire life of the service instance.<br />
<br />
=Service Manifest=<br />
{{Internal|Kubernetes Service Manifest|Service Manifest}}<br />
=<span id='Service'></span>Service (ClusterIP Service)=<br />
In its simplest form, a service is a [[Kubernetes API Resources Concepts#Service|Kubernetes API resource]] providing a single, stable [[Kubernetes_Networking_Concepts#Cluster_IP_Address|Cluster IP address]] and a set of ports that serve as point of entry to a group of pods providing the same service. A service that does not specify a type is by default a "ClusterIP" service. The service provides connection forwarding and load balancing: each connection to the service is forwarded to a '''randomly selected''' backing pod. The service also provides discovery facilities, advertising itself via DNS and environment variables, as described in the [[#Discovering_ClusterIP_Services_within_the_Cluster|Discovering ClusterIP Services within the Cluster]] section.<br />
<br />
The IP address stays unchanged for the entire lifetime of the service. A client opens a network connection to the IP address and port, and the service load-balances the connection to one of the pods associated with it. Individual pods exposed by the service may come and go, and the service makes this transparent to the clients, as explained in the [[Kubernetes_Service_Concepts#Associating_a_Service_with_Pods|Associating a Service with Pods]] section.<br />
<br />
Internal [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] clients, such as other pods, can use the service's IP address and ports to connect to the pods exposed by the service. This type of service is know as a <span id='ClusterIP'></span><span id='ClusterIP_Service'></span>'''ClusterIP service'''. Only internal clients can use a ClusterIP service, because the Cluster IP address is only routable from inside the Kubernetes cluster. Designating a service as a ClusterIP service by specifying [[Kubernetes_Service_Manifest#type|spec.type]]=ClusterIP in the service's manifest is optional.<br />
<br />
The pods may be scheduled on different physical [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes nodes]], but they are accessible to the service and each other via the virtual [[Kubernetes_Networking_Concepts#Pod_Network|pod network]].<br />
<br />
<br><br />
:[[File:Learning_Kubernetes_KubernetesServices_ClusterIPService.png]]<br />
<br><br />
<br />
The Cluster IP and the service's external port is reported by [[Kubernetes_Service_Operations#List_Services|kubectl get svc]]:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
httpd-svc ClusterIP 10.106.185.218 <none> 9898/TCP 43m<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h26m<br />
</syntaxhighlight><br />
<br />
Services operate at the TCP/UDP layer (level 4) and in consequence cannot provide application-layer routing. If such routing is needed, and [[#Ingress|Ingress]] must be used.<br />
<br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/ClusterIP}}<br />
<br />
==<span id='Service_Port'></span>Service Port(s)==<br />
<br />
A service port designates a port exposed by the service externally. This is a port the service will be available on, and the value will be used by the external clients connecting to the service. It is declared with [[Kubernetes_Service_Manifest#port|.spec.ports[*].port]] in the service manifest.<br />
<br />
A service may expose more than one port, and the requests arrived on any of these ports will be forwarded to the corresponding configured [[#Service_Target_Port|target ports]].<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 8080<br />
- name: https<br />
port: 443 <br />
targetPort: 8443<br />
...<br />
</syntaxhighlight><br />
<br />
Note that the label selector applies to the service as a whole. It cannot be configured on a per-port basis. If different ports must map to a different subset of pods, different services must be created.<br />
<br />
===Port Name===<br />
<br />
When more than one port is specified for a service, a name must be specified for each port with [[Kubernetes_Service_Manifest#name|.spec.ports[*].name]], as shown in the above example. If there is just one port per service, the name element is optional.<br />
<br />
==Service Target Port==<br />
A target port represents the port exposed by a container from the pods associated with the service, as declared by the [[Kubernetes_Pod_Manifest#ports|.spec.containers[*].ports]] array elements of the pod manifest. The service will forward requests over connections opened with this port as target. It is declared with [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] in the service manifest.<br />
<br />
In most cases, an integral port value is used as [[Kubernetes_Service_Manifest#targetPort|.spec.ports[*].targetPort]] value, but it is also it is possible to name ports in the pod's manifest using [[Kubernetes_Pod_Manifest#port_name |.spec.containers[*].ports.name]] elements, and use those names as value for [[Kubernetes_Service_Manifest#targetPort|targetPort]] in the service definition:<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
...<br />
spec:<br />
ports:<br />
- name: http<br />
port: 80 <br />
targetPort: 'http'<br />
</syntaxhighlight><br />
The benefit of using names instead of port numbers is that the port numbers can be changed by the pods without changing the service spec.<br />
<br />
==<span id='Discovering_ClusterIP_Services_within_the_Cluster'></span><span id='Discovering_ClusterIP_Services_inside_the_Cluster'></span>Service Discovery - Discovering ClusterIP Services within the Cluster==<br />
A Kubernetes cluster automatically advertises its ClusterIP services throughout the cluster by publishing their name/IP address association as they come online. The potential clients can use two mechanism to learn the cluster IP address and ports associated with a specific service: [[#Environment_Variables|environment variables]] and [[#DNS|DNS]].<br />
===Environment Variables===<br />
<br />
Any pod gets its environment automatically initialized by the cluster with special environment variables pointing to the IP address and ports for all services '''from the same namespace''' that exist at the moment of initialization. <br />
<br />
⚠️ If a service comes on-line after a potential client pod has been deployed, the potential client pod's environment will not be updated.<br />
<br />
The IP address of a service will be exposed as <SERVICE-NAME>_SERVICE_HOST and the port will be exposed as <SERVICE-NAME>_SERVICE_PORT, where all letters in the service name are uppercased and the dashes in the name are converted to underscores (some-service → SOME_SERVICE). If more than one port is declared for the service, the mandatory "name" element is uppercased and used to assemble the environment variable carrying the port value, following the pattern: <br />
<SERVICE-NAME>_SERVICE_PORT_<PORT-NAME>:<br />
<br />
<syntaxhighlight lang='text'><br />
EXAMPLE_SERVICE_HOST=10.103.254.75<br />
EXAMPLE_SERVICE_PORT_A=80<br />
EXAMPLE_SERVICE_PORT_B=81<br />
</syntaxhighlight><br />
<br />
===DNS===<br />
During a service deployment, the Kubernetes API server updates the internal database of the cluster's [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]] so the name of the newly deployed service is mapped to its ClusterIP address. Each service gets a DNS entry in the internal DNS server. Any DNS query performed by a process running in a pod will be handled by the Kubernetes [[Kubernetes_DNS_Concepts#The_DNS_Server|DNS server]], which knows all services running in the cluster. This way, a client pod that knows the name of the service can access it through its fully qualified domain name ([[DNS_Concepts#FQDN|FQDN]]).<br />
<br />
The FQDN of a service matches the following pattern:<br />
<syntaxhighlight lang='text'><br />
<service-name>.<namespace>.svc.cluster.local<br />
</syntaxhighlight><br />
where <namespace> is the namespace the service was defined in and [[Kubernetes_DNS_Concepts#svc.cluster.local|svc.cluster.local]] is a configurable suffix used in all local cluster service names. "svc.cluster.local" can be ommitted, because all pods are configured to list "svc.cluster.local" in their [[Kubernetes_DNS_Concepts#Name_Resolution_inside_a_Pod|/etc/resolv.conf search clause]]. If the client pod and the service are in the same namespace, the namespace name from the FQDN can also be omitted.<br />
<br />
⚠️ Unlike for [[#Environment_Variables|environment variables]], the service must still know the service port number, as that information is not disseminated with DNS. If the client pod and the service are deployed in the same namespace, the client pod can get the port number from the corresponding [[#Environment_Variables|environment variable]].<br />
<br />
More details about DNS support in a Kubernetes cluster are available in: {{Internal|Kubernetes DNS Concepts|Kubernetes DNS Concepts}}<br />
<br />
==Session Affinity==<br />
With no additional configuration, successive requests to a service, even if they are made by the same client, will be forwarded to different pods, randomly. By default, there is no session affinity, and this corresponds to a [[Kubernetes_Service_Manifest#sessionAffinity|.spec.sessionAffinity]] configuration of "None". <br />
<br />
However, even if session affinity is set to "None" but the client uses keep-alive connections, successive requests over the same connection will be sent to the same pod. This is because the services work at connection level, so when a connection to a service is first opened, it extends all the way to the target pod, and the same pod will be used until the connection is closed. <br />
<br />
The only kind of session affinity that can be configured is "ClientIP": all requests made by a certain client, identified by its IP, are proxied to the same pod every time.<br />
<br />
Services do not provide cookie-based session affinity because they operate at transport level (TCP/UDP) and do not inspect the payload carried by the packets. To have cookie-based session affinity, the proxy must be capable of understanding HTTP, which is not the case for services.<br />
<br />
==Kubernetes API Server Service==<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7h47m<br />
</syntaxhighlight><br />
==ClusterIP Service Implementation Details==<br />
===Pinging ClusterIP Services===<br />
The cluster IP address associated with a service cannot be ping-ed. This is because the service's cluster IP is a virtual IP, and it only has meaning when combined with the service port.<br />
<br />
=Associating a Service with Pods=<br />
The pods are loosely associated with the service exposing them: a ClusterIP service identifies the pods to forward request to by declaring a [[Kubernetes_Service_Manifest#selector|selector]]:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Service<br />
spec:<br />
...<br />
selector:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pods that intend to serve requests proxied by the service must declare '''all''' key/value pairs listed by the service selector as metadata labels to be considered:<br />
<br />
<syntaxhighlight lang='yaml'><br />
kind: Pod<br />
metadata:<br />
...<br />
labels:<br />
key1: value1<br />
key2: value2<br />
...<br />
</syntaxhighlight><br />
<br />
The pod may declare extra labels in additions with those specified by the service selector, and those will not interfere with the service selection process. The label selector needs to match some of the labels on a pod, but for a pod to match a service, in must have all of the labels the service is looking for. If the number of pods increases as result of scaling up the deployment, no service modification is required - the service dynamically identifies the new pods and starts to load-balance requests to them, as soon as they are [[#Readiness_Probe|ready to serve requests]].<br />
<br />
The list of active pods associated with the service is maintained by [[#Endpoints|Endpoints]], which is another [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]]. All pods capable of handling requests for the service are represented by entries in the [[#Endpoints|Endpoints]] instance associated with the service.<br />
<br />
==Endpoints==<br />
Although the pod selector is defined in the service spec, it is not used directly when redirecting incoming connections. Instead, the selector is used to build a list of container endpoints (IP:port pairs), which is then stored in the associated [[Kubernetes API Resources Concepts#Endpoints|Kubernetes API resource]] Endpoints resource. The Endpoints instance has the same name as the service it is associated with. <br />
<br />
When a client connects to the ClusterIP service, the service randomly selects one of the endpoints and redirects the incoming connection to the containerized server listening at that location. The list is dynamic, as new pods may be deployed, or suddenly become unavailable. The Endpoints instances are updated by the [[#Endpoints_Controller|endpoints controller]]. Various factors are taken into consideration: label match, pod availability, the state of each of the pod's container [[#Readiness_Probe|readiness probe]], etc. There is a one-to-one association between a service and its Endpoints instance, but the Endpoints is a separate resource and not an attribute of the service.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_Endpoints.png]]<br />
<br><br />
<br />
An endpoint stays in the list as long as its corresponding readiness probe succeeds. If a readiness probe fails, the corresponding endpoint, and the endpoints for all containers from the same pod are removed. The relationship between Endpoints, containers, pods and readiness probes is explained in: {{Internal|Kubernetes_Container_Probes#Readiness_Probe_Operations|Readiness Probe Operations}}<br />
<br />
The endpoint is also removed if the pod is deleted.<br />
<br />
An Endpoints instance can be inspected with 'kubectl get endpoints <name>':<br />
<br />
<syntaxhighlight lang='text'><br />
kubectl get endpoints example<br />
<br />
NAME ENDPOINTS AGE<br />
example 10.1.0.67:80,10.1.0.69:80 7m53s<br />
</syntaxhighlight><br />
<syntaxhighlight lang='yaml'><br />
kubectl -o yaml get endpoints example<br />
<br />
apiVersion: v1<br />
kind: Endpoints<br />
metadata:<br />
name: example<br />
subsets:<br />
- addresses:<br />
- ip: 10.1.0.67<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd<br />
- ip: 10.1.0.69<br />
nodeName: docker-desktop<br />
targetRef:<br />
kind: Pod<br />
name: httpd-3<br />
ports:<br />
- port: 80<br />
protocol: TCP<br />
</syntaxhighlight><br />
<br />
<br />
If the service is created without a pod selector, the associated Endpoints won't even be created. However, it can be created, configured and updated manually. To create a service with manually managed endpoints, you need to create both the service and the Endpoints resource. They are connected via name: the name of the service and of the Endpoints instance must be the same.<br />
===Endpoints Controller===<br />
The endpoints controller is part of the [[Kubernetes Control Plane and Data Plane Concepts#Controller_Manager|controller manager]].<br />
<br />
==Readiness Probe==<br />
By default, a container endpoint (IP address and port pair) is added to the service's [[#Endpoints|Endpoints]] list only if its parent container is ready, as reflected by the result of all its containers' readiness probe (there is configuration that can be deployed to change this behavior, as described in [[Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check]]). The probes are executed periodically and the container endpoint is taken out of the list if any of the probe fails - and possibly re-added if the probe starts succeeding again. Usually, there's just one container per pod, so "ready pod" and "ready container" are equivalent. More details about readiness probes:<br />
{{Internal|Kubernetes_Container_Probes#Container_and_Pod_Readiness_Check|Container and Pod Readiness Check}}<br />
<br />
=<span id='Connecting_to_External_Services_from_inside_the_Cluster'></span><span id='ExternalName'></span><span id='ExternalName_Service'></span>Connecting to External Services from within the Cluster (ExternalName Service)=<br />
An ExternalName service is a service that proxies to an external service instead of a set of pods.<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: 'external-service'<br />
spec:<br />
type: ExternalName<br />
externalName: someservice.someserver.com<br />
ports:<br />
- port: 80 <br />
</syntaxhighlight><br />
ExternalName services are implemented in DNS - a [[DNS_Concepts#CNAME_.28Alias.29|CNAME]] DNS record is created for the service. Therefore, clients connecting to the service will connect to the external service directly, bypassing the service proxy completely. For this reason, these type of service do not even get a cluster IP.<br />
<br />
The benefit of using an ExternalService is that it allows to modify the service definition and point to a different service any time later by only changing the <code>externalName</code>, or by changing the type back to ClusterIP and creating an Endpoints object for the service - either manually or via a selector.<br />
<br />
=Exposing Services outside the Cluster=<br />
==<span id='NodePort'></span>NodePort Service==<br />
<br />
A NodePort service is a type of service that builds upon [[#Service|ClusterIP service]] functionality and provides an additional feature: it allows clients external to the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] to connect to the service's pods. The feature is implemented by having each [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Node|Kubernetes node]] open a port on the node itself (hence the name) and forwarding incoming node port connections to the underlying ClusterIP service. All features of the underlying ClusterIP service are also available on the NodePort service: internal clients can use the NodePort service as a ClusterIP service.<br />
<br />
The port reserved by a NodePort service has the same value on all nodes.<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_NodePort.png]]<br />
<br><br />
<br />
A NodePort service can be created posting a manifest similar to:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-nodeport<br />
spec:<br />
type: NodePort <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
nodePort: 30080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
What makes the manifest different form a ClusterIP service manifest are the [[Kubernetes_Service_Manifest#type|type]] (NodePort) and the optional [[Kubernetes_Service_Manifest#nodePort|nodePort]] configuration elements. Everything else is the same. <br />
<br />
The valid range for node ports is 30000-32767. Setting the nodePort is not mandatory, if not specified, an random port will be chosen.<br />
<br />
More details about the service manifest are available in: {{Internal|Kubernetes_Service_Manifest#Example|Service Manifest}}<br />
<br />
Inspection of a NodePort service reveals that it also exposes a cluster IP address, as a ClusterIP service. The PORT(S) column reports both the internal port of the cluster IP (80) and the node port (30080). The service is available inside the cluster at 10.107.165.237:80 and externally, on any node, at <node-ip>:30080.<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example NodePort 10.107.165.237 <none> 80:30080/TCP 6s<br />
</syntaxhighlight><br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/NodePort}}<br />
===Node/Pod Relationship===<br />
Without additional configuration, a connection received on a particular node may or may be not forwarded to a pod deployed on that node, because the connection is forwarded to the ClusterIP service first, and the ClusterIP service chooses a pod randomly.<br />
<br />
An extra network hop may not be desirable, and it can be prevented by configuring the service to redirect external traffic only to pods running on the node that receives the connection. This is done by setting [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] to "Local" in the service's specification section. If [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is configured to "Local" and external connection is opened through the service's node port, the service proxy will choose a locally running pod. ⚠️ If no local pod exists, the connection will hang, so only nodes that have qualified pods deploying on them must be chosen. Using this annotation has the additional drawback that will prevent the service to evenly spread load across pod, and some pods may be get more load than others.<br />
===Client IP Preservation===<br />
When an internal cluster connection is handled by a [[#Service|ClusterIP service]], the pods backing the service can obtain the client IP address. However, when the connection is proxied by a Node Port service, the packets' source IP is changes by Source Network Address Translation (SNAT). The backing pod will not be able to see the client IP address anymore. This is not the case when [[Kubernetes_Service_Manifest#externalTrafficPolicy|externalTrafficPolicy]] is set to "Local", because there is no additional hop between the node receiving the connection and the node hosting the target pod - SNAT is not performed.<br />
<br />
==<span id='LoadBalancer'></span>LoadBalancer Service==<br />
{{External|https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer}}<br />
A LoadBalancer service is a type of service that builds upon [[#NodePort_Service|NodePort service]] functionality and provides an additional feature: it interacts with the cloud infrastructure the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|Kubernetes cluster]] is deployed on and triggers the provisioning of an external load balancer. The load balancer proxies traffic to the Kubernetes nodes, using the node port opened by the underlying NodeService. External clients connect to the service through the load balancer's IP.<br />
<br />
Note that each LoadBalancer service requires its own external load balancer. If this is a limitation, consider using an [[Kubernetes_Ingress_Concepts#Need_for_Ingress|ingress]].<br />
<br />
<br><br />
::[[File:Learning_Kubernetes_KubernetesServices_LoadBalancer.png]]<br />
<br><br />
<br />
A LoadBalancer service can be created posting a manifest similar to the one below:<br />
<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-loadbalancer<br />
spec:<br />
type: LoadBalancer <br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
<br />
It is similar to a NodePort manifest.<br />
<br />
If Kubernetes is running in an environment that does not support LoadBalancer services, the load balancer will not be provisioned, and the service will behave as NodePort service.<br />
===Playground===<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/LoadBalancer}}<br />
===Limit Client IP Allowed to Access the Load Balancer===<br />
Specify:<br />
<syntaxhighlight lang='yaml'><br />
apiVersion: v1<br />
kind: Service<br />
spec:<br />
type: LoadBalancer <br />
[...]<br />
loadBalancerSourceRanges:<br />
- "143.231.0.0/16"<br />
</syntaxhighlight><br />
<br />
===LoadBalancers and Different Kubernetes Implementations===<br />
====Docker Desktop Kubernetes====<br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) on the port declared as "port" (not nodePort) because Docker Desktop Kubernetes implementation will automatically tunnel to the node port, at the Docker Desktop Kubernetes VM's IP address:<br />
<syntaxhighlight lang='text'><br />
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE<br />
example LoadBalancer 10.104.90.206 localhost 80:31353/TCP 10s<br />
<br />
curl http://localhost/<br />
IP address: 10.1.0.75<br />
</syntaxhighlight><br />
<br />
====minikube====<br />
<font color=darkgray><br />
A physical load balancer will not be provisioned. However, the service will become accessible on localhost (127.0.0.1) because minikube implementation will automatically tunnel to the node port, at the minikube VM's IP address.<br />
</font><br />
<br />
====EKS====<br />
{{Internal|Amazon_EKS_Concepts#Using_a_NLB|Using an AWS NLB as LoadBalancer}}<br />
<br />
==Ingress==<br />
<br />
An Ingress resource is a different mechanism for exposing multiple services through a single IP address. Unlike the other services presented so far, which operate at layer 4, an Ingress operates at HTTP protocol layer (layer 7).<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Headless Service=<br />
<br />
A headless service is even a lower-level mechanism than a [[#Service|ClusterIP service]]. A headless service does not expose any stable Cluster IP address on its own, nor it directly load balances any requests. It simply encapsulates logic to publish the matching pods' IP addresses in the [[Kubernetes_DNS_Concepts#The_DNS_Service|internal DNS]], leaving the interested clients to look up the names in DNS, obtain the pod's IP addresses and connect to pods directly. The headless service facilitates load balancing across pods through the DNS round-robin mechanism.<br />
<br />
A headless service can be created posting a manifest similar to the one below - the key element is "clusterIP: None":<br />
<syntaxhighlight lang='yaml'> <br />
apiVersion: v1<br />
kind: Service<br />
metadata:<br />
name: example-headless<br />
spec:<br />
clusterIP: None<br />
ports:<br />
- port: 80<br />
targetPort: 8080<br />
selector:<br />
function: serves-http<br />
</syntaxhighlight><br />
==Playground==<br />
{{External|https://github.com/ovidiuf/playground/tree/master/kubernetes/services/headless}}<br />
<br />
==How Headless Services Work==<br />
<br />
Upon service deployment, the associated [[#Endpoints|Endpoints]] instance builds the list of matching ready pods, and the pod IP addresses are added to DNS. Label matching and pod's [[Kubernetes_Service_Concepts#Readiness_Probe|container readiness probes]] are handled as for any other service. <br />
<br />
The DNS is updated to resolve the service name to multiple [[DNS_Concepts#A_.28Host.29|A records]] for the service, each pointing to the IP of an individual pod backing the service '''at the moment''':<br />
<br />
<syntaxhighlight lang='text'><br />
nslookup example-headless.default.svc.cluster.local<br />
<br />
Server: 10.96.0.10<br />
Address: 10.96.0.10#53<br />
<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.188<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.189<br />
Name: example-headless.default.svc.cluster.local<br />
Address: 10.1.1.187<br />
</syntaxhighlight><br />
<br />
For pods deployed by [[Kubernetes StatefulSet|StatefulSets]], individual pods can be looked up by name - this is a feature particular to the StatefulSets.<br />
<br />
=Ingress=<br />
{{Internal|Kubernetes Ingress Concepts|Ingress Concepts}}<br />
<br />
=Service Operations=<br />
* [[Kubernetes_Service_Operations#List_Services|List services]]<br />
* [[Kubernetes_Service_Operations#Describe_a_Service|Describe a service]]<br />
* [[Kubernetes_Service_Operations#With_CLI|Create a service with CLI]]<br />
* [[Kubernetes_Service_Operations#With_Metadata|Create a service with Metadata]]<br />
* [[Kubernetes_Service_Operations#Service_Troubleshooting|Service troubleshooting]]</div>Ovidiu