Blog: Kubernetes 1.29: Cloud Provider Integrations Are Now Separate Components

Publicada el por Stack Over Cloud

Authors: Michael McCune (Red Hat), Andrew Sy Kim (Google)

For Kubernetes v1.29, you need to use additional components to integrate your
Kubernetes cluster with a cloud infrastructure provider. By default, Kubernetes
v1.29 components abort if you try to specify integration with any cloud provider using
one of the legacy compiled-in cloud provider integrations. If you want to use a legacy
integration, you have to opt back in – and a future release will remove even that option.

In 2018, the Kubernetes community agreed to form the Cloud Provider Special
Interest Group (SIG), with a mission to externalize all cloud provider
integrations and remove all the existing in-tree cloud provider integrations.
In January 2019, the Kubernetes community approved the initial draft of
KEP-2395: Removing In-Tree Cloud Provider Code. This KEP defines a
process by which we can remove cloud provider specific code from the core
Kubernetes source tree. From the KEP:

Motiviation [sic] behind this effort is to allow cloud providers to develop and
make releases independent from the core Kubernetes release cycle. The
de-coupling of cloud provider code allows for separation of concern between
“Kubernetes core” and the cloud providers within the ecosystem. In addition,
this ensures all cloud providers in the ecosystem are integrating with
Kubernetes in a consistent and extendable way.

After many years of development and collaboration across many contributors,
the default behavior for legacy cloud provider integrations is changing.
This means that users will need to confirm their Kubernetes configurations,
and in some cases run external cloud controller managers. These changes are
taking effect in Kubernetes version 1.29; read on to learn if you are affected
and what changes you will need to make.

These updated default settings affect a large proportion of Kubernetes users,
and will require changes for users who were previously using the in-tree
provider integrations. The legacy integrations offered compatibility with
Azure, AWS, GCE, OpenStack, and vSphere; however for AWS and OpenStack the
compiled-in integrations were removed in Kubernetes versions 1.26 and 1.27,
respectively.

What has changed?

At the most basic level, two feature gates are changing their default
value from false to true. Those feature gates, DisableCloudProviders and
DisableKubeletCloudCredentialProviders, control the way that the
kube-apiserver, kube-controller-manager, and kubelet
invoke the cloud provider related code that is included in those components.
When these feature gates are true (the default), the only recognized value for
the --cloud-provider command line argument is external.

Let’s see what the official Kubernetes documentation says about these
feature gates:

DisableCloudProviders: Disables any functionality in kube-apiserver,
kube-controller-manager and kubelet related to the --cloud-provider
component flag.

DisableKubeletCloudCredentialProviders: Disable the in-tree functionality
in kubelet to authenticate to a cloud provider container registry for image
pull credentials.

The next stage beyond beta will be full removal; for that release onwards, you
won’t be able to override those feature gates back to false.

What do you need to do?

If you are upgrading from Kubernetes 1.28+ and are not on Azure, GCE, or
vSphere then there are no changes you will need to make. If
you are on Azure, GCE, or vSphere, or you are upgrading from a version
older than 1.28, then read on.

Historically, Kubernetes has included code for a set of cloud providers that
included AWS, Azure, GCE, OpenStack, and vSphere. Since the inception of
KEP-2395 the community has been moving towards removal of that
cloud provider code. The OpenStack provider code was removed in version 1.26,
and the AWS provider code was removed in version 1.27. This means that users
who are upgrading from one of the affected cloud providers and versions will
need to modify their deployments.

Upgrading on Azure, GCE, or vSphere

There are two options for upgrading in this configuration: migrate to external
cloud controller managers, or continue using the in-tree provider code.
Although migrating to external cloud controller managers is recommended,
there are scenarios where continuing with the current behavior is desired.
Please choose the best option for your needs.

Migrate to external cloud controller managers

Migrating to use external cloud controller managers is the recommended upgrade
path, when possible in your situation. To do this you will need to
enable the --cloud-provider=external command line flag for the
kube-apiserver, kube-controller-manager, and kubelet components. In
addition you will need to deploy a cloud controller manager for your provider.

Installing and running cloud controller managers is a larger topic than this
post can address; if you would like more information on this process please
read the documentation for Cloud Controller Manager Administration
and Migrate Replicated Control Plane To Use Cloud Controller Manager.
See below for links to specific cloud provider
implementations.

Continue using in-tree provider code

If you wish to continue using Kubernetes with the in-tree cloud provider code,
you will need to modify the command line parameters for kube-apiserver,
kube-controller-manager, and kubelet to disable the feature gates for
DisableCloudProviders and DisableKubeletCloudCredentialProviders. To do
this, add the following command line flag to the arguments for the previously
listed commands:

--feature-gates=DisableCloudProviders=false,DisableKubeletCloudCredentialProviders=false

Please note that if you have other feature gate modifications on the command
line, they will need to include these 2 feature gates.

Note: These feature gates will be locked to true in an upcoming
release. Setting these feature gates to false should be used as a last
resort. It is highly recommended to migrate to an external cloud controller
manager as the in-tree providers are planned for removal as early as Kubernetes
version 1.31.

Upgrading on other providers

For providers other than Azure, GCE, or vSphere, good news, the external cloud
controller manager should already be in use. You can confirm this by inspecting
the --cloud-provider flag for the kubelets in your cluster, they will have
the value external if using external providers. The code for AWS and OpenStack
providers was removed from Kubernetes before version 1.27 was released.
Other providers beyond the AWS, Azure, GCE, OpenStack, and vSphere were never
included in Kubernetes and as such they began their life as external cloud
controller managers.

Upgrading from older Kubernetes versions

If you are upgrading from a Kubernetes release older than 1.26, and you are on
AWS, Azure, GCE, OpenStack, or vSphere then you will need to enable the
--cloud-provider=external flag, and follow the advice for installing and
running a cloud controller manager for your provider.

Please read the documentation for
Cloud Controller Manager Administration and
Migrate Replicated Control Plane To Use Cloud Controller Manager. See
below for links to specific cloud provider implementations.

Where to find a cloud controller manager?

At its core, this announcement is about the cloud provider integrations that
were previously included in Kubernetes. As these components move out of the
core Kubernetes code and into their own repositories, it is important to note
a few things:

First, SIG Cloud Provider offers a reference framework for developers who
wish to create cloud controller managers for any provider. See the
cloud-provider repository for more information about how
these controllers work and how to get started creating your own.

Second, there are many cloud controller managers available for Kubernetes.
This post is addressing the provider integrations that have been historically
included with Kubernetes but are now in the process of being removed. If you
need a cloud controller manager for your provider and do not see it listed here,
please reach out to the cloud provider you are integrating with or the
Kubernetes SIG Cloud Provider community for help and advice. It is
worth noting that while most cloud controller managers are open source today,
this may not always be the case. Users should always contact their cloud
provider to learn if there are preferred solutions to utilize on their
infrastructure.

Cloud provider integrations provided by the Kubernetes project

If you are looking for an automated approach to installing cloud controller
managers in your clusters, the kOps project provides a convenient
solution for managing production-ready clusters.

Want to learn more?

Cloud providers and cloud controller managers serve a core function in
Kubernetes. Cloud providers are often the substrate upon which Kubernetes is
operated, and the cloud controller managers supply the essential lifeline
between Kubernetes clusters and their physical infrastructure.

This post covers one aspect of how the Kubernetes community interacts with
the world of cloud infrastructure providers. If you are curious about this
topic and want to learn more, the Cloud Provider Special Interest Group (SIG)
is the place to go. SIG Cloud Provider hosts bi-weekly meetings to discuss all
manner of topics related to cloud providers and cloud controller managers in
Kubernetes.

SIG Cloud Provider

Originally posted on Kubernetes – Production-Grade Container Orchestration
Author:

Deja una respuesta

Tu dirección de correo electrónico no será publicada. Los campos obligatorios están marcados con *