Migrating from Azure Kubernetes Service to Container Apps: Guidelines and Considerations

Introduction

“Rameshwar, this is the second time we had issues upgrading the cluster and almost came close to having application downtime. We live in day and age of cloud, I am not comfortable hearing we might be having issues with application because AKS cluster upgrade is having trouble; and what’s with these frequent upgrades, we upgraded the cluster just last quarter. Can’t you just choose a stable version, so we don’t need to plan upgrades, at least for a year.” — this was the comment from my supervisor which kickstarted our container apps journey.

How to choose a container orchestration service

AKS usage related challenges

Azure Kubernetes Service (AKS) cluster upgrade and its general maintenance have always been a pain point for us. This is partly due to the configuration complexity involved in ensuring zero downtime (we deploy a new application every few weeks on our cluster) and partly due to the frequency Kubernetes releases.

Requirements for a container orchestration service

The idea was to look for a container orchestration service within Azure other than AKS that requires close to zero maintenance, offers the same or better scalability and performance, and is cost effective and easy to learn and manage. Any migration effort should involve no or minimal code changes.

If we can eliminate the frequent upgrades, simplify configuration complexity, and bring down costs, we will be able to onboard applications more frequently with virtually unlimited scalability. That can help make the business happy, and the team’s life much easier.

So, per my boss’s instructions, I embarked on a POC to find an alternative option to AKS.

Comparison of a few container orchestration services

I started by looking at what container orchestration services are available. In addition to AKS, I looked at Service Fabric and Azure Container Apps (ACA). ACA was launched by Microsoft in May 2022 as a fully managed container orchestration service.

Below is a high-level comparison of the factors that were important to us.

Since we were already using Kubernetes (AKS), and the team had a good understanding of and experience with it, we chose to use Azure Container Apps; it is Kubernetes platform which translates to minimal training and migration efforts.

Before we dive into migration design and architecture, let’s take a quick look at container apps architecture and some of its key features and components.

Overview of Azure Container Apps

Azure Container Apps ACA, like AKS, is a container orchestration platform. Unlike AKS, it is fully managed and is serverless. With AKS, Microsoft is responsible for the management and maintenance of the control plane, and users are responsible for worker nodes. In ACA, both the control and data planes are managed by Microsoft. ACA is still AKS, but managed by Microsoft.

Image source: https://learn.microsoft.com/en-us/azure/architecture/example-scenario/serverless/microservices-with-container-apps

Below are some of the key features that we needed to replace our AKS infrastructure:

• IaC support
• Enable HTTPS or TCP ingress without having to manage backend infrastructure
• Option to choose specialized hardware
• Run multiple container revisions
• Autoscaling
• Internal ingress and service discovery
• Native support for blue-green deployments
• VNet integration support
• Logs monitoring
• Generous quotas
• Low / no code migration

ACA available plans

ACA can be deployed using one of two plans:

• Consumption plan: A serverless architecture that allows your applications to scale in and out on demand. Applications can scale to zero, and you only pay for running apps. The consumption plan is preferrable when you don't have specific hardware requirements for your container app.
• Dedicated plan: This plan allows a series of workload profiles that range from the default consumption profile to profiles that feature dedicated hardware customized for specialized computing needs.

ACA environment

Although it is a serverless platform and we don’t have access to the underlying k8s cluster, one of the most important features of ACA is VNet integration through the ACA environment.

The Azure Container Apps environment is a secure boundary around one or more container apps and jobs. The Container Apps runtime manages each environment by handling OS upgrades, scale operations, failover procedures, and resource balancing.

There are two different types of Container Apps environments:

• Workload profiles environments — support both the consumption and dedicated plans.
• Consumption only environments — support only the consumption plan.

Image source: https://learn.microsoft.com/en-us/azure/container-apps/environment

Comparison of ACA and AKS features

Our AKS cluster was VNet integrated, and all traffic to AKS services was controlled through subnets and NSGs. We wanted to continue with that same approach. We chose to deploy a workload profiles environment in our VNet that allowed us to test both consumption and dedicated workload profiles.

ACA Terminology

• Container App — A container app in ACA is equivalent of a service in Kubernetes.
• Container — You don’t run pods in ACA, you run containers.
• Revision — A revision is a snapshot of a version of your container app. You can run multiple revisions of the same container app concurrently.

Migration considerations and design guidelines

The most important aspect of migration was a seamless switch from AKS to ACA without any downtime. Our infrastructure / application footprint included the following decision points:

Plan type

We considered the consumption or consumption + dedicated plans. Although it is costly, the consumption + dedicated plan offers the flexibility to switch between the consumption plan and the dedicated plan without the need for environment re-creation. This results in considerable configuration reduction. Ultimately, the choice depends on your existing infrastructure and cost considerations. If your main motive is cost savings, then the consumption plan is the way to go.

Environments

Every environment is exposed through a load balancer. If you have multiple applications and all applications are deployed to a single environment, the load balancer becomes the single point of failure. If you have a single application with multiple microservices, then it is better to use single environment for better performance. Don’t confuse an application — a set of microservices that interact with each other to provide application functionality — with container app, which is the mechanism for exposing a single microservice.

Managed identity / service principal

A container apps environment is a secure logical boundary. Container apps deployed within it cannot communicate outside that boundary unless explicitly configured to do so. To minimize maintenance overhead, you can choose to use single service principal / managed identity per environment that is used by all of the container apps within it.

VNet integration and subnet sizing

The decision to use an existing VNet with additional subnets, or to create a new VNet, depends on your existing VNet space and IP address availability. If you can utilize an existing VNet with additional subnet(s) for the environment, the migration will be much smoother.

Ingress

• If you are exposing your application to the internet, then choose “Accept traffic from anywhere.”
• If you are exposing your application to be reachable only from the VNet, then choose “limited to VNet.” This will limit access to the container app environment’s VNet.
• If you don’t want apps to be reachable outside the container apps environment, then choose “limited to container apps environment.”
• Alternately, you have the option to disable ingress, which will prevent all access to your container apps.

Revisions

Revisions allow you to introduce new functionality in your application. You can run multiple revisions simultaneously, and split traffic between them. This is useful if you want to do blue-green deployment.

DNS

Any application exposed through ingress settings is accessible through the container app environment’s endpoint (unique_identifier.region_name.azurecontainerapps.io). To resolve this, you can create a private DNS zone, point the environment IP to the endpoint, and then link the DNS zone to other subnets / VNets for resolution. If you use custom DNS, then set up a forwarder to resolve FQDN using a private DNS zone.

Availability

To mitigate zone failures, deploy every container app with a minimum of 3 replicas which will be distributed across 3 AZs.

Observability / monitoring

A container apps environment can be integrated with a log analytics workspace to send logs. These logs can be used to configure alerts and emails based on specific conditions.

Dapr

Dapr provides service mesh capabilities. The decision to enable it depends on your requirements.

Secrets

Container apps can use secrets stored in key vaults. Use service principals / managed identity for apps and configure appropriate RBAC permissions on key vaults to enable the integration with a key vault.

Maintenance

ACA is a fully managed PaaS service in Azure. Even the control plane is managed by Microsoft. As a result, maintenance overhead, such as node OS updates and AKS version upgrades, are handled by Microsoft. If you are using specific deployment strategies like blue-green, you can achieve almost zero downtime for your application.

Cost considerations

• ACA is billed based on per-second resource consumption and requests. The first 1,80,000 vCPU-seconds, 3,60,000 GiB-seconds and 2 million requests each month are free. Beyond that, you pay for what you use on a per second basis calculated on the number of vCPU-s and GiB-s your applications consume.
• Also, one standard load balancer will be provisioned per environment and its charges are $0.025/hr for the first 5 rules (we will be using only 2 rules per LB), and data processing charges are $0.005/GB.

Conclusion

The migration from AKS to Azure Container Apps represents a strategic shift in pursuit of a more efficient and streamlined container orchestration solution. The guidelines and considerations presented here offer a high-level roadmap if you are looking to make a similar transition. Assessing the above-mentioned factors can help your business ensure a smooth migration experience. ACA offers a compelling alternative for reducing operational complexity as well as the advantage of a cloud-native container environment. As the cloud landscape continues to evolve, these guidelines provide a foundation for embracing the future of container orchestration with confidence and agility.