Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer’s view in any way.
Horizontally scaleable, on-demand Azure Pipelines backed by Kubernetes!
Posted on July 9, 2022 | 2 minute read
Many enterprise customers run their own Kubernetes clusters either on-premise or in managed kubernetes environments in the cloud. Azure DevOps Services and Server agents can run from containers hosted in these Kubernetes clusters, but what if you do not want to run your agents 24/7? What if you need to be able to scale the number of agents dynamically as pipelines jobs are queued?
I’ve been working on a project that provides an application that can monitor a configurable set of agent pools, when pipeline jobs are queued up it will automagically provision Kubernetes Jobs for each job that is queued up. The Kubernetes Jobs will run and process only a single Pipelines Job and then be cleaned up by Kubernetes.
This allows for horizontally scaleable, on-demand agent pools backed by Kubernetes!