Executive Summary
Three decisions shape the choice of a CI/CD platform: which code host it must work with, whose machines run the builds, and what the bill is metered on.
A continuous integration (CI) platform runs an automated build and test every time a developer pushes a change to the shared codebase. A CI/CD platform also carries the tested change on toward release: it packages the artifact, promotes it through test environments and, if you choose, deploys it. Martin Fowler defines continuous integration as a software development practice where each member of a team merges their changes into a codebase together with their colleagues changes at least daily, with each integration verified by an automated build, including test, to detect integration errors as quickly as possible. Buyers search for the category as CI/CD platforms, continuous integration platforms or CI platforms.
The platforms differ on three decisions. The first is the code host: some platforms are part of the service that holds your repositories, and others work with several. The second is who runs the build machines: the vendor, your own team, or both, with the vendor scheduling jobs on machines you operate. The third is the meter: seats, compute minutes, credits, parallel jobs or action minutes.
This guide defines the category from Martin Fowler’s writing on continuous integration and from NIST’s guidance on supply chain security in CI/CD pipelines, explains where builds can run and what to test, sets out seven approaches to buying, names the eight platforms in the table that represent them, and lists published prices. Repository hosting is covered in the Source Code Management Buyer Guide. Deployment from Git to Kubernetes is covered in the GitOps & Continuous Delivery Buyer Guide.
What Is a CI/CD Platform?
Continuous integration is a team practice, and a CI platform automates it. Fowler writes that the whole point of continuous integration is to provide rapid feedback. Fowler states that continuous integration can only work if the mainline is kept in a healthy state, and that a failed integration build needs to be fixed right away. The platform watches the repository, starts a build on every push, runs the tests and reports the result to the developer and the pull request.
Martin Fowler writes that for many people continuous integration is about integrating code to the mainline in the development team’s environment, and continuous delivery is the rest of the deployment pipeline heading to a production release. Some treat the two as one practice; others treat them as linked steps. Fowler states that the aim of continuous delivery is that the product should always be in a state where the latest build can be released, which makes the release to production a business decision. A CI/CD platform automates both halves, from the first build to a release candidate, and often the deployment as well.
Speed decides whether developers wait for the result or move on and forget it. Fowler describes a two stage deployment pipeline in which the first stage runs fast, localized unit tests within the ten minute guideline, and a second stage runs tests that hit a real database and can take a couple of hours. Test each platform against your slowest pipeline at your busiest hour, not a sample project.
Security decides how much you can trust what the pipeline produces. NIST SP 800-204D sets out strategies for the integration of software supply chain security in DevSecOps CI/CD pipelines. NIST describes CI/CD pipelines as flow processes that take software through stages such as build, test, package and deploy, through operations that constitute the software supply chain. The pipeline holds credentials for cloud accounts, registries and production systems, and it writes the artifacts you ship. The SLSA framework grades how far a build can be trusted. SLSA states that at Build L2, in practice, builds run on a hosted platform that generates and signs the provenance. SLSA states that at Build L3, in practice, builds run on a hardened build platform that offers strong tamper protection. SLSA states that Build L3 usually requires significant changes to existing build platforms.
Where Should Your Builds Run: Hosted, Self-Hosted or Hybrid?
Every platform in this guide splits the work in two. A scheduler decides which jobs run and in what order. Runners, also called agents or executors, are the machines that run the jobs. The choice is who operates each half.
Vendor-hosted runners remove the machines from your list of things to patch. GitLab states that GitLab-hosted runners are managed by GitLab and available on GitLab.com, while self-managed runners are GitLab Runner instances that you install, configure, and manage in your own infrastructure. Hosted runners are billed by the minute, by credit or by parallel job, and the vendor decides which images and machine sizes you can pick.
Self-hosted runners keep source code, secrets and build traffic inside your network, and reach internal systems a hosted runner cannot. GitHub states that self-hosted runners are free to use with GitHub Actions, but that you are responsible for the cost of maintaining your runner machines. CircleCI does not charge per build minute on self-hosted runners. Some platforms are self-hosted in full, scheduler included. Jenkins describes itself as a self-contained Java-based program, ready to run out-of-the-box, with packages for Windows, Linux, macOS and other Unix-like operating systems.
A hybrid model keeps the scheduler with the vendor and the machines with you. Buildkite states that Buildkite Pipelines schedules builds through a hosted control plane, and that jobs run on agents, either on infrastructure you manage or on Buildkite-hosted infrastructure. Buildkite states that agents send job logs and build results to Buildkite, and that source code or secrets included in those outputs can leave your environment. Ask every vendor the same question about its self-hosted option: which data leaves your network, and where is it stored?
| If you need | Look at | Why |
|---|---|---|
| No build machines to run | Vendor-hosted runners | The vendor patches and scales them; you choose from its images and sizes |
| Builds that reach internal systems | Self-hosted runners or agents | Jobs run on your network; you patch, scale and secure the machines |
| Vendor scheduling, your machines | A hosted control plane with self-hosted agents | Agents connect out to the vendor; check which logs and artifacts leave |
| No external service at all | A self-hosted server, or a Kubernetes-native engine | Everything stays on your infrastructure, and so does the upkeep |
What Should You Test in a CI/CD Platform?
Start with feedback time. Port your slowest real pipeline to each finalist and measure the time from push to result, with caches warm and cold. Then measure it again with ten pipelines queued at once.
Test selection and test splitting cut feedback time on large suites. Harness states that with Test Intelligence, AI analyzes your codebase and selects only the tests impacted by a code change. CircleCI states that intelligent test selection, dynamic test splitting, auto-rerun failed tests, and dynamic parallelism are billed by test atoms evaluated, a test atom usually being a test file. Ask each vendor what a skipped test costs you if the selection is wrong, and how a skipped test is reported.
Reusable configuration decides whether fifty teams can share one tested way to build. Most platforms in this guide offer it, under names described in the approaches below. Ask how a shared component is versioned, who can publish one, and how a team is told that the version it uses is out of date.
Credentials are the part of the pipeline attackers want. GitHub states that with OIDC, the cloud provider issues a short-lived access token that is only valid for a single job and then automatically expires. GitLab states that pipeline inputs close off variable injection by default instead of leaving pipeline customization open to anyone with trigger access. Ask each vendor which clouds and secret stores accept its short-lived tokens, and who can change a pipeline’s inputs.
Provenance is the signed record of how an artifact was built. GitHub states that artifact attestations by themselves provide SLSA v1.0 Build Level 2. Buildkite states that signed pipelines prevent unauthorized pipeline modifications by signing step definitions and verifying signatures before running jobs. Ask each vendor which SLSA build level its provenance meets, where the signing key is held, and how a consumer verifies the signature.
| Capability | What it does | Buyer translation |
|---|---|---|
| Pipeline as code | Stores the pipeline definition as a file in the repository | Can a pipeline change be reviewed like any other code change? |
| Reusable components | Shares versioned build and test steps across projects | Does one fix reach every team without copying files? |
| Test selection and splitting | Runs only affected tests, or spreads them across machines | How long does a developer wait at your peak hour? |
| Short-lived credentials | Exchanges a job identity for a cloud token that expires with the job | Can you remove long-lived cloud keys from the CI system? |
| Build provenance | Signs a record of the source, steps and builder for each artifact | Which SLSA build level can you show an auditor? |
| Self-hosted runners | Runs jobs on machines you operate | Can builds reach internal systems without opening inbound ports? |
Vendors in CI/CD & Continuous Integration Platforms
Representative vendors include Argo Workflows / Events, Azure DevOps Pipelines, Buildkite, CircleCI, GitHub Actions, GitLab CI/CD, Harness CI and Jenkins / CloudBees. The list is alphabetical, is not a ranking, and is not for sale. How we choose vendors
The approaches below follow two questions: where the pipeline lives, and who runs the build machines. Two platforms in the table share the first approach.
CI built into the code host runs pipelines next to the repositories, pull requests and permissions it builds from. GitHub lists hosted runners for Linux, macOS, Windows, ARM, GPU, and containers, and states that you can use your own VMs, in the cloud or on-prem, with self-hosted runners. GitLab defines a CI/CD component as a reusable single pipeline configuration unit that can be released and used with a specific version. The fit is closest when all your code is on that host.
An open source automation server is software you install and run. The Jenkins project states that with 2300+ community contributed plugins, Jenkins integrates with practically every tool in the continuous integration and continuous delivery toolchain. The Jenkins project states that it does not develop or distribute Jenkins for commercial purposes and is part of the Continuous Delivery Foundation. The Jenkins project states that the core of Jenkins is entirely under the MIT license. A commercial edition is sold on top of the project. CloudBees describes CloudBees CI as enterprise Jenkins, built on the same open source project. CloudBees states that CloudBees CI is built on Jenkins, so pipelines and Jenkins workflows carry over, and that it is adopted controller by controller, with no forklift migration.
A hosted CI service works with code on several hosts and runs jobs on its own machines or yours. CircleCI describes orbs as reusable packages of parameterizable configuration.
A hosted control plane with self-hosted agents keeps scheduling with the vendor and the machines with you. Buildkite states that its agent works by polling Buildkite’s agent API over HTTPS, with no need to forward ports or provide incoming firewall access.
A software delivery platform sells CI as one module next to deployment, security testing and infrastructure as code. Harness states that Harness Software Delivery Agent uses AI to accelerate continuous integration. Harness states that builds run on Kubernetes, VMs, bare metal, or Harness Cloud. Harness states that its DevOps Essentials package includes CI, CD, Code Repository, Security Test Orchestration, and Infrastructure as Code Management. Harness states that DevOps Essentials has no on-premise support.
Cloud providers sell pipeline services billed through the cloud account. Microsoft states that Azure Pipelines is the part of Azure DevOps that combines continuous integration, continuous testing, and continuous delivery to automatically build, test, and deploy code projects to any destination. Microsoft states that Azure Pipelines supports several forms of version control, including GitHub and Azure Repos. Microsoft states that public projects are retired, that new public projects can no longer be created, and that existing public projects keep their free parallelism grants until they convert to private in 2027. AWS describes AWS CodePipeline as a continuous delivery service you can use to model, visualize, and automate the steps required to release your software.
A Kubernetes-native workflow engine runs each pipeline step as a container on your own cluster. The Argo project describes Argo Workflows as an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes, implemented as a Kubernetes CRD. The Argo project states that Argo Workflows can run CI/CD pipelines natively on Kubernetes without configuring complex software development products. The Argo project describes Argo Events as an event-based dependency manager for Kubernetes that triggers Kubernetes objects from event sources such as webhooks, S3, schedules and streams. CNCF states that Argo was accepted to CNCF on March 26, 2020 and moved to the Graduated maturity level on December 6, 2022.
One more product runs pipelines written for another platform on its own compute. Depot states that Depot CI is a programmable CI system for engineers and agents, with workflows that run entirely on Depot compute. Depot states that GitHub Actions is the first syntax Depot CI supports, so existing GitHub Actions workflows can be migrated. Depot states that original GitHub workflows continue running on GitHub in parallel until you are ready to switch over to Depot CI completely.
| Vendor | Approach | Where it fits |
|---|---|---|
| GitHub Actions | CI built into the code host | Hosted Linux, macOS, Windows, ARM and GPU runners; self-hosted runners at no charge; artifact attestations |
| GitLab CI/CD | CI built into the code host | Versioned CI/CD components; GitLab-hosted or self-managed runners; compute minutes by tier |
| Azure DevOps Pipelines | Cloud provider pipeline service | Builds from GitHub or Azure Repos; priced per parallel job |
| CircleCI | Hosted CI service | Credits for users, compute and add-ons; orbs; no build-minute charge on self-hosted runners |
| Buildkite | Hosted control plane, your agents | Agents poll over HTTPS; signed pipelines; per-active-user plans and hosted agents by the vCPU minute |
| Argo Workflows / Events | Kubernetes-native workflow engine | Open source, CNCF graduated; steps as containers on your cluster |
| Jenkins / CloudBees | Open source automation server | MIT-licensed core and 2300+ plugins; CloudBees CI as the commercial edition |
| Harness CI | Software delivery platform module | Test Intelligence; Kubernetes, VMs, bare metal or Harness Cloud; DevOps Essentials bundle |
Eight platforms across seven approaches; GitHub Actions and GitLab CI/CD share one. The approaches were written first, and each row was then selected from the eight platforms the previous edition covered. Rows are not for sale. AWS CodePipeline, Depot and other CI services can present their own case in the Spotlight below.
How Are CI/CD Platforms Priced?
Five meters appear in this category: seats, compute minutes or their dollar value, credits, parallel jobs, and action minutes. Most plans combine two. Self-hosted runners move the compute cost to your own infrastructure, where it is real but outside the vendor’s invoice.
Seats plus compute. GitHub lists Team at $4 USD per user per month. GitHub lists per-minute rates of $0.006 USD for Linux 2-core (x64) and $0.062 USD for macOS 3-core or 4-core (M1 or Intel). GitLab lists Free at $0 per user/month, with 5 users per top-level group and 400 compute minutes per month. GitLab lists Premium at $29 per user/month, billed annually, with 10,000 compute minutes per month. GitLab lists Ultimate at custom pricing, with 50,000 compute minutes per month.
Credits. CircleCI states that credits are the currency of CircleCI, purchased up-front and used to pay for platform usage in real time. CircleCI states that Performance plan users may purchase additional credits at $15 for every 25,000 credits. CircleCI gives the example that 30,000 credits at a rate of 10 credits per minute on Linux Medium is 3,000 minutes.
Active users plus hosted compute. Buildkite lists the Pro plan at $30 per active user/month. Buildkite lists Linux hosted agents at $0.004 USD per vCPU minute.
Parallel jobs and action minutes. Microsoft states that for private projects, the Azure DevOps free tier provides one parallel job that can take up to 60 minutes to run, up to 1,800 minutes per month, for organizations with billing set up through a valid Azure subscription. Microsoft lists $40 per extra Microsoft-hosted CI/CD parallel job and $15 per extra self-hosted CI/CD parallel job with unlimited minutes. AWS lists V1-type pipelines at $1.00 per active pipeline per month and V2-type pipelines at $0.002 per action execution minute. AWS states that V2-type pipelines get 100 free action execution minutes per month, shared between all V2 pipelines in the account.
Where CI is sold inside a bundle, ask for the CI line separately. All prices are US list prices as published when this guide was checked in November 2026.
| Meter | You are charged for | Grows with | Where it goes wrong |
|---|---|---|---|
| Seats | Each user of the platform | Team size | Occasional committers and reviewers counted as full users |
| Compute minutes | Runner time, priced by operating system and size | Build length and frequency | macOS and large runners priced at many times the Linux rate |
| Credits | Users, compute and add-on features from one balance | Everything at once | Add-ons that draw credits per job, on top of compute |
| Parallel jobs | The number of jobs that can run at the same time | Peak concurrency | Queues at peak hours when the count is set for the average |
| Action minutes | Time each pipeline action runs | Actions per pipeline and runs per month | Long waits counted as billable action time |
How long does implementation take for CI/CD & Continuous Integration Platforms?
Most of the work in a CI migration is not the pipeline syntax. It is secrets, runner images, caches, approval rules and the habits of the teams who own each pipeline.
List every pipeline, the secrets it reads, the machines it needs and the systems it deploys to. Mark the pipelines that hold production credentials.
Move the slowest and the most sensitive pipeline first. Replace long-lived cloud keys with short-lived tokens, turn on provenance, and run old and new systems side by side on the same commits.
Publish shared, versioned components for build, test, scan and release, and a default runner image. Set who may publish a component and who may change a pipeline’s inputs.
Move teams in waves, each with a named owner. Remove each old pipeline and its credentials once the new one has run cleanly for two weeks, then shut down the old servers and runners.
What should you ask vendors about CI/CD & Continuous Integration Platforms?
Ask each vendor to answer in writing and to demonstrate on one of your repositories.
-
Is all your code on one code host?Yes Test that host’s built-in CI first, against one platform that works across hosts.No Shortlist platforms that build from every host you use.
-
Must builds run inside your network?Yes Require self-hosted runners or agents, and ask which data leaves your network.No Compare hosted runner prices for the machine types you use.
-
Do you already run Kubernetes clusters for your platform teams?Yes Include a Kubernetes-native engine, and price the staff time to run it.No Weight hosted or vendor-supported options higher.
CI/CD Pipeline Platforms RFP package: 128 questions, 10 demo scenarios and a vendor scorecard
RFP packages are a separate paid product. Vendors do not pay to appear in buyer guides.