Why Your CI/CD Pipeline Is Probably Slower Than It Needs to Be (And Three Things You Can Fix Today)
Practical guide to cutting CI/CD pipeline time by 60-73% using Docker caching, test parallelization, and conditional execution.
Tegar Santosa
Deel op:
There's a special kind of frustration that comes from pushing a one-line fix to production and then watching your CI/CD pipeline churn for 45 minutes before it finally deploys. You know the feeling—you've fixed a typo in a button label, committed the change, and now you're stuck waiting while your pipeline rebuilds every Docker image, reruns your entire test suite, and scans dependencies for the thousandth time this week.
I've been there. In fact, I built one of those painfully slow pipelines. At my previous company, our deployment process took so long that developers would push their code, go to lunch, and come back to check if it had finished. We joked that it was a feature—it forced us to take breaks. But the reality was that our slow pipeline was killing productivity and discouraging us from deploying frequently.
The worst part? About 80% of that time was completely unnecessary waste.
Each addition makes sense in isolation. But nobody ever asks: "Do we need to run all of this on every commit?" We just keep stacking checks on top of each other until the pipeline becomes a bottleneck instead of an accelerator.
After auditing dozens of CI/CD pipelines, I've found that the biggest time wasters usually fall into three categories:
1. Rebuilding everything from scratch every time
Most pipelines I've seen rebuild Docker images completely on every commit, even when 90% of the dependencies haven't changed. They reinstall npm packages, re-download Maven dependencies, and recompile code that hasn't been touched in weeks.
2. Running the entire test suite sequentially
I've watched pipelines run 2,000 unit tests one after another, taking 15 minutes, when those same tests could run in parallel across multiple workers in under 3 minutes.
3. Running expensive checks that rarely find issues
Security scans and comprehensive E2E tests are important, but does your typo fix really need a full penetration test before it can go live?
# Slow Dockerfile - reinstalls everything every time
FROM node:18
WORKDIR /app
COPY . .
RUN npm install
RUN npm run build
CMD ["npm", "start"]
Structure your Dockerfile to maximize cache hits:
# Fast Dockerfile - only rebuilds what changed
FROM node:18
WORKDIR /app
# Copy dependency files first
COPY package*.json ./
RUN npm ci --only=production
# Copy source code last (changes most frequently)
COPY . .
RUN npm run build
CMD ["npm", "start"]
This simple reordering means that if you only changed application code, Docker reuses the cached npm install layer instead of downloading all your dependencies again. I've seen this single change cut build times from 8 minutes to 2 minutes.
For even better results, use multi-stage builds:
# Build stage
FROM node:18 AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
# Production stage
FROM node:18-slim
WORKDIR /app
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
CMD ["node", "dist/index.js"]
Now your final image is smaller and your builds are faster because you're not carrying build tools into production.
Most CI platforms support parallel execution, but developers rarely use it effectively. Here's a GitHub Actions example that runs tests in parallel:
# Slow approach - sequential
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- run: npm test # All 2000 tests run sequentially: 15 min
- run: npm run test:e2e # E2E tests after unit tests: +10 min
These jobs run simultaneously instead of waiting for each other. Your total pipeline time becomes the duration of your slowest job, not the sum of all jobs.
Here's why this matters beyond just saving time: when your pipeline is fast, developers actually want to deploy frequently. When it takes 45 minutes, you batch changes together to avoid the pain, which ironically makes deployments riskier and harder to debug.
Fast pipelines enable:
Deploying small changes confidently
Quick hotfix rollouts when production breaks
Actual continuous deployment instead of "deploy when we feel like waiting"
Better developer happiness (seriously, this matters)
Why Your CI/CD Pipeline Is Probably Slower Than It Needs to Be (And Three Things You Can Fix Today)