# CI/CD security with GitLab > Build a GitLab CI/CD pipeline with scoped tokens, protected variables, OIDC cloud access and a security scan that blocks merges. DevSecOps Engineer roadmap · Secure the pipeline · 22 min read · https://levelcode.org/roadmaps/devsecops/ci-gitlab Your CI/CD system runs every commit, holds your deploy credentials and can push to production: a prime target, and your best place to enforce security checks. In this lesson you'll build a GitLab CI/CD pipeline for a Node.js service from scratch: its structure and triggers, where jobs run, how to limit what each job can reach, how to cache safely, how to deploy to AWS without storing any keys, and how to make a failing security scan block the merge. (If your team uses GitHub, the GitHub Actions lesson covers the same ground for that platform.) ## Pipelines, stages and jobs A pipeline is defined in `.gitlab-ci.yml` at the root of your repository. It contains **jobs**, each of which runs a `script` inside a container image on a **runner**. Jobs are grouped into **stages** that run in order; jobs in the same stage run in parallel. `needs:` lets a job start as soon as specific earlier jobs finish, instead of waiting for the whole previous stage. ```yaml # .gitlab-ci.yml stages: [test, scan, deploy] default: image: node:22.23.3-bookworm-slim@sha256:c3de60bf2f9dd0ac6370e6117950ff62d6e339527e7472301c9c78a017978392 interruptible: true timeout: 15m test: stage: test script: - npm ci - npm test ``` By default every job runs in a pinned Node 22 image, can be cancelled when a newer pipeline starts for the same ref, and is killed after 15 minutes. If any script line exits non-zero, the job and the pipeline fail. The image reference includes both a tag and a **digest** (`@sha256:...`). Tags are mutable: whoever controls the image repository, or compromises it, can point `node:22` at different contents tomorrow. A digest identifies exactly one image, so your pipeline runs the bytes you reviewed. Let a dependency bot such as Renovate propose digest updates as merge requests. ## Triggers: rules and workflow GitLab decides *whether* to create a pipeline with `workflow:rules`, and *which jobs* to include with each job's `rules:`. Rules are checked top to bottom; the first match wins, and `when: never` excludes. A common, safe setup runs merge request pipelines for MRs, branch pipelines otherwise, and avoids duplicate pipelines when both would fire: ```yaml workflow: rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH ``` Job-level rules shape each job, for example deploying only from the default branch and running a nightly audit from a pipeline schedule: ```yaml deploy-production: rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH nightly-audit: rules: - if: $CI_PIPELINE_SOURCE == "schedule" script: - npm audit --audit-level=high ``` `when: manual` turns a job into a button someone must press, a lightweight approval step. ### Forks and untrusted code A merge request from a fork runs its pipeline **in the fork** by default, with the fork's runners and variables. A parent-project maintainer can choose to run it in the parent project instead, which executes the contributor's code and their `.gitlab-ci.yml`, so review the diff first. Branch names, MR titles (`$CI_MERGE_REQUEST_TITLE`) and commit messages are chosen by whoever pushed. The shell expands them as data, so quoting (`"$CI_MERGE_REQUEST_TITLE"`) is enough; never pass them to `eval`, `sh -c` or a generated script. ## Runners **GitLab-hosted runners** on GitLab.com run each job in a fresh virtual machine that is deleted afterwards, so nothing a job (or a malicious dependency) leaves behind reaches the next one. Tags such as `saas-linux-medium-amd64` select bigger machines. **Self-managed runners** are machines you register yourself, and they need care: - **Choose the executor carefully.** The `docker` and `kubernetes` executors isolate each job in a container. The `shell` executor runs jobs directly on the host, so jobs can read each other's files and tokens. - **Avoid `privileged = true`.** Privileged containers can escape to the host; build images with a rootless builder instead. - **Protect deployment runners.** A **protected** runner only runs jobs from protected branches and tags, which keeps the runner that can reach production away from feature-branch code. ## Least privilege for tokens and includes ### CI_JOB_TOKEN Every job gets `CI_JOB_TOKEN`, valid only while the job runs and masked in logs. It has the permissions of the user who triggered the pipeline, but reaches far fewer endpoints than a personal token: cloning, the registries and a limited set of APIs. By default, a job token can only access its own project. If another project's pipeline needs to pull from yours, add that project to your **CI/CD job token allowlist** (Settings > CI/CD > Job token permissions) rather than widening access globally. Fine-grained permissions on that allowlist restrict which API endpoints an allowlisted project's token may call. Don't replace it with a personal access token: a PAT carries one person's access to every project and outlives the job. When a job genuinely needs more (for example pushing a tag), use a **project access token** with the lowest role, minimal scopes and a short expiry, stored as a protected, masked variable. ### Pinning includes and components Shared configuration arrives through `include:`. Pin it like any other dependency: ```yaml include: - project: acme/ci-templates ref: 3f9c2a7e1b4d5c6a8e0f1a2b3c4d5e6f7a8b9c0d # commit SHA, not a branch file: /templates/node.yml - component: $CI_SERVER_FQDN/acme/components/sbom@1.4.2 ``` With a branch ref like `main`, anyone who can push to `acme/ci-templates` changes what runs in your pipeline, with your variables; a commit SHA can't change. For components, prefer exact versions over `~latest`. ## Caching without poisoning Re-downloading dependencies on every job is slow. A cache keyed on the lockfile reuses them until the lockfile changes: ```yaml test: stage: test cache: key: files: [package-lock.json] paths: [.npm/] script: - npm ci --cache .npm --prefer-offline - npm test ``` `npm ci` still verifies every package against the lockfile's integrity hashes, so a corrupted cache can't change what you install. Jobs that only read a cache can set `policy: pull`. Caches are a best-effort speed-up shared between pipelines; **artifacts** pass a job's outputs to later jobs in the same pipeline. Never rely on a cache for correctness, or put secrets or deployable builds in one. Think about who can write caches. GitLab keeps **separate caches for protected and non-protected branches** by default (keys get a `-protected` or `-non_protected` suffix), so a feature branch can't poison the cache used on `main`. Keep that setting on, and don't use caches in release or deploy jobs at all: build releases from clean, verified inputs. ## Variables, protected environments and OIDC ### CI/CD variables Store credentials under **Settings > CI/CD > Variables**, never in `.gitlab-ci.yml`, which everyone with read access can see. The options that matter: | Option | Effect | |---|---| | **Protected** | Only exposed to pipelines on protected branches and tags. A feature branch, even one that edits `.gitlab-ci.yml` to print it, never receives it. | | **Masked** | Replaced with `[MASKED]` in job logs. The value must be one line with no spaces, at least 8 characters. | | **Masked and hidden** | Masked, and the value can never be revealed in the settings UI again. Chosen at creation. | | **Environment scope** | Only exposed to jobs deploying to a matching environment, such as `production`. | | **Type: File** | Written to a temporary file; the variable holds its path, keeping the value out of `env` output. | > [!IMPORTANT] > Masking is a safety net for accidents, not a boundary: a malicious script can base64-encode a value or send it over the network. **Protected** is the real control, because it decides which code receives the value at all, so whoever can push to a protected branch can read its protected variables. **Protected environments** (Settings > CI/CD > Protected environments) add another gate: only listed users or roles can run deploy jobs for `production`, and you can require approvals before each deployment. ### OIDC instead of long-lived cloud keys A cloud access key stored as a variable never expires, and anyone who can run code on a protected branch can exfiltrate it. **OpenID Connect (OIDC)** removes it. The `id_tokens:` keyword makes GitLab issue the job a short-lived JSON Web Token, signed by GitLab and describing the project, branch, whether that branch is protected, and more. AWS verifies it and returns temporary credentials for a role. On the AWS side, create an IAM OIDC identity provider with URL `https://gitlab.com` and audience `sts.amazonaws.com`, and a role whose **trust policy** accepts only your project's protected default branch: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::123456789012:oidc-provider/gitlab.com" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "gitlab.com:aud": "sts.amazonaws.com", "gitlab.com:sub": "project_path:acme/shop:ref_type:branch:ref:main", "gitlab.com:project_id": "41237788", "gitlab.com:ref_protected": "true" } } } ] } ``` The `sub` condition is the security boundary; without it, any project on GitLab.com could assume your role. `project_id` survives renames, so a recycled project path can't match. Self-managed instances support only `sub` and `aud` as condition keys. The job requests the token, writes it to a file and lets the AWS CLI exchange it via its standard web-identity variables: ```yaml deploy-production: stage: deploy image: name: amazon/aws-cli:2.37.12@sha256:ce44b800602264c12298bbf3399055a4cefdb9ed4a098abe564afe26e7071bce entrypoint: [""] environment: production resource_group: production id_tokens: AWS_ID_TOKEN: aud: sts.amazonaws.com variables: AWS_ROLE_ARN: arn:aws:iam::123456789012:role/shop-deploy-production AWS_ROLE_SESSION_NAME: gitlab-$CI_PROJECT_ID-$CI_PIPELINE_ID AWS_REGION: eu-west-1 rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH script: - export AWS_WEB_IDENTITY_TOKEN_FILE="$(mktemp)" - printf '%s' "$AWS_ID_TOKEN" > "$AWS_WEB_IDENTITY_TOKEN_FILE" - aws sts get-caller-identity - ./scripts/deploy.sh ``` ```text { "UserId": "AROA3EXAMPLEROLEID:gitlab-41237788-1873345521", "Account": "123456789012", "Arn": "arn:aws:sts::123456789012:assumed-role/shop-deploy-production/gitlab-41237788-1873345521" } ``` Step through the exchange behind that job. Unlike some CI systems, the job never asks for its token: GitLab mints it when the runner picks the job up and hands it over as the `AWS_ID_TOKEN` variable. Watch where each check happens and how short-lived every credential is. ```sequence title: GitLab CI to AWS with OIDC participants: Runner, GitLab (OIDC issuer), AWS STS, AWS APIs Runner -> GitLab (OIDC issuer): Request next job > The runner polls GitLab for work and is assigned deploy-production on main. GitLab (OIDC issuer) --> Runner: Job + signed JWT > Because of id_tokens:, GitLab signs a JWT with aud=sts.amazonaws.com and claims such as sub, project_id and ref_protected, and passes it in as the AWS_ID_TOKEN variable. Runner -> Runner: Save token > The script writes the JWT to $AWS_WEB_IDENTITY_TOKEN_FILE. Together with AWS_ROLE_ARN, that tells the AWS CLI to use web identity. Runner -> AWS STS: AssumeRoleWithWebIdentity + JWT > The AWS CLI makes this call itself, with the role from AWS_ROLE_ARN, before its first API request. No AWS secret is involved. AWS STS -> GitLab (OIDC issuer): Get signing keys > STS fetches the issuer's published public keys (JWKS) from https://gitlab.com, and can cache them. AWS STS -> AWS STS: Check claims > STS verifies the signature and expiry, then the trust policy: aud, sub, project_id and ref_protected must all match. A merge request branch or another project fails here. AWS STS --> Runner: Temporary credentials (1 h) > An access key ID, secret key and session token for shop-deploy-production, with the session name gitlab--. Runner -> AWS APIs: aws sts get-caller-identity > This call and the deploy script's calls are signed with the temporary credentials and limited to the role's permissions policy. AWS APIs --> Runner: assumed-role/shop-deploy-production ``` No AWS key exists anywhere in GitLab; the role ARN isn't secret. `resource_group` allows one production deployment at a time. Google Cloud, Azure and Vault accept GitLab ID tokens the same way. ## Blocking merges on a security scan Add two scan jobs: static analysis with Semgrep, and a secret scan with gitleaks. Each exits non-zero when it finds a problem, which fails the job and the pipeline: ```yaml security-scan: stage: scan needs: [] image: semgrep/semgrep:1.179.0@sha256:93963d9295a366f59e4850127b1550400ee7b388f04fe144e4a1f6325d96e01b script: - semgrep scan --config p/default --error --metrics=off --gitlab-sast-output=gl-sast-report.json artifacts: when: always reports: sast: gl-sast-report.json secret-scan: stage: scan needs: [] image: name: ghcr.io/gitleaks/gitleaks:v8.30.1@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f entrypoint: [""] variables: GIT_DEPTH: "0" # full history, so old commits are scanned too script: - gitleaks git --redact --verbose . ``` `--error` makes Semgrep exit 1 on findings, and the report artifact shows the results in the merge request (the security widget is a paid-tier feature, but the job log and failed status work on every tier). `needs: []` starts the scans immediately rather than after tests. A failing run looks like this (abridged; exact wording varies by Semgrep version): ```text src/admin/ping.py ❯❯❱ python.lang.security.audit.subprocess-shell-true.subprocess-shell-true ❰❰ Blocking ❱❱ Found 'subprocess' function 'run' with 'shell=True'. This is dangerous because this call will spawn the command using a shell process. Doing so propagates current shell settings and variables, which makes it much easier for a malicious actor to execute commands. Use 'shell=False' instead. 4┆ return subprocess.run(f"ping -c 1 {host}", shell=True) ERROR: Job failed: exit code 1 ``` > [!WARNING] > GitLab's built-in scanning templates (for example `Jobs/SAST.gitlab-ci.yml`) set `allow_failure: true`: their jobs never fail the pipeline, they only report. If you include them, gate merges with your own failing job like the ones above, or with security policies on paid tiers. Never add `allow_failure: true` to a scan you rely on as a gate. A failed pipeline still doesn't stop someone pressing Merge. Configure the gate: 1. **Settings > Merge requests > Merge checks:** enable **Pipelines must succeed**, and leave "Skipped pipelines are considered successful" off. 2. **Settings > Repository > Protected branches:** protect `main` with *Allowed to merge: Maintainers*, *Allowed to push and merge: No one*, and force push off. All changes now arrive through merge requests. 3. **Approvals** (Settings > Merge requests, Premium): require at least one approval and reset approvals when new commits are pushed. 4. **Code owners:** add a `CODEOWNERS` file so pipeline changes need the platform team, and enable "Require approval from code owners" on the protected branch (Premium and above). ```text # CODEOWNERS /.gitlab-ci.yml @acme/platform-security /CODEOWNERS @acme/platform-security ``` Watch out for rules that quietly skip the gate: with `rules: - changes: [src/**/*]` on `security-scan`, an MR adding a vulnerable script under `tools/` passes unscanned. Keep gating jobs unconditional. ## Practice In a GitLab project you control (a new private one is fine): 1. Write a `.gitlab-ci.yml` with `workflow:rules`, a cached `test` job, `security-scan` and `secret-scan` jobs, and digest-pinned images. 2. Enable **Pipelines must succeed** and protect `main` so nobody can push directly. 3. Open an MR adding `subprocess.run(cmd, shell=True)` to a Python file, confirm the merge is blocked, then fix it with an argument list. 4. Create a protected, masked variable `DEPLOY_WEBHOOK_TOKEN` and verify that a pipeline on your feature branch can't see it (print its length, never its value). 5. Sketch the `deploy-production` job and IAM trust policy for OIDC deployment from `main`. **Solution** ```yaml # .gitlab-ci.yml workflow: rules: - if: $CI_PIPELINE_SOURCE == "merge_request_event" - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS when: never - if: $CI_COMMIT_BRANCH stages: [test, scan, deploy] default: image: node:22.23.3-bookworm-slim@sha256:c3de60bf2f9dd0ac6370e6117950ff62d6e339527e7472301c9c78a017978392 interruptible: true timeout: 15m test: stage: test cache: key: files: [package-lock.json] paths: [.npm/] script: - npm ci --cache .npm --prefer-offline - npm test security-scan: stage: scan needs: [] image: semgrep/semgrep:1.179.0@sha256:93963d9295a366f59e4850127b1550400ee7b388f04fe144e4a1f6325d96e01b script: - semgrep scan --config p/default --error --metrics=off --gitlab-sast-output=gl-sast-report.json artifacts: when: always reports: sast: gl-sast-report.json secret-scan: stage: scan needs: [] image: name: ghcr.io/gitleaks/gitleaks:v8.30.1@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f entrypoint: [""] variables: GIT_DEPTH: "0" script: - gitleaks git --redact --verbose . show-protected-variable: stage: test script: - echo "DEPLOY_WEBHOOK_TOKEN length is ${#DEPLOY_WEBHOOK_TOKEN}" ``` The vulnerable and fixed code for step 3: ```python # Fails security-scan subprocess.run(f"convert {name} out.png", shell=True) # Passes subprocess.run(["convert", name, "out.png"], check=True) ``` With the vulnerable MR, the merge widget shows the failed pipeline and the Merge button is unavailable until a new pipeline passes. Step 4: on the feature branch the job prints `length is 0` because protected variables are only exposed to protected refs; after merging, the pipeline on `main` prints the real length. (Delete this job afterwards; it exists only to prove the point.) Step 5: reuse the OIDC section's job and trust policy with your own project path and `project_id`, and add `production` as a protected environment with an approval rule. ## Key takeaways - A pipeline is stages of jobs running scripts in images on runners; pin images (and includes) to digests and SHAs, not mutable tags or branches. - Use `workflow:rules` and job `rules:` to control when pipelines and jobs run; fork MR code and event data such as branch names are untrusted. - Prefer ephemeral runners; avoid the shell executor and privileged mode on shared runners, and mark deployment runners as protected. - Keep `CI_JOB_TOKEN` scoped with the allowlist and avoid personal access tokens; rely on separate protected caches and keep caches out of deploys. - Store secrets as protected (the real control), masked and hidden variables; deploy with `id_tokens` and a trust policy pinned to `sub` and `project_id` instead of stored cloud keys. - Make scans fail the job (no `allow_failure`), then enforce them with Pipelines must succeed, protected branches, approvals and code owners for CI files. ## Quiz 1. A CI/CD variable holding a deploy key is masked but not protected. What can a developer with push access to a feature branch do? A. Nothing; masked variables are never exposed to jobs B. Edit .gitlab-ci.yml on their branch to send the value to an external server, since the job receives it C. Only see the value as [MASKED], which is unusable D. Read it only after a maintainer approves the pipeline **Answer: B.** Masking only redacts the value in logs. Without Protected, every pipeline (including feature branches) receives the variable, and a script can exfiltrate it in a transformed form or over the network. 2. You include GitLab's SAST template, and a merge request introduces a high-severity finding. Why might the MR still be mergeable with Pipelines must succeed enabled? A. SAST findings are only shown on the default branch B. Pipelines must succeed ignores the scan stage C. The template's jobs use allow_failure: true, so they don't fail the pipeline D. Templates run only on schedules **Answer: C.** The built-in scanning jobs report findings without failing. To gate merges, add a job that exits non-zero on findings (or use security policies on paid tiers). 3. Which condition in an AWS trust policy for GitLab OIDC ensures only your project's main branch can assume the role? A. gitlab.com:sub set to project_path:acme/shop:ref_type:branch:ref:main (ideally plus project_id) B. gitlab.com:aud set to sts.amazonaws.com alone C. A condition on the role's session name D. Listing the GitLab runner's IP address **Answer: A.** Every GitLab.com ID token shares the issuer, and the audience is whatever the job requests. The sub claim names the project and ref; project_id additionally survives renames so a recycled path can't match. 4. Why mark the self-managed runner that can reach production as protected? A. Protected runners run jobs faster B. Protected runners encrypt job logs C. It hides the runner from the project's settings page D. It will only run jobs from protected branches and tags, so feature-branch code never executes on it **Answer: D.** A protected runner refuses jobs from unprotected refs. Combined with protected branches, only reviewed, merged code can run on the machine with production network access.