Semgrep for Enterprise DevSecOps: Architecture, Deployment, and Secure CI/CD Integration

Overview

Semgrep is a lightweight static application security testing (SAST) and code analysis platform that combines pattern-based matching with semantic awareness. Enterprises use it to detect common vulnerabilities, insecure coding patterns, secrets exposure, and policy violations early in the software development lifecycle.

Its core value is speed and flexibility. Security teams can run Semgrep locally with the open source CLI, integrate it into CI/CD pipelines, and use Semgrep AppSec Platform for centralized rule management, findings triage, and developer workflows. Compared with traditional SAST tools, Semgrep is typically easier to customize, faster to deploy, and more practical for modern polyglot engineering organizations.

Architecture

Core components

• Semgrep CLI: Local or pipeline scanner that analyzes source code, infrastructure-as-code, and configuration files.
• Rules engine: Uses YAML-defined rules with language-aware parsing to identify patterns and taint flows.
• Semgrep AppSec Platform: SaaS control plane for findings management, policy administration, dashboards, and team collaboration.
• Rule registry: Semgrep-maintained and custom enterprise rule packs.

Deployment models

• CLI-only: Best for isolated environments or teams starting with repository-level scanning.
• CI/CD integrated: Runs in GitHub Actions, GitLab CI, Jenkins, or Azure DevOps to block risky merges.
• Hybrid enterprise: Local scanning in build runners with findings uploaded to Semgrep AppSec Platform for centralized governance.

Data flow

1. Developer pushes code or opens a pull request.
2. CI runner invokes semgrep scan against the repository.
3. Rules are pulled from local files or the Semgrep registry.
4. Findings are emitted as JSON, SARIF, or uploaded to the platform.
5. Security teams triage issues, tune rules, and feed exceptions back into policy.

Implementation Guide

1. Install Semgrep CLI

python3 -m pip install --upgrade semgrep
semgrep --version

2. Authenticate to Semgrep AppSec Platform

export SEMGREP_APP_TOKEN="sgp_xxxxxxxxxxxxxxxxx"
semgrep login

3. Create a local rules directory

mkdir -p semgrep-rules
cat > semgrep-rules/no-dangerous-subprocess.yaml <<'EOF'
rules:
  - id: python-dangerous-subprocess-shell
    languages: [python]
    message: Avoid subprocess with shell=True when arguments include user input.
    severity: ERROR
    patterns:
      - pattern: subprocess.$FUNC(..., shell=True, ...)
      - metavariable-regex:
          metavariable: $FUNC
          regex: "(run|Popen|call|check_output)"
    metadata:
      category: security
      technology: [python]
      cwe: ["CWE-78"]
      owasp: ["A03:2021-Injection"]
EOF

4. Run a repository scan

semgrep scan --config semgrep-rules/ --json --output semgrep-findings.json .

5. Integrate into CI

Use a fail-on-findings threshold for high-confidence rules.

semgrep scan --config p/security-audit --error

6. Add a project configuration file

version: v1
secrets:
  enabled: true
scan:
  autofix: false
  baseline: main
rules:
  - p/security-audit
  - p/secrets
paths:
  exclude:
    - vendor/
    - node_modules/
    - dist/

7. Export SARIF for code scanning platforms

semgrep scan --config p/security-audit --sarif --output semgrep.sarif .

Code Examples

Example 1: GitHub Actions pipeline

name: semgrep
on:
  pull_request:
  push:
    branches: [main]
jobs:
  semgrep:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      security-events: write
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install semgrep
      - run: semgrep scan --config p/security-audit --sarif --output semgrep.sarif .
      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: semgrep.sarif

Example 2: Custom Semgrep rule

rules:
  - id: hardcoded-aws-secret
    languages: [python, javascript]
    message: Possible hardcoded AWS secret access key.
    severity: ERROR
    patterns:
      - pattern-regex: "(?i)aws(.{0,20})?(secret|access).{0,10}[=:].{0,5}[A-Za-z0-9/+=]{40}"
    metadata:
      category: secrets
      confidence: medium

Example 3: Vulnerable Python pattern

import subprocess

def handler(user_cmd):
    subprocess.run(user_cmd, shell=True, check=True)

Security Hardening

• Store SEMGREP_APP_TOKEN in a secrets manager such as AWS Secrets Manager, HashiCorp Vault, or GitHub Actions Secrets.
• Restrict token scope to the minimum required workspace and rotate tokens regularly.
• Use private runners or isolated build agents for sensitive codebases.
• Encrypt CI workspace disks and artifact storage; if exporting SARIF or JSON, ensure encryption at rest and in transit.
• Limit outbound network access from runners to source control, package mirrors, and Semgrep endpoints only.
• Enforce RBAC in the Semgrep AppSec Platform so only security engineers can modify enterprise rule packs.

Comparison

Troubleshooting

1. Parse error on unsupported syntax

Log sample:

[ERROR] semgrep.error: ParsingError: Could not parse file services/api/app.py
[WARN] Skipping file due to parse failure: services/api/app.py

Fix: Upgrade Semgrep, verify the language parser supports the syntax version, and exclude generated or templated files.

2. Authentication failure with platform upload

Log sample:

[ERROR] API error: 401 Unauthorized
[ERROR] Failed to upload findings: invalid API token

Fix: Reissue the app token, confirm SEMGREP_APP_TOKEN is available in the runner environment, and verify workspace binding.

3. CI job exits non-zero unexpectedly

Log sample:

ran 128 rules on 642 files: 7 findings
semgrep scan: exiting with error status because findings were found

Fix: This is expected when --error is used. Adjust branch protection policy, severity filtering, or use separate informational and blocking rule sets.

Best Practices

Do

• Start with curated packs such as p/security-audit before adding custom rules.
• Separate blocking and non-blocking rules to reduce developer friction.
• Tune path exclusions for vendor/, generated code, and large binaries.
• Map findings to ownership using repository CODEOWNERS and triage workflows.
• Version-control custom rules and test them in a staging pipeline before broad rollout.

Don't

• Do not block all findings on day one; begin with high-confidence rules like command injection or hardcoded secrets.
• Do not let developers embed tokens in pipeline YAML; use managed secrets.
• Do not scan unnecessary artifacts such as dependency caches or compiled assets.
• Do not maintain unreviewed custom rules; poor patterns can create false positives and alert fatigue.