CI/CD pipelines using GitHub Actions and monitoring with Prometheus support automated software delivery and system observability. GitHub Actions enables continuous integration and continuous deployment by automating tasks such as code testing, building, and deployment directly from version control.
Prometheus is a monitoring and alerting system that collects and stores metrics from applications and infrastructure, helping teams track system health and performance over time.
GitHub Actions Basics
GitHub Actions automates workflows directly in your repository, handling CI/CD from code push to deployment. Workflows run on events like pushes or pull requests, using YAML files in .github/workflows/.
Key components include jobs, steps, and actions—reusable units from the marketplace. For web APIs, this means automated testing of endpoints and database migrations.
1. Events: Triggers like push or pull_request.
2. Runners: GitHub-hosted (Ubuntu, Windows) or self-hosted for custom needs.
3. Matrix strategy: Tests across Python versions or OS for comprehensive coverage.
Setting Up CI Pipeline
Start your CI pipeline by creating a workflow file, say ci.yml, to lint, test, and build your web API.
This ensures code quality before merging. Use Python actions for FastAPI projects.
1. Checkout code: actions/checkout@v4.
2. Set up Python: actions/setup-python@v5 with matrix for versions 3.9-3.12.
3. Install dependencies: pip install -r requirements.txt.
4. Run tests: pytest --cov=api/.
5. Build Docker image: Optional for containerized APIs.
Example YAML snippet
name: CI
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.9', '3.10']
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: pip install pytest
- run: pytestImplementing CD Pipeline
CD extends CI to deploy your API to staging or production, using environments for approval gates.
Deploy to platforms like Heroku, AWS, or Kubernetes. Secrets store API keys securely via GitHub settings.
1. Trigger on main branch push.
2. Build and push Docker image to registry (e.g., Docker Hub).
3. Deploy: Use kubectl for K8s or aws ecs update-service.
4. Notify on success/failure via Slack action.
Best practice: Use OIDC for AWS authentication, avoiding long-lived keys.
Prometheus Fundamentals
Prometheus is an open-source monitoring system using time-series data for metrics like CPU, latency.
It scrapes HTTP endpoints (e.g., /metrics) every 15s by default. Pairs with Grafana for dashboards.
1. Pull model: Server fetches metrics—no agents needed.
2. PromQL: Query language, e.g., rate(http_requests_total[5m]).
3. Recording rules: Precompute aggregates for efficiency.
For APIs, instrument with prometheus-client in Python: @prometheus.middleware.
Integrating Prometheus with Pipelines
Expose metrics in your web API and deploy Prometheus via GitHub Actions for end-to-end observability.
Actions can deploy Prometheus/Grafana stacks or validate configs with promtool. Monitor workflows themselves using GitHub API scrapers.
1. Add /metrics endpoint in FastAPI: Use prometheus-fastapi-instrumentator.
2. In CD workflow: Deploy Prometheus config targeting your API service.
3. Scrape workflow metrics: Use exporters like github-actions-exporter.
Alert rules: e.g., build_duration_seconds > 300 for slow pipelines.
Example: Deploy Prometheus Docker in workflow
- name: Deploy Prometheus
run: |
docker run -d -p 9090:9090 prom/prometheus --config.file=prometheus.ymlConfig scrapes API at your-api:8000/metrics.
Monitoring Workflow Metrics
Track GitHub Actions runs: duration, failures, runner usage with custom exporters.
Prometheus scrapes GitHub API via tokens for metrics like runner_nb_idle. Visualize in Grafana: job success rate, MTTR
Use GitHub's built-in metrics tab, enhanced by Prometheus federation.
Best Practices and Examples
Secure pipelines: Pin action versions (e.g., @v4), use minimal permissions.
For full-stack: Test frontend/backend integration in matrix jobs. Scale with self-hosted runners for heavy ML models.
1. Cache pip/ Docker layers to cut times 50%.
2. Use artifacts for test reports.
3. Canary deploys: Roll 10% traffic first.
Real example: Node.js API (adaptable to Python) pushes metrics to Pushgateway in workflow.
