CT-CVE

CT-CVE is the standalone CarrTech vulnerability service. It owns vulnerability feed sync, CVE/advisory catalog storage, package matching, vulnerability enrichment, and the CT Ops integration boundary.

This repository is being bootstrapped from the CT Ops migration plan. The first increment provides:

• A Go service entrypoint with /healthz.
• Configuration for the service, feed sync cadence, feed HTTP client, NVD, CISA KEV, Ubuntu OSV, and Red Hat Security Data sources.
• Initial PostgreSQL schema for CVE records, affected packages, and integration findings.
• Embedded database migrations applied on service startup.
• NVD, CISA KEV, Ubuntu OSV, and Red Hat Security Data feed sync that persists CVE metadata, known-exploited indicators, affected package metadata, and source status.
• Extracted distro package version comparison and matching logic.
• Docker and Compose definitions for local development.
• GitHub Actions CI running the Go test suite.
• Release automation that opens release-please PRs and publishes Docker Hub images when releases are created.
• A built-in operational status GUI, editable source configuration forms, and a JSON status endpoint with recent feed/API activity logs.
• A CT Ops connector API that accepts signed inventory snapshots, matches installed packages, and delivers signed finding batches back to CT Ops.

Local Development

Run the unit tests:

go test ./...

Start the service and database:

docker compose up --build

The service listens on http://localhost:8080 by default.

Open http://localhost:8080/status for the CT-CVE operational status page, or read http://localhost:8080/api/status for the same source health and configuration summary as JSON. The status surface is for feed configuration and observability only; host findings and customer-facing vulnerability reporting remain in CT Ops.

The first time the status page is opened, CT-CVE redirects to /signup so the initial Admin user can be created. After that user exists, /signup is no longer available and unauthenticated users are redirected to /login. Sessions are stored server-side, protected with an HttpOnly same-site cookie, and configuration changes require a per-session CSRF token.

To pair CT-CVE with CT Ops, set CT_CVE_PAIRING_USERNAME and CT_CVE_PAIRING_PASSWORD, start the container, then enter the CT-CVE URL and those initial credentials in CT Ops Administration. CT Ops generates the service tokens and CT-CVE stores them encrypted; operators do not need to manage token secrets or copy connector JSON for new installs.

During pairing, CT Ops sends CT-CVE the CT Ops callback URL used for finding delivery. That URL must be reachable from the CT-CVE container and should match the public CT Ops URL, for example https://ct-ops.example.com. If CT Ops uses a private or self-signed TLS certificate, mount a CA bundle into CT-CVE that contains the CT Ops certificate before pairing. A TLS failure at this stage can leave CT-CVE accepting inventory but unable to post matched findings back to CT Ops.

The status page can enable or disable the NVD, CISA KEV, Ubuntu OSV, and Red Hat Security Data sources, change their feed endpoints, adjust the NVD request delay, and set or clear the NVD API key. Saved source settings are stored in the CT-CVE database and are applied to the next scheduled feed sync cycle.

The Ubuntu OSV source downloads and parses Canonical's full OSV tarball. Keep CT_CVE_FEED_HTTP_TIMEOUT at 5m or higher for first syncs and slower links; the previous 30s timeout is usually too short for this feed.

Docker Resource Limits and Memory Usage

CT-CVE can use significant memory when importing, parsing, scanning, or updating CVE data. The Ubuntu OSV feed is especially large, so CT-CVE streams that feed and persists records in bounded batches controlled by CT_CVE_FEED_BATCH_SIZE. Keep the default 250 on 4-8 GiB hosts unless you have measured headroom.

Set Docker memory limits to protect the host. For an 8 GiB host running CT-CVE, PostgreSQL, and other services, start with a 3-4 GiB CT-CVE limit and keep the database outside that budget:

services:
  ct-cve:
    mem_limit: 4g
    memswap_limit: 4g
    restart: unless-stopped
    environment:
      CT_CVE_FEED_BATCH_SIZE: "250"

For Swarm or platforms that enforce deploy.resources, use:

services:
  ct-cve:
    deploy:
      resources:
        limits:
          memory: 4G
        reservations:
          memory: 1G

Plain Docker Compose does not enforce deploy.resources in every mode; use mem_limit and memswap_limit for non-Swarm Compose deployments.

Recommended starting points:

Deployment	Host RAM	CT-CVE limit	Swap	Notes
Small/test	4 GiB	2 GiB	4 GiB	Keep CT_CVE_FEED_BATCH_SIZE=250; consider CT_CVE_FEED_SYNC_ON_STARTUP=false after first import.
Medium	8 GiB	3-4 GiB	4 GiB	Recommended baseline for scheduled CVE processing.
Larger	16 GiB+	6-8 GiB	Optional	Increase batch size only after observing stable memory use.

Swap is optional but recommended on small 4-8 GiB VMs to absorb transient spikes. It is not a replacement for container limits or bounded feed processing.

Monitor live container memory with:

docker stats

Check recent Linux OOM kills with:

journalctl -k --since "7 days ago" | grep -Ei "oom|out of memory|killed process|ct-cve"

Recent feed sync outcomes and source configuration API changes are recorded as operational logs and shown on the status page and JSON endpoint. Logs report whether an NVD API key is configured but never include the key value.

CT Ops pushes inventory to CT-CVE at POST /api/v1/ct-ops/inventory-snapshots. CT-CVE verifies the signed service request, stores the org-scoped host and software inventory, immediately matches active packages against the persisted vulnerability catalog, and posts finding batches back to CT Ops at /api/integrations/ct-cve/v1/finding-batches.

Container Images

Release-please manages CT-CVE GitHub releases from Conventional Commit history. When a release is created from main, GitHub Actions builds the service image from this repository and publishes it to Docker Hub as a development image:

docker pull docker.io/carrtechdev/ct-cve:dev-0.1.0

After testing, run the Promote Docker image to production workflow with the same version number. Promotion retags the same digest as docker.io/carrtechdev/ct-cve:0.1.0 and docker.io/carrtechdev/ct-cve:latest. Only promotion updates latest, so customer installs that use latest receive production images.

Customer Install Bundle

Each GitHub release also publishes a downloadable ct-cve-single.zip bundle for single-host customer installs. Unzip it on a host with Docker and run:

unzip ct-cve-single.zip
cd ct-cve
./start.sh

The bundle includes docker-compose.yml, .env.example, start.sh, and upgrade.sh. On first run, start.sh creates .env, generates a local PostgreSQL password, starts the database and CT-CVE containers, and waits for the service health check. CT-CVE applies embedded database migrations during startup before the health check succeeds.

To upgrade an existing install, run:

./upgrade.sh

The upgrade helper backs up local install files, preserves .env, replaces the bundle-managed files, and restarts CT-CVE unless --no-start is passed.

Configuration

Variable	Required	Default	Description
CT_CVE_DATABASE_URL	Yes		PostgreSQL connection string for CT-CVE.
CT_CVE_HTTP_ADDR	No	:8080	HTTP bind address for the combined API/worker service.
CT_CVE_ENCRYPTION_KEY	Yes for pairing		Secret used to encrypt paired CT Ops token secrets at rest. The customer bundle generates this automatically.
CT_CVE_PAIRING_USERNAME	No		Initial username accepted by the CT Ops pairing flow.
CT_CVE_PAIRING_PASSWORD	No		Initial password accepted by the CT Ops pairing flow.
CT_CVE_FEED_SYNC_INTERVAL	No	6h	Periodic feed refresh interval used by enabled feed sources.
CT_CVE_FEED_SYNC_ON_STARTUP	No	true	Runs an immediate feed sync when the service starts.
CT_CVE_FEED_HTTP_TIMEOUT	No	5m	Per-request timeout for vulnerability feed HTTP calls. Keep this at 5m or higher for the Ubuntu OSV tarball.
CT_CVE_FEED_BATCH_SIZE	No	250	Maximum CVE records or affected package rows persisted per database batch during feed sync.
CT_CVE_NVD_ENABLED	No	true	Enables the NVD CVE source.
CT_CVE_NVD_BASE_URL	No	https://services.nvd.nist.gov/rest/json/cves/2.0	NVD CVE API endpoint.
CT_CVE_NVD_API_KEY	No		Optional NVD API key. Store this as a secret in deployed environments.
CT_CVE_NVD_REQUEST_DELAY	No	6s without an API key, 600ms with an API key	Minimum delay between NVD API requests.
CT_CVE_CISA_KEV_ENABLED	No	true	Enables the CISA Known Exploited Vulnerabilities source.
CT_CVE_CISA_KEV_BASE_URL	No	https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json	CISA KEV JSON feed URL.
CT_CVE_UBUNTU_OSV_ENABLED	No	true	Enables Canonical Ubuntu OSV advisory ingestion for Ubuntu package matching.
CT_CVE_UBUNTU_OSV_BASE_URL	No	https://security-metadata.canonical.com/osv/osv-all.tar.xz	Canonical Ubuntu OSV tarball URL.
CT_CVE_REDHAT_SECURITY_DATA_ENABLED	No	true	Enables Red Hat Security Data CSAF advisory ingestion for RHEL-compatible RPM package matching.
CT_CVE_REDHAT_SECURITY_DATA_BASE_URL	No	https://access.redhat.com/hydra/rest/securitydata/csaf.json	Red Hat Security Data CSAF list endpoint.
CT_CVE_CT_OPS_CONNECTIONS	No	[]	Legacy JSON array of CT Ops connector definitions with org-scoped inbound inventory tokens and outbound CT Ops callback token. Prefer CT Ops pairing for new installs.

The status page reports whether the NVD API key is configured, but it never returns the key value in HTML or JSON responses.

Example CT Ops connector configuration:

[
  {
    "name": "Primary CT Ops",
    "orgId": "org_123",
    "ctOpsBaseUrl": "https://ctops.example.com",
    "inventoryTokens": [
      {
        "id": "ctops-inventory",
        "secret": "replace-with-at-least-32-bytes-of-secret",
        "scopes": ["inventory:write", "connection:read"]
      }
    ],
    "ctOpsToken": {
      "id": "ctcve-outbound",
      "secret": "replace-with-at-least-32-bytes-of-secret",
      "scopes": ["findings:write", "connection:read"]
    }
  }
]

The same HMAC request-signing contract is used in both directions: Authorization: CT-ServiceToken <token-id>, X-CT-Timestamp, X-CT-Nonce, X-CT-Content-SHA256, and X-CT-Signature: v1=<base64url-hmac-sha256>.

Migration Status

This bootstrap now includes feed sync workers for the NVD, CISA KEV, Ubuntu OSV, and Red Hat Security Data catalogs, including persistence of CVE metadata, known-exploited CVE metadata, Linux affected package metadata, and source status, plus the first CT-CVE status GUI/API slice with editable source configuration and feed/API activity logs. The first CT Ops connector path is implemented for signed inventory ingestion, immediate matching, and signed finding delivery. CT-CVE subscription status from CT Ops remains outstanding migration work.