Skip to main content

CLI Command Reference

Now that you understand how to execute evaluations in automated CI/CD pipelines and directly from your local machine via the Hybrid Bridge, here is the complete reference for the certops CLI.

The CLI is a Python package:

pip install certops-cli

Global option​

  • --api-url: the CertOps API base URL (env: CERTOPS_API_URL). Precedence is flag → env → the URL stored at login → https://api.certops.ai.

Exit codes​

certops run and certops status follow a strict contract: 0 Certified, 1 Rejected (a blocking gate failed), 2 System Error (no verdict — timeout, API down, auth, or Ctrl+C).

Authentication​

certops login​

Authenticates the user and caches the session token and secure tunnel configuration required for Local Bridge evaluations.

The username is a tenant\email pair:

certops login --username 'acme\alice@acme.com'

certops whoami / certops logout​

whoami prints the active session (user, API URL, relay prefix); logout clears the cached credentials in ~/.certops.

certops whoami
certops logout

Execution & Lifecycle​

certops run​

The primary execution command. It manages the full lifecycle of a certification run: parsing the manifest, spinning up local bridges (if applicable), triggering the SaaS evaluation engine, and polling for the final verdict.

Arguments:

  • --manifest, -f: Path to your certops.yaml manifest.
  • --host: Target host mapping. Accepts explicit mappings (target_id=url), Local Bridge ports (local:8080), or a bare URL/local:PORT that applies to every target. This option can be provided multiple times.
  • --tag: Tag the run at trigger time (e.g. prod) so it becomes a baseline — saves a follow-up certops tag call.
  • --notes: Free-text note attached to the run.
  • --dataset, -d: Overrides the dataset ID defined within the manifest for all targets (useful when testing against a quick subset before running a massive prod dataset).
  • --timeout: Maximum seconds to wait for the run to complete before aborting the CLI watcher (default: 1800). Exceeding it exits 2.
  • --no-wait: Triggers the run on the SaaS and immediately returns the Run ID in a fire-and-forget fashion without polling the UI.

Example: Standard CI/CD Run

certops run -f ./certops.yaml --host "https://staging.acme.com"

Example: Multi-Target Hybrid Run

certops run -f ./certops.yaml --host "retriever=https://prod.acme.com" --host "generator=local:8080"

certops status​

Watches the status of an existing, asynchronous certification run. Ideal for reattaching to a run triggered with --no-wait or checking on long-running evaluations. Output streams the Live UI directly to the terminal.

Arguments:

  • --timeout: Maximum seconds to wait for the run to complete (default: 1800).
certops status <run_uuid>

certops runs​

Lists recent certification runs — run number, suite, verdict, tag, source, and run ID.

certops runs --limit 20

Certificates​

Once a run is Certified, you can issue a signed Certificate of Conformity — the auditable artifact behind the "Trust, But Verify" badge.

certops certificate generate​

Generates (or fetches) a certificate for a run number (the short #42 id, not the UUID). The run must be Certified.

certops certificate generate 42

certops certificate show​

Fetches the existing certificate for a run UUID.

certops certificate show <run_uuid>

certops certificate verify​

Publicly verifies a certificate by ID — no login required. This is the endpoint behind a public Trust Badge.

certops certificate verify <certificate_id>

Asset Management​

certops upload​

Uploads a dataset file (CSV or JSON) directly to the CertOps Media Hub and returns the unified Dataset ID to be referenced in your certops.yaml.

certops upload data/golden_set.csv

Output will contain the Dataset ID, row/column count, and inferred schema types.

certops tag​

Tags a historically completed certification run. This is extremely useful for manually marking an approved candidate run as the new prod baseline that future regression comparisons resolve against.

certops tag <run_uuid> prod

You can also tag at trigger time with certops run --tag prod.