Skip to main content

Suites and Targets

CertOps models your AI applications around the Testing Lifecycle. Whether you are testing a simple monolith or a complex multi-component pipeline, CertOps represents this architecture using Suites and Targets.

The Target (Component Under Test)​

A Target represents a specific external service or endpoint being audited.

  • Definition: An HTTP endpoint that accepts inputs and returns AI-generated outputs.
  • Context: A target can literally be an entire "Monolithic Agent", or it could just be a single "Microservice" step, such as just the Retriever element in a RAG pipeline.
  • Attributes: A target consists of an ID, a Name, a relative Endpoint path, and a RequestTemplate to dictate how tests are transmitted.

The Suite (The System Definition)​

A Suite is a logical collection of one or more Targets that together comprise the AI System you want to certify.

  • Identity: e.g., CustomerSupportBot-Suite
  • Structure: Simply a list of Targets.
  • Use Cases:
    • Single-Target System: You are testing a simple "Chatbot" API endpoint. Your Suite contains exactly 1 Target.
    • Multi-Target System: You are testing a RAG pipeline. You want to independently certify the Retriever API (Did it find the right documents?) and the Generator API (Did it answer the question well given the documents?). Your Suite contains 2 Targets, running in the same pass.

Submitting a Run​

A Run (or The System Audit) is an execution instance of a Suite against a specific version of its defined Targets.

  • Orchestration: The CertOps Engine executes tests for ALL targets defined in the Suite (either sequentially or in parallel).
  • Aggregation: The final system "Certificate" is issued only if ALL Targets pass their respective Quality Gates. If your generator passes, but your retriever fails... the system is rejected.