Integrate Verging Memory CI with GitHub Actions

This guide shows you how to add Verging Memory CI to a GitHub repository with the Verging Memory CI GitHub Action. This is the default integration for teams that use GitHub. If your repository is hosted elsewhere or your CI system cannot use GitHub Actions, use the API integration guide.

To interpret a delivered report, see Read a regression report.

Get your onboarding details

Open the private setup link in your onboarding email before you begin. The link has four steps, and two of them hand a prompt to an agent. The prompt for step 3 of 4, Set up the Action, contains only the integration and uses the account defaults; omit agent_setups and suites and set wiring_check: "true". After that wiring page is committed, the prompt for step 4 of 4, Connect your product, supplies these account values:

• Agent setups: the exact names to copy into agent_setups.
• Test suites: the exact request values to copy into suites, or an instruction to omit suites and select all suites chosen during onboarding that every named agent setup supports and has set up.

There are two kinds of agent setup, and your own step 4 prompt says which kind or kinds your account holds. Follow that prompt where it differs from this page.

An agent setup that reaches your product directly is fitted with an adapter. Its step 4 prompt supplies the product-specific adapter contract and one more account value:

• Test endpoint: the exact non-production HTTPS origin you create, then submit and verify with its scoped credential in step 4 of 4, Connect your product. Never use a production deployment for Memory CI testing; a release cannot override the verified origin.

A documentation-led agent setup is one Verging Labs sets up from instructions rather than from a connection. It has no adapter contract and no address of its own: there is nothing to build and no endpoint to submit for it. Where those instructions are your product's own published documentation, its step 4 prompt has the agent confirm the agent setup and the documentation address on record for it. Where they are a private file Verging Labs already holds, there is no public address and nothing to confirm, and Verging Labs starts the first release rather than your agent. Either way the prompt asks for your product key when that setup needs one, and your own step 4 prompt says which case you are in.

Your Verging Memory CI API key is issued at step 2 of 4, Add your Verging API key to GitHub, and shown once. If you no longer have the setup link or key, contact Verging Labs before continuing.

What gets tested

For an agent setup that reaches your product directly, Verging Labs tests whatever your product serves at your test endpoint when a release starts. Deploy the version you want tested before this job starts. A typical CI pipeline builds the release, deploys it to the test endpoint, and then starts the Verging Memory CI job.

For a documentation-led agent setup there is no test endpoint. Verging Labs sets your product up from instructions, your own published documentation or a private file it already holds, and tests that set-up, so nothing of yours has to be deployed for the tests to reach it.

vendor_version is the label shown in the report. For an agent setup that reaches your product directly it does not deploy or verify the version at your endpoint, so make sure the label matches the deployed version: if the job starts before the new version is live, the report will describe the old version under the new label. For a documentation-led agent setup the label names the report and selects nothing; there is no deployment of yours for it to point at.

A standard release returns a report when testing finishes. A final report follows if grading corrections are needed. Storage-only releases are delivered once. The same applies to a wiring check.

Private setup file

File-based setups for documented integrations are coming soon. You can upload a private instructions file to POST /v1/setup-files with your API key; it stays on your account. A documented release that names its digest is declined until this path is available.

Activation ID

When activation is available for your account, use the activation_id value your report gives next. An unused value starts an activation and is billed as one. A value you used before tests this release on that setup at the release price. A value we did not issue, one for another setup, or one whose activation is already underway is refused with a plain explanation. If you send no value, your release uses your newest setup. A selected suite that has never been tested on that setup is activated there and billed. Starting an activation without Onboarding adds Onboarding automatically. Your report names the value used for it, your earlier values with their dates, and the value to send next: a new one, with the most that activation can cost for the test suites the quote covers, or, when Verging Labs did not finish setting an activation up, the value you have already paid for, which is never charged again. A release that sets up test suites outside that list is priced on its own.

The GitHub Action

The example below is a permanent release workflow, for after onboarding: it pushes on main. Neither step of onboarding installs it. For step 3 of 4, Set up the Action, use that step's generated prompt: omit agent_setups and suites, set wiring_check: "true", and stop once its committed wiring page is verified. Step 4 of 4, Connect your product, keeps that same verification-branch workflow; its prompt says whether the workflow changes, and when it does, it changes once, in one commit, and the prompt names exactly which lines change and when. Choose a permanent workflow like the one below only after that first report, replacing the example names and suite values with your own.

name: Verging Memory CI
on:
  push:
    branches: [main]
  workflow_dispatch:
    inputs:
      fetch_only_release_id:
        description: "Fetch an existing release's report instead of submitting"
        required: false
        default: ""
permissions:
  contents: write
  checks: write
  pull-requests: write
concurrency:
  group: verging-memory-ci-${{ github.ref }}
  cancel-in-progress: false
jobs:
  memory-ci:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          sparse-checkout: "Verging Memory CI"
          sparse-checkout-cone-mode: true
      - uses: verginglabs/memory-ci-action@v1
        with:
          api_key: ${{ secrets.VERGING_API_KEY }}
          agent_setups: "Claude Code Opus 5,Hermes GPT-5.6 Luna"
          product_name: "Acme Recall"
          suites: core-recall,preference-adherence,truth-maintenance
          fetch_only_release_id: ${{ inputs.fetch_only_release_id }}

Set the repository secret VERGING_API_KEY to the API key issued during onboarding. Never paste the key directly into the workflow or commit it to the repository. Nothing else needs the key. The free wiring check belongs to the onboarding workflow, which sets wiring_check: "true": that is the workflow your coding agent adds and pushes at step 3 of 4, and its first push performs that free wiring check. The example above is the later release workflow and carries no such input, so each of its pushes submits a release; while a test suite that release names is not set up yet, it is declined and the Action submits the free wiring check in its place.

Set agent_setups to one or more agent setup names from your onboarding page, separated by commas. One name is valid. One Action invocation submits one release and produces one combined report with a section for each agent setup.

Set suites to the comma-separated request values from your onboarding page. Scoping a release to selected suites is a normal way to test different sub-repositories or kinds of pushes. If your onboarding page says to omit suites, remove the line to select all suites chosen for your account and set up on every agent setup in agent_setups. Onboarding is available on Claude Code setups; Hermes agent setups are coming soon. When several setups are named, a suite set up on some but not all of them is left out, and the receipt names the setup that has it and the release that covers it. See API setup for every suite value.

The workflow_dispatch block supplies GitHub's Run workflow button and the fetch_only_release_id field used for recovery.

What the Action does

For each push or manual start, the Action:

1. Submits one release. It performs the free wiring check instead when the workflow sets wiring_check: "true", and when a release is declined because a test suite it names is not set up yet. A manual start that carries fetch_only_release_id submits nothing and fetches that release's report instead.
2. Commits a report to your repository when it is ready, records a release that is not ready yet as pending, or commits the wiring page when one is made.
3. Posts a GitHub status check named "Verging Memory CI" on the tested commit.
4. On pull requests, maintains one comment with the verdict and report link.

The GitHub status check is success when the Release verdict is Ready and neutral otherwise. On a first report, which records a baseline instead of a verdict, the status check and the pull request comment read "Verging Memory CI: baseline recorded". It does not block your pipeline. To create a blocking gate, read latest/diff.json: gate on release_verdict == "not_ready" for the report's call, or on verdict and cost_verdict separately. A first report's release_verdict is null with release_verdict_reasons ["first_report"], so such a gate lets it through.

The Action commits the report when it is ready, or the pending record when it is not, to the report folder on the branch the job ran on and pushes it with the workflow's own token, so that branch must accept pushes from this workflow. If the push is refused (a ruleset or branch protection that does not let the workflow push, or a competing push that keeps winning), the job fails with an error that names the branch and the permission to allow; the Action writes no other branch and never opens a pull request into your default branch on its own. To have a refused report commit delivered on the branch verging-memory-ci/reports with a pull request titled "Verging Memory CI reports" into your default branch instead, set the input fallback_pull_request: "true". It is off by default and never applies to a wiring check, whose page must land on the branch the job ran on.

The sparse checkout gives the Action the report folder and files at the repository root, but not your source directories. The permissions allow it to commit the report, post the status check and comment, and, with fallback_pull_request on, open the pull request.

Recovery after a failed job

Each accepted submission creates a new release. If a job fails after submission for any reason other than failed testing, do not submit the same version again. Copy the release_id from the receipt in the job log, select Run workflow, and enter it under fetch_only_release_id. The Action fetches that release's report and commits it when ready, without submitting another release.

When testing failed, the release is voided and not billed; push again to start a new release.

By default the job does not wait for the report: it checks once whether the report is ready, and then ends. If it is ready, the job commits it. If testing has already failed, the job fails. Otherwise the job ends green with the verdict Pending, and the report is committed once it is ready by the next job on any push, or by running the Action with mode: sync. Set poll_timeout_minutes to a number of minutes to have the job wait for the report instead (a waiting job spends GitHub Actions minutes).

After editing a workflow, use Run workflow for the next attempt. GitHub's Re-run all jobs command uses the workflow file from the original attempt.

Collect reports that land later

At the start of a later release, the Action collects any reports now ready for releases on record in the report folder: the report of a release that was still pending, and the final report of a release still on its preliminary report. To collect them on demand or on a schedule without submitting a release, add this second workflow.

Add this second workflow file next to the first. It uses the same VERGING_API_KEY secret:

name: Verging Memory CI sync
on:
  workflow_dispatch: {}
  # Optional: also collect ready reports between releases, every 3 hours.
  schedule:
    - cron: "17 */3 * * *"
permissions:
  contents: write
  checks: write
  pull-requests: write
concurrency:
  group: verging-memory-ci-${{ github.ref }}
  cancel-in-progress: false
jobs:
  sync-finals:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          sparse-checkout: "Verging Memory CI"
          sparse-checkout-cone-mode: true
      - uses: verginglabs/memory-ci-action@v1
        with:
          api_key: ${{ secrets.VERGING_API_KEY }}
          mode: sync

A sync job submits nothing. It commits the report of each pending release that is now ready, replaces preliminary reports with available final reports, and leaves reports that are already final unchanged.

Inputs

api_key (required) : Your API key, supplied through the VERGING_API_KEY repository secret.

agent_setups : One or more agent setup names from your onboarding page, separated by commas or new lines. One release produces one combined report.

mode : release by default. sync submits nothing and collects the reports now ready for releases already in the report folder, pending or preliminary.

suites : Comma-separated suite values from your onboarding page. Omit this input to select all suites chosen for your account and set up on every agent setup in agent_setups. A suite not yet set up on every named setup is left out; the receipt says which setup has it and how to cover it.

wiring_check : "false" by default. Set it to "true" for one job to perform the free wiring check instead of submitting a release.

vendor_version : The release identifier (e.g. 2.4.1). Defaults to the root VERSION file, or otherwise the short commit SHA.

product_name : The product name used in the report title (e.g. Acme Recall). Free-text inputs (this, vendor_version and agent-setup names) accept letters, digits, spaces, dots, underscores, plus signs and hyphens only; no parentheses, quotes or other punctuation. Agent-setup names follow the same rule, except that a name may end in " (documented integration)" exactly as your account records it.

activation_id : The activation value your report gives next, when activation is available for your account. Empty by default, which sends none. The Action reads nothing into it: the value is passed to Verging Labs unchanged, and a wiring check never carries one. The Activation ID section above says what each kind of value does.

fetch_only_release_id : An existing release to fetch without submitting another release. Do not combine it with mode: sync.

poll_timeout_minutes : Minutes to wait for a report. Defaults to 0.

api_base : API base URL. Defaults to https://ci.verginglabs.com.

folder : Report folder at the repository root. Defaults to Verging Memory CI.

Outputs

release_id : The release whose report the job fetched.

verdict : Ready, Not ready with the reason, or Pending. When the job performs a wiring check, this output is Wiring check.

report_path : The repository-relative path to the committed REPORT.md.

The wiring check

With the GitHub Action, the first push performs the wiring check on its own: the job commits its page under Verging Memory CI/releases/<date>-<version>-wiring-check/, ends green, and its outputs say Wiring check in place of a verdict. It executes no product tests or model sessions, so a complete green job within seconds is normal. When your wiring page names an activation id, every push does the same until you send it, and sending it on your next release starts the set-up of the one agent setup that id was issued for. When it names none, Verging Labs sets your suites up and tells you when they are ready; until then every push does the same instead of a release, and after that pushes submit real releases. To repeat it at any time, set the input wiring_check: "true" for one job. With any other CI system, submit one wiring check through the API yourself, then commit its files through the same path your integration uses for regression reports.

A wiring check does not test your product and does not require an activated environment. Its report uses "format": "wiring-check/v1" and contains no release verdict. Do not gate CI on a wiring report.

Report folder

The Action commits each ready release report under:

Verging Memory CI/
  latest/
    REPORT.md
    diff.json
    release.json
    evidence/
  releases/
    <release_id>/
      REPORT.md
      diff.json
      release.json
      evidence/

latest/ is the stable path for humans and CI. The matching directory under releases/ preserves the same files for that release. Git history preserves a preliminary report after the final report replaces it.

For field definitions and report stages, see Read a regression report. For raw requests, status polling, errors and retry behavior, see API setup. Account setup names and activation are covered in Agent setups. Plan arithmetic is in Billing.

Agent handoff and feedback

The generated setup prompt supplies an onboarding API base and a short-lived, stage-scoped agent key. Use that key only as the bearer value on the two routes below:

Authorization: Bearer <stage-scoped-agent-key>
Content-Type: application/json

Never print, log or commit that key. It authorizes only the current onboarding flow and stage; it is not the product credential or the VERGING_API_KEY.

Submit agent feedback

At the end of each of those two steps, POST <onboarding-api-base>/agent-feedback with:

{
  "stage": "action",
  "outcome": "success",
  "problems": [],
  "unclear": [],
  "improvements": [],
  "wished_for": []
}

• stage: exactly action during Action wiring or test-endpoint during product integration. It must match the stage-scoped key.
• outcome: a short lowercase label. Use success, blocked or failed as appropriate.
• problems: concrete failures encountered, with the operation and a sanitized error when useful.
• unclear: instructions or expected behavior that required guessing.
• improvements: specific changes that would make this step faster or more reliable.
• wished_for: missing documentation, automation, access, diagnostics or tooling.

The last four fields are arrays of strings; send [] when there is nothing to report. Never include credentials, tokens, repository contents or customer data. A successful submission returns HTTP 202 with {"accepted":true,"feedback_id":"..."}. Invalid data returns 400; an invalid or expired agent key returns 401. A feedback failure does not block onboarding.

Test deployment and release ordering

Before creating a deployment, identify the repository's existing non-production test deployment and the path that deploys to it. Reuse it when it can safely host the required adapter without affecting production. If no suitable test deployment exists, create the smallest isolated non-production deployment using the repository's established deployment tooling and conventions. Never modify, replace or target production.

This is how a PERMANENT release workflow is wired, after onboarding. During onboarding the workflow stays on the verification branch, started by a push to it and nothing else, and its prompt says when that push happens; nothing below asks you to add a trigger during onboarding.

Find or create a reliable successful-deployment signal, then make Verging Memory CI causally downstream of it. Prefer the repository's existing deployment job, deployment-status event or post-deploy hook. If none exists, add the smallest native post-deploy signal supported by its deployment system. The signal must identify the immutable release that was deployed, and vendor_version must identify that same release. Do not start Memory CI independently from the source push and do not approximate deployment readiness with a fixed sleep or polling loop.

If creating the signal requires deployment-account access that is unavailable, complete the repository-side wiring and ask the user for the single missing configuration step. Execute the focused adapter contract tests and live smoke cycle required by the generated prompt. Do not wait for an unrelated upstream full test matrix unless the changes affect it.

Submit and verify the product connection

During product integration, POST <onboarding-api-base>/agent-connection with:

{
  "endpoint": "https://memory-ci-staging.example.com",
  "credential": "<scoped-product-credential>"
}

• endpoint: the exact non-production HTTPS origin serving the adapter. Send only its scheme and host, with no route, query or fragment.
• credential: the scoped bearer credential accepted by that deployment. Keep it in a secret environment variable and never print, log or commit it.

Do not ask the user to copy either value into the onboarding page. Verging stores the credential securely and verifies the live adapter behind the scenes. Success returns HTTP 200 with verified:true, the normalized endpoint, a correlation_id, and a next instruction. A live verification failure returns HTTP 422 with verified:false, correlation_id, failed_operation, http_status, error_class, a bounded diagnostic, and retry:true. Fix the adapter or deployment and submit the same request again. Invalid input returns 400, an invalid or expired key returns 401, and a request before the product-integration stage is open returns 409.

After verified:true, do the rest of the work your generated prompt names before you hand over: on this call the response's next field says what is left. Only then tell the user to refresh the onboarding page and click Continue. The button unlocks after the refresh; the page does not advance automatically.

Help, legal and privacy

For integration help, write to contact@verginglabs.com and include the release_id, the request without credentials, the HTTP status, and the response body.

Use of Verging Memory CI is governed by the Terms of Service. Privacy questions can be sent to Matthew Overing at legal@verginglabs.com.