Repository navigation
VAP15-77 docs: rename the PR check status to Vapi Simulations - #1330
Merged
Merged
Conversation
Follows VapiAI/gitops#82, which renames the status the gitops PR check posts from Vapi Evals to Vapi Simulations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scott-lowe-vapi
marked this pull request as ready for review
October 14, 2026 04:16
Lightsage docs evalsLightsage could not queue docs evals for this PR. Docs URL: https://vapi-preview-01a138a0-f178-7470-a1d2-36ea4c1e79ed.docs.buildwithfern.com |
Contributor
|
🌿 Preview your docs: https://vapi-preview-01a138a0-f178-7470-a1d2-36ea4c1e79ed.docs.buildwithfern.com |
stephenvapiai
approved these changes
Oct 14, 2026
scott-lowe-vapi
added a commit
to VapiAI/gitops
that referenced
this pull request
Oct 14, 2026
…tatus (#82) ## Value **V.A.L.U.E. tier:** small. It changes the name of a commit status that customers may require in branch protection. Not micro: it changes behavior, not just intended behavior restored. - **Problem:** the PR check posts a status called `Vapi Evals`, but every run is a simulation, and the public docs now separate Simulations (whole conversations) from Evals (single decisions). Next to "4 of 4 simulations passed", the name blurs exactly that line. Dogfooding also showed a single-target repo gets an overall status plus a per-target status that say the same thing and link the same run: three Vapi rows on a PR where two would do. - **Who it affects:** every repo that turns on PR checks; most have one target. - **What changes:** - **The status is now `Vapi Simulations`,** and per-target statuses are `Vapi Simulations / <check> / <target>`. The job summary heading matches. - **With one target, an `--all` run posts only the overall status.** It's pending with the run link while the run is going, then the verdict, still linking the run. With several targets, each still gets its own status linking its run, and the overall status links the workflow run. Running one named check by hand is unchanged. - **Docs:** the README and PR checks guide use the new name, with a migration note: a repo that already requires `Vapi Evals` must require `Vapi Simulations` after merging the update, or PRs will wait on a status that no longer posts. **Alignment needed:** Agent Loop (PAL-608 specifies `Vapi Evals` for the planned shared GitHub Action). Merge after they agree, or update PAL-608 to match, so the two don't diverge. The public docs page (docs.vapi.ai/config-as-code) also names `Vapi Evals`: VapiAI/docs#1330 renames it there, and should merge after this PR. ## Evidence of value | Repo with | Vapi rows on a PR, before | After | | --- | --- | --- | | 1 target | 3 (job, overall, 1 per-target) | 2 (job, overall linking the run) | | N targets | N + 2 | N + 2, unchanged | - **One target:** `check-cmd.test.ts` runs a live `--all` against a stub. It posts only `Vapi Simulations` (pending, then success), both linking the run. - **Several targets:** a second check makes two targets. Each gets a pending and a success status, and the overall status links the workflow run. - **Manual named run:** the existing test still sees per-target statuses only. ## Testing plan - `npm test` (547) and `npx tsc --noEmit` pass. - **After merge:** pull the update into the dogfood repo and check the PR shows `Vapi Simulations`, with two rows for the `goat` and `lebron` checks (two targets). - **Known flaky test (not introduced here):** a check-cmd status test failed once in about 28 full runs and didn't reproduce. A similar one-off happened on an earlier branch. Worth a dedicated look; not blocking. Refs TEST-141 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Vapi Simulationsinstead ofVapi Evals, in both places it appears ("What the repository gives you" and "Test every pull request").evals-quickstart,evals-advanced) also say "Vapi Evals", but there it means the product, so they're unchanged.Merge after VapiAI/gitops#82, so the docs never name a status the repo doesn't post yet.
Testing Steps
fern/(git grep "Vapi Evals"finds only the Evals product pages)./config-as-codeand check both sentences read correctly.🤖 Generated with Claude Code