Repository navigation
Disable vitest 5 tracker warning when running under codspeed - #99
GuillaumeLagrange merged 2 commits into
Conversation
With FORCE_COLOR set, as in an interactive terminal, console.log colors the numbers it prints, so the `done <before> <after>` regex never matched and both optimization tests failed. Print a single interpolated string, which is never colored. Co-Authored-By: Claude <noreply@anthropic.com>
…dspeed Vitest 5 wraps every module export getter with a diagnostic counter (a Map lookup and write per access) unless benchmark.suppressExportGetterWarnings is set. That counter runs inside the measured code, inflating results: a bench reading an export 100k times measures 109M instructions on Vitest 5 versus 38M on Vitest 4. Default the option to true when the plugin is active, keeping any value the user sets explicitly. Plain `vitest bench` runs are unaffected. With this, the same bench measures the same instruction count on Vitest 4 and 5. Drop the import hoisting from the benches and examples: it only worked around the tracker. Closes COD-3892 Closes #98 Co-Authored-By: Claude <noreply@anthropic.com>
[Medium impact] The PR appears safe to merge; no actionable defects were found. SummarySets
No actionable issues found. Tests were not run. Reviews (1) · Last reviewed commit: "fix(vitest-plugin)!: disable vitest 5 ex..." · Reviewed by Greptile |
Merging this PR will regress 6 benchmarks
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | iterative |
516 ns | 612 ns | -15.69% |
| ❌ | WallTime | short body |
1.5 µs | 1.8 µs | -13.7% |
| ❌ | Simulation | iterative |
14.1 µs | 16.3 µs | -13.45% |
| ❌ | WallTime | switch 2 |
84 ns | 96 ns | -12.5% |
| ❌ | WallTime | iterative |
528 ns | 600 ns | -12% |
| ❌ | Simulation | iterative |
16.9 µs | 18.9 µs | -10.44% |
| ⚡ |
Simulation | recursive fibo 10 |
1,372.6 µs | 300.1 µs | ×4.6 |
| ⚡ |
Simulation | test_recursive_fibo_10 |
722.8 µs | 280.7 µs | ×2.6 |
| ⚡ | Simulation | short body |
242.6 µs | 155 µs | +56.52% |
| ⚡ | Simulation | short body |
242.5 µs | 155.1 µs | +56.34% |
| ⚡ | WallTime | test_iterative_fibo_10 |
144 ns | 120 ns | +20% |
| ⚡ | WallTime | switch 1 |
84 ns | 72 ns | +16.67% |
| ⚡ | Simulation | long body |
639.6 µs | 549.7 µs | +16.36% |
| ⚡ | Simulation | long body |
639.6 µs | 550.1 µs | +16.26% |
| ⚡ | WallTime | test_iterative_fibo_10 |
120 ns | 108 ns | +11.11% |
| ⚡ | WallTime | test_recursive_cached_fibo_10 |
780 ns | 708 ns | +10.17% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing cod-3892-vitest-5-getter-tracking-adds-overhead-to-codspeed (b943512) with main (7f4107e)
Footnotes
-
56 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Fixes #98