Skip to content

Disable vitest 5 tracker warning when running under codspeed - #99

Merged
GuillaumeLagrange merged 2 commits into
mainfrom
cod-3892-vitest-5-getter-tracking-adds-overhead-to-codspeed
Oct 16, 2026
Merged

GuillaumeLagrange merged 2 commits into
mainfrom
cod-3892-vitest-5-getter-tracking-adds-overhead-to-codspeed

Conversation

@GuillaumeLagrange

@GuillaumeLagrange GuillaumeLagrange commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #98

GuillaumeLagrange and others added 2 commits October 15, 2026 14:20
With FORCE_COLOR set, as in an interactive terminal, console.log colors
the numbers it prints, so the `done <before> <after>` regex never matched
and both optimization tests failed. Print a single interpolated string,
which is never colored.

Co-Authored-By: Claude <noreply@anthropic.com>
…dspeed

Vitest 5 wraps every module export getter with a diagnostic counter (a Map
lookup and write per access) unless benchmark.suppressExportGetterWarnings
is set. That counter runs inside the measured code, inflating results: a
bench reading an export 100k times measures 109M instructions on Vitest 5
versus 38M on Vitest 4.

Default the option to true when the plugin is active, keeping any value
the user sets explicitly. Plain `vitest bench` runs are unaffected. With
this, the same bench measures the same instruction count on Vitest 4
and 5.

Drop the import hoisting from the benches and examples: it only worked
around the tracker.

Closes COD-3892
Closes #98
Co-Authored-By: Claude <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable defects were found.

Summary

Sets suppressExportGetterWarnings to true by default for Vitest 5 runs under CodSpeed.

  • CodSpeed suppresses Vitest 5 export getter warnings by default.
  • Benchmark examples call functions through direct imports.
  • Optimization test snippets print V8 statuses in one line.

No actionable issues found. Tests were not run.

Reviews (1) · Last reviewed commit: "fix(vitest-plugin)!: disable vitest 5 ex..." · Reviewed by Greptile

@codspeed

codspeed Bot commented Oct 15, 2026

Copy link
Copy Markdown

Merging this PR will regress 6 benchmarks

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 14 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

ℹ️ 25 benchmarks have no flame graph

The flamegraph could not be generated because too much time was spent in runtime internals, making the visualization not useful for performance analysis.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 10 improved benchmarks
❌ 6 regressed benchmarks
✅ 248 untouched benchmarks
⏩ 56 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ WallTime iterative 516 ns 612 ns -15.69%
❌ WallTime short body 1.5 µs 1.8 µs -13.7%
❌ Simulation iterative 14.1 µs 16.3 µs -13.45%
❌ WallTime switch 2 84 ns 96 ns -12.5%
❌ WallTime iterative 528 ns 600 ns -12%
❌ Simulation iterative 16.9 µs 18.9 µs -10.44%
⚡ ⚠️ Simulation recursive fibo 10 1,372.6 µs 300.1 µs ×4.6
⚡ ⚠️ Simulation test_recursive_fibo_10 722.8 µs 280.7 µs ×2.6
⚡ Simulation short body 242.6 µs 155 µs +56.52%
⚡ Simulation short body 242.5 µs 155.1 µs +56.34%
⚡ WallTime test_iterative_fibo_10 144 ns 120 ns +20%
⚡ WallTime switch 1 84 ns 72 ns +16.67%
⚡ Simulation long body 639.6 µs 549.7 µs +16.36%
⚡ Simulation long body 639.6 µs 550.1 µs +16.26%
⚡ WallTime test_iterative_fibo_10 120 ns 108 ns +11.11%
⚡ WallTime test_recursive_cached_fibo_10 780 ns 708 ns +10.17%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cod-3892-vitest-5-getter-tracking-adds-overhead-to-codspeed (b943512) with main (7f4107e)

Open in CodSpeed

Footnotes

  1. 56 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@GuillaumeLagrange
GuillaumeLagrange merged commit b943512 into main Oct 16, 2026
48 of 49 checks passed
@GuillaumeLagrange
GuillaumeLagrange deleted the cod-3892-vitest-5-getter-tracking-adds-overhead-to-codspeed branch October 16, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Vitest 5 getter tracking adds overhead to CodSpeed measurements

2 participants