Skip to content

Improve the GC harness Summary output#517

Merged
eightbitraptor merged 1 commit into
mainfrom
mvh-gc-handler-improvemnets
Jun 18, 2026
Merged

Improve the GC harness Summary output#517
eightbitraptor merged 1 commit into
mainfrom
mvh-gc-handler-improvemnets

Conversation

@eightbitraptor

Copy link
Copy Markdown
Member

This PR adds more information about GC counts and timing to the summary output of the GC handler to avoid having to scroll up to the individual tables and infer information. It adds:

  • Ratios of the average total time spent marking/sweeping. So the ratio is mean(total_control_time) / mean(total_experiment_time). Above 1 represents a speed-up in the experiment branch.
  • Ratios of the average time spend marking/sweeping per GC (across all GC's major and minor).. Each branch is calculated using mean(total_marking_time) / (mean(major_count) + mean(minor_count))
  • Comparison values of average major and minor count per branch. 1.4 -> 6.0 says that the control branch ran an average of 1.4 GC cycles across all runs, and the experiment ran an average of 6.0.
  • Percentage of total GC time spent doing Minor GC's per branch. Intended to give an indication as to what the major/minor time split per branch is.

the time / iter and time / GC ratios are intended to help determine whether the marking/sweeping time improved because marking/sweeping got faster, or because we're running fewer. For example, If "mark time/iter" improves but "mark time/GC" stays flat then you know that the win came from running fewer GC cycles.

New Summary output looks like this:

master: ruby 4.1.0dev (2026-06-17T10:00:21Z master 45fd5d58f1) +YJIT +PRISM [x86_64-linux]
experiment: ruby 4.1.0dev (2026-06-17T10:00:21Z mvh-obspace-malloc.. 45fd5d58f1) +PRISM [x86_64-linux]

----------  -------------  ------------  ---------------  ------------  ------------------  -----------------  ---------------------
bench         master (ms)     RSS (MiB)  experiment (ms)     RSS (MiB)  experiment 1st itr  master/experiment  RSS master/experiment
lobsters     977.1 ± 1.2%  352.3 ± 0.0%    1760.3 ± 5.0%  307.2 ± 0.2%               1.588              0.555                  1.147
railsbench  1838.6 ± 1.5%  139.1 ± 0.3%    3506.9 ± 0.5%  121.1 ± 1.2%               0.728              0.524                  1.149
gcbench      699.4 ± 5.8%   47.3 ± 0.0%    2904.4 ± 1.4%   46.8 ± 0.0%               0.253              0.241                  1.010
----------  -------------  ------------  ---------------  ------------  ------------------  -----------------  ---------------------

GC summary:
----------  --------------  ---------------  ------------  -------------  -------------  -------------  -------------
bench       mark time/iter  sweep time/iter  mark time/GC  sweep time/GC     major/iter     minor/iter     minor GC %
lobsters             0.957            0.929         0.957          0.929   0.0  →   0.0   0.9  →   0.9  100%  →  100%
railsbench           0.179            1.020         0.134          0.768   0.2  →   1.8   9.5  →   5.5   98%  →   75%
gcbench              0.963            0.987         0.963          0.987   0.0  →   0.0  52.0  →  52.0  100%  →  100%
----------  --------------  ---------------  ------------  -------------  -------------  -------------  -------------

Legend:
- experiment 1st itr: ratio of master/experiment time for the first benchmarking iteration.
- master/experiment: ratio of master/experiment time. Higher is better for experiment. Above 1 represents a speedup.
- RSS master/experiment: ratio of master/experiment RSS. Higher is better for experiment. Above 1 means lower memory usage.
- GC summary compares master → comparison. Ratio columns are master/comparison; above 1 means the comparison spent less GC time.
- mark time/iter and sweep time/iter compare total GC phase time per benchmark iteration, so they include both per-GC cost and GC frequency changes.
- mark time/GC and sweep time/GC compare average phase time per GC, isolating whether each GC became cheaper or more expensive.
- major/iter, minor/iter, and minor GC % show master → comparison values, not ratios. Rows with no GC activity are omitted.

@peterzhu2118 peterzhu2118 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I really like this!

@eightbitraptor eightbitraptor merged commit 26bef91 into main Jun 18, 2026
11 checks passed
@eightbitraptor eightbitraptor deleted the mvh-gc-handler-improvemnets branch June 18, 2026 13:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants