Skip to content

Benchmark parameter disposal #6

Description

@mawosoft

See https://github.com/mawosoft/BenchmarkDotNet.Tests/tree/master/BDNParamDisposal
for parameter/argument creation and disposal in BDN host and in generated projects.

The generated code does not dispose the actually used parameters/arguments at all. Disposal occurs only while enumerating a ParamsSource or ArgumentsSource to skip to the current one and only for the ones skipped:

foreach (T parameter in parameters)
{
if (count == index)
{
return parameter;
}
if (parameter is IDisposable disposable)
{
// parameters might contain locking finalizers which might cause the benchmarking process to hung at the end
// to avoid that, we dispose the parameters that were created, but won't be used
// (for every test case we have to enumerate the underlying source enumerator and stop when we reach index of given test case)
// See https://github.com/dotnet/BenchmarkDotNet/issues/1383 and https://github.com/dotnet/runtime/issues/314 for more
disposable.Dispose();
}
count++;
}

BDN host only disposes benchmarks it actually runs and only after they all have been run:

// some benchmarks might be using parameters that have locking finalizers
// so we need to dispose them after we are done running the benchmarks
// see https://github.com/dotnet/BenchmarkDotNet/issues/1383 and https://github.com/dotnet/runtime/issues/314 for more
foreach (var benchmarkInfo in benchmarkRunInfos)
{
benchmarkInfo.Dispose();
}

The IDisposable chain goes: BenchmarkRunInfo -> BenchmarkCase(s) -> ParameterInstances -> ParameterInstance(s) -> (Value as IDisposable)?

However, BenchmarkCase(s) - and therefore ParameterInstance(s) - have been already created early and have gone through filters and validators.

  • If some are filtered out, they are not disposed.
  • If any validator reports a critical error, none are disposed at all.

The problem is that ParameterInstance(s) are constructed per method. Multiple job definitions share the same cached instances. Thus a filtered BenchmarkCase cannot be disposed because it might share ParameterInstance(s) with benchmarks that are about to be run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions