@fluid-tools/benchmark

This package contains benchmarking tools to profile runtime and memory usage. There's the pieces you can use to profile your own code (described below) and various utilities for customizing/configuring benchmark, as well as authoring alternative reporters and profiling functions (ex: to support test frameworks other than mocha).

General use

This package exports a few functions that you'll use instead of mocha's it() to define profiling tests:

• benchmark() for runtime tests
• benchmarkCustom() for custom usage tests
• benchmarkMemory() for memory usage tests

More details particular to each can be found in the sections below.

The tests you write using these functions can also act as correctness tests. When you run mocha on a package that contains profiling tests, they'll behave like any other mocha test defined with it().

To run them as profiling tests, invoke mocha as you normally would for your package but pass some additional arguments, like this:

--v8-expose-gc --perfMode --fgrep @Benchmark --fgrep @ExecutionTime --reporter @fluid-tools/benchmark/dist/MochaReporter.js

--perfMode (required)

Indicates that the tests should be run as profiling instead of just correctness tests. When run like this, many iterations will be run and measured, but when run as correctness tests only one iteration will be run and no measuring will take place.

--v8-expose-gc (required)

This is necessary so the package can perform explicit garbage collection between tests to help reduce cross-test contamination.

--fgrep @Benchmark

All tests created with the tools in this package get tagged with @Benchmark in their name by default, so most of the time you'll want to use --fgrep @Benchmark to only run tests that were defined with the tools provided here. You can change the @Benchmark tag to a few other values (like @Measurement, @Perspective, or @Diagnostic) with one of the arguments to the functions exposed in this package, and if you do, you can be more specific about which tests want to run by passing a different filter, e.g. --fgrep @Measurement.

--fgrep @ExecutionTime or --fgrep @MemoryUsage

You'll also want to use one of these to only run execution-time/runtime or memory usage tests. You can technically run them both at the same time, but the custom mocha reporters (one for runtime tests, one for memory usage tests) expect to only see tests of their corresponding type, so in order to use those you'll have to use --fgrep as described here and do two separate test runs, one for each type of test.

`--reporter