Methodology
I tried as much as possible to get stable and consistent benchmarking results. These are
the main things I did to get there.
Benchmark itself
- Version 0.35.3 of rawloader was used throughout the test
- A set of 60 files from a broad set of cameras was chosen manually to try and exercise most types of decompression algorithm. All the sample files are available at raw.pixls.us
- Benchmarking is done by first loading the file into memory and only then benchmarking 50 iterations of decoding the file
- All the file and rust version combinations were benchmarked five times in random order (to try to eliminate any systematic biases during the run)
- Between every two runs a one second sleep was done to prevent thermal throttling
Data aggregation
- The value for each file and rust version combination is the median of the five runs
- The comparisons are simply the ratio between the time of the two runs
- The box plots were generated in R with the default boxplot() function
Test Machine
- All benchmarks were run on a Lenovo T460s with a i5-6200U CPU
- Benchmarks were ran overnight with no other user programs running
- The CPU governor was set to performance during the runs
Copyright 2018 Pedro CĂ´rte-Real