Benchmarks (cont)
Benchmarking allows us to compare alternative programs for the same task.
Things to beware of with benchmarks:
- must compare programs in similar (identical) envrionments
- eliminate system effects ... system load, caching
- avoid data sample bias ... use likely/varied sample data
- benchmarks give coarse view of program performance
Benchmarks are not useful as a basis for performance tuning.
(because they provide no (direct) information on reasons for inefficiency)
|