[prev] [index] [next]

Benchmarks (cont)

Benchmarking allows us to compare alternative programs for the same task.

Things to beware of with benchmarks:

  • must compare programs in similar (identical) envrionments
    • eliminate system effects ... system load, caching
  • avoid data sample bias ... use likely/varied sample data
  • benchmarks give coarse view of program performance

Benchmarks are not useful as a basis for performance tuning.
(because they provide no (direct) information on reasons for inefficiency)