In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it. The term 'benchmark' is also mostly utilized for the purposes of elaborately-designed benchmarking programs themselves.
Benchmarking is usually associated with assessing performance characteristics of computer hardware, for example, the floating point operation performance of a CPU, but there are circumstances when the technique is also applicable to software. Software benchmarks are, for example, run against compilers or database management systems.
Benchmarks provide a method of comparing the performance of various subsystems across different chip/system architectures.
Types Of Benchmark:
- Real program
- word processing software
- tool software of CAD
- user's application software (i.e.: MIS)
- Microbenchmark
- Designed to measure the performance of a very small and specific piece of code.
- Kernel
- contains key codes
- normally abstracted from actual program
- popular kernel: Livermore loop
- linpack benchmark (contains basic linear algebra subroutine written in FORTRAN language)
- results are represented in MFLOPS
- Component Benchmark/ micro-benchmark
- programs designed to measure performance of a computer's basic components [3]
- automatic detection of computer's hardware parameters like number of registers, cache size, memory latency
- Synthetic Benchmark
- Procedure for programming synthetic benchmark:
- take statistics of all types of operations from many application programs
- get proportion of each operation
- write program based on the proportion above
- Types of Synthetic Benchmark are:
- These were the first general purpose industry standard computer benchmarks. They do not necessarily obtain high scores on modern pipelined computers.
- Procedure for programming synthetic benchmark:
- I/O benchmarks
- Database benchmarks: to measure the throughput and response times of database management systems (DBMS')
- Parallel benchmarks: used on machines with multiple cores, processors or systems consisting of multiple machines
Common benchmarks
Industry standard (audited and verifiable)
- Business Applications Performance Corporation (BAPCo)
- Embedded Microprocessor Benchmark Consortium (EEMBC)
- Standard Performance Evaluation Corporation (SPEC), in particular their SPECint and SPECfp
- Transaction Processing Performance Council (TPC)
- Coremark: Embedded computing standard benchmark
Open source benchmarks
- Bonnie++: filesystem and hard drive benchmark
- DEISA Benchmark Suite: scientific HPC applications benchmark
- Dhrystone: integer arithmetic performance
- Fhourstones: an integer benchmark
- HINT: designed to measure overall CPU and memory performance
- Iometer: I/O subsystem measurement and characterization tool for single and clustered systems.
- Linpack, traditionally used to measure FLOPS
- LAPACK[citation needed]
- Livermore loops
- NAS parallel benchmarks
- NBench: synthetic benchmark suite measuring performance of integer arithmetic, memory operations, and floating-point arithmetic
- PAL: a benchmark for realtime physics engines
- Phoronix Test Suite: open-source cross-platform benchmarking suite for Linux,OpenSolaris, FreeBSD, OSX and Windows. It includes a number of other benchmarks included on this page to simplify execution.
- POV-Ray: 3D render
- Tak (function): a simple benchmark used to test recursion performance
- TATP Benchmark: Telecommunication Application Transaction Processing Benchmark
- TPoX: An XML transaction processing benchmark for XML databases
- Whetstone: floating-point arithmetic performance
- Geek Benchmark: For finding integer,Floating point performance and memory performance
- Rodinia: Benchmark for parallel architectures based on accelerators.
- Parsec: Benchmark for parallel shared memory systems.
- Splash2: Benchmark for parallel systems.
- STREAM: Benchmark for measuring memory bandwidth of a system.
- LLCbench: Low Level Architectural Characterization Benchmark Suite, used to measure CPU and memory performance of a system.
Microsoft Windows benchmarks
- BAPCo: MobileMark, SYSmark, WebMark
- Futuremark: 3DMark, PCMark
- Whetstone
- Worldbench (discontinued)
- PiFast
- SuperPrime
- Super PI
- Windows System Assessment Tool, included with Microsoft Windows Vista and later Windows operating systems, providing an index for consumers to rate their systems easily
Others
- BRL-CAD
- Khornerstone
- iCOMP, the Intel comparative microprocessor performance, published by Intel
- Performance Rating, modeling scheme used by AMD and Cyrix to reflect the relative performance usually compared to competing products.
- VMmark, a virtualization benchmark suite.
- Sunspider, a Browser speed test
- BreakingPoint Systems, modeling and simulation of network application traffic for benchmarking servers and network equipment
- Glaesemann, K. R.; van Dam, H. J. J.; Carr, J. F. (2011). "MSC Benchmark 1.0". Pacific Northwest National Lab., a benchmark for testing massively parallel computer systems under simultaneously heavy network, memory, and CPU loads.
What is benchmark testing?
Benchmark testing is the process of load testing a component or an entire end to end IT system to determine the performance characteristics of the application. The benchmark test is repeatable in that the performance measurements captured will vary only a few percent each time the test is run. This enables single changes to be made to the application or infrastructure in an attempt to determine if there is a performance improvement or degradation.
Benchmark testing can combine aspects of security testing.- An example in case is benchmark testing firewalls. This requires system and or user loads combined with security violations concurrently executed against the component to determine its performance benchmark.
The goals of benchmark testing typically fall into two categories;
To test the system to measure how a change affects its performance characteristics.
To test and tune the system to reach a performance requirement or service level agreement (SLA). In this case a series of benchmark tests are conducted in conjunction with iterative cycles of performance tuning.
Benchmark testing services
Benchmarking End To End IT Systems
Benchmarking Networks
Benchmarking Firewalls
Benchmarking Databases
No comments:
Post a Comment