Testing numbench
8 August 2011 2 Comments
This article explains how to install the numbench script and to run the benchmarks.
Install app-benchmarks/numbench
First of all, the science overlay has to be added through layman; therefore, install layman on your Gentoo system if you don’t have it already, and do:
layman -L layman -a science
After that you will be able to install the package app-benchmarks/numbench. Remember that numbench is still unstable and therefore I will need to install the ~x86/~amd64 version.
The package installs the executable “numbench”, some Python data and a man page numbench(1).
I recommend to add the bicatali overlay through layman too, because it contains many numerical libraries that can be benchmarked, even if we are migrating them into the science overlay.
Run the benchmarks
In order to run the benchmarks you have to provide a configuration file. The man page explains how to write one, and you will find some examples under /usr/share/numbench/samples. Once you have your configuration file (say conf.in), and you have chosen the module to test (e.g. blas, lapack or lapack_accuracy; see man numbench or numbench -h), just run the command
numench module conf.in -s
The documentation explains how to run the test with more parameters in order to choose the tests that have to be performed.
After the execution you will find interesting directories under ~/.benchmarks:
- log contains the log, obviously; they are divided in subfolders in case of multiple runs
- packages contains the packages that are useful if you decide to install some tested one (the documentation here lacks)
- reports contains for each run a set of images, an HTML page and a copy of the logs; they are ready to be published somewhere, just copy the whole folder in your www directory
- roots and tests are two directories which are used by the script for storing data; they are keeped in order not tu run the tests again if the results already exist
Hi! I just wanted to try your benchmark numbench from the science overlay. I called it by
numbench blas blastests.in
================================================================================
The following tests will be run:
——————————-
Test: blas-reference
– Package: sci-libs/blas-reference-20110418
– Environment: FFLAGS=”-O3 -pipe -march=opteron” FC=”gfortran-4.5.3″
Test: eigen
– Package: dev-cpp/eigen-3.0.3
– Environment: CC=”gcc-4.5.3″ CXX=”g++-4.5.3″ CFLAGS=”-O3 -pipe -march=opteron” CXXFLAGS=”-O3 -pipe -march=opteron”
Test: goto
– Package: sci-libs/gotoblas2-1.13
– Environment: USE=”incblas” TARGET=”OPTERON” CFLAGS=”-O3 -march=opteron”
Test: openblas
– Package: sci-libs/openblas-9999
– Environment: CC=”gcc-4.5.3″ USE=”incblas” TARGET=”OPTERON” CFLAGS=”-O3 -pipe -march=opteron”
Test: acml
– Package: sci-libs/acml-4.4.0-r1
– Skip implementations: acml32-gfortran acml32-gfortran-openmp acml64-gfortran-openmp
================================================================================
The script is located in the directory /usr/lib64/python2.7/site-packages/numbench
The script is run from the directory /usr/lib64/python2.7/site-packages/numbench
The BTL is located in the directory /usr/include/btl
The logs will be available in the directory /home/martinez/.benchmarks/log/blas_2011-10-12_5
The results will be available in the directory /home/martinez/.benchmarks/reports/blas_2011-10-12_5
BEGIN TEST 1 – blas-reference
– Emerging dependencies
Traceback (most recent call last):
File “/usr/bin/numbench”, line 21, in
import numbench.main
File “/usr/lib64/python2.7/site-packages/numbench/main.py”, line 287, in
logdir=tlogdir)
File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 110, in install_dependencies
install_package(d, env, root, pkgdir, logfile)
File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 139, in install_package
pkg = normalize_cpv(package)
File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 76, in normalize_cpv
if cpv[-1] != ‘r0′:
TypeError: ‘NoneType’ object is not subscriptable
And obtain the listed error. Is this a knows bug? I’m using python2.7
I was able to fix this error, however, I think that there are a lot more present. E.G. the script tries to test acml, but have not listed in in the config…