Testing numbench

This article explains how to install the numbench script and to run the benchmarks.

Install app-benchmarks/numbench

First of all, the science overlay has to be added through layman; therefore, install layman on your Gentoo system if you don’t have it already, and do:

layman -L
layman -a science

After that you will be able to install the package app-benchmarks/numbench. Remember that numbench is still unstable and therefore I will need to install the ~x86/~amd64 version.

The package installs the executable “numbench”, some Python data and a man page numbench(1).

I recommend to add the bicatali overlay through layman too, because it contains many numerical libraries that can be benchmarked, even if we are migrating them into the science overlay.

Run the benchmarks

In order to run the benchmarks you have to provide a configuration file. The man page explains how to write one, and you will find some examples under /usr/share/numbench/samples. Once you have your configuration file (say conf.in), and you have chosen the module to test (e.g. blas, lapack or lapack_accuracy; see man numbench or numbench -h), just run the command

numench module conf.in -s

The documentation explains how to run the test with more parameters in order to choose the tests that have to be performed.

After the execution you will find interesting directories under ~/.benchmarks:

  • log contains the log, obviously; they are divided in subfolders in case of multiple runs
  • packages contains the packages that are useful if you decide to install some tested one (the documentation here lacks)
  • reports contains for each run a set of images, an HTML page and a copy of the logs; they are ready to be published somewhere, just copy the whole folder in your www directory
  • roots and tests are two directories which are used by the script for storing data; they are keeped in order not tu run the tests again if the results already exist
Please let me know if you find any bugs!

2 Responses to Testing numbench

  1. Hi! I just wanted to try your benchmark numbench from the science overlay. I called it by

    numbench blas blastests.in
    The following tests will be run:

    Test: blas-reference
    – Package: sci-libs/blas-reference-20110418
    – Environment: FFLAGS=”-O3 -pipe -march=opteron” FC=”gfortran-4.5.3″

    Test: eigen
    – Package: dev-cpp/eigen-3.0.3
    – Environment: CC=”gcc-4.5.3″ CXX=”g++-4.5.3″ CFLAGS=”-O3 -pipe -march=opteron” CXXFLAGS=”-O3 -pipe -march=opteron”

    Test: goto
    – Package: sci-libs/gotoblas2-1.13
    – Environment: USE=”incblas” TARGET=”OPTERON” CFLAGS=”-O3 -march=opteron”

    Test: openblas
    – Package: sci-libs/openblas-9999
    – Environment: CC=”gcc-4.5.3″ USE=”incblas” TARGET=”OPTERON” CFLAGS=”-O3 -pipe -march=opteron”

    Test: acml
    – Package: sci-libs/acml-4.4.0-r1
    – Skip implementations: acml32-gfortran acml32-gfortran-openmp acml64-gfortran-openmp


    The script is located in the directory /usr/lib64/python2.7/site-packages/numbench
    The script is run from the directory /usr/lib64/python2.7/site-packages/numbench
    The BTL is located in the directory /usr/include/btl
    The logs will be available in the directory /home/martinez/.benchmarks/log/blas_2011-10-12_5
    The results will be available in the directory /home/martinez/.benchmarks/reports/blas_2011-10-12_5

    BEGIN TEST 1 – blas-reference
    — Emerging dependencies
    Traceback (most recent call last):
    File “/usr/bin/numbench”, line 21, in
    import numbench.main
    File “/usr/lib64/python2.7/site-packages/numbench/main.py”, line 287, in
    File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 110, in install_dependencies
    install_package(d, env, root, pkgdir, logfile)
    File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 139, in install_package
    pkg = normalize_cpv(package)
    File “/usr/lib64/python2.7/site-packages/numbench/PortageUtils.py”, line 76, in normalize_cpv
    if cpv[-1] != ‘r0’:
    TypeError: ‘NoneType’ object is not subscriptable

    And obtain the listed error. Is this a knows bug? I’m using python2.7

  2. I was able to fix this error, however, I think that there are a lot more present. E.G. the script tries to test acml, but have not listed in in the config…

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: