Benchmarking suite – Report 10

This is the report of the project “Automated benchmark suite for numerical libraries in Gentoo” for the week 25 July-1 August.

Project description

The project aims to develop a simple yet powerful automated system of benchmarking for numerical libraries. The Gentoo software system provides many implementations of widely used standards such as BLAS, CBLAS, LAPACK, ScaLAPACK and some other numerical library such as FFTW, MKL. The developed tools will aid the system maintener to choose the best suited implementation with respect to the machine hardware and to test the samer implementation or different ones with different compilers, compiler versions and compile flags.

Progrees during the week

This week has been devoted to the following features:

  • Management of the dependencies
  • Improvement of the PBLAS/ScaLAPACK tests and investigation of some BTL-related problem — see below
  • 3-Dimensional FFTW tests

Before this week’s work, only packages without unsatisfied dependencies on the main system could be tested. For example, if one did not have cmake installed in the system, he could not test eigen, as it requires cmake to be installed in order to compile. This is not a problem anymore. The dependencies are emerged into the “sandbox” root along with the package to be tested, and the environment is adjusted in order to make use of the software installed. The delay of this report is due to this work, which has required more work as expected — a small rewriting of the preliminary part of the script has been done –; now the report can be complete.

Some investigation has been done regarding the input matrices and vectors for the tests. In order to be fair, the tests should produce the same inputfor the libraries, as different input could require different work by the libraries. The linear congruential random number generatorhas been written, along with the new-style matrix generator. This will not completely solve the problem, but it represents a step in the right direction. Now the PBLAS/ScaLAPACK tests use this method, and the results are satisfactory.

The work with FFTW is finished for now. 1-dimensional, 2-dimensional and 3-dimensional complex discrete Fourier transformations are tested in both forward and backward versions and with the “estimate” and “measure” strategies. If the time will allow it, I will also add sinus and cosinus tranformations.

Plan for the next week

During the following week I will implement the tests for METIS. This is a set of executables and libraries that compute graph and mesh partitioning — along with other numerical functions –, which are widely used in the field of numerical simulations and other scientific computations.

The project has intensively gained features in the last weeks. After the METIS tests implementation, the quality of the whole project will be tested and improved. A considerable part of the time has been devoted to the BTL library, and more work will be performed in order to submit the new code upstream. Some work has to be also done in order to make the script more user-friendly; this means managing the exceptions that occur sometimes, avoid conflicts between tests already done and the currect ones, reduce the probability of errors. Much documentation has to be written. The ebuild will be splitted in two part, a Gentoo-specific one and a general one containing the modified BTL.

All this work will eventually provide a finished and usable project, which is my primary objective for the programme. I do not want to provide an incomplete or undocumented software; I rather prefer pospone the inclusion of new features after the programme end, if the time is not enough.

Best regards
Andrea Arteaga


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: