Tag - testing

Aunit vs Ahven

By Stephane Carrez

AUnit and Ahven are two testing frameworks for Ada. Both of them are inspired from the well known JUnit Java framework. Having some issues with the Aunit testing framework, I wanted to explore the use of Ahven. This article gives some comparison elements between the two unit test frameworks. I do not pretend to list all the differences since both frameworks are excellent.

Writing a unit test is equally simple in both frameworks. They however have some differences that may not be visible at the first glance.

AUnit

AUnit is a unit test framework developped by Ed Falis and maintained by AdaCore. It is distributed under the GNU GPL License.

Some good points:

  • AUnit has a good support to report where a test failed. Indeed, the Assert procedures will report the source file and line number where the assertion failed.
  • AUnit is also able to dump the exception stack trace in symbolic form. This is useful to find out quickly the source of a problem.

Some bad points:

  • AUnit has several memory leaks which is quite annoying when you want to track memory links with valgrind.
  • AUnit does not integrate easily with JUnit-based XML tools. In particular the XML file it creates can be invalid in some cases (special characters in names). More annoying is the fact that the XML format is not compatible with JUnit XML format.

Ahven

Ahven is another unit test framework developed by Tero Koskinen. It is distributed under the permissive ISC License.

Some good points:

  • Ahven license is a better model for proprietary unit tests.
  • Ahven generates XML result files which are compatible with Junit XML result files. Integration with automatic build tools such as Jenkins is easier.
  • Ahven XML result files can integrate the test output (as in JUnit). This is useful to analyze a problem.
  • Ahven has a test case timeout which is useful to detect and stop blocking tests.

Some bad points:

  • The lack of precise information in message (source line, exception trace) can be annoying to find out why a test failed.

Don't choose and be prepared to use both with Ada Util!

The unit tests I've written were done for AUnit and I had arround 329 tests to migrate. To help the migration to Ahven, I wrote a Util.XUnit package which exposes a common interface on top of AUnit or Ahven. It turns out that this is easy and quite small. The package has one specific implementation (spec+body) for both frameworks. All the unit tests have to use it instead of the AUnit or Ahven packages.

The Aunit implementation (util-xunit.ads) defines several types which are also defined in the Ahven implementation.

package Util.XUnit is
...
   subtype Status is AUnit.Status;

   Success : constant Status := AUnit.Success;
   Failure : constant Status := AUnit.Failure;

   subtype Message_String is AUnit.Message_String;
   subtype Test_Suite is AUnit.Test_Suites.Test_Suite;
   subtype Access_Test_Suite is AUnit.Test_Suites.Access_Test_Suite;

   type Test_Case is abstract new AUnit.Simple_Test_Cases.Test_Case with null record;
   type Test is abstract new AUnit.Test_Fixtures.Test_Fixture with null record;
...
end Util.XUnit;

The XUnit implementation for Ahven is a little bit more complex because all my tests were using AUnit interface, I decided to keep almost that API and thus I had to simulate what is missing or is different.

Ahven implementation: util-xunit.ads

package Util.XUnit is
...
   type Status is (Success, Failure);

   subtype Message_String is String;
   subtype Test_Suite is Ahven.Framework.Test_Suite;
   type Access_Test_Suite is access all Test_Suite;

   type Test_Case is abstract new Ahven.Framework.Test_Case with null record;
   type Test is new Ahven.Framework.Test_Case with null record;
...
end Util.XUnit;

The choice of the unit test framework is done when the Ada Utility library is configured.

To add a comment, you must be connected. Login to add a comment