Version 2.7.0 of Ada Utility Library brings the following improvements:

• Used spdx-tool to use SPDX-License-Identifier in headers
• New package Util.Files.Walk to iterate over directory trees and honor .gitignore
• Add support for custom log formatter (allow to translate log messages, filter messages, ...)
• Feature #48: Change the log time from UTC to Local Time (configurable)
• Fix #49: Perf report generates incorrect XML title attributes
• Fix #50: 128Bit AES-CTR Encoding doesn't work (thanks Adam Jasinski)
• Fix #53: Executor does not always stop the worker tasks

This article highlights two new features provided by the new version.

Walking directory trees with support of .gitignore

It is sometimes necessary to walk a directory tree while taking into account some inclusion or exclusion patterns or more complex ignore lists. The Util.Files.Walk package provides a support to walk such directory tree while taking into account some possible ignore lists such as the .gitignore file (see util-files-walk.ads package specification). The implementation is based on the Ada.Directories support. The package defines the Filter_Type tagged type to represent and control the exclusion or inclusion filters and a second tagged type Walker_Type to walk the directory tree.

The Filter_Type provides two operations to add patterns in the filter and one operation to check against a path whether it matches a pattern. A pattern can contain fixed paths, wildcards or regular expressions. Similar to .gitignore rules, a pattern which starts with a / will define a pattern that must match the complete path. Otherwise, the pattern is a recursive pattern. Example of pattern setup:

 Filter : Util.Files.Walk.Filter_Type;
 ...
 Filter.Exclude ("*.o");
 Filter.Exclude ("/alire/");
 Filter.Include ("/docs/*");

The Match function looks in the filter for a match. The path could be included, excluded or not found. For example, the following paths will match:

To scan a directory tree, the Walker_Type must have some of its operations overriden:

• The Scan_File should be overriden to be notified when a file is found and handle it.
• The Scan_Directory should be overriden to be notified when a directory is entered.
• The Get_Ignore_Path is called when entering a new directory. It can be overriden to indicate a path of a file which contains some patterns to be ignored (ex: the .gitignore file).

The example below shows a possible overriding definition for Walker_Type to scan a directory tree and print the files while ignoring files and directories described in the .gitignore of each directory when they are defined:

type Walker_Type is new Util.Files.Walk.Walker_Type with null record;

overriding
function Get_Ignore_Path (Walker : Walker_Type;
                          Path   : String) return String
   is (Util.Files.Compose (Path, ".gitignore"));

overriding
procedure Scan_File (Walker : in out Walker_Type;
                     Path   : String);
overriding
procedure Scan_File (Walker : in out Walker_Type;
                     Path   : String) is
begin
   Ada.Text_IO.Put_Line (Path);
end Scan_File;

With the above declarations, the directory is scanned by calling the Scan procedure giving the starting directory and the default root filters which allow to complete the definition of .gitignore files:

Walker : Walker_Type;
  ...
  Walker.Scan (".", Filter);

This flexible walk directory support is the basis for spdx-tool to identify the files for the analysis of language and license headers in a project.

Refer to the Directory tree walk documentation and also have a look at the tree.adb example available in the Ada Utility Library samples.

Custom log formatter

The Ada Utility Library provides a logging framework giving a flexible, extensible, and efficient mechanism to write logs in an Ada application. The framework is built arround the following concepts used in other logging frameworks:

• A logger is the abstraction that provides operations to emit a message. The message is composed of a text, optional formatting parameters, a log level and a timestamp.
• A formatter is the abstraction that takes the information about the log and its parameters to create the formatted message.
• An appender is the abstraction that writes the message either to a console, a file or some other final mechanism. A same log can be sent to several appenders at the same time.

A typical example to add log messages can be defined with the following code extract. The Log instance is assigned a name SPDX_Tool.Y which is used for the configuration of that logger to control the format, level and appenders which define where messages are written.

with Util.Log.Loggers;
...
  Log : constant Util.Log.Loggers.Logger := Util.Log.Loggers.Create ("SPDX_Tool.Y");
  ...
     Log.Info ("exclude file {0}", Path);

The log configuration can be defined programatically or from a configuration file that is loaded by the program. The next code extract shows a configuration defined in Ada. Property names are prefixed by a string which is customized when the logging framework is initialized ("spdx_tool." in the example). A first rootCategory property defines the global log level which is applied (DEBUG) followed by a list of appenders where the log messages are written (errorConsole and verbose). Each appender has its own configuration represented by properties prefixed with appender and the appender name. A main appender property defines the type of appender which could be Console, File, RollingFile (and syslog for the syslog_appenders.adb example, you can add your own types).

The errorConsole appender will write the messages on the Console and it will write on the standard error (stderr=true), it will filter to only report errors (level=ERROR), only the message will be printed (layout=message) with a spdx-tool: prefix. The utf8 configuration tells to not use the Ada.Text_IO package but instead write strings directly, hence assuming that message strings contain UTF-8 encoding.

The verbose appender will use the File appender that writes messages in the file configured as File=verbose.log. It will print every message (level=DEBUG) and format them with date and level (layout=full). The utc=false configuration indicates to write dates using the local timezone (the default being to print dates in UTC).

Log_Config  : Util.Properties.Manager;
   ...
   Log_Config.Set ("spdx_tool.rootCategory", "DEBUG,errorConsole,verbose");
   Log_Config.Set ("spdx_tool.appender.errorConsole", "Console");
   Log_Config.Set ("spdx_tool.appender.errorConsole.level", "ERROR");
   Log_Config.Set ("spdx_tool.appender.errorConsole.layout", "message");
   Log_Config.Set ("spdx_tool.appender.errorConsole.stderr", "true");
   Log_Config.Set ("spdx_tool.appender.errorConsole.prefix", "spdx-tool: ");
   Log_Config.Set ("spdx_tool.appender.errorConsole.utf8", "true");
   Log_Config.Set ("spdx_tool.appender.verbose", "File");
   Log_Config.Set ("spdx_tool.appender.verbose.level", "DEBUG");
   Log_Config.Set ("spdx_tool.appender.verbose.layout", "full");
   Log_Config.Set ("spdx_tool.appender.verbose.File", "verbose.log");
   Log_Config.Set ("spdx_tool.appender.verbose.utc", "false");
   Util.Log.Loggers.Initialize (Log_Config, "spdx_tool.");

The new version of Ada Utility Library introduces the support to customize the formatter. The formatter is responsible for preparing the message to be displayed by log appenders. It takes the message string and its arguments and builds the message. The same formatted message is given to each log appender (file, console, syslog, ...). This step is handled by a Format_Message procedure that can be overriden. Then, each log appender can use it to format the log event which is composed of the log level, the date of the event, the logger name. This last formatting step is handled by a Format_Event function called from each log appender.

Using a custom formatter can be useful to change the message before it is formatted, translate messages, filter messages to hide sensitive information and so on. Implementing a custom formatter is made in three steps:

• first by extending the Util.Log.Formatters.Formatter tagged type and overriding the Format_Message operation (and Format_Event if necessary). The procedure gets the log message passed to the Debug, Info, Warn or Error procedure as well as every parameter passed to customize the final message. It must populate a Builder object with the formatted message.
• second by writing a Create function that allocates an instance of the formatter and customizes it with some configuration properties.
• third by instantiating the Util.Log.Formatters.Factories generic package. It contains an elaboration body that registers automatically the factory.

For example, spdx-tool defines a custom log formatter that translates the message before it is printed on the console. Messages are translated by the gettext (3) and the NLS thin Ada binding library.

The two first steps could be implemented as follows (method bodies are not shown):

 type NLS_Formatter (Length : Positive) is
    new Util.Log.Formatters.Formatter (Length) with null record;
 function Create_Formatter (Name       : in String;
                            Properties : in Util.Properties.Manager)
               return Util.Log.Formatters.Formatter_Access;

Then, the package is instantiated as follows. The factory is given a name "NLS" which is used by the configuration to make a reference of it.

 package NLS_Factory is
   new Util.Log.Appenders.Factories (Name   => "NLS",
                                     Create => Create_Formatter'Access)
   with Unreferenced;

To use the new registered formatter, it is necessary to declare some minimal configuration. A formatter.<name> definition must be declared for each named formatter where <name> is the logical name of the formatter. The property must indicate the factory name that must be used (example: NLS). The named formatter can have custom properties and they are passed to the Create procedure when it is created. Such properties can be used to customize the behavior of the formatter. Here, we declare a logical "nlsFormatter" that will use our "NLS" formatter factory.

Log_Config.Set ("spdx_tool.formatter.nlsFormatter", "NLS");

Once the named formatter is declared, it can be selected for one or several logger by appending the string :<name> after the log level. For example:

Log_Config.Set ("spdx_tool.logger.SPDX_Tool", "INFO:nlsFormatter");

With the above configuration, the SPDX_Tool and its descendant loggers will use the formatter identified by nlsFormatter. When we write the message Log.INFO ("exclude file {0}", Path) it will first be translated by the nlsFormatter in the native user's language, then it will be formatted to include the Path parameter in the translated message and the final formatted message passed to the appenders. This simple mechanism allows to support several native languages, provided you have translations of these messages (extraction of messages from source code and their translation is another story that deserves a specific article !!!).

Have a look at the Logging documentation and the spdx_tool.adb use case for more information and detailed example.