Java 2 Ada

Using MAT the Memory Analysis Tool

By stephane.carrez 2015-05-25 11:50:13

MAT is a memory analysis tool that monitors calls to malloc, realloc and free calls. It works with a small shared library libmat.so that is loaded into the program with the LD_PRELOAD dynamic linker feature (See the ld.so(8) man page). The library overrides the malloc, realloc and free function to monitor calls to these functions. It then writes or sends probe events which contain enough information for mat to tell what, when, where and by whom the memory allocation was done.

mat will assign a unique number to each event that is collected. The tool will reconcile the events to find those that are related based on the allocation address so that it becomes possible to find forward and backward who allocates or releases the memory. When started, the tool provides a set of interactive commands that you can enter with the readline editing capabilities.

Instrumenting your application: the file event mode

The first method to instrument your application is by running your program and collecting the information into a file. Later, when the program is finished, use the mat tool to analyze the results.

mat-analyse.png

To instrument a program and save the results into a file, you can use the matl launcher as follows:

matl -o name command

where command is the command to instrument and name is the prefix file name. The generated file will have the process ID in its name with the .mat extension.

To start the analysis, you have to launch mat with the name of the generated file:

mat name-xxx.mat

Instrumenting live application: the TCP/IP socket mode

You can also instrument your application and do some analysis while your program is running. For this, you will use the TCP/IP socket mode provided by libmat.so and mat. You must first start the mat tool so that the TCP/IP server provided by mat is started before the program connects to it through the libmat.so shared library.

mat-analyse-tcp.png

To use this mode, start mat in a first terminal console with either the -s or the -b option to start the TCP/IP server and wait for a program to connect. For example:

mat -b 192.168.0.17:4096

Then, in a second terminal console start your program through the matl launcher as follows:

matl -s 192.168.0.17:4606 command

Here you will give the IP address (you may use localhost or any IP) and the TCP/IP port (the default port used being the port 4606). The mat server may run on a different host with a different architecture.

General information

When mat reads the events, it detects the endianness and the size of pointers so that it is able to analyze 32-bit as well as 64-bit applications with little endian or big endian formats. The info command gives information about the program and the events that have been collected.

The output presented below come from the analysis of gdb 7.9.1 that was debugging mat.

mat>info
Pid              : 5291
Path             : /data/ext/gnu/i386/gdb-7.9.1/gdb/gdb
Endianness       : LITTLE_ENDIAN
Memory regions   : 24
Events           : 0..586514
Duration         : 41.26s
Stack frames     :  0
Malloc count     :  279768
Realloc count    :  36272
Free count       :  270462
Memory allocated : 8341613
Memory slots     :  10484

The number of collected events can be quite high, 586514 in the above example, and it can easily exceed several millions.

Timeline

With so many events, you may not know where to start. You can use the timeline command to analyse the events and find interesting groups and report information about them. The command takes a duration parameter to control the groups by defining the maximum duration in seconds of a group. For each group, the command indicates the event ID range, the number of malloc, realloc and free calls as well as the memory growth or shrink during the period.

mat>timeline 5
Start     End time  Duration  Event range         # malloc  # realloc # free    Memory
0us       4.32s     4.32s     0..533645           261393    20423     251923    +4618592
15.60s    20.57s    4.96s     533646..542619      4495      102       4379      +438803
20.62s    21.05s    432.68ms  542620..581167      11555     15663     11338     +3198744
26.16s    28.86s    2.70s     581168..582388      539       4         678       -9425
31.40s    32.41s    1.01s     582389..583133      288       11        446       +19246

In this sample, the first group correspond to gdb's startup during which it loads the symbol table. The second group correspond to the command b main followed by run in gdb. The third group is when the program reaches the breakpoint and gdb handles it and read the DWARF2 debugging information. The two last groups correspond to a step, where, cont and quit commands.

Looking at allocation sizes

The sizes command is another command that helps with many events as it counts and groups the events by their allocation size. Let's say we want to look further in the second group identified by the timeline 5 command, we can use the event range ID to filter out the events so that the sizes command only takes into account these events. By using the -c option, only a summary is printed by the command:

mat>sizes -c 542620..581167
Found 9550 different sizes, +3198744 bytes, with 11555 malloc, 15663 realloc, 11338 free

There are still many allocation and we can change the filter to report only the memory allocations greater than 75000 bytes. Now, we want to have all the information and the -c option is not given:

mat>sizes 542620..581167 and size > 75000
Event Id range                Time      Event               Size        Count   Total size  Memory
554191..554243                20.75s    realloc             75248        4      300992      -75240
565067                        20.88s    malloc              161323       1      161323      +161323
562816                        20.85s    malloc              179035       1      179035      +179035
564877                        20.87s    malloc              213328       1      213328      +213328
Found 7 different sizes, +730796 bytes, with 7 malloc, 1 realloc, 2 free

Looking at the event

When you identify some interesting event, you can use the event command and give it the event ID to dump the information with the complete stack frame. This is probably the most useful command as the stack frame helps you point out where in the code and by which flow the allocation call is made.

mat>event 554191
75248 bytes reallocated after 20.75s, freed 817us after by event 554243 +8 bytes
Id Frame Address         Function
 1 0x0000000000408898    _start
 2 0x00007F688D18AEC5    __libc_start_main (libc-start.c:321)
 3 0x0000000000408855    main (gdb.c:33)
 4 0x000000000054999B    gdb_main (main.c:1161)
 5 0x00000000005459F5    catch_errors (exceptions.c:237)
 6 0x0000000000549506    captured_main (main.c:1150)
 7 0x00000000005459F5    catch_errors (exceptions.c:237)
 8 0x00000000005484C3    captured_command_loop (main.c:329)
 9 0x000000000054EABE    start_event_loop (event-loop.c:334)
10 0x000000000054EA25    gdb_do_one_event (event-loop.c:296)
11 0x000000000054E755    gdb_wait_for_event (event-loop.c:773)
12 0x0000000000550482    inferior_event_handler (inf-loop.c:57)
13 0x000000000053A218    fetch_inferior_event (infrun.c:3273)
14 0x0000000000537E22    handle_signal_stop (infrun.c:4264)
15 0x000000000061B750    get_current_frame (frame.c:1486)
16 0x000000000054582C    catch_exceptions_with_msg (exceptions.c:189)
17 0x000000000061E36C    unwind_to_current_frame (frame.c:1451)
18 0x000000000061E081    get_prev_frame (frame.c:2212)
19 0x000000000061D949    get_prev_frame_always_1 (frame.c:1954)
20 0x000000000061B63B    compute_frame_id (frame.c:454)
21 0x000000000061EFAF    frame_unwind_find_by_frame (frame-unwind.c:157)
22 0x000000000061EBF6    frame_unwind_try_unwinder (frame-unwind.c:106)
23 0x00000000005CCEB4    dwarf2_frame_sniffer (dwarf2-frame.c:1405)
24 0x00000000005CCB38    dwarf2_frame_find_fde (dwarf2-frame.c:1772)
25 0x00000000005CC8E9    dwarf2_build_frame_info (dwarf2-frame.c:2313)
26 0x00000000005CAC4B    add_fde (dwarf2-frame.c:1812)

This event is a realloc call that reallocates a memory slot to 75248 bytes, the previous slot size was 75240 bytes. The operation that makes the call is add_fde in gdb.

Looking at frames

The frames command analyzes all the event stack frames and report the function with the number of calls and memory growth that they created. The command takes a level parameter that indicates the stack frame level to take into account. The level counts the stack frame starting from the bottom, so that level 1 should give you functions that directly call malloc, realloc and free functions.

A filter can be defined to control the allocation events to take into account. For example if we want to look at functions that allocate or free large memory blocks and which are in the second group reported by the timeline command, we can use the following command:

mat>frames 1 542620..581167 and size > 50000
Level Size      Count   Function
 1    -53328    1       bfd_elf64_slurp_symbol_table (elfcode.h:1172)
 1    +743794   8       __GI__obstack_newchunk (obstack.c:269)
 1    -187272   3       dwarf2_build_frame_info (dwarf2-frame.c:2451)
 1    +53328    1       bfd_elf_get_elf_syms (elf.c:419)
 1    -71104    1       _bfd_elf_canonicalize_dynamic_symtab (elf.c:7215)
 1    +461056   4       bfd_alloc (opncls.c:956)
 1    +25248    3156    add_fde (dwarf2-frame.c:1812)
 1    =75248    2       dwarf2_build_frame_info (dwarf2-frame.c:2422)
 1    -75248    1       dwarf2_frame_find_fde (dwarf2-frame.c:1772)
 1    +71104    1       bfd_elf_get_elf_syms (elf.c:453)

Download

You can download the first release of MAT and get mat sources at http://download.vacs.fr/mat/mat-1.0.0.tar.gz and follow the Build instructions.

Ubuntu 14.04 packages for 64-bit platforms are available:

http://download.vacs.fr/mat/mat_1.0.0_amd64.deb and http://download.vacs.fr/mat/libmat_1.0.0_amd64.deb

Ubuntu 14.04 packages for 32-bit platforms are available:

http://download.vacs.fr/mat/mat_1.0.0_i386.deb and http://download.vacs.fr/mat/libmat_1.0.0_i386.deb

Conclusion

Unlike valgrind, mat does not instrument the program. Instead, it overrides the malloc, realloc and free calls and only monitors these events. This makes the implementation easier to port and allows to use mat in some embedded systems where valgrind is more difficult to use (due to portability and memory resource constraints).

I've used mat on several Mips boards (BCM6362, BCM63168, Vox185) and it was very useful to understand the memory allocations and reduce the memory used by several programs. It does not yet provide a graphical front end but this will come one day.

Give it a try, you may find some interesting features in it!

Ada BFD 1.1.0 is available

By stephane.carrez 2015-05-16 13:19:52

Ada BFD is an Ada binding for the GNU Binutils BFD library. It allows to read binary ELF, COFF files by using the GNU BFD and allows your program to read ELF sections, get access to the symbol table and use the disassembler.

I've added the support to the GNU demangler so that it is now possible to demangle symbols. You can demangle C++, Java and Ada symbols. To use the demangler you need a BFD file which is a limited type.

with Bfd.Files;
with Bfd.Symbols;
  ...
  File : Bfd.Files.File_Type;

The BFD file is opened as follows:

  Bfd.Files.Open (File, Path, "");

Then, you may convert any symbol name using the Demangle function:

  Name : String := Bfd.Symbols.Demangle (File, "bfd__symbols__get_name",
                                         Constants.DMGL_GNAT);

Sources are now moved to GitHub: https://github.com/stcarrez/ada-bfd

You can download the release at: http://download.vacs.fr/ada-bfd/ada-bfd-1.1.0.tar.gz

NetBSD 6.1.5 upgrade

By stephane.carrez 2015-01-10 10:03:07

Once every year I try to upgrade one of my virtual machine which is running NetBSD. This description is a short reminder for the major steps for the upgrade process.

System upgrade

The NetBSD system is upgraded by using the following command:

poseidon$ sudo sysupgrade auto ftp://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1/i386

During the upgrade it will ask whether some system files have to be replaced, merged or kept unmodified.

GCC Ada Package Upgrade

The GCC Ada compiler is now based on GCC 4.9. I did the upgrade by using the following command:

poseidon$ sudo pkg_add -uu gcc-aux-20140422nb3 pkg_add: Warning: package `gcc-aux-20140422nb3' was built for a platform: pkg_add: NetBSD/i386 6.0 (pkg) vs. NetBSD/i386 6.1.4 (this host)

The gprbuild package must also be upgrade:

poseidon$ sudo pkg_add -u gprbuild-aux-20130416 pkg_add: Warning: package `gprbuild-aux-20130416' was built for a platform: pkg_add: NetBSD/i386 6.0 (pkg) vs. NetBSD/i386 6.1.4 (this host) pkg_add: Warning: package `gnat_util-20140422' was built for a platform: pkg_add: NetBSD/i386 6.0 (pkg) vs. NetBSD/i386 6.1.4 (this host)

And because I also use some other packages such as xmlada, the following package is also upgrade:

poseidon$ sudo pkg_add -u xmlada-4.4.0.0nb1 pkg_add: Warning: package `xmlada-4.4.0.0nb1' was built for a platform: pkg_add: NetBSD/i386 6.0 (pkg) vs. NetBSD/i386 6.1.4 (this host)

Before running an Ada program compiled by GCC 4.9

The GCC 4.9 Ada compiler works very well but it comes with a specific libgcc_s.so file installed in /usr/pkg/gcc-aux/lib. By default libgcc_s.so is installed on the system in /usr/lib/libgcc_s.so or /usr/pkg/lib/libgcc_s.so and they were compiled by GCC 4.5.3 or GCC 4.6.4.

If you use the wrong libgcc_s.so, the program will almost work except when a exception is raised: none of the exception can be caught and the program terminates as though there was no exception handler.

What happens is that the GCC 4.6 frame unwinder is unable to correctly identify the frames generated by GCC 4.9. The solution is of course to use the correct library and we can do this by setting the following environment variable before starting any program:

poseidon$ export LD_LIBRARY_PATH=/usr/pkg/gcc-aux/lib

Extending an ext4 LVM partition

By stephane.carrez 2015-01-03 10:34:55

From time to time a disk partition becomes full and it becomes desirable to grow the partition. Since I often don't remember how to do this, I wrote this short description to keep track of how to do it.

Extending the LVM partition

The first step is to extend the size of the LVM partition. This is done easily by using the lvextend (8) command. You just need to specify the amount and the LVM partition. In my case, the vg02-ext volume was using 60G and the +40G command will grow its size to 100G. Note that you can run this command while your file system is mounted (if you grow the size of your LVM partition).

$ sudo lvextend --size +40G /dev/mapper/vg02-ext Extending logical volume ext to 100.00 GiB Logical volume ext successfully resized

Preparing the ext4 filesystem for resize

Before resizing the ext4 filesystem, you must make sure it is not mounted:

$ sudo umount /ext

The file system must be clean and you should run the e2fsck (8) command to check it:

$ sudo e2fsck -f /dev/mapper/vg02-ext e2fsck 1.42 (29-Nov-2011) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/mapper/vg02-ext: 1974269/3932160 files (0.1% non-contiguous), 14942044/15728640 blocks 5.392u 1.476s 0:48.25 14.2% 0+0k 3208184+48io 2pf+0w

Resizing the ext4 filesystem

The last step is to resize the ext4 file system by using the resize2fs (8) command. The command can enlarge or shrink an unmounted file system of type ext2, ext3 or ext4. The command only needs the block device path to operate.

$ sudo resize2fs /dev/mapper/vg02-ext resize2fs 1.42 (29-Nov-2011) Resizing the filesystem on /dev/mapper/vg02-ext to 26214400 (4k) blocks. The filesystem on /dev/mapper/vg02-ext is now 26214400 blocks long.

After the resize, we can re-mount the ext4 partition:

$ sudo mount -a
2 comments

Ada BFD 1.0.1 is available

By stephane.carrez 2014-12-24 15:12:14

Ada BFD is an Ada binding for the GNU Binutils BFD library.

It allows to read binary ELF, COFF files by using the GNU BFD and allows your program to read ELF sections, get access to the symbol table and use the disassembler.

The new version fixes build and compilation issues with recent releases of GNU Binutils and it also provides support to build Debian packages.

http://download.vacs.fr/ada-bfd/ada-bfd-1.0.1.tar.gz

Ada Web Application 1.0.0 is available

By stephane.carrez 2014-07-27 18:48:33

Ada Web Application is a framework to build web applications.

The new version of AWA provides:

  • New countries plugin to provide country/region/city data models
  • New settings plugin to control application user settings
  • New tags plugin to easily add tags in applications
  • New <awa:tagList> and <awa:tagCloud> components for tag display
  • Add tags to the question and blog plugins
  • Add comments to the blog post

AWA can be downloaded at http://blog.vacs.fr/vacs/download.html

A live demonstration of various features provided by AWA is available at http://demo.vacs.fr/atlas

A small tutorial explains how you can easily setup a project, design the UML model, and use the features provided by the Ada Web Application framework.

New releases available for Ada Utility, Ada EL, Ada Security, Ada Server Faces, ADO, Dynamo

By stephane.carrez 2014-07-27 14:41:25

A maintenance release is available for the following Ada packages:

Ada Utility Library: Version 1.7.1

  • Support XmlAda 2014
  • Fixed Get_Week_Start/Get_Week_End when the system timezone is different than the asked timezone

Download: http://download.vacs.fr/ada-util/ada-util-1.7.1.tar.gz

Ada EL: Version 1.5.1

  • Fix minor configuration issue with GNAT 2014

Download: http://download.vacs.fr/ada-el/ada-el-1.5.1.tar.gz

Ada Security: Version 1.1.1

  • Fix minor configuration issue with GNAT 2014

Download: http://download.vacs.fr/ada-security/ada-security-1.1.1.tar.gz

Ada Server Faces: Version 1.0.1

  • Fix minor configuration issue with GNAT 2014
  • Fix concurrent issues in facelet and session cache implementation

Download: http://download.vacs.fr/ada-asf/ada-asf-1.0.1.tar.gz

Ada Database Objects: Version 1.0.1

  • Fix minor configuration issue with GNAT 2014

Download: http://download.vacs.fr/ada-ado/ada-ado-1.0.1.tar.gz

Dynamo: Version 0.7.1

  • Fix minor configuration issue with GNAT 2014

Download: http://download.vacs.fr/dynamo/dynamo-0.7.1.tar.gz

Ubuntu 14.04 LTS Ada build node installation

By stephane.carrez 2014-07-25 20:39:29

This short article is a reminder to know the steps and actions in order to add a Ubuntu 14.04 build machine for Jenkins.

The steps are very similar to what I've described in Installation of FreeBSD for a jenkins build node. The virtual machine setup is the same (20G LVM partition, x86_64 CPU, 1Gb memory) and Ubuntu is installed from the ubuntu-14.04.1-server-i386.iso image.

Packages to build Ada software

The following commands install the GNAT Ada compiler with the libraries and packages to build various Ada libraries and projects including AWA.

# GNAT Compiler Installation
sudo apt-get install gnat-4.6 libaws2.10.2-dev libxmlada4.1-dev gprbuild gdb

# Packages to build Ada Utility Library
sudo apt-get install libcurl4-openssl-dev libssl-dev

# Packages to build Ada Database Objects
sudo apt-get install sqlite libsqlite3-dev
sudo apt-get install libmysqlclient-dev
sudo apt-get install mysql-server mysql-client

# Packages to build libaws2-2-10
sudo apt-get install libasis2010-dev libtemplates-parser11.6-dev
sudo apt-get install texinfo texlive-latex-base \
 texlive-generic-recommended texlive-fonts-recommended 

The libaws2-2-10 package was not functional for me (see bug 1348902) so I had to rebuild the Debian package from the sources and install it.

Packages to create Debian packages

When the Ada build node is intended to create Debian packages, the following steps are necessary:

sudo apt-get install dpkg-dev gnupg reprepro pbuilder debhelper quilt chrpath
sudo apt-get install autoconf automake autotools-dev

Packages and setup for Jenkins

Before adding the build node in Jenkins, the JRE must be installed and a jenkins user must exist:

sudo apt-get install openjdk-7-jre subversion
sudo useradd -m -s /bin/bash jenkins

Jenkins will use ssh to connect to the build node so it is good practice to setup a private/public key to allow the Jenkins master node to connect to the slave. On the master, copy the jenkins user's key:

ssh-copy-id target-host

The Ada build node is then added through the Jenkins UI in Manage Jenkins/Manage Nodes.

Jenkins jobs

The jenkins master is now building 7 projects automatically for Ubuntu 14.04: Trusty Ada Jobs

2 comments

Review Web Application: Listing the reviews

By stephane.carrez 2014-07-20 15:50:15

After the creation and setup of the AWA project and the UML model design we have seen how to create a review for the review web application. In this new tutorial, you will understand the details to list the reviews that have been created and published. This tutorial has three steps:

  • First the definition of the database query,
  • The implementation of the Ada review list bean,
  • The writing of the XHTML facelet presentation file.

Step 1: Database query to list the reviews

Let's start with the database query that we will use to retrieve the reviews.

Since we need to access the list of reviews from the XHTML files, we will map the SQL query result to a list of Ada Beans objects. For this, an XML query mapping is created to tell how to map the SQL query result into some Ada record. The XML query mapping is then processed by Dynamo to generate the Ada Beans implementation. The XML query mapping is also read by AWA to get the SQL query to execute.

A template of the XML query mapping can be added to a project by using the dynamo add-query command. The first parameter is the module name (reviews) and the second parameter the name of the query (list). The command will generate the file db/reviews-list.xml.

dynamo add-query reviews list

The generated XML query mapping is an example of a query. You can replaced it or update it according to your needs. The first part of the XML query mapping is a class declaration that describes the type to represent each row returned by our query. Within the class, a set of property definition describes the class attributes with their type and name.

<query-mapping package='Atlas.Reviews.Models'>
    <class name="Atlas.Reviews.Models.List_Info" bean="yes">
        <comment>The list of reviews.</comment>
        <property type='Identifier' name="id">
            <comment>the review identifier.</comment>
        </property>
        <property type='String' name="title">
            <comment>the review title.</comment>
        </property>
        ...
    </class>
</query-mapping>

Following the class declaration, the query declaration describes a query by giving it a name and describing the SQL statement to execute. By having the SQL statement separate and external to the application, we can update, fix and tune the SQL without rebuilding the application. The Dynamo code generator will use the query declaration to generate a query definition that can be referenced and used from the Ada code.

The SQL statement is defined within the sql XML entity. The optional sql-count XML entity is used to associate a count query that can be used for the pagination.

We want to display the review with the author's name and email address. The list will be sorted by date to show the newest reviews first. The SQL to execute is the following:

<query-mapping package='Atlas.Reviews.Models'>
   ...
    <query name='list'>
       <comment>Get the list of reviews</comment>
       <sql>
SELECT
      r.id,
      r.title,
      r.site,
      r.create_date,
      r.allow_comments,
      r.reviewer_id,
      a.name,
      e.email,
      r.text
FROM atlas_review AS r
INNER JOIN awa_user AS a ON r.reviewer_id = a.id
INNER JOIN awa_email AS e ON a.email_id = e.id
ORDER BY r.create_date DESC
    LIMIT :first, :last
       </sql>
       <sql-count>
    SELECT
      count(r.id)
    FROM atlas_review AS r
       </sql-count>
    </query>
</query-mapping>

The query has two named parameters represented by :first and :last. These parameters allow to paginate the list of reviews.

The complete source can be seen in the file: db/reviews-list.xml.

Once the XML query is written, the Ada code is generated by Dynamo by reading the UML model and all the XML query mapping defined for the application. Dynamo merges all the definitions into the target Ada packages and generates the Ada code in the src/model directory. You can use the generate make target:

make generate

or run the following command manually:

dynamo generate db uml/atlas.zargo

From the List_Info class definition, Dynamo generates the List_Info tagged record. The record contains all the data members described in the class XML entity description. The List_Info represents one row returned by the SQL query. The attributes of the List_Info can be accessed from the XHTML files by using UEL expression and the property name defined for each attribute.

To describe the list of rows, Dynamo generates the List_Info_Beans package which instantiates the Util.Beans.Basic.Lists generic package. This provides an Ada vector for the List_Info type and an Ada bean that gives access to the list.

package Atlas.Reviews.Models is
  ...
  type List_Info is new Util.Beans.Basic.Readonly_Bean with record
  ...
   package List_Info_Beans is
      new Util.Beans.Basic.Lists (Element_Type => List_Info);
   package List_Info_Vectors renames List_Info_Beans.Vectors;
   subtype List_Info_List_Bean is List_Info_Beans.List_Bean;
   subtype List_Info_Vector is List_Info_Vectors.Vector;
   Query_List : constant ADO.Queries.Query_Definition_Access;
   ...
end Atlas.Reviews.Models;

The generated code can be seen in src/model/atlas-reviews-models.ads.

Step 2: The review list bean

In order to access the list of reviews from the XHTML facelet file, we must create an Ada bean that provides the list of reviews. This Ada bean is modelized in the UML model and we define:

  • A set of attributes to manage the review list pagination (page, page_size, count)
  • An Ada bean action that can be called from the XHTML facelet file (load)

The Review_List_Bean tagged record will hold the list of reviews for us:

package Atlas.Reviews.Beans is
  ...
   type Review_List_Bean is new Atlas.Reviews.Models.Review_List_Bean with record
      Module       : Atlas.Reviews.Modules.Review_Module_Access := null;
      Reviews      : aliased Atlas.Reviews.Models.List_Info_List_Bean;
      Reviews_Bean : Atlas.Reviews.Models.List_Info_List_Bean_Access;
   end record;
   type Review_List_Bean_Access is access all Review_List_Bean'Class;
end Atlas.Reviews.Beans;

We must now implement the Load operation that was described in the UML model and we are going to use our list query. For this, we use the ADO.Queries.Context to setup the query to retrieve the list of reviews. A call to Set_Query indicates the query that will be used. Since that query needs two parameters (first and last), we use the Bind_Param operation to give the two values. The list of reviews is then retrieved easily by calling the Atlas.Reviews.Models.List operation that was generated by Dynamo.

package body Atlas.Reviews.Beans is
...
   overriding
   procedure Load (Into    : in out Review_List_Bean;
                   Outcome : in out Ada.Strings.Unbounded.Unbounded_String) is
      Session     : ADO.Sessions.Session := Into.Module.Get_Session;
      Query       : ADO.Queries.Context;
      Count_Query : ADO.Queries.Context;
      First       : constant Natural  := (Into.Page - 1) * Into.Page_Size;
      Last        : constant Positive := First + Into.Page_Size;
   begin
      Query.Set_Query (Atlas.Reviews.Models.Query_List);
      Count_Query.Set_Count_Query (Atlas.Reviews.Models.Query_List);
      Query.Bind_Param (Name => "first", Value => First);
      Query.Bind_Param (Name => "last", Value => Last);
      Atlas.Reviews.Models.List (Into.Reviews, Session, Query);
      Into.Count := ADO.Datasets.Get_Count (Session, Count_Query);
   end Load;
end Atlas.Reviews.Beans;

Review list bean creation

The AWA framework must be able to create an instance of the Review_List_Bean type. For this, we have to declare and implement a constructor function that allocates an instance of the Review_List_Bean type and setup some pre-defined values. When the instance is returned, the list of reviews is not loaded.

package body Atlas.Reviews.Beans is
   ...
   function Create_Review_List_Bean (Module : in Atlas.Reviews.Modules.Review_Module_Access)
                                     return Util.Beans.Basic.Readonly_Bean_Access is
      Object  : constant Review_List_Bean_Access := new Review_List_Bean;
   begin
      Object.Module       := Module;
      Object.Reviews_Bean := Object.Reviews'Access;
      Object.Page_Size    := 20;
      Object.Page         := 1;
      Object.Count        := 0;
      return Object.all'Access;
   end Create_Review_List_Bean;
end Atlas.Reviews.Beans;

The constructor function is then registered in the Atlas.Reviews.Modules package within the Initialize procedure. This registration allows to give a name for this constructor function and be able to specify it in the managed-bean bean declaration.

package body Atlas.Reviews.Modules is
   ...
   overriding
   procedure Initialize (Plugin : in out Review_Module;
                         App    : in AWA.Modules.Application_Access;
                         Props  : in ASF.Applications.Config) is
   begin
      ...
      Register.Register (Plugin => Plugin,
                         Name   => "Atlas.Reviews.Beans.Review_List_Bean",
                         Handler => Atlas.Reviews.Beans.Create_Review_List_Bean'Access);
   end Initialize;
end Atlas.Reviews.Modules;

Review list bean declaration

The managed-bean XML declaration associates a name to a constructor function that will be called when the name is needed. The scope of the Ada bean is set to request so that a new instance is created for each HTTP GET request.

  <managed-bean>
    <description>The list of reviews</description>
    <managed-bean-name>reviewList</managed-bean-name>
    <managed-bean-class>Atlas.Reviews.Beans.Review_List_Bean</managed-bean-class>
    <managed-bean-scope>request</managed-bean-scope>
  </managed-bean>

Step 3: Listing the reviews: the XHTML facelet presentation file

To load the reviews to be displayed we will use a JSF 2.2 view action. The review list page has a parameter page that indicates the page number to be displayed. The f:viewParam allows to retrieve that parameter and configure the reviewList Ada bean with it. Then, the f:viewAction defines the action that will be executed after the view parameters are extracted, validated and passed to the Ada bean. In our case, we will call the load operation on our reviewList Ada bean.

<f:metadata>
    <f:viewParam id='page' value='#{reviewList.page}' required="false"/>
    <f:viewAction action="#{reviewList.load}"/>
</f:metadata>

To summarize, the reviewList Ada bean is created, then configured for the pagination and filled with the current page content by running our SQL query.

The easy part is now to render the list of reviews. The XHTML file uses the <h:list> component to iterate over the list items and render each of them. At each iteration, the <h:list> component initializes the Ada bean review to refer to the current row in the review list. We can then access each attribute defined in the XML query mapping by using the property name of that attribute. For example review.title returns the title property.

<h:list var="review" value="#{reviewList.reviews}">
    <div class='review' id="p_#{review.id}">
        <div class='review-title'>
            <h2><a href="#{review.site}">#{review.title}</a></h2>
            <ul class='review-info'>
                <li><span>By #{review.reviewer_name}</span></li>
                <li>
                    <h:outputText styleClass='review-date' value="#{review.date}" converter="dateConverter"/>
                </li>
                <h:panelGroup rendered="#{review.reviewer_id == user.id}">
                    <li>
                        <a href="#{contextPath}/reviews/edit-review.html?id=#{review.id}">#{reviewMsg.review_edit_label}</a>
                    </li>
                    <li>
                        <a href="#"
                           onclick="return ASF.OpenDialog(this, 'deleteDialog', '#{contextPath}/reviews/forms/delete-review.html?id=#{review.id}');">
                           #{reviewMsg.review_delete_label}
                        </a>
                    </li>
                </h:panelGroup>
            </ul>
        </div>
        <awa:wiki styleClass='review-text post-text' value="#{review.text}" format="dotclear"/>
    </div>
</h:list>

Understanding the request flow

Let's see the whole request flow to better understand what happens.

To display the list of reviews, the user's browser makes an HTTP GET request to the page /reviews/list.html. This page maps to the XHTML file web/reviews/list.xhtml that we created in the previous tutorial.

The Ada Server Faces framework handles the request by first reading the XHTML file and building a tree of components that represent the view to render. Within that tree of component, the <f:metadata> component allows to make a pre-initialization of components and beans before the component tree is rendered.

For the pre-initialization, the reviewList Ada bean is created because it is referenced in an EL expression used by the <f:viewParam> component or by the <f:viewAction>. For this creation, the Create_Review_List_Bean constructor that we registered is called. The page attribute is set on the reviewList Ada bean if it was passed as a URL request parameter.

The load action is then called by Ada Server Faces and the current review list page is retrieved by executing the SQL query.

As soon as the load action terminates, the rendering of the component tree can be processed. The reviewList Ada bean contains the information to display and the <h:list> component iterates over the list and renders each row at a time.

C

Conclusion

After the previous tutorial we were able to create a review and populate our database with one or several reviews. We are now able to display the list of reviews to our users.

The next tutorial will focus on using the Votes module to bring some voting capabilities in the review web application. Meanwhile, you may browse and study the sources:

db/reviews-list.xml
atlas-reviews-beans.ads
atlas-reviews-beans.adb
review-list.xhtml

Review Web Application: Creating a review

By stephane.carrez 2014-06-14 20:22:45

In previous tutorials we have seen how to create and setup the project, design the UML model to generate the Ada implementation and the database schema. In this tutorial we will see how to design the page to create a review, implement the operations to create and populate the database with the new review.

Adding the review creation form

We will start with the presentation layer by adding two pages in our web application. A first page will contain the list of reviews and the second page will contain a form to create or update a review.

AWA uses the Facelets technology to allow developers write and design the presentation layer of the web application. This technology is commonly used in J2EE applications. A page is represented by an XML file that contains HTML code, includes some stylesheets, Javascript files and makes the link between the presentation and the web application.

Adding pages

Dynamo provides at least two commands that help in adding presentation files. The add-page command adds a simple page that can be edited and filled with real content. We will use it for the creation of the page to display the list of reviews.

dynamo add-page reviews/list

The add-form command creates another template of page that includes an HTML form to let a user submit some data to the web application.

dynamo add-form reviews/edit-review

These two commands will create the following files and they can now be modified.

./web/reviews/list.xhtml
./web/reviews/edit-review.xhtml
./web/reviews/forms/edit-review-form.xhtml

The create review form

In Facelets, an HTML form is created by using the <h:form> component from the HTML JSF namespace. This component will generate the HTML form tag and it will also manage the form submission.

The ASF framework provides a set of widget components that facilitate the design of web application. The <w:inputText> component renders a title field with an HTML <label> and an HTML <input> text. We will use it to let the user enter the review title and the site URL being reviewed. The HTML <textarea> is provided by the JSF component <h:inputTextArea>. The review submit form is defined by the following XML extract:

<h:form xmlns:h="http://java.sun.com/jsf/html
  xmlns:w="http://code.google.com/p/ada-asf/widget">
  <h:inputHidden id='entity-id' value='#{review.id}' required='false'/>
  <w:inputText title='Title' value='#{review.title}'/>
  <w:inputText title='Site' value='#{review.site}'/>
  <h:inputTextArea rows='20' value='#{review.text}'/>
  <h:commandButton value='Save'
     action='#{review.save}'/>
</h:form>

Before closing the <h:form> component, we will put a <h:commandButton> that will render the form submit button.

How it works

Before going further, let's see how all this works. The principle below is exactly the same for a Java Server Faces application.

First, when the page is rendered the UEL expressions that it contains are evaluated. The #{review.title}, #{review.site} and #{review.text} are replaced by the content provided by the review object which is an Ada Bean provided by the Review_Bean tagged record.

When the page is submitted by the user, the input values submitted in the form are saved in the review bean, again by using the UEL expression. The <h:commandButton> action is then executed. This is also an UEL that indicates a method to invoke on the bean.

To sum up, the UEL makes the binding between the presentation layer in Facelets files and the Ada or Java beans.

The Ada Bean layer provides getter and setter to allow the UEL to retrieve and set values. For this, the Review_Bean tagged record implements two operations that are defined in the Bean interface:

overriding
function Get_Value (From : in Review_Bean;
                    Name : in String) return Util.Beans.Objects.Object;

overriding
procedure Set_Value (From : in out Review_Bean;
                    Name : in String;
                    Value : in Util.Beans.Objects.Object);

The Get_Value operation is called to retrieve one of the Ada Bean member attribute and the Set_Value operation is called during form submission to set the member attribute.

demo-awa-request-flow.png

Then the form button is pressed, the HTML form is submitted and received by the server. The <h:form> component identifies the form submission and each input component will validate the input fields. When everything has been validated, the <h:commandButton> component invokes the Save procedure that is declared as follows in the Review_Bean tagged record:

overriding
procedure Save (Bean : in out Review_Bean;
                Outcome : in out Ada.Strings.Unbounded.Unbounded_String);

In the Ada Bean layer, we have to call the business logic to perform the save operation.

The business logic part is provided by the Ada module whose initial skeleton was generated by Dynamo. That layer is responsible for defining how the data is created, retrieved and modified. As far as we are concerned, this is rather simple since we only have to verify the permission and save the review object within some transaction. In other modules, several objects may be envolved and more complex rules may be defined for the integrity and validity of these objects.

The last part of the architecture is the data model layer that was in fact generated by Dynamo from the UML model. It is responsible for loading and saving Ada objects into the database.

The Review_Bean type declaration

When we designed our UML model, we have created the Review_Bean UML class and gave that class the Bean stereotype. We also declared two operations (save and delete) on that class. With this definition, Dynamo has generated in the Atlas.Reviews.Models package the Review_Bean abstract type. This type is abstract because we have to implement the Save and Delete operations. These are the two operations that can be called by an action such as used by the <h:commandButton> component.

The Atlas.Reviews.Models package is a generated package and it must not be modified. To implement our Ada Bean, we will add the Review_Bean type in our own package: the Atlas.Reviews.Beans package.

For this the Review_Bean type will inherit from the Atlas.Reviews.Models.Review_Bean type and it will implement the required operations. The type declaration looks like this:

package Atlas.Reviews.Beans is
...
type Review_Bean is new Atlas.Reviews.Models.Review_Bean with record
   Module : Atlas.Reviews.Modules.Review_Module_Access := null;
end record;
...

The Review_Bean implementation

The Save and Delete procedure must be implemented and since the whole business logic is managed by the module layer, we just have to call the associated module procedure as follows:

overriding
procedure Save (Bean : in out Review_Bean;
                Outcome : in out Ada.Strings.Unbounded.Unbounded_String);
begin
   Bean.Module.Save (Bean);
end Save;

overriding
procedure Delete (Bean : in out Review_Bean;
                Outcome : in out Ada.Strings.Unbounded.Unbounded_String);
begin
   Bean.Module.Delete (Bean);
end Delete;

The Review_Bean creation

The AWA framework must be able to create the review bean instance when a page is processed. For this, there are three steps that are necessary:

  • we must define a create function whose role is to allocate the Review_Bean instance and return it. At the same time, the function can setup some pre-defined values for the object. The Dynamo tool has generated for us an example of such function so that there is nothing to do.
function Create_Review_Bean (Module : in Atlas.Reviews.Modules.Review_Module_Access)
   return Util.Beans.Basic.Readonly_Bean_Access is
   Object : constant Review_Bean_Access := new Review_Bean;
begin
   Object.Module := Module;
   return Object.all'Access;
end Create_Review_Bean;
  • the creation function must be registered in the AWA framework under a name that identifies the create function. Again, an example of this registration has been generated by Dynamo and we are going to use it as is.
Register.Register (Plugin => Plugin,
                 Name   => "Atlas.Reviews.Beans.Reviews_Bean",
                 Handler => Atlas.Reviews.Beans.Create_Review_Bean'Access);
  • the last step is the configuration step. In the module XML configuration file, we must declare the Ada bean name and indicate what create function must be called to create it. We will use the managed-bean XML declaration that comes from Java Server Faces. We can declare as many Ada beans as we want each of them with a different name.
  <managed-bean>
    <description>An example of a bean (change description and bean name)</description>
    <managed-bean-name>review</managed-bean-name>
    <managed-bean-class>Atlas.Reviews.Beans.Reviews_Bean</managed-bean-class>
    <managed-bean-scope>request</managed-bean-scope>
  </managed-bean>

When the UEL expression #{review.title} is used, the AWA framework looks for the Ada bean represented by review and identified by the managed-bean-name entry. It then calls the create function defined by the managed-bean-class. The Ada bean object is then stored either in the request context, a session context or an application context. This is defined by the managed-bean-scope entry. The request scope means that the Ada bean object is created once for each request. Concurrent page accesses will use their own Ada bean object instance. The session scope means that the Ada bean object is shared between requests on the same session. The application scope means that the Ada bean object is global to the application, shared by every request and every user.

Adding the module operations

Now, we must add two operations on the business logic to save a review and delete a review. The Dynamo code generator provides the add-module-operation command that will help us in this task. Let's run it:

dynamo add-module-operation reviews review Save
dynamo add-module-operation reviews review Delete

The first parameter is the name of the module where the new operation is added. This is the name of the module that was created by using the add-module operation. In our case, this is the reviews module.

The second parameter is the name of the database entity or database table if you prefer.

The add-module-operation command modifies the Ada module specification and body to define and implement the following operation:

package Atlas.Reviews.Modules is
...
procedure Save (Model  : in Review_Module;
                Entity : in out Atlas.Reviews.Models.Review_Ref'Class);
...

The object to save in the Review table is passed as parameter to the Save operation. The procedure body that was generated is rather simple but functional: it just saves the object in the database within a transaction. In many cases it is ready to use but you may also need to modify the operation to either change the implementation or even add new parameters.

Saving our review

Before saving our review entity object, we want to associate it with the current user. We have to know who is the current user and for this we can use the AWA service context. The AWA service context is an object that is provided by the AWA.Services.Contexts package and that provides some useful contextual information for the business logic:

  • It indicates the optional user that is authenticated and is doing the call,
  • It gives access to the database connections that the business logic can use,
  • It allows to manage database transactions.

The current service context is retrieved by using the AWA.Services.Contexts.Current function and we can use the Get_User function to know the current user. The Save procedure implementation is the following:

package ASC renames AWA.Services.Contexts;
procedure Save (Model  : in Review_Module;
                  Entity : in out Atlas.Reviews.Models.Review_Ref'Class) is
   Ctx   : constant ASC.Service_Context_Access := ASC.Current;
   DB    : ADO.Sessions.Master_Session := AWA.Services.Contexts.Get_Master_Session (Ctx);
begin
   Ctx.Start;
   if not Entity.Is_Inserted then
      Entity.Set_Reviewer (Ctx.Get_User);
      Entity.Set_Create_Date (Ada.Calendar.Clock);
   end if;
   Entity.Save (DB);
   Ctx.Commit;
end Save;

Setting up the permissions

Because we want to bring some minimal security to the Review Web Application, we are going to setup some permissions that will be enforced by the business logic layer when a save or delete operation is done. The AWA framework uses the Ada Security to implement and enforce permissions. For this we need:

  • An Ada definition of the permission,
  • Adding a verification to enforce the permission in the new module operations,
  • A definition of the permission rules.
Generating the permission

Dynamo provides the add-permissions command to help us in the first task. It generates some Ada code that declares the permissions. It also provides a default configuration for the new permissions.

dynamo add-permissions reviews review

The first parameter is the name of our module where the new permissions are declared and the second parameter is the name of the database entity. The command will modify the Ada module specification and add the following lines:

package Atlas.Reviews.Modules is
...
package ACL_Create_Reviews is new Security.Permissions.Definition ("review-create");
package ACL_Delete_Reviews is new Security.Permissions.Definition ("review-delete");
package ACL_Update_Reviews is new Security.Permissions.Definition ("review-update");

Each of these package instantiation, declares a single permission identified by a name.

Enforcing security

Now that we have our permission, we can enforce the security in the Save and Delete operation. This is done by using the Check operation provided by the AWA.Permissions package.

To verify that the user has the permission to create a new review, we can use the following call:

AWA.Permissions.Check (Permission => ACL_Create_Reviews.Permission);

Now, if we have a review to modify, we will use the update permission and also give the review object to the Check operation so that it can verify if that particular review can be modified.

AWA.Permissions.Check (Permission => ACL_Update_Reviews.Permission,
                       Entity => Entity);
Configuring the permission

Until now we have created the permission and enforced it in the business logic. We have not defined the rules that tell what is really checked to verify the permission. The configuration part is defined in the XML file config/reviews.xml that was generated when the reviews module was created. The add-permissions command has modified the XML file to provide some default configuration. It has generated a XML permission for the review-create, review-update and review-delete permissions.

The review-create permission is defined as follows:

<auth-permission>
    <name>review-create</name>
</auth-permission>

This XML definition associate the Authenticated Permission controller to the review-create permission. With that controller the permission is granted if the security context has a principal (ie, a user is authenticated).

The review-update permission has another definition that we must change. Basically, we want that only the reviewer that created the review can update the review. For this we will use the entity permission controller provided by AWA. The XML definition is the following:

<entity-permission>
    <name>review-update</name>
    <entity-type>altas_review</entity-type>
    <sql>
       SELECT r.id FROM atlas_review AS r
       WHERE r.id = :entity_id AND r.reviewer_id = :user_id
    </sql>
</entity-permission>

When the permission is checked, the entity permission controller will use the SQL statement to verify the permission. The SQL statement has three parameters:

  1. user_id is the ID of the user associated with the security context. If there is no authentified user, the permission is refused.
  2. entity_id is the ID of the database entity as passed to the Check procedure and propagated to the permission controller.
  3. entity_type is a unique number that identifies the database entity type or database table if you prefer. It is created and setup automatically according to the entity type defined in the entity-type XML member. It is not used in our example.

The above SQL statement verifies that the review exists and was created by the current user.

To learn more about permission, you may look at the AWA permissions documentation.

A word about navigation rules

We have seen that when the review creation form is submitted the <h:commandButton> component has invoked the Save procedure of our Review_Bean object. The review object has been created and saved in the database and we kept the relation between the new review and the user.

We must now decide what should happen for the user to see the result. We could display a new form, update some page content or redirect to a new page. All this is defined by the navigation rules.

The navigation rules is the Java Server Faces mechanism that controls and defines what is the next page or view that must be displayed to a user.

In the definition below, the navigation rule defines that the user is redirected to the page /reviews/list.xhtml if the current page was /reviews/edit-review.xhtml and the operation returned success.

<navigation-rule>
  <from-view-id>/reviews/edit-review.xhtml</from-view-id>
    <navigation-case>
      <from-outcome>success</from-outcome>
      <to-view-id>/reviews/list.xhtml</to-view-id>
      <redirect/>
    </navigation-case>
</navigation-rule>

The Review Web Application video

To help you in creating the review page and see how the whole process looks like in reality, I've created the following short video that details the above tutorial steps.

Conclusion

We have created a review form for the web application and we made the link between the presentation layer and the Ada code through the use of Ada Beans.

We implemented the business logic and saw how the review object is saved in the database. We defined the security of the application by creating specific permissions and we enforced the security in the Save and Delete operations.

The next tutorial will describe how to display a list of reviews for our application.

Ada Web Application: Building the UML model

By stephane.carrez 2014-05-18 14:39:12

In the Ada Web Application: Setting up the project we have seen how to create a new AWA project. In this second article, we will see how to design the UML model, generate the Ada code and create the database tables from our UML design.

Introduction

A Model driven engineering or MDE promotes the use of models to ease the development of software and systems. The Unified Modeling Language is used to modelize various parts of the software. UML is a graphical type modelling language and it has many diagrams but we are only going to use one of them: the Class Diagram.

The class diagram is probably the most powerful diagram to design, explain and share the data model of any application. It defines the most important data types used by an application with the relation they have with each other. In the class diagram, a class represents an abstraction that encapsulates data member attributes and operations. The class may have relations with others classes.

For the UML model, we are going to use ArgoUML that is a free modelization tool that works pretty well. For the ArgoUML setup, we will use two profiles:

  • The Dynamo profile that describes the base data types for our UML model. These types are necessary for the code generator to work correctly.
  • The AWA profile that describes the tables and modules provided by AWA. We will need it to get the user UML class definition.

These UML profiles are located in the /usr/share/dynamo/base/uml directory after Dynamo and AWA are installed. To configure ArgoUML, go in the Edit -> Settings menu and add the directory in the Default XMI directories list. Beware that you must restart ArgoUML to be able to use the new profiles.

demo-awa-argouml-setup.png

Modelize the domain model in UML

The UML model must use a number of Dynamo artifacts for the code generation to work properly. The artifact describes some capabilities and behavior for the code generator to perform its work. Stereotype names are enclosed within << and >> markers. Dynamo uses the following stereotypes:

  • The DataModel stereotype must be applied on the package which contains the model to generate. This stereotype activates the code generation (other packages are not generated).
  • The Table stereotype must be applied to the class. It controls which database table and Ada type will be generated.
  • The PK stereotype must be defined in at most one attribute of the class. This indicates the primary key for the database table. The attribute type must be an integer or a string. This is a limitation of the Ada code generator.
  • The Version stereotype must be applied on the attribute that is used for the optimistic locking implementation of the database layer.

demo-awa-uml-review-table.png

In our UML model, the Review table is assigned the Table stereotype so that an SQL table will be created as well as an Ada tagged type to represent our table. The id class attribute represents the primary key and thus has the PK stereotype. The version class attribute is the database column used by the optimistic locking implementation provided by ADO. This is why is has the Version stereotype. The title, site, create_date, text and allow_comments attributes represent the information we want to store in the database table. They are general purpose attributes and thus don't need any specific stereotype. For each attribute, the Dynamo code generator will generate a getter and a setter operation that can be used in the Ada code.

To tune the generation, several UML tagged values can be selected and added on the table or on a table attribute. By applying a stereotype to the class, several tagged values can be added. By selecting the Tagged Values tab in ArgoUML we can edit and setup new values. For the Review table, the dynamo.table.name tagged value defines the name of the SQL database table, in our case atlas_review.

demo-awa-argouml-review-tagged.png

The text attribute in the Review table is a string that can hold some pretty long text. To control the length of the SQL column, we can set the dynamo.sql.length tagged value and tell what is that length.

demo-awa-argouml-text-tagged.png

Once the UML model is designed, it is saved in the project directory uml. Dynamo will be able to read the ArgoUML file format (.zargo extension) so there is no need to export the UML in XMI.

The Review application UML model

The final UML model of our review application is fairly simple. We just added a table and a bean declaration. To benefit from the user management in AWA, we can use the AWA::Users::Models::User class that is defined in the AWA UML model. The reviewed-by association will create an attribute reviewer in our class. The code generator will generate a Get_Reviewer and Set_Reviewer operation in the Ada code. The SQL table will contain an additional column reviewer that will hold the primary key of the reviewer.

demo-awa-uml-review-model.png

The Review_Bean class is an Ada Bean abstract class that will be generated by the code generator. The Bean stereotype activates the bean code generator and the generator will generate some code support that is necessary to turn the Review_Bean tagged record into an Ada Bean aware type. We will see in the next tutorial that we will only have to implement the save and delete operation that are described in this UML model.

Makefile setup

The Makefile.in that was generated by the Dynamo create-project command must be updated to setup a number of generation arguments for the UML to Ada code generator. Edit the Makefile.in to change:

DYNAMO_ARGS=--package Atlas.Reviews.Models db uml/atlas.zargo

The --package option tells Dynamo to generate only the model for the specified package. The db directory is the directory that will contain the SQL model files.

Once the Makefile.in is updated, the Makefile must be updated by using the following command:

./config.status

Or if you prefer, you may run again the configure script to re-configure the whole project.

We need the code

To run the generator, we can use the generate make target:

make generate

The Dynamo code generator reads the file uml/atlas.zargo and the UML model it contains and generates:

  • the Ada package Atlas.Reviews.Models which contains the definition of the Review table. The model files are created in the directory src/models which is separate from your Ada sources.
  • the SQL files to create the MySQL or SQLite database. Depending on the AWA modules which are used, the generated SQL files will contain additional tables that are used by the AWA modules. The SQL files are generated in the db/mysql and db/sqlite directories.

Let's create the database

Until now we designed our application UML model, we have our Ada code generated, but we need a database with the tables for our application. We can do this by using the create-database command in Dynamo. This command needs several arguments:

  1. The directory that contains the SQL model files. In our case, this is db.
  2. The information to connect to the database, the database name, the user and its password. This information is passed in the form of a database connection string.
  3. The name of the database administration account to connect to the server and create the new database.
  4. The optional password for the database administration account.

If the MySQL server is running on your host and the admin account does not have any password, you can use the following command:

dynamo create-database  db 'mysql://localhost/demo_atlas?user=demo&password=demo' root

The create-database creates the database (demo_atlas) with the tables that are necessary for the application. It also creates the demo user and give it the necessary MySQL grants to connect to the demo_atlas database.

The Review Web Application UML video

To help you in building the UML model and see who the whole process looks like in reality, I've created the following short video that details the above tutorial steps.

Conclusion

Thanks to ArgoUML and Dynamo, generating the Ada model and database tables becomes a simple and fun task. We have not written any line of code yet in this Review Web Application project, everything has been generated but we achieved a big progress:

  • The Review Web Application server is built and can be launched,
  • The database is initialized and contains our application data model schema.

The next tutorial will explain how to design the review form, implement the operations to create and populate the database with the new review.

Dynamo 0.7.0 is available

By stephane.carrez 2014-05-12 19:24:03

Dynamo is a code generator used to generate Ada Web Application or database mappings.

  • New project template to generate Gtk Ada application
  • Register the new module in the application when they are added
  • Update the current testsuite when new tests are added
  • New stereotype <<Limited_Bean>> for Ada bean generation
  • Support for the creation of Debian packages
  • New command add-form and add-module-operation

You can download the new version at http://download.vacs.fr/dynamo/dynamo-0.7.0.tar.gz

Ada Web Application: Setting up the project

By stephane.carrez 2014-05-10 17:50:39

Ada Web Application is a complete framework that allows to write web applications using the Ada language. Through a complete web application, the tutorial explains various aspects in setting up and building an application by using AWA. The tutorial is split in several articles and they are completed by short videos to show how easy the whole process is.

The tutorial assumes that you have already installed the following software on your computer:

The review web application

The review web application allows users to write reviews about a product, a software or a web site and share them to the Internet community. The community can read the review, participate by adding comments and voting for the reviewed product or software.

demo-awa-use-case.png

The AWA framework provides several modules that are ready to be used by our application. The login and user management is handled by the framework so this simplifies a lot the design of our application. We will see in the tutorial how we can leverage this to our review application.

Because users of our review web application have different roles, we will need permissions to make sure that only reviewers can modify a review. We will see how the AWA framework leverages the Ada Security library to enforce the permissions.

The AWA framework also integrates three other modules that we are going to use: the tags, the votes and the comments.

Since many building blocks are already provided by the Ada framework, we will be able to concentrate on our own review application module.

Project creation with Dynamo

The first step is to create the new project. Since creating a project from scratch is never easy we will use the Dynamo tool to build our initial review web application. Dynamo is a command line tool that provides several commands that help in several development tasks. For the project creation we will give:

  • the output directory,
  • the project name,
  • the license to be used for the project,
  • the project author's email address.

Choose the project name with care as it defines the name of the Ada root package that will be used by the project. For the license, you have the choice between GPL v2, GPL v3, MIT, BSD 3 clauses, Apache 2 or some proprietary license.

dynamo -o atlas create-project -l apache atlas Stephane.Carrez@gmail.com

(Of course, change the above email address by your own email address, this is an example!)

The Dynamo project creation will build the atlas directory and populate it with many files:

  • A set of configure, Makefile, GNAT project files to build the project,
  • A set of Ada files to build your Ada web application,
  • A set of presentation files for the web application.

Once the project is created, we must configure it to find the Ada compiler, libraries and so on. This is done by the following commands:

cd atlas
./configure

At this step, you may even build your new project and start it. The make command will build the Ada files and create the bin/atlas-server executable that represents the web application.

make
bin/atlas-server

Once the server is started, you may point your browser to the following location: http://localhost:8080/atlas/index.html

Creating the review module with Dynamo

With the Ada Web Application framework, a web application is composed of modules where each module brings a specific functionality to the application. AWA provides a module for user management, another for comments, tags, votes, and many others. The application can decide to use these modules or not. The AWA module helps in defining the architecture and designing your web application.

For the review web application we will create our own module dedicated for the review management. The module will be an Ada child package of our root project package. From the Ada point of view, the final module will be composed of the following packages:

  • A Modules package represents the business logic of the module. It is provides operations to access and manage the data owned by the module.
  • A Beans package holds the Ada beans that make the link between the presentation layer and business logic.
  • A Models package holds the data model to access the database content. This package is generated from UML and will be covered by a next tutorial.

To help in setting up a new AWA module, the Dynamo tool provides the add-module command. You just have to give the name of the module, which is the name of the Ada child package. Let's create our reviews module now:

dynamo add-module reviews

The command generates the new AWA module and modifies some existing files to register the new module in the application. You can build your web application at this stage even though the new module will not do anything yet for you.

Eclipse setup

Launch you Eclipse and create the new project by going to the File -> New -> Project menu. Choose the Ada Project and uncheck the Use default location checkbox so that you can browse your file system and select the atlas directory.

That's it. If everything went well, you should be able to see the projects files in the Eclipse project explorer.

demo-awa-eclipse-project-explorer.png

The Review Web Application setup video

To help you in setting up and see how the whole process looks like in reality, I've created the following short video that details the above tutorial steps.

Conclusion

The whole process takes less than 3 minutes and gives you the basis to setup and build your new web application. The next tutorial will explain how to use the UML to design and generate the data model for our Review Web Application.

Ada Database Objects 1.0.0 is available

By stephane.carrez 2014-04-27 15:11:01

The Ada Database Objects is a library that allows to easily access database contents for Ada applications.

From a UML class diagram model, the Dynamo code generator generates the Ada mapping files for UML classes. A UML class represents a database table and the generator defines an Ada tagged record in the mapping file to give access to the table. Saving a database record is as simple as invoking the Set procedure generated for each class attribute and calling the Save operation to persist the record in the database.

with Samples.User.Model;
with ADO.Sessions;
...
   DB    : ADO.Sessions.Master_Session
   User  : Samples.User.Model.User_Ref;
...
   User.Set_Name (Name);
   User.Set_Email (Email);
   User.Set_Description ("My friend " & Name);
   User.Set_Status (0);
   User.Save (DB);

The version 1.0.0 of the library brings the following improvements:

  • Support to load query results in Ada bean datasets
  • Added support to load dynamic database drivers
  • Port on FreeBSD
  • Support for the creation of Debian packages

Debian packages are provided in the http://apt.vacs.fr/ubuntu-raring and http://apt.vacs.fr/ubuntu-precise repository.

You can download the new version at http://download.vacs.fr/ada-ado/ada-ado-1.0.0.tar.gz

Upgrading to NetBSD 6.1.4

By stephane.carrez 2014-04-26 20:43:45

I'm using NetBSD for few years now but I've never took time to upgrade the system to a new version. To remember what I did for the upgrade, I've collected below the main steps.

Setup

The system upgrade can be made from the running NetBSD system by using the sysupgrade tool. I have installed the tool by using:

sudo pkgin install sysupgrade

Edit the file /usr/pkg/etc/sysupgrade.conf and setup the RELEASEDIR to point to the new release:

RELEASEDIR="ftp://ftp.NetBSD.org/pub/NetBSD/NetBSD-6.1.4/$(uname -m)"

NetBSD upgrade

Now, we just have to run the sysupgrade command to upgrade the base system and NetBSD kernel and then upgrade the packages by using the pkgin command.

sudo sysupgrade auto
sudo pkgin upgrade
sudo pkgin full-upgrade

And after the upgrade reboot the new kernel:

sudo shutdown -r now

Upgrading FreeBSD for a GCC 4.9 Ada compiler

By stephane.carrez 2014-04-26 10:03:34

After the recent announcement of the GCC 4.9 Ada compiler availability on FreeBSD by John Marino, I decided to do the upgrade and give it some try.

After a quick investigation, I´ve performed the following two simple steps on my FreeBSD host:

sudo pkg update
sudo pkg upgrade

Among several upgrade notifications, I've noted the following messages. The gcc-aux package corresponds to the GCC 4.9 compiler and the gnat-aux package contains the GCC 4.6.4 compiler.

Upgrading gcc-aux: 20130411_3 -> 20140416
Upgrading gnat-aux: 20130412_1 -> 20130412_2
Upgrading aws: 3.1.0.0 -> 3.1.0.0_2

The GCC 4.9 Ada compiler is located in /usr/local/gcc-aux/bin and the GCC 4.6.4 Ada compiler is located in /usr/local/bin.

Once the upgrade was finished, I've rebuilt all my FreeBSD jenkins projects and... it's done.

It worked so well that I wasn't sure whether the right compiler was used. Looking at the generated ALI file there was the V "GNAT Lib v4.9" tag that identifies the new compiler.

Next step is to perform a similar upgrade on NetBSD...

New debian repository with Ada packages

By stephane.carrez 2014-04-06 15:54:30

I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries. The Debian repository includes the binary and development packages for Ada Utility Library, Ada EL, Ada Security, and Ada Server Faces.

Access to the repository

The repository packages are signed with PGP. To get the verification key and setup the apt-get tool, you should run the following command:

wget -O - http://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -

Ubuntu 13.04 Raring

A first repository provides Debian packages targeted at Ubuntu 13.04 raring. They are built with the gnat-4.6 package and depend on libaws-2.10.2-4 and libxmlada4.1-dev. Add the following line to your /etc/apt/sources.list configuration:

deb http://apt.vacs.fr/ubuntu-raring raring main

Ubuntu 12.04 LTS Precise

A second repository contains the Debian packages for Ubuntu 12.04 precise. They are built with the gnat-4.6 package and depend on libaws-2.10.2-1 and libxmlada4.1-dev. Add the following line to your /etc/apt/sources.list configuration:

deb http://apt.vacs.fr/ubuntu-precise precise main

Installation

Once you've added the configuration line, you can install the packages:

sudo apt-get update
sudo apt-get install libada-asf1.0

For the curious, you may browse the repository here.

Ada Server Faces 1.0.0 is available

By stephane.carrez 2014-04-05 19:05:21

Ada Server Faces is a framework that allows to create Web applications using the same design patterns as the Java Server Faces (See JSR 252, JSR 314, or JSR 344). The presentation pages benefit from the Facelets Web template system and the runtime takes advantages of the Ada language safety and performance.

A new release is available with several features that help writing online applications:

  • Add support for Facebook and Google+ login
  • Javascript support for popup and editable fields
  • Added support to enable/disable mouseover effect in lists
  • New EL function util:iso8601
  • New component <w:autocomplete> for input text with autocompletion
  • New component <w:gravatar> to render a gravatar image
  • New component <w:like> to render a Facebook, Twitter or Google+ like button
  • New component <w:panel> to provide collapsible div panels
  • New components <w:tabView> and <w:tab> for tabs display
  • New component <w:accordion> to display accordion tabs
  • Add support for JSF <f:facet>, <f:convertDateTime>, <h:doctype>
  • Support for the creation of Debian packages

You can try the online demonstration of the new widget components and download this new release at http://download.vacs.fr/ada-asf/ada-asf-1.0.0.tar.gz

Ada Security 1.1.0 is available

By stephane.carrez 2014-03-22 14:05:36

The Ada Security library provides a security framework which allows applications to define and enforce security policies. This framework allows users to authenticate by using OpenID Authentication 2.0, OAuth 2.0 or OpenID Connect protocols.

The new version brings the following improvements:

  • New authentication framework that supports OpenID, OpenID Connect, OAuth, Facebook login
  • AWS demo for a Google, Yahoo!, Facebook, Google+ authentication
  • Support to extract JSON Web Token (JWT)
  • Support for the creation of Debian packages

The library can be downloaded at http://download.vacs.fr/ada-security/ada-security-1.1.0.tar.gz

Ada EL 1.5.0 is available

By stephane.carrez 2014-03-19 21:49:33

Ada EL is a library that implements an expression language similar to JSP and JSF Unified Expression Languages (EL). The expression language is the foundation used by Java Server Faces and Ada Server Faces to make the necessary binding between presentation pages in XML/HTML and the application code written in Java or Ada.

The presentation page uses an UEL expression to retrieve the value provided by some application object (Java or Ada). In the following expression:

#{questionInfo.question.rating}

the EL runtime will first retrieve the object registered under the name questionInfo and look for the question and then rating data members. The data value is then converted to a string.

The new release is available for download at http://download.vacs.fr/ada-el/ada-el-1.5.0.tar.gz

This version brings the following improvements:

  • EL parser optimization (20% to 30% speed up)
  • Support for the creation of Debian packages