Bacula maintains a catalog of files in a database. Over time, the database grows and despite some automatic purge and job cleanup, some information remains that is no longer necessary. This article explains how to remove some dead records from the Bacula catalog.

Bacula maintains a list of backup jobs that have been executed in the job table. For each job, it keeps the list of files that have been saved in the file table. When you do a restore, you somehow select the job to restore and pick files from that job. There should not exist any file entry associated with a non existing job. Unfortunately this is not the case. I've found that some files (more than 2 millions entries) were pointing to some job that did not exist.

Discovering dead jobs still referenced

The first step is to find out which job has been deleted and is still referenced by the file table. First, let's create a temporary table that will hold the job ids associated with the files.

mysql> create temporary table job_files (id bigint);

The use of a temporary table was necessary in my case because the file table is so big and the ReadyNAS so slow that scanning the database takes too much time.

Now, we can populate the temporary table with the job ids:

mysql> insert into job_files select distinct file.jobid from file;
Query OK, 350 rows affected (8 min 53.26 sec)
Records: 350  Duplicates: 0  Warnings: 0

The list of jobs that have been removed but are still referenced by a file is obtained by:

mysql> select job_files.id from job_files
 left join job on job_files.id = job.jobid
 where job.jobid is null;
+------+
| id   |
+------+
| 2254 | 
| 2806 | 
+------+
2 rows in set (0.05 sec)

Deleting Dead Files

Deleting all the file records in one blow was not possible for me because there was too many files to delete and the mysql server did not have enough resources on the ReadyNAS to do it. I had to delete these records in batch of 100000 files, the process was repeated several times (each delete query took more than 2mn!!!).

mysql> delete from file where jobid = 2254 limit 100000;

Conclusion

This cleanup process allowed me to reduce the size of the file table from 10 millions entries to 7 millions. This improves the database performance and speeds up the Bacula catalog backup process.

Memory optimization reveals sometimes some nice surprise. I was interested to analyze the memory used by the Ada Server Faces framework. For this I've profiled the unit tests program. This includes 130 tests that cover almost all the features of the framework.

Memory analysis with Valgrind Massif

Massif is a Valgrind tool that is used for heap analysis. It does not require the application to be re-compiled and can be used easily. The application is executed by using Valgrind and its tool Massif. The command that I've used was:

valgrind --tool=massif --threshold=0.1 \
   --detailed-freq=1 --alloc-fn=__gnat_malloc \
   bin/asf_harness -config test.properties

The valgrind tool creates a file massif.out.NNN which contains the analysis. The massif-visualizer is a graphical tool that reads the file and allows you to analyze the results. It is launched as follows:

massif-visualizer massif.out.19813

(the number is the pid of the process that was running, replace it accordingly).

The tool provides a graphical representation of memory used over the time. It allows to highlight a given memory snapshot and understand roughly where the memory is used.

While looking at the result, I was intrigued by a 1MB allocation that was made several times and then released (It creates these visual spikes and it correspond to the big red horizontal bar that appears visually). It was within the sax-utils.adb file that is part of the XML/Ada library. Looking at the implementation, it turns out that it allocates a hash table with 65536 entries. This allocation is done each time the sax parser is created. I've reduced the size of this hash table to 1024 entries. If you want to do it, change the following line in sax/sax-symbols.ads (line 99):

   Hash_Num : constant := 2**16;

by:

   Hash_Num : constant := 2**10;

After building, checking there is no regression (yes, it works), I've re-run the Massif tool and here are the results.

The peak memory was reduced from 2.7Mb to 2.0Mb. The memory usage is now easier to understand and analyse because the 1Mb allocation is gone. Other memory allocations have more importance now. But wait. There is more! My program is now faster!

Cache analysis with cachegrind

To understand why the program is now faster, I've used Cachegrind that measures processor cache performance. Cachegrind is a cache and branch-prediction profiler provided by Valgrind as another tool. I've executed the tool with the following command:

valgrind --tool=cachegrind \
    bin/asf_harness -config test.properties

I've launched it once before the hash table correction and once after. Similar to Massif, Cachegrind generates a file cachgrind.NNN that contains the analysis. You analyze the result by using either cg_annotate or kcachegrind. Having two Cachegrind files, I've used cg_diff to somehow get diff between the two executions.

cg_diff cachegrind.out.24198 cachegrind.out.23286 > cg.out.1
cg_annotate cg.out.1

Before the fix, we can see in Cachegrind report that the most intensive memory operations are performed by Sax.Htable.Reset operation and by the GNAT operation that initializes the Sax.Symbols.Symbol_Table_Record type which contains the big hash table. Dr is the number of data reads, D1mr the L1 cache read miss and Dw is the number of writes with D1mw representing the L1 cache write miss. Having a lot of cache miss will slow down the execution: L1 cache access requires a few cycles while main memory access could cost several hundreds of them.

--------------------------------------------------------------------------------
         Dr      D1mr          Dw      D1mw 
--------------------------------------------------------------------------------
212,746,571 2,787,355 144,880,212 2,469,782  PROGRAM TOTALS

--------------------------------------------------------------------------------
        Dr      D1mr         Dw      D1mw  file:function
--------------------------------------------------------------------------------
25,000,929 2,081,943     27,672       244  sax/sax-htable.adb:sax__symbols__string_htable__reset
       508       127 33,293,050 2,080,768  sax/sax-htable.adb:sax__symbols__symbol_table_recordIP
43,894,931   129,786  7,532,775     8,677  ???:???
15,021,128     4,140  5,632,923         0  pthread_getspecific
 7,510,564     2,995  7,510,564    10,673  ???:system__task_primitives__operations__specific__selfXnn
 6,134,652    41,357  4,320,817    49,207  _int_malloc
 4,774,547    22,969  1,956,568     4,392  _int_free
 3,753,930         0  5,630,895     5,039  ???:system__task_primitives__operations(short,...)(long, float)

With a smaller hash table, the Cachegrind report indicates a reduction of 24,543,482 data reads and 32,765,323 data writes. The cache read miss was reduced by 2,086,579 (74%) and the cache write miss was also reduced by 2,056,247 (83% reduction!).

With a small hash table, the Sax.Symbols.Symbol_Table_Record gets initialized quicker and its cleaning needs less memory accesses, hence CPU cycles. By having a smaller hash table, we also benefit from less cache miss: using a 1Mb hash table flushes a big part of the data cache.

--------------------------------------------------------------------------------
         Dr    D1mr          Dw    D1mw 
--------------------------------------------------------------------------------
188,203,089 700,776 112,114,889 413,535  PROGRAM TOTALS

--------------------------------------------------------------------------------
        Dr    D1mr        Dw   D1mw  file:function
--------------------------------------------------------------------------------
43,904,760 120,883 7,532,577  8,407  ???:???
15,028,328      98 5,635,623      0  pthread_getspecific
 7,514,164     288 7,514,164  9,929  ???:system__task_primitives__operations__specific__selfXnn
 6,129,019  39,636 4,305,043 48,446  _int_malloc
 4,784,026  18,626 1,959,387  3,261  _int_free
 3,755,730       0 5,633,595  4,390  ???:system__task_primitives__operations(short,...)(long, float)
 2,418,778      65 2,705,140     14  ???:system__tasking__initialization__abort_undefer
 3,839,603   2,605 1,283,289      0  malloc

Conclusion

Running massif and cachegrind is very easy but it may take some time to figure out how to understand and use the results. A big hash table is not always a good thing for an application. By creating cache misses it may in fact slow down the application. To learn more about this subject, I recommend the excellent document What Every Programmer Should Know About Memory written by Ulrich Drepper.

Ada Web Application is a framework to build web applications.

• AWA uses Ada Server Faces for the web framework. This framework is using several patterns from the Java world such as Java Server Faces and Java Servlets.
• AWA provides a set of ready to use and extendable modules that are common to many web application. This includes managing the login, authentication, users, permissions.
• AWA uses an Object Relational Mapping that helps in writing Ada applications on top of MySQL or SQLite databases. The ADO framework allows to map database objects into Ada records and access them easily.
• AWA is a model driven engineering framework that allows to design the application data model using UML and generate the corresponding Ada code.

The new version of AWA provides:

• New jobs plugin to manage asynchronous jobs,
• New storage plugin to manage a storage space for application documents,
• New votes plugin to allow voting on items,
• New question plugin to provide a general purpose Q&A.

AWA can be downloaded at http://code.google.com/p/ada-awa/downloads/list

A live demonstration of various features provided by AWA is available at http://demo.vacs.fr/atlas

Dynamo is a tool to help developers write some types of Ada Applications which use the Ada Server Faces or Ada Database Objects frameworks. Dynamo provides several commands to perform one specific task in the development process: creation of an application, generation of database model, generation of Ada model, creation of database.

The new version of Dynamo provides:

• A new command build-doc to extract some documentation from the sources,
• The generation of MySQL and SQLite schemas from UML models,
• The generation of Ada database mappings from UML models,
• The generation of Ada beans from the UML models,
• A new project template for command line tools using ADO,
• A new distribution command to merge the resource bundles.

The most important feature is probably the Ada code generation from a UML class diagram. With this, you can design the data model of an application using ArgoUML and generate the Ada model files that will be used to access the database easily through the Ada Database Objects library. The tool will also generate the SQL database schema so that everything is concistent from your UML model, to the Ada implementation and the database tables.

The short tutorial below indicates how to design a UML model with ArgoUML, generate the Ada model files, the SQL files and create the MySQL database.

The Dynamo tool is available at http://code.google.com/p/ada-gen.

To build Dynamo, you will need:

• Ada ASF 0.5.0
• Ada ADO 0.4.0
• Ada EL 1.4.2
• Ada Util 1.6.1
• XML Ada 4.x or 3.x

The Ada Database Objects is an Object Relational Mapping for the Ada05 programming language. It allows to map database objects into Ada records and access databases easily. Most of the concepts developped for ADO come from the Java Hibernate ORM. ADO supports MySQL and SQLite databases.

The new version brings:

• Support to reload query definitions,
• It optimizes session factory implementation,
• It allows to customize the MySQL database connection by using MySQL SET

This version can be downloaded at http://code.google.com/p/ada-ado/downloads/list.

Ada Server Faces is an Ada implementation of several Java standard web frameworks.

• The Java Servlet (JSR 315) defines the basis for a Java application to be plugged in Web servers. It standardizes the way an HTTP request and HTTP response are represented. It defines the mechanisms by which the requests and responses are passed from the Web server to the application possibly through some additional filters.
• The Java Unified Expression Language (JSR 245) is a small expression language intended to be used in Web pages. Through the expressions, functions and methods it creates the link between the Web page template and the application data identified as beans.
• The Java Server Faces (JSR 314 and JSR 344) is a component driven framework which provides a powerful mechanism for Web applications. Web pages are represented by facelet views (XHTML files) that are modelized as components when a request comes in. A lifecycle mechanism drives the request through converters and validators triggering events that are received by the application. Navigation rules define what result view must be rendered and returned.

Ada Server Faces gives to Ada developers a strong web framework which is frequently used in Java Web applications. On their hand, Java developers could benefit from the high performance that Ada brings: apart from the language, they will use the same design patterns.

The new version of Ada Server Faces is available and brings the following changes:

• The Security packages was moved in a separate project: Ada Security,
• New demo to show OAuth and Facebook API integration,
• Integrated jQuery 1.8.3 and jQuery UI 1.9.2,
• New converter to display file sizes,
• Javascript support was added for click-to-edit behavior,
• Add support for JSF session beans,
• Add support for servlet error page customization,
• Allow navigation rules to handle exceptions raised by Ada bean actions,
• Support the JSF 2.2 conditional navigation,
• New functions fn:escapeXml and fn:replace.

The new version can be downloaded on the Ada Server Faces project page. A live demo is available at http://demo.vacs.fr/demo.

Ada Utility Library is a collection of utility packages for Ada 2005. This release is provided as workarround release to avoid the gcc 4.7 bug 53737. It allows to build Ada Utility Library with gcc 4.7.2.

Ada Security is a security framework which allows web applications to define and enforce security policies. The framework allows users to authenticate by using OpenID Authentication 2.0. Ada Security also defines a set of client methods for using the OAuth 2.0 protocol.

• A security policy manager defines and implements the set of security rules that specify how to protect the system or the resources.
• A user is authenticated in the application. Authentication can be based on OpenID or another system.
• A security context holds the contextual information that allows the security policy manager to verify that the user is allowed to access the protected resource according to the policy rules.

The Ada Security framework can be downloaded at Ada Security project page.

The framework is the core security framework used by Ada Server Faces and Ada Web Application to protect access to resources.

Ada Utility Library is a collection of utility packages for Ada 2005. A new version is available which provides:

• Support for HTTP clients (curl, AWS, ...)
• Support for REST APIs using JSON
• New operations To_JSON and From_JSON for easy object map serialization
• Added a listeners to help implementing the observer/listener design patterns
• Added support for wildcard mapping in serialization framework
• New option -d <dir> for the unit test harness to change the working directory,
• New example facebook.adb to show the REST support.

It has been compiled and ported on Linux, Windows and Netbsd (gcc 4.4, GNAT 2011, gcc 4.6.3). You can download this new version at http://code.google.com/p/ada-util/downloads/list.

Since Ubuntu 12.04 upgrade, the Thunderbird scrollbar position were not visible any more. The scrollbar works but you have no visual feedback to know where you are in your long lists. Annoying!!! It turns out that this was a feature of the overlay scrollbar.

To restore the Thunderbird scrollbar, remove the following packages which are causing these troubles:

sudo apt-get remove overlay-scrollbar liboverlay-scrollbar-0.2-0 liboverlay-scrollbar3-0.2-0

Since I've disabled it, I realized many others have the issue. The How do I disable overlay scrollbars? Q&A gives other hints.

Ada BFD is an Ada binding for the GNU Binutils BFD library.

It allows to read binary ELF, COFF files by using the GNU BFD. The Ada BFD library allows to:

• list and scan the ELF sections of an executable or object file,
• get the content of the ELF sections,
• get access to the symbol table,
• use the BFD disassembler

Version 1.0 of this Ada binding library is now available on Ada BFD. This new release bring the following changes:

• Fixed installation of library
• Added examples for BfdAda
• Add support to use the disassembler

The GNU Binutils provides support for reading and writing program files in various formats such as ELF, COFF. This support is known as the BFD, the Binary File Descriptor library (or the Big F*cking Deal). This article illustrates how an Ada application can use the BFD library to have access to a program symbol table.

Declarations

The Ada BFD library provides a set of Ada bindings that give access to the BFD library. A binary file such as an object file, an executable or an archive is represented by the Bfd.Files.File_Type limited type. The symbol table is represented by the Bfd.Symbols.Symbol_Table limited type. These two types hold internal data used and managed by the BFD library.

with Bfd.Files;
with Bfd.Sections;
with Bfd.Symbols;
...
  File    : Bfd.Files.File_Type;
  Symbols : Bfd.Symbols.Symbol_Table;

Opening the BFD file

The Open procedure must be called to read the object or executable file whose path is given as argument. The File_Type parameter will be initialized to get access to the binary file information. The Check_Format function must then be called to let the BFD library gather the file format information and verify that it is an object file or an executable.

Bfd.Files.Open (File, Path, "");
if Bfd.Files.Check_Format (File, Bfd.Files.OBJECT) then
    ...
end if;

Loading the symbol table

The symbol table is loaded by using the Read_Symbols procedure.

   Bfd.Symbols.Read_Symbols (File, Symbols);

The resources used by the symbol table will be freed when the symbol table instance is finalized.

Looking for a symbol

Once the symbol table is loaded, we can use the Get_Symbol function to find a symbol knowing its name. If the symbol is not found, a Null_Symbol is returned.

Sym  : Bfd.Symbols.Symbol := Bfd.Symbols.Get_Symbol (Symbols, "_main");
...
if Sym /= Bfd.Symbols.Null_Symbol then
   --  Symbol found
end if;

Each symbol has the following set of information:

• A name (it may not be unique),
• A set of flags that describe the symbol (global, local, weak, constructor, TLS, ...),
• A symbol value,
• A section to which the symbol belongs.

Sec   : constant Bfd.Sections.Section := Bfd.Symbols.Get_Section (Sym);
Flags : constant Bfd.Symbol_Flags     := Bfd.Symbols.Get_Flags (Sym);
Value : constant Bfd.Symbol_Value     := Bfd.Symbols.Get_Value (Sym);

Before interpreting and using the symbol value returned by Get_Value, you must look at the section to check for an undefined symbol. Indeed, undefined symbols being not yet resolved by the linker they have no value and no section. You can check that by using the Is_Undefined_Section function:

if Bfd.Sections.Is_Undefined_Section (Sec) then
   Ada.Text_IO.Put_Line ("undefined symbol");
end if;

When the symbol is defined, you must look at the flags and also the section information to know more about it.

if (Flags and Bfd.Symbols.BSF_GLOBAL) /= 0 then
  Ada.Text_IO.Put_Line ("global symbol in section "
               & Bfd.Sections.Get_Name (Sec)
               & " := " & Bfd.Symbol_Value'Image (Value));

elsif (Flags and Bfd.Symbols.BSF_LOCAL) /= 0 then
  Ada.Text_IO.Put_Line ("local symbol in section "
               & Bfd.Sections.Get_Name (Sec)
               & " := " & Bfd.Symbol_Value'Image (Value));

else
  Ada.Text_IO.Put_Line ("other symbol in section "
                                     & Bfd.Sections.Get_Name (Sec)
                                     & " := " & Bfd.Symbol_Value'Image (Value));
end if;

Conclusion and references

Reading an executable symbol table has been made fairly simple with the use of the Ada BFD library. Furthermore, the library allows to scan the sections, read their content and even use the BFD disassembler.

symbol.adb
BFD Documentation

If you see the following message when an application stops, the fix is for you.

Error in my_thread_global_end(): 5 threads didn't exit

The message comes from the MySQL C connector when the library cleans up before the program exits.

The MySQL C client connector has a bug that exists for many years in multi-threaded applications: it does not cleanup correctly threads that use the library. Interestingly, MySQL documentation says you must call mysql_thread_end yourself before a thread stops. This is a shame as it forces developers to find impossible and dirty workaround, one of them is nicely explained in Bug 846602 - MySQL C API missing THR_KEY_mysys.

Root cause

The origin of the bug comes from a wrong use or a miss-understanding of the POSIX Threads. Indeed, the library uses the thread-specific data management through pthread_getspecific and pthread_setspecific to store some dynamically allocated object associated with each thread. The POSIX 1003.1c standard has defined a cleanup mechanism that can be used to automatically reclaim storage that was allocated for thread specific data.

The pthread_key_create is the operation called to create a new thread specific data key. When the key is created, it allows to register a cleanup handler that will be called with the thread specific data before the thread exits. A cleanup handler is as simple as the following function:

static void thread_cleanup(void* data)
{
    free(data);
}

With such cleanup handler, the pthread data key must be created as follows:

static pthread_key_t  THR_KEY_mysys;
...
  pthread_key_create(&THR_KEY_mysys, thread_cleanup);

The MySQL C connector does not specify any cleanup handler when creating the pthread data key. The patch attached to this post defines the cleanup handler and installs it as described above.

Fixing MySQL C connector

To fix the MySQL connector, you will need the sources, apply the patch and build the sources. Download the Connector/C sources from MySQL site. The patch is against 6.0.2 but you should be able to apply it to other versions easily (it patches the file mysys/my_thr_init.c that was not changed since 2009). Extract the sources and apply the patch:

tar xzf mysql-connector-c-6.0.2.tar.gz
patch < fix-mysql-connector.diff

To build the connector, you will need CMake. The build process is explained in the sources (read BUILD.unix).

cmake -G "Unix Makefiles"
make
make install

Beware that before building you may have to change some configuration setup according to your system (check include/mysql_version.h).

Patch

fix-mysql-connector.diff

Through this article you will learn how to use the OAuth 2.0 framework to let an application access service provider APIs such as Facebook API, Google+ API and others. Althought this article uses Ada as programming language and Facebook as service provider, most part also applies to other programming languages and other service providers.

Overview

OAuth 2.0 is an open standard for authorization. It is used by service providers as authorization mechanism for most of their APIs. The authorization workflow is depicted below:

• [1], first a user goes in the application which displays a link to the OAuth API provider asking the user to grant access to his data for the application,
• [2], the user clicks on the authenticate link and grants access to the application,
• [3.1], The OAuth server redirects the user to a callback URL and it provides an application grant code,
• [3.3], The application ask the API provider to transform the grant code to an access token,
• [4] The application invokes the API provider with the access token

Registering the application

The first step is to register the application in the service provider (Facebook, Google+, Twitter, ...). This registration process is specific to the provider. From this registration, several elements will be defined:

• An application id is allocated, This identifier is public. This is the client_id parameter in OAuth 2.0.
• An application secret is defined. It must be kept private to the application. This is the secret parameter in OAuth 2.0.
• A callback URL or domain name is registered in the service provider. As far as I'm concerned, I had to register the domain demo.vacs.fr.

Facebook OAuth

For the OAuth authorization process, we will use the Ada Security library and its Application type. We will extend that type to expose some EL variables and an EL method that will be used in the authorization process. The Ada Server Faces Application Example part 3: the action bean explains how to do that and many details will no be covered by this article.

type Facebook_Auth is new Security.OAuth.Clients.Application
     and Util.Beans.Basic.Readonly_Bean
     and Util.Beans.Methods.Method_Bean with private;

FB_Auth      : aliased Facebook.Facebook_Auth;

Before anything we have to initialize the Application type to setup the application identifier, the application secret, the provider URL and a callback URL.

FB_Auth.Set_Application_Identifier ("116337738505130");
FB_Auth.Set_Application_Secret ("xxxxxxxxxxx");
FB_Auth.Set_Application_Callback ("http://demo.vacs.fr/oauth/localhost:8080/demo/oauth_callback.html");

The first step in the OAuth process is to build the authorization page with the link that redirects the user to the service provider OAuth authorization process. The link must be created by using the Get_State and Get_Auth_Params functions provided by the Application type. The first one generates a secure unique key that will be returned back by the service provider. The second one builds a list of request parameters that are necessary for the service provider to identify the application and redirect the user back to the application once the authentication is done.

Id : constant String := "...";
State  : constant String := FB_Auth.Get_State (Id);
Params : constant String := FB_Auth.Get_Auth_Params (State, "read_stream");

For a Facebook authorization process, the URI would be created as follows:

URI : constant String := "https://www.facebook.com/dialog/oauth?" & Params;

For another service provider, the process is similar but the URL is different.

OAuth callback

When the user has granted access to his data, he will be redirected to the callback defined by the application. Most service providers will require that the OAuth callback be a public URL. If you want to run you application on localhost (which is the case when you are developing), you will need a second redirection. If you are using the Apache server, you can easily setup a rewrite rule:

RewriteRule ^/oauth/localhost:([0-9]+)/(.*) http://localhost:$1/$2 [L,R=302]

With the above rewrite rule, the callback given to the OAuth provider would look like:

http://demo.vacs.fr/oauth/localhost:8080/demo/oauth_callback.html

The OAuth provider will first redirect to the public internet site which will redirect again to localhost and port 8080.

Getting the OAuth access token

The next step is to receive the code parameter from the callback which grants the application access to the service provider API. For this, we will use an XHTML view file and a view action that will be executed when the page is displayed. When this happens, the EL method authenticate will be called on the facebook bean (ie, our FB_Auth instance).

<f:view xmlns:f="http://java.sun.com/jsf/core">
    <f:metadata>
        <f:viewAction action="#{facebook.authenticate}"/>
    </f:metadata>
</f:view>

The Authenticate procedure extracts from the request the OAuth state and code parameters. It verifies that the state parameter is a valid key that we submitted and it makes a HTTP POST request on the OAuth service provider to transform the code into an access token. This step is handled by the Ada Security library through the Request_Access_Token operation.

procedure Authenticate (From    : in out Facebook_Auth;
                        Outcome : in out Ada.Strings.Unbounded.Unbounded_String) is
   use type Security.OAuth.Clients.Access_Token_Access;

   F       : constant ASF.Contexts.Faces.Faces_Context_Access := ASF.Contexts.Faces.Current;
   State : constant String := F.Get_Parameter (Security.OAuth.State);
   Code  : constant String := F.Get_Parameter (Security.OAuth.Code);
   Session : ASF.Sessions.Session := F.Get_Session;
begin
   Log.Info ("Auth code {0} for state {1}", Code, State);
   if Session.Is_Valid then
      if From.Is_Valid_State (Session.Get_Id, State) then
         declare
            Acc : Security.OAuth.Clients.Access_Token_Access
              := From.Request_Access_Token (Code);
         begin
            if Acc /= null then
               Log.Info ("Access token is {0}", Acc.Get_Name);
               Session.Set_Attribute ("access_token",
                                      Util.Beans.Objects.To_Object (Acc.Get_Name));
            end if;
         end;
      end if;
   end if;
end Authenticate;

The access token must be saved in the user session or another per-user safe storage so that it can be retrieved later on. The access token can expire and if this happens a fresh new access token must be obtained.

Getting the Facebook friends

Until now we have dealt with the authorization process. Let's look at using the service provider API and see how the Ada Utility Library will help in this task.

Defining the Ada beans

To represent the API result, we will use an Ada bean object that can easily be used from a presentation page. For the Facebook friend, a name and an identifier are necessary:

type Friend_Info is new Util.Beans.Basic.Readonly_Bean with record
   Name : Util.Beans.Objects.Object;
   Id   : Util.Beans.Objects.Object;
end record;
type Friend_Info_Access is access all Friend_Info;

Having a bean type to represent each friend, we will get a list of friends by instantiating the Ada bean Lists package:

package Friend_List is new Util.Beans.Basic.Lists (Element_Type => Friend_Info);

Mapping JSON or XML to Ada

The Ada Utility library provides a mechanism that parses JSON or XML and map the result in Ada objects. To be able to read the Facebook friend definition, we have to define an enum and implement a Set_Member procedure. This procedure will be called by the JSON/XML parser when a given data field is recognized and extracted.

type Friend_Field_Type is (FIELD_NAME, FIELD_ID);

procedure Set_Member (Into : in out Friend_Info;
                      Field : in Friend_Field_Type;
                      Value : in Util.Beans.Objects.Object);

The Set_Member procedure is rather simple as it just populates the data record with the value.

procedure Set_Member (Into : in out Friend_Info;
                      Field : in Friend_Field_Type;
                      Value : in Util.Beans.Objects.Object) is
begin
   case Field is
      when FIELD_ID =>
         Into.Id := Value;

      when FIELD_NAME =>
         Into.Name := Value;

   end case;
end Set_Member;

The mapper is a package that defines and controls how to map the JSON/XML data fields into the Ada record by using the Set_Member operation. We just have to instantiate the package. The Record_Mapper generic package will map JSON/XML into the Ada record and the Vector_Mapper will map a list of JSON/XML elements following a given structure into an Ada vector.

package Friend_Mapper is
  new Util.Serialize.Mappers.Record_Mapper (Element_Type        => Friend_Info,
                                            Element_Type_Access => Friend_Info_Access,
                                            Fields              => Friend_Field_Type,
                                            Set_Member          => Set_Member);

package Friend_Vector_Mapper is
   new Util.Serialize.Mappers.Vector_Mapper (Vectors        => Friend_List.Vectors,
                                             Element_Mapper => Friend_Mapper);

Now we need to control how the JSON/XML fields are mapped to our Ada fields. For this we have to setup the mapping. The Facebook JSON structure is so simple that we can use the default mapping provided by the mapper. For this we use the Add_Default_Mapping procedure. We also have to tell what is the JSON mapping used by the friend vector mapper.

Friend_Map        : aliased Friend_Mapper.Mapper;
Friend_Vector_Map : aliased Friend_Vector_Mapper.Mapper;
...
   Friend_Map.Add_Default_Mapping;
   Friend_Vector_Map.Set_Mapping (Friend_Map'Access);

Creating the REST client

Now it would be nice if we could get an operation that invokes the service provider API through an HTTP GET operation and put the result in our Ada object. The Facebook friends API returns a list of friends which correspond to our Friend_List.Vectors. To get our operation, we just have to instantiate the Rest_Get_Vector operation with our vector mapper (the generic parameter is a package name).

procedure Get_Friends is
  new Util.Http.Rest.Rest_Get_Vector (Vector_Mapper => Friend_Vector_Mapper);

Calling the REST client

Invoking the service provider API is now as simple as calling a procedure. The URI must include the access token as parameter. The HTTP GET operation must be made using SSL/TLS since this is part of OAuth 2.0.

 List  : Friend_List.List_Bean;
...
   Get_Friends ("https://graph.facebook.com/me/friends?access_token="
                      & Token,
                      Friend_Vector_Map'Access,
                      "/data",
                      List.List'Access);

Now you are ready to use and access the user's data as easily as other information...

References

facebook.ads
facebook.adb
Facebook API

Ada Web Application is a framework to build web applications easily on top of Ada Server Faces, Ada Database Objects and Ada Web Server.

The new version of the framework provides:

• A new event framework with configurable action listeners,
• Persistent event queues for the event framework,
• A new blog module and wiki engine supporting Google Wiki, Creole, MediaWiki, phpPP and Dotclear syntax,
• New mail UI components allowing to generate and send email easily with the ASF presentation pages,
• A new Javascript plugin Markedit with jQuery Markedit (MIT License)

This new version can be downloaded at http://code.google.com/p/ada-awa/downloads/list (downloading the awa-all package is recommended to get the project and its dependencies).

A demo of an AWA application is available at http://demo.vacs.fr/atlas/

AWA is a framework to build web applications on top of various robust components.

• AWA uses Ada Server Faces for the web framework. This framework is using several patterns from the Java world such as Java Server Faces and Java Servlets.
• AWA is architectured arround modules and plugins that allow to build, re-use and extend modules made of Ada code, Web pages or Javascript.
• AWA provides a set of ready to use and extendable plugins that are common to many web application. This includes managing the login, authentication, users, permissions, a mail plugin, a blog plugin, a Javascript light editor.
• AWA uses an Object Relational Mapping that helps in writing Ada applications on top of MySQL or SQLite databases. The ADO framework allows to map database objects into Ada records and access them easily.
• AWA integrates a configurable event service which allows plugins to easily interact and connect with each other (either synchronously or asynchronously). The event service provided by AWA is heavily inspired from the Java Message Service.

To learn more on how to create easily a web application using AWA, look at the 4 minutes video.

The Atlas Web Application Demonstrator is a demonstration of an application using this AWA framework. Sources of the Atlas demonstrator are available on GitHub project: Atlas.

Dynamo is a tool to help developers write an Ada Web Application using the Ada Server Faces and the Ada Database Objects frameworks. Dynamo provides several commands to perform one specific task in the development process: creation of an application, generation of database model, generation of Ada model, creation of database.

The new version of Dynamo provides:

• Support multi-line comments in XML mappings,
• Generate List_Bean types for the XML mapped queries,
• Add support for Ada enum generation,
• Add test template generation,
• Add AWA service template generation,
• Add support for blob model mapping,
• New command 'add-ajax-form', 'add-query', 'dist', 'create-plugin'

The Dynamo tool is available at http://code.google.com/p/ada-gen. To build Dynamo, you will need:

• Ada ASF 0.4.0
• Ada ADO 0.3.0
• Ada EL 1.4.0
• XML Ada 4.x or 3.x

The Ada Database Objects is an Object Relational Mapping for the Ada05 programming language. It allows to map database objects into Ada records and access databases easily. Most of the concepts developped for ADO come from the Java Hibernate ORM. ADO supports MySQL and SQLite databases.

The new version brings:

• Support to update database records when a field is really modified,
• Customization of the SQLite database connection by using SQLite PRAGMAs,
• Escape of MySQL or SQLite reserved keywords,
• Support for blob type.

This version can be downloaded at http://code.google.com/p/ada-ado/downloads/list.

Ada Server Faces is a web framework which uses the Java Server Faces design patterns (See JSR 252, JSR 314 or the latest one the JSR 344).

This new version brings a serious step ahead towards JSF compatibility. This new version provides:

• Support for shared or static build configuration,
• Support for file upload,
• New components <h:inputFile>, <f:metadata>, <f:viewParam>, <f:viewAction>,
• New EL function util:hasMessage,
• ASF now Implements the JSF phase events and phase listeners,
• Implements the JSF/Ruby on Rails flash context,
• Adds the pre-defined JSF beans: initParam, flash,
• Support for locales and honors the Accept-Language,
• New demos are available in French and English

It has been compiled and ported on Linux, Windows and Netbsd (gcc 4.4, GNAT 2011, gcc 4.6.2). You can download this new version at http://code.google.com/p/ada-asf/downloads/list.

A live demo is available at: http://demo.vacs.fr.

Feel free to play with the OpenID stuff!!!

Ada Utility Library is a collection of utility packages for Ada 2005. A new version is available which provides:

• Concurrent fifo queues and arrays
• Changed Objects.Maps to use a String instead of an Unbounded_String as the key
• Support for shared or static build configuration
• Implementation of input/output/error redirection to a file for process launch

It has been compiled and ported on Linux, Windows and Netbsd (gcc 4.4, GNAT 2011, gcc 4.6.2). You can download this new version at http://code.google.com/p/ada-util/downloads/list.