Java 2 Ada2023-10-01T16:33:41+00:00Stephane Carrezurn:md5:d12e23c53b2436d6becce3d51ddbdf38AWAUnlocking the Power of OpenAI in Ada programsurn:md5:689bff7708f38b8dc2fdd5cf1d282af22023-10-01T16:33:41+00:002023-10-01T16:33:41+00:00Stephane CarrezAdaOpenAITutorial
<div class="post-text"><h5>[Ada/openai-img.jpg](Ada/openai-img.jpg)</h5><p>The <a href="OpenAI">OpenAI</a>(https://openai.com/) provides a service that can be queried using REST requests. The service API can be classified as follows:</p><ul><li>OpenAI's GPT (generative pre-trained transformer) is the well-known text generation based on text inputs.</li><pre><code> It gives access to several AI models that have been trained to understand natural language and code.
</code></pre><li>Image generation gives access to the <a href="DALL.E">DALL.E</a>(https://platform.openai.com/docs/models/dall-e) for both</li><pre><code> generating images based on a text description, or, modify an existing image based on a text description.
</code></pre><li>Speech to text gives access to the <a href="Whisper model">Whisper model</a>(https://openai.com/research/whisper) that converts</li><pre><code> audio records into text (several languages are supported).
</code></pre><li>OpenAIâs text embeddings measure the relatedness of text strings.</li></ul><p>For a detailed description of these API have a look at the <a href="OpenAI API Introduction">OpenAI API Introduction</a>(https://platform.openai.com/docs/introduction) document. To use these APIs, you'll first need to register on their service and get an API key that will grant access to the operations.</p><p>The library was generated by the <a href="OpenAPI">OpenAPI</a>(https://github.com/OpenAPITools/openapi-generator) code generator from the OpenAPI description of the OpenAI service and it uses the <a href="OpenAPI Ada">OpenAPI Ada</a>(https://github.com/stcarrez/swagger-ada) library to make HTTP requests and perform JSON serialization and de-serialization. Each OpenAI operation is made available through an Ada procedure which takes care of building the request that was filled within some Ada record, submitting it to the OpenAI server and extract the response in another Ada record. Every request submitted to the OpenAI server is therefore strongly typed!</p><ol><li><ol><li>Setup</li></ol></li></ol><p>To use the library, you should use Alire to setup your project and use:</p><p>``` alr index <del>add git+https://gitlab.com/stcarrez/awa-alire-index.git </del>name awa alr with openai ```</p><p>For the HTTP connection, you can either use <a href="AWS">AWS</a>(https://github.com/AdaCore/aws) or <a href="curl">curl</a>(https://curl.se/) and run one of the following commands:</p><p>``` alr with utilada_curl alr with utilada_aws ```</p><ol><li><ol><li>Initialization</li></ol></li></ol><p>First, make sure you import at least the following Ada packages:</p><p>``` with Util.Http.Clients.Curl; -- replace Curl with AWS if needed with OpenAPI.Credentials.OAuth; with OpenAI.Clients; with OpenAI.Models; ```</p><p>If you want to use <a href="curl">curl</a>(https://curl.se/), the initialization should use the following:</p><p>``` Util.Http.Clients.Curl.Register; ```</p><p>But if you want to use <a href="AWS">AWS</a>(https://github.com/AdaCore/aws), you will initialize with:</p><p>``` Util.Http.Clients.AWS.Register; ```</p><p>After the initialization is done, you will declare the `OpenAI` client instance to access the API operations. The <a href="OpenAI">OpenAI</a>(https://openai.com/) service uses an OAuth bearer API key to authenticate requests made on the server. We will need an `OAuth2_Credential_Type` instance represented by `Cred` below.</p><p>``` Cred : aliased OpenAPI.Credentials.OAuth.OAuth2_Credential_Type; Client : OpenAI.Clients.Client_Type; ```</p><ol><li><ol><li>Credential setup</li></ol></li></ol><p>For the credential setup you will first need to get your access key from your account. Once you have your key as a `String`, you can configure the `Cred` object and tell the client connection entry point which credentials to use:</p><p>``` Api_Key : constant String := ...;</p><pre><code> Cred.Bearer_Token (Api_Key);
Client.Set_Credentials (Cred'Unchecked_Access);
</code></pre><p>```</p><ol><li><ol><li>OpenAPI client setup</li></ol></li></ol><p>The last step necessary before you can make requests, is to setup the server base URL to connect for the REST requests:</p><p>```</p><pre><code> Client.Set_Server ("https://api.openai.com/v1");
</code></pre><p>```</p><ol><li><ol><li>API for chat</li></ol></li></ol><p>The `Create_Chat` is the main operation for the conversation chat generation. The request is represented by the `ChatRequest_Type` type which describes the OpenAI chat model that must be used and the query parameters. The request can be filled with a complete conversation chat which means it is possible to call it several times with previous queries and responses to proceed in the chat conversation.</p><p>``` C : OpenAI.Clients.Client_Type; Req : OpenAI.Models.ChatRequest_Type; Reply : OpenAI.Models.ChatResponse_Type; ...</p><pre><code> Req.Model := OpenAPI.To_UString ("gpt-3.5-turbo");
Req.Messages.Append ((Role => Openapi.To_Ustring ("user"),
Content => Prompt,
others => <>));
Req.Temperature := 0.3;
Req.Max_Tokens := (Is_Null => False, Value => 1000);
Req.Top_P := 1.0;
Req.Frequency_Penalty := 0.0;
Req.Presence_Penalty := 0.0;
C.Create_Chat (Req, Reply);
</code></pre><p>```</p><p>Upon successful completion, we get a list of choices in the reply that contains the text of the conversation. You can iterate over the list with the following code extract. Beware that the returned string is using UTF-8 encoding and it may need a conversion to a `Wide_Wide_String` if necessary.</p><p>```</p><pre><code> for Choice of Reply.Choices loop
declare
Utf8 : constant String := OpenAPI.To_String (Choice.Message.Content);
begin
Put_Line (Ada.Strings.UTF_Encoding.Wide_Wide_Strings.Decode (Utf8));
end;
end loop;
</code></pre><p>```</p><p>The complete chat example is available at: <a href="OpenAI Chat">OpenAI Chat</a>(https://gitlab.com/stcarrez/openai-chat) .</p><ol><li><ol><li>Generating an image</li></ol></li></ol><p>The image generation is based on the <a href="DALL.E">DALL.E</a>(https://platform.openai.com/docs/models/dall-e) model generator. The creation is made by populating a request object, making the call (which is an HTTP POST) and getting the result in a response. Both request and response are represented by full Ada types and are strongly typed:</p><p>The request contains a prompt string which must be provided and is the textual description of the image to create. An optional parameter allows to control the number of images which are created (from 1 to 10). Another optional parameter controls the dimension of the final image. The OpenAI API limits the possible values to: `256x256`, `512x512` and `1024x1024`. The image creation is represented by the `Create_Image` procedure and we can call it with our request instance:</p><p>``` Req : OpenAI.Models.CreateImagesRequest_Type; Reply : OpenAI.Models.ImagesResponse_Type; ...</p><pre><code> Req.Prompt := Prompt;
Req.N := (Is_Null => False, Value => 3);
Req.Size := (Is_Null => False, Value => "512x512");
C.Create_Image (Req, Reply);
</code></pre><p>```</p><p>Once it has finished, it produces a response which basically contains a list of URLs for each generated image.</p><p>```</p><pre><code> for Url of Reply.Data loop
if not Url.Url.Is_Null then
Ada.Text_IO.Put_Line (OpenAPI.To_String (Url.Url.Value));
end if;
end loop;
</code></pre><p>```</p><p>The complete image generation example is available at: <a href="OpenAI Image Generation">OpenAI Image Generation</a>(https://gitlab.com/stcarrez/openai-image) .</p><p>For more information about the image creation, have a look at the <a href="OpenAI Images API reference">OpenAI Images API reference</a>(https://platform.openai.com/docs/api-reference/images).</p><ol><li><ol><li>Conclusion</li></ol></li></ol><p>The <a href="Ada OpenAI">Ada OpenAI</a>(https://gitlab.com/stcarrez/ada-openai) library is eagerly awaiting your Ada programming creativity. You can try using the chat generation example to ask the AI to generate some Ada code, but you'll likely be disappointed by the poor quality of Ada programs that <a href="OpenAI">OpenAI</a>(https://openai.com/) generates. However, you may still find some small benefits in using it across different domains. I encourage you to give it a try for your enjoyment and pleasure.</p></div> Ada BFD 1.3.0urn:md5:0f5fbfe07c452321316d3dea42e681502023-08-20T14:14:31+00:002023-08-20T14:14:31+00:00Stephane CarrezreleaseELFCOFFbinutilsAda
<div class="post-text"><h5>[Ada/ada-bfd-1.3.jpg](Ada/ada-bfd-1.3.jpg)</h5><ol><li><ol><li>Integration with Alire</li></ol></li></ol><p>For Linux users only, the <a href="Ada BFD">Ada BFD</a>(https://github.com/stcarrez/ada-bfd) has an associated Alire crate which allows you to use it easily. To get access to the Alire crate, you should add the <a href="AWA Alire index">AWA Alire index</a>(https://github.com/stcarrez/awa-alire-index) in your Alire configuration as follows:</p><p>``` alr index <del>add=https://github.com/stcarrez/awa-alire-index.git </del>name awa ```</p><p>Then, you can get access to the crate by using</p><p>``` alr with bfdada ```</p><p>Let's see how to use this library...</p><ol><li><ol><li>Declarations</li></ol></li></ol><p>The <a href="Ada BFD">Ada BFD</a>(https://github.com/stcarrez/ada-bfd) library provides a set of Ada bindings that give access to the BFD library. A binary file such as an object file, an executable or an archive is represented by the `Bfd.Files.File_Type` limited type. The symbol table is represented by the `Bfd.Symbols.Symbol_Table` limited type. These two types hold internal data used and managed by the BFD library.</p><p>```<a href="ada">ada</a> with Bfd.Files; with Bfd.Sections; with Bfd.Symbols; ...</p><pre><code> File : Bfd.Files.File_Type;
Symbols : Bfd.Symbols.Symbol_Table;
</code></pre><p>```</p><ol><li><ol><li>Opening the BFD file</li></ol></li></ol><p>The first step is to use the `Open` procedure to read the object or executable file whose path is given as argument. The `File_Type` parameter will be initialized to get access to the binary file information. The `Check_Format` function must then be called to let the BFD library gather the file format information and verify that it is an object file or an executable.</p><p>```<a href="ada">ada</a> Bfd.Files.Open (File, Path, ""); if Bfd.Files.Check_Format (File, Bfd.Files.OBJECT) then</p><pre><code> ...
</code></pre><p>end if; ```</p><p>The `File_Type` uses finalization so that it will close and reclaim resources automatically.</p><ol><li><ol><li>Loading the symbol table</li></ol></li></ol><p>The symbol table is loaded by using the `Read_Symbols` procedure.</p><p>```<a href="ada">ada</a></p><pre><code> Bfd.Symbols.Read_Symbols (File, Symbols);
</code></pre><p>```</p><p>The resources used by the symbol table will be freed when the symbol table instance is finalized.</p><ol><li><ol><li>Find nearest line</li></ol></li></ol><p>Once the symbol table is loaded, we can use the `Find_Nearest_Line` function to find the nearest line of a function knowing some address. This is almost a part of that function that the <a href="addr2line (1)">addr2line (1)</a>(https://www.man7.org/linux/man-pages/man1/addr2line.1.html) command is using.</p><p>```<a href="ada">ada</a> File_Name, Func_Name : Ada.Strings.Unbounded.Unbounded_String; Text_Section : Bfd.Sections.Section; Line : Natural; Pc : constant Bfd.Vma_Type := ...; ...</p><pre><code> Text_Section := Bfd.Sections.Find_Section (File, ".text");
Bfd.Symbols.Find_Nearest_Line (File => File,
Sec => Text_Section,
Symbols => Symbols,
Addr => Pc,
Name => File_Name,
Func => Func_Name,
Line => Line);
</code></pre><p>```</p><p>One tricky aspect of using `Find_Nearest_Line` is the fact that the address we are giving must **sometimes** be converted to an offset within the text region. With <a href="Address space layout randomization (ASLR)">Address space layout randomization (ASLR)</a>(https://en.wikipedia.org/wiki/Address_space_layout_randomization) a program is mapped at a random address when it executes. Before calling `Find_Nearest_Line`, we must subtract the base address of the memory region. We must now find the virtual address of the start of the text region that is mapped in memory. While the program is running, you can find the base address of the program by looking at the `/proc/self/maps` file. This special file indicates the list of memory regions used by the process with the addresses, flags and other information. Without ASLR, the program is almost always loaded at the `0x00400000` address.</p><p>``` 00400000-007f9000 r-xp 00000000 fd:01 12067645 /home/... 009f8000-009fa000 r--p 003f8000 fd:01 12067645 /home/... 009fa000-00a01000 rw-p 003fa000 fd:01 12067645 /home/... ```</p><p>But when it is mapped at a random address, we get a different address each time the program is launched:</p><p>``` 55d5983d9000-55d598592000 r--p 00000000 fc:02 1573554 /... 55d598592000-55d599376000 r-xp 001b9000 fc:02 1573554 /... 55d599376000-55d5997ed000 r--p 00f9d000 fc:02 1573554 /... 55d5997ee000-55d5998bb000 r--p 01414000 fc:02 1573554 /... 55d5998bb000-55d5998c6000 rw-p 014e1000 fc:02 1573554 /... ```</p><p>In that case, the value to use it the first address of first `r--p` region associated with the program (here `0x55d5983d9000`).</p><p>Another method to know the virtual base address is to use the <a href="dl_iterate_phdr (3)">dl_iterate_phdr (3)</a>(https://man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html) function and look at the shared objects which are loaded. This function must be executed by the program itself: it gets as parameter a callback function which is called for each loaded shared object and a data parameter that will be passed to the callback.</p><p>```</p><ol><li>include <dlfcn.h></li></ol><p>static int dl_callback (struct dl_phdr_info* info, size_t size, void* data) {</p><pre><code> /* VM base address is: info->dlpi_addr */
return 0;
</code></pre><p>} ...</p><pre><code> dl_iterate_phdr (dl_callback, 0);
</code></pre><p>```</p><p>When the callback is called, you can get the name of the shared object by looking at `info->dlpi_name` and the virtual base address by looking at `info->dlpi_addr`.</p><p><a href="Ada BFD">Ada BFD</a>(https://github.com/stcarrez/ada-bfd) is a very specific library that is not always easy to use due to the complexity of binary program representation (ELF, DWARF, ...) and program execution. It is however used in very specific contexts such as the <a href="Muen Separation Kernel">Muen Separation Kernel</a>(https://muen.codelabs.ch/) and the <a href="Memory Analysis Tool">Memory Analysis Tool</a>(https://github.com/stcarrez/mat).</p></div> Reentrant scanner and parser with Aflex and Ayaccurn:md5:549bf23cdcd37b17183eaafe7543f19c2023-05-14T19:02:00+00:002023-05-14T19:02:00+00:00Stephane CarrezAdalexyaccreleasegeneratorbison
<div class="post-text"><h5>[Aflex and Ayacc](Ada/Aflex-Ayacc-code.jpg)</h5><ol><li><ol><li>What's new in Aflex 1.6</li></ol></li></ol><p>- Support the flex options `%option output`, `%option nooutput`, `%option yywrap`, `%option noinput`,</p><pre><code> `%option noyywrap`, `%option unput`, `%option nounput`, `%option bufsize=NNN` to better control the
generated `_IO` package.
</code></pre>- <a href="Aflex">Aflex</a>(https://github.com/Ada-France/aflex) templates provide more control for tuning the code generation and<pre><code> they are embedded with [Advanced Resource Embedder](https://gitlab.com/stcarrez/resource-embedder)
</code></pre>- Support to define Ada code block in the scanner that is inserted in the generated scanner - New option -P to generate a private Ada package for DFA and IO - New directive `%option reentrant` and `%yyvar` to generate a recursive scanner - New directive `%yydecl` to allow passing parameters to `YYLex`<pre><code> or change the default function name
</code></pre><p>Example of `%option` directives to tell <a href="Aflex">Aflex</a>(https://github.com/Ada-France/aflex) to avoid generating several function or procedures and customize the buffer size.</p><p>```<a href="Ada">Ada</a> %option nounput %option noinput %option nooutput %option noyywrap %option bufsize=1024 ```</p><p>The tool supports some code block injection at various places in the generated scanner. The code block has the following syntax where `<block-name>` is the name of the code block:</p><p>```<a href="Ada">Ada</a> %<block-name> {</p><pre><code> -- Put Ada code here
</code></pre><p>} ```</p><p>The `%yytype` code block can contain type declaration, function and procedure declarations. It is injected within the `YYLex` function in the declaration part. The `%yyinit` code block can contain statements that are executed at beginning of the `YYLex` function. The `%yyaction` code block can contain statements that are executed before running any action. The `%yywrap` code block can contain statements which are executed when the end of current file is reached to start parsing a next input.</p><ol><li><ol><li>What's new in Ayacc 1.4</li></ol></li></ol><p>- Support the Bison `%define variable value` option to configure the parser generator - Support the Bison `%code name { ... }` directive to insert code verbatim into the output parser - Recognize some Bison variables `api.pure`, `api.private`, `parse.error`, `parse.stacksize`,</p><pre><code> `parse.name`, `parse.params`, `parse.yyclearin`, `parse.yyerrok`, `parse.error`
</code></pre>- New option `-S skeleton` to allow using an external skeleton file for the parser generator - <a href="Ayacc">Ayacc</a>(https://github.com/Ada-France/ayacc) templates provide more control for tuning the code generation and<pre><code> they are embedded with [Advanced Resource Embedder](https://gitlab.com/stcarrez/resource-embedder)
</code></pre>- New option `-P` to generate a private Ada package for the tokens package - Improvement to allow passing parameters to `YYParse` for the grammar rules - New `%lex` directive to control the call of `YYLex` function - Fix #6: ayacc gets stuck creating an infinitely large file after encountering a comment in an action<p>The generator supports two code block injections, the first one `decl` is injected in the `YYParse` procedure declaration and the `init` is injected as first statements to be executed only once when the procedure is called. The syntax is borrowed from the Bison parser:</p><p>```<a href="Ada">Ada</a> %code decl {</p><pre><code> -- Put Ada declarations
</code></pre><p>} %code init {</p><pre><code> -- Put Ada statements
</code></pre><p>} ```</p><p>Some other Bison like improvements have been introduced to control the generation of the parser code.</p><p>``` %define parse.error true %define parse.stacksize 256 %define parse.yyclearin false %define parse.yyerrok false %define parse.name MyParser ```</p><ol><li><ol><li>How to use</li></ol></li></ol><p>The easiest way to use <a href="Ayacc">Ayacc</a>(https://github.com/Ada-France/ayacc) and <a href="Aflex">Aflex</a>(https://github.com/Ada-France/aflex) is to use <a href="Alire">Alire</a>(https://github.com/alire-project/alire), get the sources, build them and install them. You can do this as follows:</p><p>``` alr get aflex cd aflex_1.6.0_b3c21d99 alr build alr install alr get ayacc cd ayacc_1.4.0_c06f997f alr build alr install ```</p><ul><li>UPDATE*: the `alr install` command is available only with <a href="Alire">Alire</a>(https://github.com/alire-project/alire) 2.0.</li></ul><p>Using these tools is done in two steps:</p><p>1. a first step to call `aflex` or `ayacc` command with the scanner file or grammar file, 2. a second step to call `gnatchop` to split the generated file in separate Ada files</p><p>For example, with a `calc_lex.l` scanner file, you would use:</p><p>``` aflex calc_lex.l gnatchop -w calc_lex.ada ```</p><p>And with a `calc.y` grammar file:</p><p>``` ayacc calc.y gnatchop -w calc.ada ```</p><p>To know more about how to write a scanner file or grammar file, have a look at <a href="Aflex 1.5 and Ayacc 1.3.0">Aflex 1.5 and Ayacc 1.3.0</a>(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/12/18/Aflex-1.5-and-Ayacc-1.3.0) which explains more into details some of these aspects.</p><ol><li><ol><li>Highlight on reentrancy</li></ol></li></ol><p>By default <a href="Aflex">Aflex</a>(https://github.com/Ada-France/aflex) and <a href="Ayacc">Ayacc</a>(https://github.com/Ada-France/ayacc) generate a scanner and a parser which use global variables declared in a generated Ada package. These global variables contain some state about the scanner such as the current file being scanned. The <a href="Ayacc">Ayacc</a>(https://github.com/Ada-France/ayacc) parser generates on its side two global variables `YYLVal` and `YYVal`.</p><p>Using global variables creates some strong restrictions when using the generated scanner and parser: we can scan and parse only one file at a time. It cannot be used in a multi-thread environment unless the scan and parse is protected from concurrent access. We cannot use easily some grammars that need to recurse and parse another file such as an included file.</p><ol><li><ol><li><ol><li>Reentrant scanner</li></ol></li></ol></li></ol><p>The reentrant scanner is created by using the `-R` option or the `%option reentrant` directive. The scanner will then need a specific declaration with a context parameter that will hold the scanner state and variables. The context parameter has its type generated in the `Lexer_IO` package. The `%yydecl` directive in the scanner file must be used to declare the `YYLex` function with its parameters. By default the name of the context variable is `Context` but you can decide to customize and change it to another name by using the `%yyvar` directive.</p><p>``` %option reentrant %yyvar Context %yydecl function YYLex (Context : in out Lexer_IO.Context_Type) return Token ```</p><p>When the `reentrant` option is activated, <a href="Aflex">Aflex</a>(https://github.com/Ada-France/aflex) will generate a first `Context_Type` limited type in the `Lexer_DFA` package and another one in the `Lexer_IO` package. The generator can probably be improved in the future to provide a single package with a single type declaration. The `Lexer_DFA` package contains the internal data structures for the scanner to maintain its state and the `Lexer_IO` package holds the input file as well as the `YYLVal` and `YYVal` values.</p><ol><li><ol><li><ol><li>Reentrant parser</li></ol></li></ol></li></ol><p>On its side, <a href="Ayacc">Ayacc</a>(https://github.com/Ada-France/ayacc) uses the `YYLVal` and `YYVal` variables. By default, it generates them in the `_tokens` package that contains the list of parser symbols. It must not generate them and it must now use the scanner `Context_Type` to hold them as well as the scanner internal state. The setup requires several steps:</p><p>1. The reentrant parser is activated by using the `%define api.pure`</p><pre><code> directive similar to the [bison %define](https://www.gnu.org/software/bison/manual/html_node/_0025define-Summary.html).
</code></pre><p>2. The `%lex` directive must be used to define how the `YYLex` function must be called since it now has some</p><pre><code> context parameter.
</code></pre><p>3. The scanner context variable must be declared somewhere, either as parameter to the `YYParse`</p><pre><code> procedure or as a local variable to `YYParse`. This is done using the new `%code decl` directive
and allows to customize the local declaration part of the `YYParse` generated procedure.
</code></pre><p>4. We must give visibility of the `YYLVal` and `YYVal` values defined in the scanner context variable.</p><pre><code> Again, we can do this within the `%code decl` directive.
</code></pre><p>A simple reentrant parser could be defined by using:</p><p>```<a href="Ada">Ada</a> %define api.pure true %lex YYLex (Scanner) %code decl {</p><pre><code> Scanner : Lexer_IO.Context_Type;
YYLVal : YYSType renames Scanner.YYLVal;
YYVal : YYSType renames Scanner.YYVal;
</code></pre><p>} ```</p><p>However, this simple form is not really useful as you may need to open the file and setup the scanner to read from it. It is probably better to pass the scanner context as parameter to the `YYParse` procedure. For this, we can use the `%define parse.params` directive to control the procedure parameters. The reentrant parser is declared as follows:</p><p>```<a href="Ada">Ada</a> %lex YYLex (Scanner) %define api.pure true %define parse.params "Scanner : in out Lexer_IO.Context_Type" %code decl {</p><pre><code> YYLVal : YYSType renames Scanner.YYLVal;
YYVal : YYSType renames Scanner.YYVal;
</code></pre><p>} ```</p><p>To use the reentrant parser and scanner, we only need to declare the scanner context, open the file by using the `Lexer_IO.Open_Input` procedure and call the `YYParse` procedure as follows:</p><p>```<a href="Ada">Ada</a></p><pre><code> Scanner : Lexer_IO.Context_Type;
...
Lexer_IO.Open_Input (Scanner, "file-to-scan");
YYParse (Scanner);
</code></pre><p>```</p><ol><li><ol><li><ol><li>Grammar examples:</li></ol></li></ol></li></ol><p>To have a more complete example of a reentrant parser, you may have a look at the following files:</p><ul><li><a href="porion-templates-parser-lexer.l">porion-templates-parser-lexer.l</a>(https://gitlab.com/stcarrez/porion/-/blob/master/src/parser/porion-templates-parser-lexer.l)</li><li><a href="porion-templates-parser-parser.y">porion-templates-parser-parser.y</a>(https://gitlab.com/stcarrez/porion/-/blob/master/src/parser/porion-templates-parser-parser.y)</li></ul></div> AWA 2.4.0urn:md5:d54e438ac8145e0e1e6b4ce8524b75272022-08-07T21:12:20+00:002022-08-07T21:12:20+00:00Stephane CarrezAdawebRESTOAuthrelease
<div class="post-text"><p>The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.</p><p>AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.</p><p>A typical architecture of an AWA application is represented by the picture below:</p><h5>[Ada Web Application architecture](Ada/awa_architecture_overview.png)</h5><ol><li><ol><li>Using AWA with Alire</li></ol></li></ol><p>To use AWA with Alire, you should first register a new index to get access to the AWA crates and its dependencies.</p><p>``` alr index <del>add git+https://github.com/stcarrez/awa-alire-index </del>name awa ```</p><p>After this command, the `alr` command should give you access to the AWA crates and you can check that with the command:</p><p>``` alr search awa ```</p><p>Once you have setup the Alire index, you can import the `awa` framework and the `dynamo` associated tool by using the following commands:</p><p>``` alr init demo cd demo alr with awa alr with dynamo alr with servletada_aws alr build ```</p><p>Note, if you are using FreeBSD, you should probably use the following command to build (because the MariaDB shared libraries are installed in `/usr/local/lib/mariadb` directory):</p><p>``` alr build -- -largs -L/usr/local/lib/mariadb ```</p><p>Once the `dynamo` tool is built, you can use it to setup your project by using the `create-project` command. To run the `dynamo` tool, you must run it by using the `alr exec` command because Alire will setup some environment variables and `PATH` that gives access to Dynamo and AWA configuration files.</p><p>``` alr exec <del> dynamo create-project -l apache </del>web demo Ada.Lovelace@planet.dev ```</p><p>Change the `demo.gpr` GNAT project that was generated by `alr` and replace `demo.adb` by `demo-server.adb`. That later source has been generated by `dynamo` and it contains everything to setup, prepare and run the Web server. Once this is done, build the application:</p><p>``` alr build ```</p><p>The <a href="Atlas AWA Demonstrator">Atlas AWA Demonstrator</a>(https://github.com/stcarrez/atlas) is also available as Alire crates. Two versions are available, the `atlas` crate is using the Ada Web Server and the `atlas_ews` is using the Embedded Web Server. To play to Atlas, you can try one of the following commands:</p><p>``` alr get atlas</p><ol><li>or</li></ol><p>alr get altas_ews ```</p><ol><li><ol><li>Debian packages</li></ol></li></ol><p>Debian packages are also available for Ubuntu 20, Ubuntu 22 and Debian 11. The repository also includes a Debian package for Alire 1.2. Choose **one** of the following configuration and add it to your `/etc/apt/sources.list` configuration.</p><p>``` deb <a href="https://apt.vacs.fr/ubuntu-focal">https://apt.vacs.fr/ubuntu-focal</a> focal main</p><ol><li>or</li></ol><p>deb <a href="https://apt.vacs.fr/ubuntu-jammy">https://apt.vacs.fr/ubuntu-jammy</a> jammy main</p><ol><li>or</li></ol><p>deb <a href="https://apt.vacs.fr/debian-bullseye">https://apt.vacs.fr/debian-bullseye</a> bullseye main ```</p><p>And you can run the following command to accept the signed packages:</p><p>``` wget -O - <a href="https://apt.vacs.fr/apt.vacs.fr.gpg.key">https://apt.vacs.fr/apt.vacs.fr.gpg.key</a> | sudo apt-key add - ```</p><ol><li><ol><li><ol><li><a href="AWA 2.4.0">AWA 2.4.0</a>(https://github.com/stcarrez/ada-awa/releases/tag/2.4.0) Download: <a href="awa-2.4.0.tar.gz">awa-2.4.0.tar.gz</a>(http://download.vacs.fr/ada-awa/awa-all-2.4.0.tar.gz)</li></ol></li></ol></li></ol><pre><code> - Add support for SQL queries embedded in applications with ARE
- Fix #20: Do not use git:// protocol
- New EasyMDE plugin to integrate the Easy Markdown Editor
- Update AWA blog and AWA wiki to use the EasyMDE editor for Markdown
- Use Dynamo 1.3.0, Ada Server Faces 1.5.0, Ada Servlet 1.6.0, OpenAPI Ada 0.6.0
- Use Ada Wiki 1.4.0, Ada Database Objects 2.3.0
- Use Ada Keystore 1.3.3, Ada EL 1.8.5, Ada Utility Library 2.5.0
</code></pre><ol><li><ol><li><ol><li><a href="Dynamo 1.3.0">Dynamo 1.3.0</a>(https://github.com/stcarrez/dynamo/releases/tag/1.3.0) Download: <a href="dynamo-1.3.0.tar.gz">dynamo-1.3.0.tar.gz</a>(http://download.vacs.fr/dynamo/dynamo-1.3.0.tar.gz)</li></ol></li></ol></li></ol><pre><code> - Fix #5: Generated body does not compile when an enumeration of another UML package is used
- Fix #7: No default type for SQL generation of a column that uses an enumeration
- Fix #9: Option or configuration to disable some SQL generation
- Fix #10: Definition of an UML datatype with a tagged value raises an exception
- Fix #12: Avoid emitting a full qualified type name for types declared in the current package
- Fix #16: Improvement in Markdown documentation generator
- Fix #17: YAML parser: accessibility check failure
- Fix #18: Generate database operation to reload an object
- Fix #19: Add dynamo configuration through environment support
- Fix #20: Give access to the man page from alire
- Fix $21: Generated procedure Create is missing overriding keyword
</code></pre><ol><li><ol><li><ol><li><a href="OpenAPI Ada 0.6.0">OpenAPI Ada 0.6.0</a>(https://github.com/stcarrez/swagger-ada/releases/tag/0.6.0) Download: <a href="openapi-ada-0.6.0.tar.gz">openapi-ada-0.6.0.tar.gz</a>(http://download.vacs.fr/openapi-ada/openapi-ada-0.6.0.tar.gz)</li></ol></li></ol></li></ol><pre><code> - Rename Swagger package into OpenAPI and provide a Swagger package for compatibility
- Update the openapi generator to version 6.0.0
- Add support for text/plain response
- Add support to use external Ada types in OpenAPI definitions
- Add support for multiple response types
- Add support for binary responses
- Add support for Ada enum generation for OpenAPI enums
- Integrate Swagger UI v4.13.0
</code></pre><ol><li><ol><li><ol><li><a href="Ada Server Faces 1.5.0">Ada Server Faces 1.5.0</a>(https://github.com/stcarrez/ada-asf/releases/tag/1.5.0) Download: <a href="ada-asf-1.5.0.tar.gz">ada-asf-1.5.0.tar.gz</a>(http://download.vacs.fr/ada-asf/ada-asf-1.5.0.tar.gz)</li></ol></li></ol></li></ol><p>- New widget <w:progress> to display horizontal/vertical progress bars</p><ol><li><ol><li><ol><li><a href="Ada Servlet 1.6.0">Ada Servlet 1.6.0</a>(https://github.com/stcarrez/ada-servlet/releases/tag/1.6.0) Download: <a href="ada-servlet-1.6.0.tar.gz">ada-servlet-1.6.0.tar.gz</a>(http://download.vacs.fr/ada-servlet/ada-servlet-1.6.0.tar.gz)</li><pre><code> - Fix #4: Alire servletada_aws GNAT project fails due to missing Naming rule
- Fix #5: The Input_Line_Size_Limit parameter is not taken into account
- Fix #6: GNAT configuration project is not correct to build with debugging
- Fix #7: Constraint error raised when matching empty path routes
- Fix #11: Support for Embedded Web Server
- Fix #12: Support for multiple response types in REST operations
</code></pre></ol></li></ol></li></ol><ol><li><ol><li><ol><li><a href="Ada Security 1.4.1">Ada Security 1.4.1</a>(https://github.com/stcarrez/ada-security/releases/tag/1.4.1) Download: <a href="ada-security-1.4.1.tar.gz">ada-security-1.4.1.tar.gz</a>(http://download.vacs.fr/ada-security/ada-security-1.4.1.tar.gz)</li></ol></li></ol></li></ol><pre><code> - Fix Alire GNAT project to build in debug mode
- Fix Security.Random that generates shorter random string
</code></pre><ol><li><ol><li><ol><li><a href="Ada Database Objects 2.3.0">Ada Database Objects 2.3.0</a>(https://github.com/stcarrez/ada-ado/releases/tag/2.3.0) Download: <a href="ada-ado-2.3.0.tar.gz">ada-ado-2.3.0.tar.gz</a>(http://download.vacs.fr/ada-ado/ada-ado-2.3.0.tar.gz)</li></ol></li></ol></li></ol><pre><code> - Fix #4: Is_Loaded predicate operation is false when an object is not yet inserted in the database
- Fix #5: Exception raised when SQLite Query_Statement is finalized if the SQL query was invalid
- Fix #7: Update SQLite support to 3.31.1
- Fix #8: Add SQlite busy handler to handle the SQLITE_BUSY error
- Fix #9: Better handling of SQLITE_BUSY error
- Fix #10: Error 'XML query file does not exist' when the query is loaded from a static embedded loader
</code></pre><ol><li><ol><li><ol><li><a href="Ada Wiki Engine 1.5.0">Ada Wiki Engine 1.5.0</a>(https://github.com/stcarrez/ada-wiki/releases/tag/1.5.0) Download: <a href="ada-wiki-1.5.0.tar.gz">ada-wiki-1.5.0.tar.gz</a>(http://download.vacs.fr/ada-wiki/ada-wiki-1.5.0.tar.gz)</li><pre><code> - Add support for Textile markup language
- Rewrote the Markdown parser to better follow the Common Mark Specification
</code></pre></ol></li></ol></li></ol><ol><li><ol><li><ol><li><a href="Ada Utility Library 2.5.0">Ada Utility Library 2.5.0</a>(https://github.com/stcarrez/ada-util/releases/tag/2.5.0) Download: <a href="ada-util-2.5.0.tar.gz">ada-util-2.5.0.tar.gz</a>(http://download.vacs.fr/ada-util/ada-util-2.5.0.tar.gz)</li></ol></li></ol></li></ol><pre><code> - New examples to illustrate the IO stream composition
- New examples for JSON parser and Util.Beans.Objects
- Add support to set environment variables when launching a process (without changing the current process environment!)
- Add support to indent XML output streams
- New package Util.Files.Rolling to provide a rolling file manager
- New package Util.Beans.Objects.Iterators to easily iterate over objects
- Add a new log appender to support rolling log files with size and time based policies
- New operation Util.Files.Delete_Tree to delete a directory tree and work arround
for GNAT bug gcc/63222 and gcc/56055
- New operation Util.Files.Realpath to find the canonicalized absolute path of a file
- New package Util.Blobs to hold binary content with reference counting
- New package Util.Http.Headers to provide some HTTP helper operations
- Add support for Blob in bean objects
- Fix compilation on NetBSD 9.2
- Fix compilation with AWS >= 22.0
</code></pre></div> Ada development on FreeBSD 13.1urn:md5:df9e3024fee610ff7280e9b4ef4d339a2022-08-05T20:22:00+00:002022-08-05T20:22:00+00:00Stephane CarrezAdaFreeBSDInstallation
<div class="post-text"><ul><li>Warning:* this is full of hacks and I don't pretend to provide any complete detailed and completely reproducible steps for getting a new Ada compiler.</li></ul><p>Before proceeding, make sure you have `gmake` installed because the BSD make uses an old Makefile syntax and is not able to handle GNU specific Makefiles.</p><p>``` pkg install gmake ```</p><ol><li><ol><li>Get gcc6-aux from an old FreeBSD installation</li></ol></li></ol><p>First step is to get the `gcc6-aux` port as is from an old FreeBSD installation. Basically, a tar-gz of the `/usr/local/gcc6-aux` directory tree is enougth. Basically, what I did is to run a `tar cf /tmp/gcc.tar /usr/local/gcc6-aux` on an old FreeBSD and then extract that tar as is on the FreeBSD 13.1 system (Of course, it must be the same architecture).</p><ol><li><ol><li>Build gcc 12</li></ol></li></ol><p>Get the GCC 12 sources from <a href="https://gcc.gnu.org">https://gcc.gnu.org</a> and extract them:</p><p>``` tar xzf gcc-12.1.0.tar.xz ```</p><p>Building GCC must be made in another directory and we must also setup the `PATH` to give access to the old Ada compiler. You must setup and run the configure script as follows:</p><p>``` export PATH=/usr/local/gcc6-aux/bin:$PATH mkdir build cd build ../gcc-12.1.0/configure <del>disable-nls </del>enable-gnu-indirect-function <del>enable-host-shared </del>with-as=/usr/local/bin/as <del>with-gmp=/usr/local </del>with-ld=/usr/local/bin/ld <del>with-system-zlib </del>without-zstd <del>enable-libada </del>localstatedir=/var <del>prefix=/build/gcc-12.1 </del>build=x86_64-portbld-freebsd13.0 --enable-languages=c,ada,c++ ```</p><p>The `<del>with-as` and `</del>with-ld` are important to use the correct `as` and `ld` commands. A subset of the above configure command is taken from the configure command that FreeBSD ports is using to build GCC 12.</p><p>After the configure, run the `gmake` command:</p><p>``` gmake ```</p><p>or to run several compilations in parallel use:</p><p>``` gmake -j4 ```</p><p>While building, the compiler is built a first time with `gcc6-aux` and another time with itself. Building the Ada libraries failed for me with:</p><p>``` /home/ciceron/src/build/./gcc/xgcc -B/home/ciceron/src/build/./gcc/ -B/build/gcc-12.1/x86_64-portbld-freebsd13.0/bin/ -B/build/gcc-12.1/x86_64-portbld-freebsd13.0/lib/ -isystem /build/gcc-12.1/x86_64-portbld-freebsd13.0/include -isystem /build/gcc-12.1/x86_64-portbld-freebsd13.0/sys-include -fchecking=1 -c -g -O2 -m32 -fpic -W -Wall -gnatpg -nostdinc -m32 s-exnllf.adb -o s-exnllf.o s-exnllf.ads:38:04: warning: in instantiation at s-dorepr.adb:82 <a href="enabled by default">enabled by default</a> s-exnllf.ads:38:04: warning: in instantiation at s-exponr.adb:54 <a href="enabled by default">enabled by default</a> s-exnllf.ads:38:04: warning: "Temp" overlays smaller object <a href="enabled by default">enabled by default</a> s-exnllf.ads:38:04: warning: program execution may be erroneous <a href="enabled by default">enabled by default</a> s-exnllf.ads:38:04: warning: size of "Temp" is 96 <a href="enabled by default">enabled by default</a> s-exnllf.ads:38:04: warning: size of "Rep64" is 64 <a href="enabled by default">enabled by default</a> gmake<a href="9">9</a>: *** <a href="../gcc-interface/Makefile:301: s-exnllf.o">../gcc-interface/Makefile:301: s-exnllf.o</a> Error 1 ```</p><p>This error is caused by some inconsistency between the floating point mantissa size returned by `Machine_Mantissa'` Ada predicate and the size of the floating point number. After hacking the `gcc/ada/libgnat/s-dorepr.adb` I've managed to build the Ada libraries.</p><p>After a successful build, installation in `/build/gcc-12.1` is done with:</p><p>``` gmake install ```</p><ol><li><ol><li>Build gprbuild</li></ol></li></ol><p>Yet, another challenge is building <a href="gprbuild">gprbuild</a>(https://github.com/AdaCore/gprbuild). First, get the following sources:</p><p>``` git clone <a href="https://github.com/AdaCore/xmlada.git">https://github.com/AdaCore/xmlada.git</a> git clone <a href="https://github.com/AdaCore/gprconfig_kb">https://github.com/AdaCore/gprconfig_kb</a> git clone <a href="https://github.com/AdaCore/gprbuild.git">https://github.com/AdaCore/gprbuild.git</a> ```</p><p>Then run the boostrap script from `gprbuild` and make sure to setup your PATH to use the new GCC Ada compiler:</p><p>``` export PATH=/build/gcc-12.1/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin cd gprbuild ./bootstrap.sh <del>with-xmlada=../xmlada </del>with-kb=../gprconfig_kb --prefix=./bootstrap ```</p><p>Then, build again and install in the same location as GCC:</p><p>``` gmake prefix=/build/gcc-12.1 SOURCE_DIR=`pwd` setup gmake all gmake install ```</p><p>Don't stop at building only `gprbuild` because you'll probably need the `libgpr` library (if you want to build <a href="Ada Web Server">Ada Web Server</a>(https://github.com/AdaCore/aws) which now depends on <a href="GNAT libcoll">GNAT libcoll</a>(https://github.com/AdaCore/gnatcoll-core) which needs libgpr).</p><p>``` gmake libgpr.build gmake libgpr.install ```</p><ol><li><ol><li>Installation of vscode</li></ol></li></ol><p>It is possible to use vscode and the Ada plugin for development under FreeBSD. The vscode Ada plugin contains the `ada_language_server` binary which is executed as a language server for vscode. No binary is provided for FreeBSD but since the binary is provided for Linux, we can use it by using the Linux support in FreeBSD. First, install vscode with:</p><p>``` pkg install vscode ```</p><p>then, launch it and install the `Language Support for Ada` extension. When the extension is activated, you will see a message indicating that the language server cannot be started. There are two things to do:</p><ul><li>create a `freebsd` directory with the `ada_language_server` binary,</li><li>activate and setup the Linux support in FreeBSD</li></ul><p>For the first part, copy</p><p>``` cd ~/.vscode-oss/extensions/adacore.ada-23.0.8 cp -rp linux freebsd ```</p><p>Now we must setup the Linux support in FreeBSD, and follow the <a href="FreeBSD Chapter 10. Linux Binary Compatibility">FreeBSD Chapter 10. Linux Binary Compatibility</a>(https://docs.freebsd.org/en/books/handbook/linuxemu/).</p><p>Add in `/etc/rc.conf`:</p><p>``` linux_enable="YES" ```</p><p>and start with:</p><p>``` sudo service linux start ```</p><p>Install the `debootsrap`:</p><p>``` sudo pkg install debootstrap sudo debootstrap focal /compat/ubuntu. ```</p><p>Add the line in `/etc/sysctl.conf`:</p><p>``` compat.linux.emul_path="/compat/ubuntu" ```</p><p>and configure with:</p><p>``` sudo sysctl -f /etc/sysctl.conf ```</p><p>Mark the binary as Linux:</p><p>``` brandelf -t Linux ~/.vscode-oss/extensions/adacore.ada-23.0.8/freebsd/ada_language_server ```</p><p>Replace the symbolic link `/compat/ubuntu/lib64/ld-linux-x86-64.so.2` to a real file otherwise the Linux linker is not found by the kernel.</p><p>Now, restart vscode and the Ada language server should be started.</p><p>The description on how to build `ada_language_server` for FreeBSD is left as an exercise to the reader :-)</p></div> IO stream composition and serialization with Ada Utility Libraryurn:md5:94a130493002050aa31349cb63f025f32022-03-05T22:48:00+00:002022-03-05T22:48:00+00:00Stephane CarrezTutorialAdaXMLJSONStreamlzmapipe
<div class="post-text"><p>To be able to provide this IO stream combination, the <a href="Ada Utility Library">Ada Utility Library</a>(https://github.com/stcarrez/ada-util) defines two Ada types: the `Input_Stream` and the `Output_Stream` limited interfaces. The `Input_Stream` interface only defines a `Read` procedure and the `Output_Stream` interface defines the `Write`, `Flush` and `Close` procedures. By implementing these interfaces, it is possible to provide stream objects that can be combined together.</p><h5>[IO Stream Composition and Serialization](Ada/ada-util-streams.png)</h5><p>The <a href="Ada Utility Library">Ada Utility Library</a>(https://github.com/stcarrez/ada-util) provides stream types that implement these interfaces so that it is possible to read or write files, sockets and system pipes. In many cases, the concrete type implements both interfaces so that reading or writing is possible. This is the case for `File_Stream` which allows to read or write on a file, the `Socket_Stream` which handles sockets by using GNAT sockets. The `Pipe_Stream` on its side allows to launch an external program and either read its output, available through the `Input_Stream`, or feed the external program with some input by using the `Output_Stream`.</p><p>The <a href="Ada Utility Library">Ada Utility Library</a>(https://github.com/stcarrez/ada-util) also provides stream objects that make transformation on the data through various data encoders. The Ada library supports the following encoders:</p><ul><li>Base 16, Base 64,</li><li>AES encryption or decryption,</li><li>LZMA compression or decompression</li></ul><p>Other encoders could be added and it is always possible to provide custom transformations by implementing the `Input_Stream` and `Output_Stream` interfaces.</p><p>The last part that completes the IO stream framework is the serialization framework. That framework defines and provides interface and types to read or write a CSV, XML, JSON or HTTP form stream. The serialization framework uses either the `Input_Stream` or the `Output_Stream` interfaces to either read or write the content. The serialization framework defines operations in a way that allows to read or write these streams independently of their representation format.</p><ol><li><ol><li>LZMA Compression</li></ol></li></ol><p>Let's have a look at compressing a file by using the `Util.Streams` framework. First we need a `File_Stream` that is configured to read the file to compress and we need another `File_Stream` configured for writing to save in another file. The first file is opened by using the `Open` procedure and the `In_File` mode while the second one is using `Create` and the `Out_File` mode. The `File_Stream` is using the Ada `Stream_IO` standard package to access files.</p><p>```Ada with Util.Streams.Files;</p><pre><code> In_Stream : aliased Util.Streams.Files.File_Stream;
Out_Stream : aliased Util.Streams.Files.File_Stream;
</code></pre><pre><code> In_Stream.Open (Mode => Ada.Streams.Stream_IO.In_File, Name => Source);
Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
</code></pre><p>```</p><p>In the middle of these two streams, we are going to use a `Compress_Stream` whose job is to compress the data and write the compressed result to a target stream. The compression stream is configured by using the `Initialize` procedure and it is configured to write on the `Out_Stream` file stream. The compression stream needs a buffer and its size is configured with the `Size` parameter.</p><p>```Ada with Util.Streams.Lzma;</p><pre><code> Compressor : aliased Util.Streams.Lzma.Compress_Stream;
</code></pre><pre><code> Compressor.Initialize (Output => Out_Stream'Unchecked_Access, Size => 32768);
</code></pre><p>```</p><p>To feed the compressor stream with the input file, we are going to use the `Copy` procedure. This procedure reads the content from the `In_Stream` and writes what is read to the `Compressor` stream.</p><p>```Ada</p><pre><code> Util.Streams.Copy (From => In_Stream, Into => Compressor);
</code></pre><p>```</p><p>Flushing and closing the files is automatically handled by a `Finalize` procedure on the `File_Stream` type.</p><p>Complete source example: <a href="compress.adb">compress.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/compress.adb)</p><ol><li><ol><li>LZMA Decompression</li></ol></li></ol><p>The LZMA decompression is very close to the LZMA compression but instead it uses the `Decompress_Stream`. The complete decompression method is the following:</p><p>```Ada procedure Decompress_File (Source : in String;</p><pre><code> Destination : in String) is
In_Stream : aliased Util.Streams.Files.File_Stream;
Out_Stream : aliased Util.Streams.Files.File_Stream;
Decompressor : aliased Util.Streams.Lzma.Decompress_Stream;
</code></pre><p>begin</p><pre><code> In_Stream.Open (Mode => Ada.Streams.Stream_IO.In_File, Name => Source);
Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
Decompressor.Initialize (Input => In_Stream'Unchecked_Access, Size => 32768);
Util.Streams.Copy (From => Decompressor, Into => Out_Stream);
</code></pre><p>end Decompress_File; ```</p><p>Complete source example: <a href="decompress.adb">decompress.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/decompress.adb)</p><ol><li><ol><li>AES Encryption</li></ol></li></ol><p>Encryption is a little bit more complex due to the encryption key that must be configured. The encryption is provided by the `Encoding_Stream` and it uses a `Secret_Key` to configure the encryption key. The `Secret_Key` is a limited type and it cannot be copied. To build the encryption key, one method consists in using the PBKDF2 algorithm described in <a href="RFC 8018">RFC 8018</a>(https://tools.ietf.org/html/rfc8018). The user password is passed to the PBKDF2 algorithm configured to use the HMAC-256 hashing. The hash method is called on itself 20000 times in this example to produce the final encryption key.</p><p>```Ada with Util.Streams.AES; with Util.Encoders.AES; with Util.Encoders.KDF.PBKDF2_HMAC_SHA256;</p><pre><code> Cipher : aliased Util.Streams.AES.Encoding_Stream;
Password_Key : constant Util.Encoders.Secret_Key := Util.Encoders.Create (Password);
Salt : constant Util.Encoders.Secret_Key := Util.Encoders.Create ("fake-salt");
Key : Util.Encoders.Secret_Key (Length => Util.Encoders.AES.AES_256_Length);
...
PBKDF2_HMAC_SHA256 (Password => Password_Key,
Salt => Salt,
Counter => 20000,
Result => Key);
</code></pre><p>```</p><p>The encoding stream is able to produce or consume another stream. For the encryption, we are going to use the first mode and use the `Produces` procedure to configure the encryption to write on the `Out_Stream` file. Once configured, the `Set_Key` procedure must be called with the encryption key and the encryption method. The initial encryption `IV` vector can be configured by using the `Set_IV` procedure (not used by the example). As soon as the encryption key is configured, the encryption can start and the `Cipher` encoding stream can be used as an `Output_Stream`: we can write on it and it will encrypt the content before passing the result to the next stream. This means that we can use the same `Copy` procedure to read the input file and pass it through the encryption encoder.</p><p>```Ada</p><pre><code> Cipher.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
Util.Streams.Copy (From => In_Stream, Into => Cipher);
</code></pre><p>```</p><p>Complete source example: <a href="encrypt.adb">encrypt.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/encrypt.adb)</p><ol><li><ol><li>AES Decryption</li></ol></li></ol><p>Decryption is similar but it uses the `Decoding_Stream` type. Below is the complete example to decrypt the file:</p><p>```Ada procedure Decrypt_File (Source : in String;</p><pre><code> Destination : in String;
Password : in String) is
In_Stream : aliased Util.Streams.Files.File_Stream;
Out_Stream : aliased Util.Streams.Files.File_Stream;
Decipher : aliased Util.Streams.AES.Decoding_Stream;
Password_Key : constant Util.Encoders.Secret_Key := Util.Encoders.Create (Password);
Salt : constant Util.Encoders.Secret_Key := Util.Encoders.Create ("fake-salt");
Key : Util.Encoders.Secret_Key (Length => Util.Encoders.AES.AES_256_Length);
</code></pre><p>begin</p><pre><code> -- Generate a derived key from the password.
PBKDF2_HMAC_SHA256 (Password => Password_Key,
Salt => Salt,
Counter => 20000,
Result => Key);
</code></pre><pre><code> -- Setup file -> input and cipher -> output file streams.
In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
Decipher.Produces (Output => Out_Stream'Access, Size => 32768);
Decipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
</code></pre><pre><code> -- Copy input to output through the cipher.
Util.Streams.Copy (From => In_Stream, Into => Decipher);
</code></pre><p>end Decrypt_File; ```</p><p>Complete source example: <a href="decrypt.adb">decrypt.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/decrypt.adb)</p><ol><li><ol><li>Stream composition: LZMA > AES</li></ol></li></ol><p>Now, if we want to compress the stream before encryption, we can do this by connecting the `Compressor` to the `Cipher` stream and we only have to use the `Compressor` instead of the `Cipher` in the call to `Copy`.</p><p>```Ada</p><pre><code> In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
Cipher.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
Compressor.Initialize (Output => Cipher'Unchecked_Access, Size => 4096);
</code></pre><pre><code> Util.Streams.Copy (From => In_Stream, Into => Compressor);
</code></pre><p>```</p><p>When `Copy` is called, the following will happen:</p><ul><li>first, it reads the `In_Stream` source file,</li><li>the data is written to the `Compress` stream,</li><li>the `Compressor` stream runs the LZMA compression and writes on the `Cipher` stream,</li><li>the `Cipher` stream encrypts the data and writes on the `Out_Stream`,</li><li>the `Out_Stream` writes on the destination file.</li></ul><p>Complete source example: <a href="lzma_encrypt.adb">lzma_encrypt.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/lzma_encrypt.adb)</p><ol><li><ol><li>More stream composition: LZMA > AES > Base64</li></ol></li></ol><p>We can easily change the stream composition to encode in Base64 after the encryption. We only have to declare an instance of the Base64 `Encoding_Stream` and configure the encryption stream to write on the Base64 stream instead of the output file. The Base64 stream is configured to write on the output stream.</p><p>```Ada In_Stream : aliased Util.Streams.Files.File_Stream; Out_Stream : aliased Util.Streams.Files.File_Stream; Base64 : aliased Util.Streams.Base64.Encoding_Stream; Cipher : aliased Util.Streams.AES.Encoding_Stream; Compressor : aliased Util.Streams.Lzma.Compress_Stream;</p><pre><code> In_Stream.Open (Ada.Streams.Stream_IO.In_File, Source);
Out_Stream.Create (Mode => Ada.Streams.Stream_IO.Out_File, Name => Destination);
Base64.Produces (Output => Out_Stream'Unchecked_Access, Size => 32768);
Cipher.Produces (Output => Base64'Unchecked_Access, Size => 32768);
Cipher.Set_Key (Secret => Key, Mode => Util.Encoders.AES.ECB);
Compressor.Initialize (Output => Cipher'Unchecked_Access, Size => 4096);
</code></pre><p>```</p><p>Complete source example: <a href="lzma_encrypt_b64.adb">lzma_encrypt_b64.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/lzma_encrypt_b64.adb)</p><ol><li><ol><li>Serialization</li></ol></li></ol><p>Serialization is achieved by using the `Util.Serialize.IO` packages and child packages and their specific types. The parent package defines the limited `Output_Stream` interface which inherit from the `Util.Streams.Output_Stream` interface. This allows to define specific operations to write various Ada types but also it provides common set of abstractions that allow to write either a JSON, XML, CSV and FORM (`x-www-form-urlencoded`) formats.</p><p>The target format is supported by a child package so that you only have to use the `Output_Stream` type declared in one of the `JSON`, `XML`, `CSV` or `Form` child package and use it transparently. There are some constraint if you want to switch from one output format to another while keeping the same code. These constraints comes from the nature of the different formats: `XML` has a notion of entity and attribute but other formats don't differentiate entities from attributes.</p><ul><li>A `Start_Document` procedure must be called first. Not all serialization method need it but it is required for JSON to produce a correct output.</li><li>A `Write_Entity` procedure writes an XML entity of the given name. When used in JSON, it writes a JSON attribute.</li><li>A `Start_Entity` procedure prepares the start of an XML entity or a JSON structure with a given name.</li><li>A `Write_Attribute` procedure writes an XML attribute after a `Start_Entity`. When used in JSON, it writes a JSON attribute.</li><li>A `End_Entity` procedure terminates an XML entity or a JSON structure that was opened by `Start_Entity`.</li><li>At the end, the `End_Document` procedure must be called to finish correctly the output and terminate the JSON or XML content.</li></ul><p>```Ada procedure Write (Stream : in out Util.Serialize.IO.Output_Stream'Class) is begin</p><pre><code> Stream.Start_Document;
Stream.Start_Entity ("person");
Stream.Write_Entity ("name", "Harry Potter");
Stream.Write_Entity ("gender", "male");
Stream.Write_Entity ("age", 17);
Stream.End_Entity ("person");
Stream.End_Document;
</code></pre><p>end Write; ```</p><ol><li><ol><li><ol><li>JSON Serialization</li></ol></li></ol></li></ol><p>With the above `Write` procedure, if we want to produce a JSON stream, we only have to setup a JSON serializer. The JSON serializer is connected to a `Print_Stream` which provides a buffer and helper operations to write some text content. An instance of the `Print_Stream` is declared in `Output` and configured with a buffer size. The JSON serializer is then connected to it by calling the `Initialize` procedure and giving the `Output` parameter.</p><p>After writing the content, the JSON is stored in the `Output` print stream and it can be retrieved by using the `To_String` function.</p><pre><code>
</code></pre><p>```Ada with Ada.Text_IO; with Util.Serialize.IO.JSON; with Util.Streams.Texts; procedure Serialize is</p><pre><code> Output : aliased Util.Streams.Texts.Print_Stream;
Stream : Util.Serialize.IO.JSON.Output_Stream;
</code></pre><p>begin</p><pre><code> Output.Initialize (Size => 10000);
Stream.Initialize (Output => Output'Unchecked_Access);
Write (Stream);
Ada.Text_IO.Put_Line (Util.Streams.Texts.To_String (Output));
</code></pre><p>end Serialize; ```</p><p>The `Write` procedure described above produces the following JSON content:</p><p>```C {"person":{"name":"Harry Potter","gender":"male","age": 17}} ```</p><p>Complete source example: <a href="serialize.adb">serialize.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/serialize.adb)</p><ol><li><ol><li><ol><li>XML Serialization</li></ol></li></ol></li></ol><p>Switching to an XML serialization is easy: replace `JSON` by `XML` in the package to use the XML serializer instead.</p><p>```Ada with Ada.Text_IO; with Util.Serialize.IO.XML; with Util.Streams.Texts; procedure Serialize is</p><pre><code> Output : aliased Util.Streams.Texts.Print_Stream;
Stream : Util.Serialize.IO.XML.Output_Stream;
</code></pre><p>begin</p><pre><code> Output.Initialize (Size => 10000);
Stream.Initialize (Output => Output'Unchecked_Access);
Write (Stream);
Ada.Text_IO.Put_Line (Util.Streams.Texts.To_String (Output));
</code></pre><p>end Serialize; ```</p><p>This time, the same `Write` procedure produces the following XML content:</p><p>```C <person><name>Harry Potter</name><gender>male</gender><age>17</age></person> ```</p><p>Complete source example: <a href="serialize_xml.adb">serialize_xml.adb</a>(https://github.com/stcarrez/ada-util/tree/master/samples/serialize_xml.adb)</p></div> Advanced Resource Embedder 1.2.0urn:md5:9d168c833ee907e95a358d21ce9436f82022-01-23T09:04:00+00:002022-01-23T09:04:00+00:00Stephane CarrezAdageneratorCGorelease
<div class="post-text"><h5>[Embedding SQL in binary](Ada/resource-embedder-bin.png)</h5><p>The new release contains the following fixes:</p><p>- Fix Ada generator to be able to use other binary content types</p><pre><code> such as `System.Storage_Elements.Storage_Array`
</code></pre>- Fix Ada generator to escape special characters in strings<p>Since the previous version, Fabien Chouteau asked to be able to use an Ada system type to represent a binary content. This is now possible by using the following XML extract for the description to tell the resource embedder how to integrate the file and generate the Ada source code:</p><p>```XML <package></p><pre><code> <resource name='Resources.Help'
format='binary'
type='access constant System.Storage_Elements.Storage_Array'>
<install mode='copy'>
<fileset dir="help">
<include name="**/*.txt"/>
</fileset>
</install>
</resource>
...
</code></pre><p></package> ```</p><p>With the above description, the Ada code generator produces the following package specification:</p><p>```Ada with System.Storage_Elements; package Resources.Help is</p><pre><code> type Content_Access is access constant System.Storage_Elements.Storage_Array;
type Name_Access is access constant String;
type Name_Array is array (Natural range <>) of Name_Access;
Names : constant Name_Array;
function Get_Content (Name : String)
access constant System.Storage_Elements.Storage_Array;
</code></pre><p>private</p><pre><code> ...
</code></pre><p>end Resources.Help; ```</p><p>This example is available in <a href="Embedding help and documentation in Ada (binary)">Embedding help and documentation in Ada (binary)</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help-binary). If you prefer to use `String` instead of a `Storage_Array`, have a look at the <a href="Embedding help and documentation in Ada">Embedding help and documentation in Ada</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help). Both examples are similar but they are exposing the file using different Ada types.</p><p>To install the tool, follow the instructions given in the initial announcement: <a href="Advanced Resource Embedder for Ada, C and Go">Advanced Resource Embedder for Ada, C and Go</a>(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/06/11/Advanced-Resource-Embedder).</p><p>If you want to know more about the tool, have a look at its documentation:</p><ul><li><a href="Resource Embedder Guide">Resource Embedder Guide</a>(https://resource-embedder.readthedocs.io/en/latest/) <a href="PDF">PDF</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)</li><li>Man page: <a href="are (1)">are (1)</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)</li></ul><p>and if you have ideas for improvements, fork the project and submit a pull request!</p></div> Aflex 1.5 and Ayacc 1.3.0urn:md5:45a1bebacf173ea548b9a06d2555561c2021-12-19T13:03:00+00:002021-12-19T13:03:00+00:00Stephane Carrezreleasegeneratorlexyaccbison
<div class="post-text"><ol><li><ol><li>Aflex Version 1.5.2021</li></ol></li></ol><p>The new release provides the following changes:</p><p>- Fix crash when the scanner file uses characters in range 128..255, - Fixed various compilation warnings, - Use `gprbuild` to build and support `alr`, - Reduced number of style compilation warnings in generated code</p><p>To install Aflex, you can use the <a href="Alire">Alire</a>(https://github.com/alire-project/alire) tool by using:</p><p>```<a href="sh">sh</a></p><pre><code> alr get aflex
cd aflex_1.5.2021_33198b8f
alr build
</code></pre><p>```</p><p>You can also build and install it as easily with the following commands:</p><p>```<a href="sh">sh</a></p><pre><code> git clone https://github.com/Ada-France/aflex.git
cd aflex
make
sudo make install
</code></pre><p>```</p><p>You can also install the following Debian package: <a href="aflex_1.5.2021">aflex_1.5.2021</a>(https://blog.vacs.fr/vacs/debian/ubuntu-focal/aflex_1.5.2021_amd64.html).</p><ol><li><ol><li>Ayacc Version 1.3.0</li></ol></li></ol><p>The Ayacc release provides a little bit more improvements:</p><p>- New option `-C` to disable the generation of `yyclearin` procedure, - New option `-E` to disable the generation of `yyerrok` procedure, - New option `-D` to write the generated files to the directory specified, - New option `-k` to keep the case of grammar symbols, - Fixed various compilation warnings, - Generate constants for shift reduce and goto arrays, - Better strong typing in the generated state tables, - Reduced number of style compilation warnings in generated code</p><p>To install Ayacc with <a href="Alire">Alire</a>(https://github.com/alire-project/alire) tool use:</p><p>```<a href="sh">sh</a></p><pre><code> alr get ayacc
cd ayacc_1.3.0_9b8bf854
alr build
</code></pre><p>```</p><p>You can also build and install it as easily with the following commands:</p><p>```<a href="sh">sh</a></p><pre><code> git clone https://github.com/Ada-France/ayacc.git
cd ayacc
make
sudo make install
</code></pre><p>```</p><p>You can also install the following Debian package: <a href="ayacc_1.3.0">ayacc_1.3.0</a>(https://blog.vacs.fr/vacs/debian/ubuntu-focal/ayacc_1.3.0_amd64.html).</p><ol><li><ol><li>Development with Aflex and Ayacc</li></ol></li></ol><p>Aflex is a tool for generating a scanner that recognizes lexical patterns in text. Ayacc uses a BNF-like specification of a grammar to generate an Ada parser for that grammar. These two tools are tightly coupled together. First the scanner's role is to identify tokens that are used by the grammar file. Second, the parser will use the scanner to ask for tokens as it parses the input content.</p><p>The generated parser provides a main procedure `YYParse` which is the entry point to parse an input content. For this, it calls the `YYLex` function provided by the generated scanner.</p><p>The development model is that you write a scanner file that describes the lexical patterns and the grammar file in the BNF-like specification. Aflex reads the scanner file and generates the scanner. Ayacc reads the grammar file and produces the parser. The scanner and parser are composed of several Ada packages and they are used by each other to exchange some types and operations.</p><h5>[Aflex and Ayacc Development Model](Ada/aflex-ayacc.jpg)</h5><p>Ayacc and Aflex need a number of Ada types for their work and the important types are controlled by the Ayacc grammar. They also share some global variables to hold a token that was parsed or describe the current parser state. The list of possible tokens is generated by Ayacc in the `<Parser>_Tokens` package and represented by the `Token` type.</p><p>```<a href="Ada">Ada</a> package <Parser>_Tokens</p><pre><code> type Token is (T_AND, T_OR, T_STRING, ...);
</code></pre><p>end <Parser>_Tokens; ```</p><p>Aflex will generate the `YYLex` function that returns a `Token` type:</p><p>```<a href="Ada">Ada</a></p><pre><code> function YYLex return <Parser>_Tokens.Token;
</code></pre><p>```</p><p>On its side Ayacc generates the `YYParse` procedure in the generated package body. It is up to you and your grammar to decide whether this procedure must be exported as is or encapsulated to another operation.</p><p>```<a href="Ada">Ada</a></p><pre><code> procedure YYParse;
</code></pre><p>```</p><ol><li><ol><li>Writing a scanner with Aflex</li></ol></li></ol><p>The scanner file describes regular expressions and Ada code which are executed when some input text matches the regular expression. The scanner file contains basically three areas separated by a line containing only `%%`. The global layout is:</p><p>```<a href="C">C</a> definitions %% rules %% user code ```</p><p>The scanner extracts below are taken from a CSS parser written in Ada, the complete scanner file is <a href="css-parser-lexer.l">css-parser-lexer.l</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-lexer.l). I highlight here some interesting parts of that lexer.</p><p>The definitions section contains a set of configuration options that tells the Aflex generator how to generate the final Ada scanner. First, the `%unit` directive allows to control the Ada package name used by the final scanner. The scanner package can be a child package which gives a lot of flexibility and control for the program organization.</p><p>```<a href="C">C</a> %unit CSS.Parser.Lexer %option case-insensitive %option yylineno ```</p><p>The `%option case-insensitive` tells Aflex to ignore case when generating the scanner and the `%option yylineno` instructs it to maintain a variable `yylineno` that indicates the current line number. It also maintains the `yylinecol` variable that indicates the column number of the current token. This is useful to keep track of token position in order to report them when an error occurs.</p><p>Because writing regular expressions is difficult, Aflex allows you to create named definitions where you give a name to a regular expression. A regular expression can use another named definition, hence simplifying the creation of complex regular expressions.</p><p>```<a href="C">C</a> h <a href="0-9a-f">0-9a-f</a> nonascii <a href="240-377">240-377</a> unicode \{h}{1,6}(rn|<a href="trnf">trnf</a>)? escape {unicode}|\<a href="^rnf0-9a-f">^rnf0-9a-f</a> string1 "(<a href="^nrf\"">^nrf\"</a>|\{nl}|{escape})*" string2 '(<a href="^nrf\'">^nrf\'</a>|\{nl}|{escape})*' string {string1}|{string2} nl n|rn|r|f ```</p><p>When all named definitions are written, the next section separated by the `%%` marker defines the rules that describe a pattern and an associated action. The pattern is a regular expression and the action corresponds to some Ada code that is executed. Aflex is not aware of the Ada grammar and does not check its validity. It uses the action code as is and write it in the final Ada file. When an action is complex and must be written in several lines, the action block can be enclosed by `{` and `}` (this syntax comes from the C lexer, the two symbols are of course omitted in the final program).</p><p>The extract below shows several pattern and actions:</p><p>```<a href="C">C</a> %% - return '-'; + return '+'; "and" { return T_AND; } "or" { return T_OR; } "not" { return T_NOT; } {string} { Set_String (YYLVal, YYText, yylineno, yylinecol); return T_STRING; } ```</p><p>Here, when the scanner finds the `-` or `+` symbols, it returns the `-` and `+` token. If it finds the word '`and`', it returns the token `T_AND`.</p><p>The last rule of this example will match a CSS string such as `"red"` or `'blue'` and we will get these results when the `YYText` function is called. The action will execute the `Set_String` procedure whose purpose is to populate the `YYLVal` variable (not shown in the example, and for the curious, that procedure is provided by the private part of the parent package `CSS.Parser`, hence it is visible by the generated scanner package `CSS.Parser.Lexer`).</p><p>The last section of the lexer file contains the user Ada code that will be used for the Ada scanner code generation. This section should contain a definition of the scanner package specification with any necessary `with` units. It must also provide the declaration of the `YYLex` function that should return a `Token` type.</p><p>```<a href="Ada">Ada</a> %% with CSS.Parser.Parser_Tokens; package CSS.Parser.Lexer is</p><pre><code> use CSS.Parser.Parser_Tokens;
</code></pre><pre><code> function YYLex return Token;
</code></pre><p>end CSS.Parser.Lexer;</p><p>with Ada.Text_IO; with CSS.Parser.Lexer_dfa; with CSS.Parser.Lexer_io; package body CSS.Parser.Lexer is</p><pre><code> use CSS.Parser.Lexer_dfa;
use CSS.Parser.Lexer_io;
</code></pre><pre><code> pragma Style_Checks (Off);
pragma Warnings (Off);
</code></pre><ol><li><ol><li></li></ol></li></ol><p>end CSS.Parser.Lexer; ```</p><p>The package body must also be written and it should contain a `##` marker which is the place where the Aflex generator will write the scanner code with the `YYLex` function body.</p><p>Because Aflex does not parse the Ada action code in the scanner file, it is not able to indent correctly the final program. The `pragma Style_Checks (Off);` is useful to remove the indentation warnings that the compiler could emit.</p><p>Aflex also generates a `<Lexer>_io` package and a `<Lexer>_dfa` package which provides helper operations for the generated scanner. The `<Lexer>_dfa` package defines two important functions that give access to the current token as a text.</p><p>```<a href="Ada">Ada</a> package CSS.Parser.Lexer_dfa is</p><pre><code> function YYText return String;
function YYLength return Integer;
</code></pre><p>end CSS.Parser.Lexer_dfa; ```</p><p>For example, if the scanner finds the input string `"red"`, the `YYLex` function will return the `T_STRING` value and the `YYText` function will contain the string `"red"`.</p><p>The `<Lexer>_io` package defines the `Open_Input` procedure that opens the file that must be parsed. The package also defines several other procedures and functions but most of them are dedicated to the scanner.</p><p>```<a href="Ada">Ada</a> package CSS.Parser.Lexer_io is</p><pre><code> procedure Open_Input (Fname : in String);
</code></pre><p>end CSS.Parser.Lexer_io; ```</p><ol><li><ol><li><ol><li>Code generation</li></ol></li></ol></li></ol><p>The Ada scanner is generated by running the `aflex` tool. The `-m` option tells the tool to avoid emitting Ada `with` clauses to the `Text_IO` package, the `-s` option disables the default lex rule that emits an echo of unmatched scanner input to the standard output and the `-L` option disables the `#line` directives.</p><p>```<a href="sh">sh</a></p><pre><code> aflex -ms -L css-parser-lexer.l
gnatchop -w css-parser-lexer.ada
</code></pre><p>```</p><p>The tool generates a single Ada file that contains the specification and the body and it must be split by using `gnatchop` to produce the <a href="css-parser-lexer.ads">css-parser-lexer.ads</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-lexer.ads) specification and <a href="css-parser-lexer.adb">css-parser-lexer.adb</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-lexer.adb) package body.</p><ol><li><ol><li>Writing a parser with Ayacc</li></ol></li></ol><p>Ayacc uses the BNF-like grammar definition to generate the parser written in Ada. The grammar file is similar to a grammar file created for <a href="Yacc">Yacc</a>(https://en.wikipedia.org/wiki/Yacc) or <a href="Bison">Bison</a>(https://www.gnu.org/software/bison/) but it contains Ada code for the grammar actions. The grammar file starts with a set of definitions that enumerates the available tokens and declares the main type to describe a parser state.</p><p>Similar to the scanner file, the `%unit` directive allows to control the name of the generated Ada package. I've found very convenient to provide a parent Ada package that defines various types and operations that the generated parser can use. By using the `%unit` and specifying a child package, the generated parser can easily access those operations and types even if they are declared in the private part.</p><p>```<a href="C">C</a> %unit CSS.Parser.Parser %token T_STRING %token T_AND %token T_OR %token T_NOT ```</p><p>The Ayacc generated parser uses a stack to keep track and maintain the state while parsing a content. Each stack entry represents a grammar rule that was recognized and it is possible to also record some specific information. For this, the declaration part must declare the `YYSType` Ada type. That type describes the state and information about a grammar rule. The parser pushes or pops values on the stack according to the grammar rules that are recognized (after a _shift_ or _reduce_ operation). The `YYSType` cannot be a limited type nor can it be abstract because the value must be copied to or from the `YYLVal` global variable.</p><p>In the following definition, the `YYSType` is in fact declared in the private part of the parent package `CSS.Parser` (again, that type is visible to the parser because it is one of its child package).</p><p>```<a href="Ada">Ada</a> {</p><pre><code> subtype YYSType is CSS.Parser.YYstype;
</code></pre><p>} ```</p><p>After the declaration part, the `%%` separator introduces a list of BNF-like rules. Some grammar rules can be associated with some actions which are executed when the grammar rule is recognized. With Ayacc, the action is an Ada code which can contain `$n` constructs which give access to values of the rule elements. In fact, the `$n` values refers to values in the parser stack and the `$$` value is the `YYVal` variable. When an action is executed, this is known as a _reduce_ action and all the rule elements are replaced by the `$$` result.</p><p>```<a href="Ada">Ada</a> %% expr :</p><pre><code> expr operator term
{ CSS.Parser.Set_Expr ($$, $1, $3); }
|
expr term
{ CSS.Parser.Set_Expr ($$, $1, $2); }
|
term
;
</code></pre><p>```</p><p>In this grammar rule extract, the `$$` refers to the current rule value (`YYVal`), the `$1` contains the value of the first element and so on. It is not possible to exceed the number of elements described by the rule. These values are of type `YYSType`. When the rule action is finished, the elements matched by the rule are removed from the stack and the current value `$$` (`YYVal`) is pushed on top of the stack.</p><p>It is possible to write complex action code in the grammar file but this is not very convenient. In many cases, it is easier to provide procedures or functions that the generated parser can use for the action to perform its work.</p><p>After the grammar rules, a last `%%` separator introduces the final Ada code used by the parser generator. That last section must contain the `##` marker which represents the place where Ayacc will emit the body of the parser. The Ada code should also declare the `yyerror` procedure that is called by the generated parser when a syntax error is detected.</p><p>Below is a partial example of such final Ada code:</p><p>```<a href="Ada">Ada</a> %% with CSS.Core.Sheets; package CSS.Parser.Parser is</p><pre><code> Error_Count : Natural := 0;
</code></pre><pre><code> function Parse (Content : in String;
Document : in CSS.Core.Sheets.CSSStylesheet_Access) return Integer;
</code></pre><p>end CSS.Parser.Parser;</p><p>with CSS.Parser.Parser_Goto; with CSS.Parser.Parser_Tokens; with CSS.Parser.Parser_Shift_Reduce; with CSS.Parser.Lexer_io; with CSS.Parser.Lexer; with CSS.Parser.Lexer_dfa; with Ada.Text_IO; package body CSS.Parser.Parser is</p><pre><code> procedure YYParse;
</code></pre><pre><code> procedure yyerror (Message : in String := "syntax error");
</code></pre><pre><code> procedure yyerror (Message : in String := "syntax error") is
begin
Error_Count := Error_Count + 1;
end yyerror;
</code></pre><pre><code> function Parse (Content : in String;
Document : in CSS.Core.Sheets.CSSStylesheet_Access) return Integer is
begin
Error_Count := 0;
CSS.Parser.Lexer_Dfa.yylineno := 1;
CSS.Parser.Lexer_Dfa.yylinecol := 1;
CSS.Parser.Lexer_IO.Open_Input (Content);
CSS.Parser.Parser.Document := Document;
YYParse;
return Error_Count;
end Parse;
</code></pre><ol><li><ol><li></li></ol></li></ol><p>end CSS.Parser.Parser; ```</p><p>To prepare the scanner to read a content, you will have to call the `Open_Input` procedure with the file name as parameter. Then, all you have to do is call the `YYParse` procedure that will call the `YYLex` function and proceed with the scanning and parsing of the input file.</p><ol><li><ol><li><ol><li>Code generation</li></ol></li></ol></li></ol><p>The Ada parser is generated by running the `ayacc` tool. The `-n` option controls the size of the parser stack (the default is 8192). The `-k` option tells the generator to keep the case of tokens as they are written in the grammar file (the default is that it converts the token using mixed case). The `-s` option prints some statistics about the grammar. This is useful to understand and check the shift/reduce and reduce/reduce conflicts that could arise in the grammar. The `-e` option controls the extension of the generated file.</p><p>```<a href="sh">sh</a></p><pre><code> ayacc -n 256 -k -s -e .ada css-parser-parser.y
gnatchop -w css-parser-parser.ada
</code></pre><p>```</p><p>The tool generates a single Ada file that contains the specification and the body and it must be split by using `gnatchop` to produce the <a href="css-parser-parser.ads">css-parser-parser.ads</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-parser.ads) specification and <a href="css-parser-parser.adb">css-parser-parser.adb</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-parser.adb) package body.</p><ol><li><ol><li>Going further</li></ol></li></ol><p>Both tools are very close to the lex/flex and yacc/bison Unix tools which means that most tips and documentation that you will find on Lex and Yacc are common and applicable to Aflex and Ayacc. Writing the grammar file is probably the most difficult task and the <a href="GNU Bison documentation">GNU Bison documentation</a>(https://www.gnu.org/software/bison/manual/html_node/index.html) seriously helps in that task.</p><p>Looking at how scanner and grammar files are written in some other projects can help. Below is a non exhaustive list of scanner and grammar files that can be processed by Aflex and Ayacc:</p><ol><li><ol><li><ol><li>Scanner examples</li></ol></li></ol></li></ol><ul><li>Aflex example <a href="example.l">example.l</a>(https://github.com/Ada-France/aflex/blob/master/doc/example.l)</li><li>Aflex example <a href="test95.l">test95.l</a>(https://github.com/Ada-France/aflex/blob/master/doc/test95.l)</li><li>Aflex example <a href="options.l">options.l</a>(https://github.com/Ada-France/aflex/blob/master/doc/options.l)</li><li>Ayacc calc example <a href="calc_lex.l">calc_lex.l</a>(https://github.com/Ada-France/ayacc/blob/master/examples/calc/calc_lex.l)</li><li>Ayacc Ada83 scanner example <a href="ada_lex.l">ada_lex.l</a>(https://github.com/Ada-France/ayacc/blob/master/examples/ada_parser/ada_lex.l)</li><li>Ada CSS, CSS 3 scanner <a href="css-parser-lexer.l">css-parser-lexer.l</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-lexer.l)</li><li>Ada CSS, CSS 3 definition rule scanner <a href="css-analysis-parser-lexer.l">css-analysis-parser-lexer.l</a>(https://github.com/stcarrez/ada-css/blob/master/tools/parser/css-analysis-parser-lexer.l)</li><li>Memory Analysis Tool, expression scanner <a href="mat-expressions-lexer.l">mat-expressions-lexer.l</a>(https://github.com/stcarrez/mat/blob/master/mat/src/parser/mat-expressions-lexer.l)</li></ul><ol><li><ol><li><ol><li>Grammar examples</li></ol></li></ol></li></ol><ul><li>Ayacc calc example <a href="calc.y">calc.y</a>(https://github.com/Ada-France/ayacc/blob/master/examples/calc/calc.y)</li><li>Ayacc Ada83 grammar example <a href="ada.y">ada.y</a>(https://github.com/Ada-France/ayacc/blob/master/examples/ada_parser/ada.y)</li><li>Ada CSS, CSS 3 parser <a href="css-parser-parser.y">css-parser-parser.y</a>(https://github.com/stcarrez/ada-css/blob/master/src/parser/css-parser-parser.y)</li><li>Ada CSS, CSS 3 definition rule parser <a href="css-analysis-parser-parser.y">css-analysis-parser-parser.y</a>(https://github.com/stcarrez/ada-css/blob/master/tools/parser/css-analysis-parser-parser.y)</li><li>Memory Analysis Tool, expression parser <a href="mat-expressions-parser.y">mat-expressions-parser.y</a>(https://github.com/stcarrez/mat/blob/master/mat/src/parser/mat-expressions-parser.y)</li></ul></div> AWA 2.3.0urn:md5:69399f402572f7858add3af5941186332021-07-31T08:53:25+00:002021-07-31T08:53:25+00:00Stephane CarrezAdaframeworkrelease
<div class="post-text"><p>The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.</p><p>AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.</p><p>A typical architecture of an AWA application is represented by the picture below:</p><p>((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)</p><p>This version of AWA integrates smoothly with <a href="https://blog.vacs.fr/vacs/blogs/post.html?post=2019/12/26/AKT-a-tool-to-store-and-protect-your-sensitive-information">Ada Keystore</a> in order to protect the server sensitive configuration.</p><h3>AWA, Version 2.3.0</h3><ul><li>Update Trumbowyg editor to version 2.23.0</li><li>Fix generation of og:image meta for blog articles written in Markdown</li><li>Fix wiki preview with latest xkhtmltoimage 0.12.6</li><li>Use Dynamo 1.2.2, Ada Server Faces 1.4.3, Ada Servlet 1.5.2, OpenAPI Ada 0.5.0</li><li>Use Ada Wiki 1.3.2, Ada Database Objects 2.2.0</li><li>Use Ada Keystore 1.3.2, Ada EL 1.8.3, Ada Utility Library 2.4.1</li></ul><p>Links: <a href="http://download.vacs.fr/ada-awa/ada-awa-2.3.0.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-awa">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-awa">GitLab</a></p><h3>Dynamo, Version 1.2.2</h3><ul><li>Fix the SQL type definition for double on PostgreSQL</li><li>Fix double support and nullable entity_type</li><li>Fix SQL generation for a foreign key with variable length</li></ul><p>Links: <a href="http://download.vacs.fr/dynamo/dynamo-1.2.2.tar.gz">Download</a> <a href="https://github.com/stcarrez/dynamo">GitHub</a> <a href="https://gitlab.com/stcarrez/dynamo">GitLab</a></p><h3>Swagger Ada, Version 0.5.0</h3><ul><li>Fix for GNAT 2021</li><li>Update the openapi generator to version 5.2.0</li></ul><p>Links: <a href="http://download.vacs.fr/swagger-ada/swagger-ada-0.5.0.tar.gz">Download</a> <a href="https://github.com/stcarrez/swagger-ada">GitHub</a> <a href="https://gitlab.com/stcarrez/swagger-ada">GitLab</a></p><h3>Ada Servlet, Version 1.5.2</h3><ul><li>Fix for GNAT 2021</li></ul><p>Links: <a href="http://download.vacs.fr/ada-servlet/ada-servlet-1.5.2.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-servlet">GitHub</a> <a href="https://github.com/stcarrez/ada-servlet">GitLab</a></p><h3>Ada Wiki Library, Version 1.3.2</h3><ul><li>Fix <hr> and <br> generation to follow HTML5 convention.</li><li>Add option -H to the render example</li><li>Fix for GNAT 2021</li></ul><p>Links: <a href="http://download.vacs.fr/ada-wiki/ada-wiki-1.3.2.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-wiki">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-wiki">GitLab</a></p><h3>Ada Server Faces, Version 1.4.3</h3><ul><li>Add jQuery 3.6.0</li><li>Add a programmer's guide</li><li>Remove very old jQuery 1.11.3, jQuery UI 1.11.4, jQuery Chosen 1.4.2</li></ul><p>Links: <a href="http://download.vacs.fr/ada-asf/ada-asf-1.4.3.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-asf">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-asf">GitLab</a></p><h3>Ada Database Objects, Version 2.2.0</h3><ul><li>Improvement of query loaders</li><li>Fix reading database schema with float/double values</li><li>Rename Get_Double into Get_Long_Float</li></ul><p>Links: <a href="http://download.vacs.fr/ada-ado/ada-ado-2.2.0.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-ado">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-ado">GitLab</a></p><h3>Ada EL Library, Version 1.8.3</h3><ul><li>Fix compilation warning with GNAT 2021</li></ul><p>Links: <a href="http://download.vacs.fr/ada-el/ada-el-1.8.3.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-el">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-el">GitLab</a></p><h3>Ada Utility Library, Version 2.4.1</h3><ul><li>Fix compilation issues with GNAT 2021</li><li>Fix serialization of Util.Beans.Objects holding arrays or maps</li></ul><p>Links: <a href="http://download.vacs.fr/ada-util/ada-util-2.4.1.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-util">GitHub</a> <a href="https://gitlab.com/stcarrez/ada-util">GitLab</a></p><h3>Ada Keystore, Version 1.3.2</h3><ul><li>Minor compilation warning fixes</li></ul><p>Links: <a href="http://download.vacs.fr/ada-keystore/ada-keystore-1.3.2.tar.gz">Download</a> <a href="https://github.com/stcarrez/ada-keystore">GitHub</a> <a href="https://github.com/stcarrez/ada-keystore">GitLab</a></p><p>All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at <a href="http://download.vacs.fr/ada-awa/awa-all-2.3.0.tar.gz">http://download.vacs.fr/ada-awa/awa-all-2.3.0.tar.gz</a></p><p>After downloading the <a href="http://download.vacs.fr/ada-awa/awa-all-2.3.0.tar.gz">awa-all-2.3.0.tar.gz</a> package, have a look at the <a href="http://ada-awa.readthedocs.io/en/latest/">Ada Web Application Programmer's Guide</a> to learn how to build, install and start using all this.</p><p>If you don't have time to build all this, a docker container is available: <a href="https://hub.docker.com/r/ciceron/ada-awa/">https://hub.docker.com/r/ciceron/ada-awa/</a></p><h2>Debian packages</h2><p>I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.</p><h3>Access to the repository</h3><p>The repository packages are signed with GPG. To get the verification key and setup the <code>apt-get</code> tool, you should run the following command:</p><pre><code>wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
</code></pre><h3>Ubuntu 18.04 Bionic Beaver</h3><p>A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the <code>gnat-7</code> package and depend on <code>libgpr1-dev</code>, <code>libaws3.3.2.2-dev</code> and <code>libxmlada-sax7-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-bionic bionic main
</code></pre><h3>Ubuntu 20.04 Focal Fossa</h3><p>A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the <code>gnat-9</code> package and depend on <code>libgnatprj7-dev</code>, <code>libaws19-dev</code> and <code>libxmlada-sax9-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-focal focal main
</code></pre><h3>Debian 10 Buster</h3><p>A third repository provides Debian packages targeted at Debian 10 buster. They are built with the <code>gnat-8</code> package and depend on <code>libgpr2-dev</code>, <code>libaws18-dev</code> and <code>libxmlada-sax8-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/debian-buster buster main
</code></pre><h3>Installation</h3><p>Once you've added the configuration line, you can install the packages:</p><pre><code>sudo apt-get update
sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
</code></pre><p>The <a href="https://github.com/stcarrez/ada-awa">Ada Web Application</a> framework contains several plugins which are provided by separate Debian packages.</p><pre><code>sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
</code></pre><p>The documentation is packaged in separate Debian packages, you can install them with:</p><pre><code>sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
</code></pre></div> Advanced Resource Embedder 1.1.0urn:md5:2087a0d7482b1a0f7a7ef5ac368f31792021-07-04T16:52:00+00:002021-07-04T16:52:00+00:00Stephane CarrezAdageneratorCGo
<div class="post-text"><p>The resource embedder allows to embed files in binaries by producing C, Ada or Go source files that contain the original files. The first version of the tool was representing the file content as an array of bytes. In Ada, they are accessed through an `Ada.Streams.Stream_Element_Array` which is not easy to use when you need the content as a string.</p><p>The new release introduces the customization of the format for each resource. The format is controlled through the XML description by the `format` attribute. The following data formats are supported:</p><ul><li>`binary` format provides the file content as a binary data,</li><li>`string` format provides the file content as string,</li><li>`lines` format splits the content in several lines and according to a set of customizable rules.</li></ul><p>With the `string` format, the Ada code generator can generate the following function:</p><p>```</p><pre><code> function Get_Content (Name : in String)
return access constant String;
</code></pre><p>```</p><p>The `lines` format tells the code generator to represent the content as an array of separate lines. For this integration, some control is available to indicate how the content must be split and optionally apply some filter on the input content. These controls are made within the XML description by using the `line-separator` and `line-filter` description: The `line-separator` indicates the characters that represent a line separation. There can be several `line-separator` definition. The `line-filter` defines a regular expression that when matched must be replaced by an empty string or a specified content. The `line-filter` are applied in the order of the XML definition.</p><p>The example below is intended to integrate an SQL scripts with:</p><ul><li>a separate line for each SQL statement,</li><li>remove spurious empty lines and SQL comments.</li></ul><p>The SQL statements are separated by `;` (semi-colon) and the `line-separator` indicates to split lines on that character. By splitting on the `;` we allow to have an SQL statement on multiple lines in the original SQL source file.</p><p>```XML <package></p><pre><code> <resource name='Scripts'
format='lines'
type='access constant String'>
<line-separator>;</line-separator>
</code></pre><pre><code> <!-- Remove new lines -->
<line-filter>[\r\n]</line-filter>
</code></pre><pre><code> <!-- Remove C comments -->
<line-filter>/\*[^/]*\*/</line-filter>
</code></pre><pre><code> <!-- Remove contiguous spaces after C comments removal -->
<line-filter replace=' '>[ \t][ \t]+</line-filter>
</code></pre><pre><code> <install mode='copy' strip-extension='yes'>
<fileset dir="sql">
<include name="**/*.sql"/>
</fileset>
</install>
</resource>
</code></pre><p></package> ```</p><p>Then the first `line-filter` will remove the `r` and `n` characters.</p><p>The regular expression `/\*[./]*\*/` matches a C style comment and remove it.</p><p>The last `line-filter` replaces multiple tabs and spaces by a single occurrence.</p><p>Below is an example of generated code with the above resource description. Each file is accessed through a separate variable whose name is built from the base name of the original file.</p><h5>[Resource Embedder Overview](Ada/are-scripts.png)</h5><p>The command used to generate this code was:</p><p>``` are <del>lang=Ada -o src </del>var-access <del>content-only </del>rule=package.xml . ```</p><p>and the `sql` directory contains only two files: `create-database.sql` and `drop-database.sql`.</p><p>The complete example is available for two languages:</p><ul><li><a href="Embedding SQL scripts in Ada and mapping them in array of String">Embedding SQL scripts in Ada and mapping them in array of String</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-lines)</li><li><a href="Embedding SQL scripts in C and mapping them in array of String">Embedding SQL scripts in C and mapping them in array of String</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/c-lines)</li></ul><p>To install the tool, follow the instructions given in the initial announcement: <a href="Advanced Resource Embedder for Ada, C and Go">Advanced Resource Embedder for Ada, C and Go</a>(https://blog.vacs.fr/vacs/blogs/post.html?post=2021/06/11/Advanced-Resource-Embedder).</p><p>If you want to know more about the tool, have a look at its documentation:</p><ul><li><a href="Resource Embedder Guide">Resource Embedder Guide</a>(https://resource-embedder.readthedocs.io/en/latest/) <a href="PDF">PDF</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)</li><li>Man page: <a href="are (1)">are (1)</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)</li></ul><p>and if you have ideas for improvements, fork the project and submit a pull request!</p></div> Advanced Resource Embedder for Ada, C and Gourn:md5:4031b79fd70c34caaaf9239de5fa43e12021-06-11T12:43:00+00:002021-06-11T12:43:00+00:00Stephane CarrezAdageneratorCGo
<div class="post-text"><p>The tool is able to apply some transformations on the collected files:</p><ul><li>it can run a Javascript minifier such as `closure`,</li><li>it can compress CSS files by running `yui-compressor`,</li><li>it can compress files by running `gzip` or another compression tool.</li></ul><p>Once these transformations are executed, it invokes a target generator to produce a source file either in C, Ada or Go language. The generated source file can then be used in the final program and taken into account during the compilation process of that program. At the end, the binary will contain the embedded files with their optional transformations.</p><h5>[Resource Embedder Overview](Ada/resource-embedder.png)</h5><p>The process to use ARE is simple:</p><ul><li>You describe the resources that you want to embed.</li><pre><code> The description is either made on command line arguments or by writing an XML file.
The XML description gives more flexibility as it allows to define a transformation rule that
must be executed on the original file before being embedded. This allows to minify a Javascript
or CSS file, compress some files and even encrypt a file before its integration.
</code></pre><li>You run the ARE command with your target language and rule description and you give the tool</li><pre><code> a list of directories that must be scanned to identify the files that must be collected.
The ARE tool scan the directories according to the patterns that you have given either on
the command line or in the XML rule description. After identifying the files, the tool applies
the rules and execute the transformations.
The ARE tool then invokes the target language generator that writes one or several files depending
on the list of resources.
</code></pre><li>Once the files are generated, you use them in your program and add them in your build process</li><pre><code> as they are now part of your sources. After building your program, it now embeds the
resource files that were collected and optionally transformed.
</code></pre></ul><ol><li><ol><li>Defining resources to embed</li></ol></li></ol><p>The first step is to write some `package.xml` file which describes a list of resources with their content. The root XML element is *`package`* and each resource is described by a `resource` XML element. The resource is assigned a name that will be used by the code generator. The C generator will use the name as a prefix for the C function and C type declaration. The Ada and Go generator will use that name for the Ada or Go package.</p><p>The following resource definition declares the `Help` resource. It contains an installation rule that will copy the files under the `help` directory in the resource set. Only files matching the `.txt` pattern will be taken into account.</p><p>```XML <package></p><pre><code> <resource name='Help'>
<install mode='copy'>
<fileset dir="help">
<include name="**/*.txt"/>
</fileset>
</install>
</resource>
...
</code></pre><p></package> ```</p><p>The next resource definition will run an external program to get the content that must be embedded. The `man` directory is scanned and it will execute the command `man #{name}` on each filename found. That directory contains the empty files `ls`, `pwd` and `sh` and this will run and embed the man page for these commands.</p><p>```XML <package></p><pre><code> ...
<resource name='Man'>
<install mode='exec'>
<command output='#{dst}'>man #{name}</command>
<fileset dir="man">
<include name="*"/>
</fileset>
</install>
</resource>
</code></pre><p></package> ```</p><p>You may run other commands such as:</p><p>``` <command>closure <del>charset UTF-8 #{src} </del>js_output_file #{dst}</command> <command>yui-compressor <del>type css </del>charset UTF-8 -o #{dst} #{src}</command> <command output="#{dst}">gzip --no-name -c #{src}</command> ```</p><ol><li><ol><li>Running the Advanced Resource Embedder</li></ol></li></ol><p>The tool has several options that allows you to control the target code generator and tune the generation according to your needs. Assuming that the files to embed are located in the current directory, you would use the following command to generate C source files in the `src` directory:</p><p>``` are <del>lang=c -o src </del>rule=package.xml <del>list-access </del>name-access . ```</p><p>For C, this would generate a `src/man.h`, `src/man.c`, `src/help.h` and `src/help.c`. You now have to include these files in the compilation of your program.</p><p>You would use the following command for Go:</p><p>``` are <del>lang=go </del>rule=package.xml <del>list-access </del>name-access . ```</p><p>and it would generate in `man/man.go` and `help/help.go` the Go source files.</p><p>You would use the following command for Ada:</p><p>``` are <del>lang=Ada -o src </del>rule=package.xml <del>list-access </del>name-access . ```</p><p>and it would generate `src/man.ads`, `src/man.adb`, `src/help.ads` and `src/help.adb`.</p><ol><li><ol><li>Using the resource</li></ol></li></ol><p>The code generator emits by default a set of type and function declaration that give access to the resource. For C, the following structure and declaration are generated in the header for the `man` resource:</p><p>```C struct man_content {</p><pre><code> const unsigned char *content;
size_t size;
time_t modtime;
int format;
</code></pre><p>};</p><p>extern const struct man_content* man_get_content(const char* name); ```</p><p>But for the Go language, the `man` generated package declares:</p><p>```Go type Content struct {</p><pre><code> Content []byte
Size int64
Modtime int64
Format int
</code></pre><p>} ... func Get_content(name string) (*Content) {...} ```</p><p>And for Ada, it will generate:</p><p>```Ada package Resources.Man is</p><pre><code> type Content_Access is access constant Ada.Streams.Stream_Element_Array;
type Name_Access is access constant String;
type Format_Type is (FILE_RAW, FILE_GZIP);
type Content_Type is record
Name : Name_Access;
Content : Content_Access;
Modtime : Interfaces.C.long := 0;
Format : Format_Type := FILE_RAW;
end record;
Null_Content : constant Content_Type;
</code></pre><pre><code> type Name_Array is array (Natural range <>) of Name_Access;
Names : constant Name_Array;
</code></pre><pre><code> function Get_Content (Name : String) return Content_Type;
</code></pre><p>private</p><pre><code> ...
</code></pre><p>end Resources.Man; ```</p><p>You can avoid the type declaration by using the `--no-type-declaration` option and in that case you have to make available these types somehow. In Ada, this can easily be made by providing these types in a parent Ada package (see the <a href="Embedding help and documentation in Ada">Embedding help and documentation in Ada</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help) for the example).</p><p>Now with the generated code, you only need to call the generated get content method to obtain the embedded content. Simple!</p><ol><li><ol><li>Examples</li></ol></li></ol><p>The project proposes several detailed examples to illustrate various integrations.</p><p>This first set of example shows how to you can embed configuration files in a C, Ada or Go program. The Advance Resource Embedder simply puts the configuration files in an array of bytes that can easily be retrieved by a generated function.</p><ul><li><a href="Embedding configuration files in C">Embedding configuration files in C</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/c-config)</li><li><a href="Embedding configuration files in Ada">Embedding configuration files in Ada</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-config)</li><li><a href="Embedding configuration files in Go">Embedding configuration files in Go</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/go-config)</li></ul><p>A second set of example is more advanced by the use of an XML file that describes what must be embedded with the transformations that must be made. It creates two distinct resource sets `help` and `man`. The `help` resource set is composed of a set of fixed documentation files provided in the example. The `man` resource set is created by running the `man` Unix command on various names to embed the man page of `ls`, `pwd` and `sh`.</p><ul><li><a href="Embedding help and documentation in C">Embedding help and documentation in C</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/c-help)</li><li><a href="Embedding help and documentation in Ada">Embedding help and documentation in Ada</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-help)</li><li><a href="Embedding help and documentation in Go">Embedding help and documentation in Go</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/go-help)</li></ul><p>More specific examples show how to make specific transformations on the files before integrating them:</p><ul><li><a href="Embedding web files in C">Embedding web files in C</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/c-web)</li><li><a href="Embedding merged properties in Ada">Embedding merged properties in Ada</a>(https://gitlab.com/stcarrez/resource-embedder/tree/master/examples/ada-bundles)</li></ul><ol><li><ol><li>License considerations</li></ol></li></ol><p>The generated code produced by <a href="Advance Resource Embedder">Advance Resource Embedder</a>(https://gitlab.com/stcarrez/resource-embedder) is not covered by any license. However, when you integrate some resource files with the tool, the generated code will contain the original file in some binary form and therefore it may be covered by the license associated with these resource files.</p><p>For example, if you include a file covered by the GNU General Public license, then the generated file is covered by the GPL.</p><ol><li><ol><li>How can you get Advanced Resource Embedder?</li></ol></li></ol><ol><li><ol><li><ol><li>Use the source Luke!</li></ol></li></ol></li></ol><p>Quick instructions to build the tool:</p><p>``` git clone --recursive <a href="https://gitlab.com/stcarrez/resource-embedder.git">https://gitlab.com/stcarrez/resource-embedder.git</a> cd resource-embedder make build test install ```</p><ol><li><ol><li><ol><li>Debian Packages for x86_64</li></ol></li></ol></li></ol><p>You can install ARE by using the Debian 10 and Ubuntu 20.04 or 18.04 packages. First, setup to accept the signed packages:</p><p>``` wget -O - <a href="https://apt.vacs.fr/apt.vacs.fr.gpg.key">https://apt.vacs.fr/apt.vacs.fr.gpg.key</a> | sudo apt-key add - ```</p><p>and choose one of the `echo` command according to your Linux distribution:</p><p>Ubuntu 20.04 ``` echo "deb <a href="https://apt.vacs.fr/ubuntu-focal">https://apt.vacs.fr/ubuntu-focal</a> focal main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```</p><p>Ubuntu 18.04 ``` echo "deb <a href="https://apt.vacs.fr/ubuntu-bionic">https://apt.vacs.fr/ubuntu-bionic</a> bionic main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```</p><p>Debian 10 ``` echo "deb <a href="https://apt.vacs.fr/debian-buster">https://apt.vacs.fr/debian-buster</a> buster main" | sudo tee -a /etc/apt/sources.list.d/vacs.list ```</p><p>Then, launch the apt update command:</p><p>``` sudo apt-get update ```</p><p>and install the tool using:</p><p>``` sudo apt-get install -y are ```</p><ol><li><ol><li>Conclusion</li></ol></li></ol><p>Embedding files, scripts, documentation and other contents in C and Ada is easily done by using the <a href="Advance Resource Embedder">Advance Resource Embedder</a>(https://gitlab.com/stcarrez/resource-embedder). Go developers are already familiar with these mechanisms thanks to the `go:embed` directive. But the tool provides custom transformations that gives more flexibility for the integration.</p><p>Some benefits in embedding contents in final binaries:</p><ul><li>the content is embedded within the readonly `.rodata` section,</li><li>you avoid at least 3 system calls to access the content: an `open`, a `read` and a `close`,</li><li>you reduce the size of your application on the disk: contents are contiguous within the `.rodata` section</li><pre><code> whereas written on a file system they each require full dedicated data blocks (4K in most cases),
</code></pre><li>you get a portable `mmap` of your files for free and without pain.</li></ul><p>If you want to know more about the tool, have a look at its documentation:</p><ul><li><a href="Resource Embedder Guide">Resource Embedder Guide</a>(https://resource-embedder.readthedocs.io/en/latest/) <a href="PDF">PDF</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are-book.pdf)</li><li>Man page: <a href="are (1)">are (1)</a>(https://gitlab.com/stcarrez/resource-embedder/blob/master/docs/are.md)</li></ul><p>and if you have ideas for improvements, fork the project and submit a pull request!</p></div> New release Ada BFD 1.2.0urn:md5:49b106f1dd4b211d749df14de73863ab2021-04-11T18:37:21+00:002021-04-11T18:37:21+00:00Stephane CarrezbinutilsreleaseAdaELF
<div class="post-text"><p>The new release is a cleanup and update of the library to support newer version of GNU binutils. The main changes are below:</p><ul><li>Cleanup build process and use gprinstall for installation</li><li>Fix build with binutils > 2.34</li><li>Remove invalid API: `Bfd.Set_Error_Handler`</li><li>Remove fake API: `Bfd.Symbols.Is_Local_Label_Name`</li></ul><ol><li><ol><li>Installation</li></ol></li></ol><p>To use the Ada BFD library, you may either build it from the sources <a href="ada-bfd-1.2.0.tar.gz">ada-bfd-1.2.0.tar.gz</a>(https://download.vacs.fr/ada-bfd/ada-bfd-1.2.0.tar.gz) or install the Debian packages.</p><p>To build from the sources, you need to have a working GNAT Ada compiler as well as the `binutils-dev` Debian package installed. Then, run the following commands:</p><p>```sh git clone <a href="https://github.com/stcarrez/ada-bfd.git">https://github.com/stcarrez/ada-bfd.git</a> cd ada-bfd ./configure make build check install ```</p><p>For the Debian package installation, use the configuration that corresponds to your setup:</p><p>``` deb <a href="https://apt.vacs.fr/ubuntu-bionic">https://apt.vacs.fr/ubuntu-bionic</a> bionic main deb <a href="https://apt.vacs.fr/ubuntu-focal">https://apt.vacs.fr/ubuntu-focal</a> focal main deb <a href="https://apt.vacs.fr/debian-buster">https://apt.vacs.fr/debian-buster</a> buster main ```</p><p>and then run:</p><p>``` sudo apt-get update</p><ol><li>Bionic: sudo apt-get install libbfdada1-dev</li><li>Focal: sudo apt-get install libbfdada2-dev</li><li>Buster: sudo apt-get install libbfdada3-dev</li></ol><p>```</p><ol><li><ol><li>Reading the ELF sections</li></ol></li></ol><p>Using the Ada BFD library in a projet is quite easy, the first step is to add the following line in your GNAT project file:</p><p>``` <a href="Ada">Ada</a> with "bfdada"; ```</p><p>To access the information of a binary, you must first define an instance of the `File_Type` and open the file. You will do this as follows:</p><p>``` <a href="Ada">Ada</a> with Bfd.Files;</p><pre><code> ...
Path : constant String := "..."; -- the binary to load
File : Bfd.Files.File_Type;
...
Bfd.Files.Open (File, Path, "");
</code></pre><p>```</p><p>Looking at the ELF section is easily made by using the `Section_Iterator` provided by the `Bfd.Sections` package.</p><p>```ada with Bfd.Sections;</p><pre><code> ...
Iter : Bfd.Sections.Section_Iterator := Bfd.Sections.Get_Sections (File);
...
while Bfd.Sections.Has_Element (Iter) loop
declare
S : constant Bfd.Sections.Section := Bfd.Sections.Element (Iter);
begin
Ada.Text_IO.Put_Line (Bfd.Sections.Get_Name (S));
end;
Bfd.Sections.Next (Iter);
end loop;
</code></pre><p>```</p><p>The library contains several examples that show different features of the Ada BFD library:</p><p>| <a href="bfdinfo.adb">bfdinfo.adb</a>(https://github.com/stcarrez/ada-bfd/blob/master/samples/bfdinfo.adb) | ./bin/bfdinfo ./bin/bfdgen | Open BFD file, get flags, list sections and symbols | | <a href="sections.adb">sections.adb</a>(https://github.com/stcarrez/ada-bfd/blob/master/samples/sections.adb) | ./bin/sections ./bin/bfdgen | Display the ELF sections with the `Bfd.Sections` | | <a href="symbol.adb">symbol.adb</a>(https://github.com/stcarrez/ada-bfd/blob/master/samples/symbol.adb) | ./bin/symbol ./bin/bfdgen main | Read the symbol table with `Bfd.Symbols` | | <a href="disassemble.adb">disassemble.adb</a>(https://github.com/stcarrez/ada-bfd/blob/master/samples/disassemble.adb) | ./bin/disassemble ./bin/bfdgen | Disassemble the text section with `Bfd.Disassemble` |</p></div> AWA 2.2.0urn:md5:7cf2c06fdc7fb95d1d33bae39809e88d2021-03-20T16:33:31+00:002021-03-20T16:33:31+00:00Stephane CarrezAdareleaseframework
<div class="post-text"><p>The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.</p><p>AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.</p><p>A typical architecture of an AWA application is represented by the picture below:</p><p>((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)</p><p>This version of AWA integrates smoothly with <a href="https://blog.vacs.fr/vacs/blogs/post.html?post=2019/12/26/AKT-a-tool-to-store-and-protect-your-sensitive-information">Ada Keystore</a> in order to protect the server sensitive configuration.</p><h3>AWA, Version 2.2.0</h3><ul><li>Fixed the Markdown js editor configuration</li><li>Send an event when a blog post is published (allows customisation such as sending e-mails)</li><li>Use Dynamo 1.2.1, Ada Server Faces 1.4.2, Ada Servlet 1.5.1, OpenAPI Ada 0.4.0</li><li>Use Ada Security 1.4.0, Ada Wiki 1.3.1, Ada Database Objects 2.1.2</li><li>Use Ada Keystore 1.3.1, Ada EL 1.8.2, Ada Utility Library 2.4.0</li></ul><p>Download: <a href="http://download.vacs.fr/ada-awa/ada-awa-2.2.0.tar.gz">http://download.vacs.fr/ada-awa/ada-awa-2.2.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-awa">https://github.com/stcarrez/ada-awa</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-awa">https://gitlab.com/stcarrez/ada-awa</a></p><h3>Dynamo, Version 1.2.1</h3><ul><li>Fix uninitialized float values in generated code</li></ul><p>Download: <a href="http://download.vacs.fr/dynamo/dynamo-1.2.1.tar.gz">http://download.vacs.fr/dynamo/dynamo-1.2.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/dynamo">https://github.com/stcarrez/dynamo</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/dynamo">https://gitlab.com/stcarrez/dynamo</a></p><h3>Swagger Ada, Version 0.4.0</h3><ul><li>Update the openapi generator to version 5.1.0</li></ul><p>Download: <a href="http://download.vacs.fr/swagger-ada/swagger-ada-0.4.0.tar.gz">http://download.vacs.fr/swagger-ada/swagger-ada-0.4.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/swagger-ada">https://github.com/stcarrez/swagger-ada</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/swagger-ada">https://gitlab.com/stcarrez/swagger-ada</a></p><h3>Ada Servlet, Version 1.5.1</h3><ul><li>Cleanup the examples</li><li>Fix registration and support of application with an empty registration URI</li></ul><p>Download: <a href="http://download.vacs.fr/ada-servlet/ada-servlet-1.5.1.tar.gz">http://download.vacs.fr/ada-servlet/ada-servlet-1.5.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-servlet">https://github.com/stcarrez/ada-servlet</a></p><p>GitLab: <a href="https://github.com/stcarrez/ada-servlet">https://github.com/stcarrez/ada-servlet</a></p><h3>Ada Wiki Library, Version 1.3.1</h3><ul><li>Minor cleanup for the build</li></ul><p>Download: <a href="http://download.vacs.fr/ada-wiki/ada-wiki-1.3.1.tar.gz">http://download.vacs.fr/ada-wiki/ada-wiki-1.3.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-wiki">https://github.com/stcarrez/ada-wiki</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-wiki">https://gitlab.com/stcarrez/ada-wiki</a></p><h3>Ada Server Faces, Version 1.4.2</h3><ul><li>Fix compilation warnings</li><li>Cleanup build and examples</li></ul><p>Download: <a href="http://download.vacs.fr/ada-asf/ada-asf-1.4.2.tar.gz">http://download.vacs.fr/ada-asf/ada-asf-1.4.2.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-asf">https://github.com/stcarrez/ada-asf</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-asf">https://gitlab.com/stcarrez/ada-asf</a></p><h3>Ada Database Objects, Version 2.1.2</h3><ul><li>Fix uninitialized float values in generated code used by unit tests</li></ul><p>Download: <a href="http://download.vacs.fr/ada-ado/ada-ado-2.1.2.tar.gz">http://download.vacs.fr/ada-ado/ada-ado-2.1.2.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-ado">https://github.com/stcarrez/ada-ado</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-ado">https://gitlab.com/stcarrez/ada-ado</a></p><h3>Ada EL Library, Version 1.8.2</h3><ul><li>Minor build cleanup</li></ul><p>Download: <a href="http://download.vacs.fr/ada-el/ada-el-1.8.2.tar.gz">http://download.vacs.fr/ada-el/ada-el-1.8.2.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-el">https://github.com/stcarrez/ada-el</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-el">https://gitlab.com/stcarrez/ada-el</a></p><h3>Ada Utility Library, Version 2.4.0</h3><ul><li>Add support to customize and provide application specific log appenders (example in ada-keystore)</li><li>Improvement of read/write streams to chain LZMA, AES, Base64</li><li>Add examples to show LZMA compress+AES encryption, AES decryption+LZMA decompress</li><li>Fix compilation with GNAT 10</li><li>New package Util.Properties.Form to help in parsing application/x-www-form-urlencoded</li><li>Fixed the Util.Tests.Get_Test_Path semantic and use the results directory</li><li>Drop detection of buggy gcc 4.7.2</li></ul><p>Download: <a href="http://download.vacs.fr/ada-util/ada-util-2.4.0.tar.gz">http://download.vacs.fr/ada-util/ada-util-2.4.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-util">https://github.com/stcarrez/ada-util</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-util">https://gitlab.com/stcarrez/ada-util</a></p><h3>Ada Security Library, Version 1.4.0</h3><ul><li>Add support to authenticate with Gitlab and GitHub</li><li>Update Yahoo! authenticate to use OpenId connect</li><li>Update the AWS demo to add Gitlab and Github</li></ul><p>Download: <a href="http://download.vacs.fr/ada-security/ada-security-1.4.0.tar.gz">http://download.vacs.fr/ada-security/ada-security-1.4.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-security">https://github.com/stcarrez/ada-security</a></p><p>GitLab: <a href="https://github.com/stcarrez/ada-security">https://github.com/stcarrez/ada-security</a></p><p>All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at <a href="http://download.vacs.fr/ada-awa/awa-all-2.2.0.tar.gz">http://download.vacs.fr/ada-awa/awa-all-2.2.0.tar.gz</a></p><p>After downloading the <a href="http://download.vacs.fr/ada-awa/awa-all-2.2.0.tar.gz">awa-all-2.2.0.tar.gz</a> package, have a look at the <a href="http://ada-awa.readthedocs.io/en/latest/">Ada Web Application Programmer's Guide</a> to learn how to build, install and start using all this.</p><p>If you don't have time to build all this, a docker container is available: <a href="https://hub.docker.com/r/ciceron/ada-awa/">https://hub.docker.com/r/ciceron/ada-awa/</a></p><h2>Debian packages</h2><p>I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.</p><h3>Access to the repository</h3><p>The repository packages are signed with GPG. To get the verification key and setup the <code>apt-get</code> tool, you should run the following command:</p><pre><code>wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
</code></pre><h3>Ubuntu 18.04 Bionic Beaver</h3><p>A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the <code>gnat-7</code> package and depend on <code>libgpr1-dev</code>, <code>libaws3.3.2.2-dev</code> and <code>libxmlada-sax7-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-bionic bionic main
</code></pre><h3>Ubuntu 20.04 Focal Fossa</h3><p>A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the <code>gnat-9</code> package and depend on <code>libgnatprj7-dev</code>, <code>libaws19-dev</code> and <code>libxmlada-sax9-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-focal focal main
</code></pre><h3>Debian 10 Buster</h3><p>A third repository provides Debian packages targeted at Debian 10 buster. They are built with the <code>gnat-8</code> package and depend on <code>libgpr2-dev</code>, <code>libaws18-dev</code> and <code>libxmlada-sax8-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/debian-buster buster main
</code></pre><h3>Installation</h3><p>Once you've added the configuration line, you can install the packages:</p><pre><code>sudo apt-get update
sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
</code></pre><p>The <a href="https://github.com/stcarrez/ada-awa">Ada Web Application</a> framework contains several plugins which are provided by separate Debian packages.</p><pre><code>sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
</code></pre><p>The documentation is packaged in separate Debian packages, you can install them with:</p><pre><code>sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
</code></pre><p>For the curious, you may browse the repository <a href="https://blog.vacs.fr/vacs/debian/ubuntu-focal/index.html">here</a>.</p></div> New release of AKT with Fuse supporturn:md5:f45e95e3497aa7a80095da3bfbede6ce2021-03-07T19:08:00+00:002021-03-07T19:08:00+00:00Stephane CarrezAdaSecurityreleaseStorage
<div class="post-text"><p>Data are stored in secure wallets protected and signed by their own AES-256 key.</p><p>Wallets are protected by a master key using AES-256-CBC and the wallet master key is protected by a user password. The wallet defines up to 7 slots that identify a password key that is able to unlock the master key. To open a wallet, it is necessary to unlock one of these 7 slots by providing the correct password. Wallet key slots are protected by the user's password and the PBKDF2-HMAC-256 algorithm, a random salt, a random counter and they are encrypted using AES-256-CBC. A wallet key slot can also be protected by a GPG private key.</p><p>Data stored in the wallet are protected by their own encryption keys using AES-256-CBC. Big contents are stored in several data blocks and each data block is encrypted by using its own AES-256-CBC encryption key. Encryption keys are protected by the master key using AES-256-CBC. Data blocks are then signed using HMAC-256.</p><p>A wallet can contain another wallet which is then protected by its own encryption keys and passwords (with 7 independent slots). Because the child wallet has its own master key, it is necessary to known the primary password and the child password to unlock the parent wallet first and then the child wallet.</p><h3>Installation</h3><p>You can install AKT by using the Debian 10 and Ubuntu 20.04 or 18.04 packages. First, setup to accept the signed packages:</p><pre><code class="lang-sh">wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
</code></pre><p>and choose one of the <code>echo</code> command according to your Linux distribution:</p><pre><code class="lang-sh"># Ubuntu 20.04
# echo "deb https://apt.vacs.fr/ubuntu-focal focal main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
# Ubuntu 18.04
# echo "deb https://apt.vacs.fr/ubuntu-bionic bionic main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
# Debian 10
# echo "deb https://apt.vacs.fr/debian-buster buster main" | sudo tee -a /etc/apt/sources.list.d/vacs.list
</code></pre><p>Then, launch the apt update command:</p><pre><code class="lang-sh">sudo apt-get update
</code></pre><p>and install the tool using:</p><pre><code class="lang-sh">sudo apt-get install -y akt
</code></pre><p>You can also build from the source by using the following commands:</p><pre><code class="lang-sh">sudo apt-get install -y make gnat-7 gprbuild git gnupg2 libfuse-dev
git clone --recursive https://github.com/stcarrez/ada-keystore.git
cd ada-keystore
./configure --enable-fuse
</code></pre><h3>Using AKT</h3><p>To setup a keystore file and protect it with your GPG key, use the following command:</p><pre><code class="lang-sh">akt create secure.akt --gpg your-gpg-key-id
</code></pre><p>The keystore content can be mounted as a filesystem so that the encrypted content are directly available to existing applications as regular files. The <code>mount</code> command starts the fuse daemon and puts the command as a background fuse daemon to serve as a fuse filesystem and make the content visible through the mount point.</p><pre><code class="lang-sh">akt mount secure.akt /mnt
</code></pre><p>After the mount is successful, you can copy or access any file stored in the secure keystore. By default the Fuse filesystem is configured to ask the kernel to avoid caching the decrypted content. This has an impact on the performance since every access requires to decrypt the content.</p><p>To unmount the filesystem, you must use the <code>umount</code> command.</p><pre><code class="lang-sh">umount /mnt
</code></pre><p>Sometimes, you may have to force the umount by running:</p><pre><code>sudo umount /mnt
</code></pre></div> Ada Web Application release 2.1.0urn:md5:7d5d71414984faf192ae0188d08cf7762020-11-15T21:12:45+00:002020-11-15T21:12:45+00:00Stephane CarrezAdareleaseframework
<div class="post-text"><p>The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.</p><p>AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.</p><p>A typical architecture of an AWA application is represented by the picture below:</p><p>((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)</p><p>This version of AWA integrates smoothly with <a href="https://blog.vacs.fr/vacs/blogs/post.html?post=2019/12/26/AKT-a-tool-to-store-and-protect-your-sensitive-information">Ada Keystore</a> in order to protect the server sensitive configuration.</p><h3>AWA, Version 2.1.0</h3><ul><li>Update Trumbowyg editor to version 2.21.0</li><li>Fix compilation issues with GNAT 2020</li><li>Update mail UI component to attach external files</li><li>Improved setup for secure configuration with Ada Keystore</li><li>Use Dynamo 1.2.0, Ada Server Faces 1.4.1, Ada Servlet 1.5.0, OpenAPI Ada 0.3.0</li><li>Use Ada Security 1.3.1, Ada Wiki 1.3.0, Ada Database Objects 2.1.1</li><li>Use Ada Keystore 1.2.1, Ada EL 1.8.1, Ada Utility Library 2.3.0</li></ul><p>Download: <a href="http://download.vacs.fr/ada-awa/ada-awa-2.1.0.tar.gz">http://download.vacs.fr/ada-awa/ada-awa-2.1.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-awa">https://github.com/stcarrez/ada-awa</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-awa">https://gitlab.com/stcarrez/ada-awa</a></p><h3>Dynamo, Version 1.2.0</h3><ul><li>Integrate ArgoUML 0.35.2-2020-07-05</li><li>Fix SQL generation with 'auto' generator</li><li>Fix XML Hibernate mapping support</li><li>Improvement in SQL schema generation</li></ul><p>Download: <a href="http://download.vacs.fr/dynamo/dynamo-1.2.0.tar.gz">http://download.vacs.fr/dynamo/dynamo-1.2.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/dynamo">https://github.com/stcarrez/dynamo</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/dynamo">https://gitlab.com/stcarrez/dynamo</a></p><h3>Swagger Ada, Version 0.3.0</h3><ul><li>Install the openapi generator</li><li>Update the openapi generator to version 5.0.0</li><li>Update to use Swagger UI 3.36.0</li><li>Fixed Ada client code generator to support `FreeFormObject` and multi-path parameters</li><li>Add support for HTTP HEAD, OPTIONS and PATCH requests</li><li>Initial implementation of Swagger OpenAPI to easily implement REST clients and servers</li></ul><p>Download: <a href="http://download.vacs.fr/swagger-ada/swagger-ada-0.3.0.tar.gz">http://download.vacs.fr/swagger-ada/swagger-ada-0.3.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/swagger-ada">https://github.com/stcarrez/swagger-ada</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/swagger-ada">https://gitlab.com/stcarrez/swagger-ada</a></p><h3>Ada Servlet, Version 1.5.0</h3><ul><li>Added support for PATCH and fix OPTIONS and HEAD requests</li><li>Added a Docker image to play with Ada Servlet</li></ul><p>Download: <a href="http://download.vacs.fr/ada-servlet/ada-servlet-1.5.0.tar.gz">http://download.vacs.fr/ada-servlet/ada-servlet-1.5.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-servlet">https://github.com/stcarrez/ada-servlet</a></p><p>GitLab: <a href="https://github.com/stcarrez/ada-servlet">https://github.com/stcarrez/ada-servlet</a></p><h3>Ada Wiki Library, Version 1.3.0</h3><ul><li>New plugin and filter to setup and expand variables in Wiki texts</li><li>Add support for Markdown tables</li><li>Fixes in the Markdown and Creole syntax parsers</li><li>New Ada Wiki Engine Programmer's Guide</li></ul><p>Download: <a href="http://download.vacs.fr/ada-wiki/ada-wiki-1.3.0.tar.gz">http://download.vacs.fr/ada-wiki/ada-wiki-1.3.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-wiki">https://github.com/stcarrez/ada-wiki</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-wiki">https://gitlab.com/stcarrez/ada-wiki</a></p><h3>Ada Server Faces, Version 1.4.1</h3><ul><li>Fix translations, compilation warnings</li></ul><p>Download: <a href="http://download.vacs.fr/ada-asf/ada-asf-1.4.1.tar.gz">http://download.vacs.fr/ada-asf/ada-asf-1.4.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-asf">https://github.com/stcarrez/ada-asf</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-asf">https://gitlab.com/stcarrez/ada-asf</a></p><h3>Ada Database Objects, Version 2.1.1</h3><ul><li>Fix using the configuration with multiple config environments</li></ul><p>Download: <a href="http://download.vacs.fr/ada-ado/ada-ado-2.1.1.tar.gz">http://download.vacs.fr/ada-ado/ada-ado-2.1.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-ado">https://github.com/stcarrez/ada-ado</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-ado">https://gitlab.com/stcarrez/ada-ado</a></p><h3>Ada EL Library, Version 1.8.1</h3><ul><li>Fix compilation issue with GNAT 2020</li></ul><p>Download: <a href="http://download.vacs.fr/ada-el/ada-el-1.8.1.tar.gz">http://download.vacs.fr/ada-el/ada-el-1.8.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-el">https://github.com/stcarrez/ada-el</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-el">https://gitlab.com/stcarrez/ada-el</a></p><h3>Ada Utility Library, Version 2.3.0</h3><ul><li>New stream operations to read/write UTF-8 sequences in Wide_Wide character</li><li>Fix AES encryption in CFB, OFB and CTR modes</li><li>Add HTTP support for HEAD, OPTIONS, PATCH requests</li></ul><p>Download: <a href="http://download.vacs.fr/ada-util/ada-util-2.3.0.tar.gz">http://download.vacs.fr/ada-util/ada-util-2.3.0.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-util">https://github.com/stcarrez/ada-util</a></p><p>GitLab: <a href="https://gitlab.com/stcarrez/ada-util">https://gitlab.com/stcarrez/ada-util</a></p><h3>Ada Security Library, Version 1.3.1</h3><ul><li>Fix AWS example</li></ul><p>Download: <a href="http://download.vacs.fr/ada-security/ada-security-1.3.1.tar.gz">http://download.vacs.fr/ada-security/ada-security-1.3.1.tar.gz</a></p><p>GitHub: <a href="https://github.com/stcarrez/ada-security">https://github.com/stcarrez/ada-security</a></p><p>GitLab: <a href="https://github.com/stcarrez/ada-security">https://github.com/stcarrez/ada-security</a></p><p>All these Ada projects can be downloaded individually but they are also packaged together to help in their download and build process. You can also download everything at <a href="http://download.vacs.fr/ada-awa/awa-all-1.1.0.tar.gz">http://download.vacs.fr/ada-awa/awa-all-1.1.0.tar.gz</a></p><p>After downloading the <a href="http://download.vacs.fr/ada-awa/awa-all-1.1.0.tar.gz">awa-all-1.1.0.tar.gz</a> package, have a look at the <a href="http://ada-awa.readthedocs.io/en/latest/">Ada Web Application Programmer's Guide</a> to learn how to build, install and start using all this.</p><p>If you don't have time to build all this, a docker container is available: <a href="https://hub.docker.com/r/ciceron/ada-awa/">https://hub.docker.com/r/ciceron/ada-awa/</a></p><h2>Debian packages</h2><p>I've created and setup a Debian repository to give access to several Debian packages for several Ada projects that I manage. The goal is to provide some easy and ready to use packages to simplify and help in the installation of various Ada libraries.</p><h3>Access to the repository</h3><p>The repository packages are signed with GPG. To get the verification key and setup the <code>apt-get</code> tool, you should run the following command:</p><pre><code>wget -O - https://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
</code></pre><h3>Ubuntu 18.04 Bionic Beaver</h3><p>A first repository provides Debian packages targeted at Ubuntu 18.04 bionic. They are built with the <code>gnat-7</code> package and depend on <code>libgpr1-dev</code>, <code>libaws3.3.2.2-dev</code> and <code>libxmlada-sax7-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-bionic bionic main
</code></pre><h3>Ubuntu 20.04 Focal Fossa</h3><p>A second repository provides Debian packages targeted at Ubuntu 20.04 focal. They are built with the <code>gnat-9</code> package and depend on <code>libgnatprj7-dev</code>, <code>libaws19-dev</code> and <code>libxmlada-sax9-dev</code>. Add the following line to your <code>/etc/apt/sources.list</code> configuration:</p><pre><code>deb https://apt.vacs.fr/ubuntu-focal focal main
</code></pre><h3>Installation</h3><p>Once you've added the configuration line, you can install the packages:</p><pre><code>sudo apt-get update
sudo apt-get install -y dynamo akt libawa2-dev libawa-unit2-dev
</code></pre><p>The <a href="https://github.com/stcarrez/ada-awa">Ada Web Application</a> framework contains several plugins which are provided by separate Debian packages.</p><pre><code>sudo apt-get install -y libawa-blogs2-dev libawa-questions2-dev libawa-wikis2-dev libawa-setup2-dev
</code></pre><p>The documentation is packaged in separate Debian packages, you can install them with:</p><pre><code>sudo apt-get install -y libawa-doc libado-doc libawa-doc libkeystoreada-doc libsecurity-doc libutilada-doc libwikiada-doc
</code></pre><p>For the curious, you may browse the repository <a href="https://blog.vacs.fr/vacs/debian/ubuntu-focal/index.html">here</a>.</p></div> Easy reading and writing files with Ada Utility Libraryurn:md5:3b8780a649dd29ae069d836ebd6c052e2020-08-09T20:49:00+00:002020-08-09T20:49:00+00:00Stephane CarrezAdaTutorialfiles
<div class="post-text"><p>The <a href="https://github.com/stcarrez/ada-util">Ada Utility Library</a> provides several simple operations that simplify the reading and writing of files through a single procedure call. These operations are not new since I've implemented most of them 10 years ago!!!</p><p>Reading a file line by line by using <code>Ada.Text_IO</code> is quite easy but annoying due to the fact that you have to open the file, iterate over the content getting one line at a time and then closing the file. To simplify this process, the <a href="https://github.com/stcarrez/ada-util/blob/master/src/base/files/util-files.ads">Util.Files</a> package of <a href="https://github.com/stcarrez/ada-util">Ada Utility Library</a> provides a simple <code>Read_File</code> procedure that uses a procedure as parameter and that procedure will be called for each line that is read.</p><pre><code class="lang-ada">with Util.Files;
procedure Read_Line (Line : in String) is ...;
Util.Files.Read_File (Path => "file.txt",
Process => Read_Line'Access);
</code></pre><p>Another <code>Read_File</code> procedure allows to read the file and get its content in a vector of strings so that once the file is read the vector contains each line. Yet, another <code>Read_File</code> procedure reads the file content in an <code>Unbounded_String</code>. You will use that later form as follows:</p><pre><code class="lang-ada">with Util.Files;
with Ada.Strings.Unbounded;
Content : Ada.Strings.Unbounded.Unbounded_String;
Util.Files.Read_File (Path => "file.txt",
Into => Content);
</code></pre><p>Very often it is also necessary to write some content in a file. Again, this is easy to do but a simple <code>Write_File</code> procedure that takes the file path and the content to write is easier to use in several cases:</p><pre><code class="lang-ada">with Util.Files;
Util.Files.Write_File (Path => "file.txt",
Content => "something");
</code></pre><p>The <a href="https://github.com/stcarrez/ada-util">Ada Utility Library</a> contains other useful operations and features that have helped me in implementing various projects such as <a href="https://github.com/stcarrez/ada-awa">Ada Web Application</a> and <a href="https://github.com/stcarrez/ada-keystore">Ada Keystore</a>. Have a look at the <a href="https://ada-util.readthedocs.io/en/latest/">Ada Utility Library Programmer's Guide</a>!</p></div> Ada Stemmer Libraryurn:md5:2609b15398e6015330ddfd83cb1d292a2020-05-16T07:55:00+00:002020-05-16T07:55:00+00:00Stephane CarrezAdaTutorialStemmerAnalysis
<div class="post-text"><p>Stemming is not new as it was first introduced in 1968 by <a href="https://en.wikipedia.org/wiki/Julie_Beth_Lovins">Julie Beth Lovis</a> who was a computational linguist that created the first algorithm known today as the <a href="http://snowball.tartarus.org/algorithms/lovins/stemmer.html">Lovins Stemming</a> algorithm. Her algorithm has significantly influenced other algorithms such as the <a href="https://tartarus.org/martin/PorterStemmer/">Porter Stemmer</a> algorithm which is now a common stemming algorithm for English words. These algorithms are specific to the English language and will not work for French, Greek or Russian.</p><p>To support several natural languages, it is necessary to have several algorithms. The <a href="https://snowballstem.org/">Snowball stemming algorithms</a> project provides such support through a specific string processing language, a compiler and a set of algorithms for various natural languages. The <a href="https://snowballstem.org/">Snowball compiler</a> has been adapted to generate Ada code (See <a href="https://github.com/stcarrez/snowball/tree/ada-support">Snowball Ada</a> on GitHub).</p><p>The <a href="https://github.com/stcarrez/ada-stemmer">Ada Stemmer Library</a> integrates stemming algorithms for: English, Danish, Dutch, French, German, Greek, Italian, Serbian, Spanish, Swedish, Russian. The Snowball compiler provides several other algorithms but they are not integrated yet: their integration is left as an exercise to the reader.</p><h3>Stemmer Overview</h3><p>Snowball is a small string processing language designed for creating stemming algorithms for use in Information Retrieval. A Snowball script describes a set of rules which are applied and checked on an input word or some portion of it in order to eliminate or replace some terms. The stemmer will usually transform a plural into a singular form, it will reduce the multiple forms of a verb, find the noun from an adverb and so on. <a href="https://snowballstem.org/algorithms/romance.html">Romance languages</a>, <a href="https://snowballstem.org/algorithms/germanic.html">Germanic languages</a>, <a href="https://snowballstem.org/algorithms/scandinavian.html">Scandinavian languages</a> share some common rules but each language will need its own snowball algorithm. The Snowball compiler provides a detailed list of several stemming algorithms for various natural languages. This list is available on: <a href="https://snowballstem.org/algorithms/">https://snowballstem.org/algorithms/</a></p><p><img src="/vacs/blogs/images/2601/901/default/ada-stemmer-overview.png" width="801" height="491" alt="C"></img></p><p>The Snowball compiler reads the Snowball script and generates the stemmer implementation for a given target programming language such as Ada, C, C#, Java, JavaScript, Go, Python, Rust. The <a href="https://github.com/stcarrez/ada-stemmer">Ada Stemmer Library</a> contains the generated algorithms for several natural languages. The generated stemmers are not able to recognize the natural language and it is necessary to tell the stemmer library which natural language you wish to use.</p><p>The <a href="https://github.com/stcarrez/ada-stemmer">Ada Stemmer Library</a> supports only UTF-8 strings which simplifies both the implementation and the API. The library only uses the Ada <code>String</code> type to handle strings.</p><h3>Setup</h3><p>To use the library, you should run the following commands:</p><pre><code class="lang-shell"> git clone https://github.com/stcarrez/ada-stemmer.git
cd ada-stemmer
make build install
</code></pre><p>This will fetch, compile and install the library. You can then add the following line in your GNAT project file:</p><pre><code class="lang-ada"> with "stemmer";
</code></pre><h3>Stemming examples</h3><p>Each stemmer algorithm works on a single word at a time. The <a href="https://github.com/stcarrez/ada-stemmer">Ada Stemmer Library</a> does not split words. You have to give it one word at a time to stem and it returns either the word itself or its stem. The <code>Stemmer.Factory</code> is the multi-language entry point. The stemmer algorithm is created for each call. The following simple code:</p><pre><code class="lang-ada"> with Stemmer.Factory; use Stemmer.Factory;
with Ada.Text_IO; use Ada.Text_IO;
...
Put_Line (Stem (L_FRENCH, "chienne"));
</code></pre><p>will print the string:</p><pre><code class="lang-ada"> chien
</code></pre><p>When multiple words must be stemmed, it may be better to declare the instance of the stemmer and use the same instance to stem several words. The <code>Stem_Word</code> procedure can be called with each word and it returns a boolean that indicates whether the word was stemmed or not. The result is obtained by calling the <code>Get_Result</code> function. For exemple,</p><pre><code class="lang-ada"> with Stemmer.English;
with Ada.Text_IO; use Ada.Text_IO;
..
Ctx : Stemmer.English.Context_Type;
Stemmed : Boolean;
..
Ctx.Stem_Word ("zealously", Stemmed);
if Stemmed then
Put_Line (Ctx.Get_Result);
end if;
</code></pre><h3>Integrating a new Stemming algorithm</h3><p>Integration of a new stemming algorithm is quite easy but requires to install the <a href="https://github.com/stcarrez/snowball/tree/ada-support">Snowball Ada</a> compiler.</p><pre><code class="lang-shell"> git clone --branch ada-support https://github.com/stcarrez/snowball
cd snowball
make
</code></pre><p>The Snowball compiler needs the path of the stemming algorithm, the target programming language, the name of the Ada child package that will contain the generated algorithm and the target path. For example, to generate the Lithuanian stemmer, the following command can be used:</p><pre><code class="lang-shell"> ./snowball algorithms/lithuanian.sbl -ada -P Lithuanian -o stemmer-lithuanian
</code></pre><p>You will then get two files: <code>stemmer-lithuanian.ads</code> and <code>stemmer-lithuanian.adb</code>. After integration of the generated files in your project, you can access the generated stemmer with:</p><pre><code class="lang-ada"> with Stemmer.Lithuanian;
..
Ctx : Stemmer.Lithuanian.Context_Type;
</code></pre><h3>Conclusion</h3><p>Thanks to the <a href="https://snowballstem.org/">Snowball compiler</a> and its algorithms, it is possible to do some natural language analysis. Version 1.0 of the <a href="https://github.com/stcarrez/ada-stemmer">Ada Stemmer Library</a> being available on GitHub, it is now possible to start doing some natural language analysis in Ada!</p></div> New version of Ada Web Applicationurn:md5:a1a6bbc8fba7d3f154dc5e44201020362020-05-01T20:49:00+00:002020-05-01T20:49:00+00:00Stephane CarrezreleaseAdaframework
<div class="post-text"><p>The framework provides several ready to use and extendable modules that are common to many web applications. This includes the login, authentication, users, permissions, managing comments, tags, votes, documents, images. It provides a complete blog, question and answers and a wiki module.</p><p>AWA simplifies the Web Application development by taking care of user management authentication and by providing the foundations on top of which you can construct your own application. AWA provides a powerful permission management that gives flexibility to applications to grant access and protect your user's resources.</p><p>A typical architecture of an AWA application is represented by the picture below:</p><p>((Ada/awa_architecture_overview.png|awa_architecture_overview.png|C|Ada Web Application architecture)</p><p>This version of AWA integrates smoothly with <a href="https://blog.vacs.fr/vacs/blogs/post.html?post=2019/12/26/AKT-a-tool-to-store-and-protect-your-sensitive-information">Ada Keystore</a> in order to protect the server sensitive configuration.</p><h4>Ada Web Application, Version 2.0</h4><ul><li>Refactoring of build process and installation</li><li>New audit manager for database auditing</li><li>Support for Postgresql</li><li>Improvements of images and storage plugins</li><li>Update Trumbowyg editor to version 2.18.0</li><li>Update Flot library to version 4.2.0</li><li>Support for commands to configure, start, stop the server</li><li>New mail UI component <mail:attachment> to send attachments</li></ul><h4>Dynamo, Version 1.0.0</h4><ul><li>Improvement and fixes in the YAML database model files</li><li>Add support for Nullable_String type</li><li>Generate Postgresql SQL files from the model files</li><li>Add support for database record auditing</li><li>Add support for floating point</li><li>Add support for CSS and Javascript merge in the dist command</li></ul><h4>Ada Database Objects, Version 2.1.0</h4><ul><li>Added Is_Modified predicate on database objects</li><li>Fix SQLite Load_Schema to avoid loading SQLite specific tables</li><li>Support for Postgresql database</li><li>Improvement for errors reported by database drivers</li><li>New audit framework to track database record changes</li><li>Added support for floating point numbers</li><li>Serialize queries in JSON/XML streams</li></ul><h4>Ada Keystore, Version 1.2.0</h4><ul><li>Added support for Fuse with a new mount command in akt (beta!)</li><li>Fix the implementation to iterate with Keystore.Properties</li></ul><h4>Ada Server Faces, Version 1.4.0</h4><ul><li>Performance improvement for the Facelet cache</li><li>Integrate jQuery 3.4.1, jQuery UI 1.12.1, jQuery Chosen 1.8.7</li><li>New <f:validateRegex> to validate an input field with a regular expression</li></ul><h4>Ada Utility Library, Version 2.2.0</h4><ul><li>New Wait_Empty operation on fifo.</li><li>Add Get_Count and Wait operation on executors</li></ul><h4>Ada EL Library, Version 1.8.0</h4><ul><li>New Expand procedure to expand the properties in place</li></ul><h4>Ada Wiki Library, Version 1.2.1</h4><ul><li>Minor configuration and code coverage support</li><li>Corrections in the Markdown syntax parser</li></ul><h4>Ada Security Library, Version 1.3.0</h4><ul><li>Add support to extend the authenticate manager and allow to use custom authentication through the Set_Default_Factory operation.</li></ul><h4>Ada Servlet, Version 1.4.0</h4><ul><li>Added support to configure the web container</li></ul></div> AKT a tool to store and protect your sensitive informationurn:md5:bab9c41baaba09dff3add71d9f8426672019-12-26T17:47:00+00:002019-12-26T17:47:00+00:00Stephane CarrezSecurityAdaStorage
<div class="post-text"><p><a href="https://github.com/stcarrez/ada-keystore">AKT</a> stores information in secure wallets and protects the stored information by encrypting the content with different keys. AKT can be used to safely store passwords, credentials, bank accounts, documents and even directories.</p><p>Wallets are protected by a master key using AES-256 and the wallet master key is protected by a user password or a user GPG encrypted key. The wallet defines up to 7 slots that identify a password key that is able to unlock the master key. To open a wallet, it is necessary to unlock one of these 7 slots by providing the correct password. Wallet key slots are protected by the user's password and the PBKDF2-HMAC-256 algorithm, a random salt, a random counter and they are encrypted using AES-256.</p><p><img src="/vacs/blogs/images/2401/701/default/akt-overview.png" width="799" height="399" alt="C"></img></p><p>Values stored in the wallet are protected by their own encryption keys using AES-256. A wallet can contain another wallet which is then protected by its own encryption keys and passwords (with 7 independent slots). Because the child wallet has its own master key, it is necessary to known the primary password and the child password to unlock the parent wallet first and then the child wallet.</p><p>The data is organized in 4K blocks whose primary content is encrypted either by the wallet master key or by the entry keys. The data block is signed by using HMAC-256. A data block can contain several values but each of them is protected by its own encryption key. Each value is also signed using HMAC-256.</p><p>The keystore uses several encryption keys at different levels to protect the content. A document stored in the keystore is split in data fragment and each data fragment is encrypted by using its own key. The data fragments are stored in specific data blocks so that they are physically separated from the encryption keys.</p><p>The data fragment encryption keys are stored in the directory blocks and they are encrypted by using a specific directory key.</p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/akt-keys.png" longdesc="AKT Keys" alt="akt-keys.png"></img></div></div></p><p>For example, a 10K document will be split in 3 data fragments, each of them encrypted by their own AES-256 key. A 5K document will be encrypted with two AES-256 keys, one for the two data fragments. All these keys are protected by the wallet data key key. The directory part of the wallet which describes entries in the wallet is also encrypted by another wallet key: the directory key.</p><p>The tool allows to separate the data blocks which contain data fragments from other blocks. This allows to keep the wallet keys separate from the data. It is then possible to export the data blocks, which are encrypted in AES-256-CBC, to the Cloud without exposing the keys used for encryption.</p><p>If you want to know more about the implementation, have a look at the <a href="https://ada-keystore.readthedocs.io/en/latest/Keystore_Design/">Ada Keystore Implementation</a> chapter.</p><h3>Using AKT</h3><p><code>akt</code> is the command line tool that you can use to protect and store your documents. It contains several commands:</p><ul><li><code>create</code>: create the keystore</li><li><code>edit</code>: edit the value with an external editor</li><li><code>get</code>: get a value from the keystore</li><li><code>help</code>: print some help</li><li><code>list</code>: list values of the keystore</li><li><code>remove</code>: remove values from the keystore</li><li><code>set</code>: insert or update a value in the keystore</li></ul><p>To create the secure file, use the following command and enter your secure password (it is recommended to use a long and complex password):</p><pre><code> akt create secure.akt
</code></pre><p>You may also protect the keystore by using your GPG key. In that case, you can use the <code>--gpg</code> option and specify one or several GPG key ids. Using GPG is probably the best method to protect your <code>akt</code> files.</p><pre><code> akt create secure.akt --gpg 0xFC15CA870BE470F9
</code></pre><p>At this step, the secure file is created and it can only be opened by providing the password you entered. To add something, use:</p><pre><code> akt set secure.akt bank.password 012345
</code></pre><p>To store a file, use the following command:</p><pre><code> akt store secure.akt contract.doc
</code></pre><p>If you want to retrieve a value, you can use one of:</p><pre><code> akt get secure.akt bank.password
akt extract secure.akt contract.doc
</code></pre><p>You can also use the <code>akt</code> command together with the <code>tar</code> command to create secure backups. You can create the compressed tar file, pipe the result to the <code>akt</code> command to store the content in the wallet.</p><pre><code> tar czf - dir-to-backup | akt store secure.akt -- backup.tar.gz
</code></pre><p>To extract the backup you can use the <code>extract</code> command and feed the result to the <code>tar</code> command as follows:</p><pre><code> akt extract secure.akt -- backup.tar.gz | tar xzf -
</code></pre><h3>Using Ada Keystore</h3><p>The <a href="https://github.com/stcarrez/ada-keystore">Ada Keystore</a> is the Ada 2012 library that is behind AKT. It should be quite easy to integrate the library in an existing Ada application to protect for example some sensitive configuration file. The <code>Keystore</code> is the main package that provides operations to store information in secure wallets and protect the stored information by encrypting the content. To use it, add the following <code>with</code> clause at beginning of your GNAT project:</p><pre><code class="lang-ada"> with "keystoreada";
</code></pre><h5>Creation</h5><p>To create a keystore you will first declare a <code>Wallet_File</code> instance. You will also need a password that will be used to protect the wallet master key.</p><pre><code class="lang-ada">with Keystore.Files;
...
WS : Keystore.Files.Wallet_File;
Pass : Keystore.Secret := Keystore.Create ("There was no choice but to be pioneers");
</code></pre><p>You can then create the keystore file by using the <code>Create</code> operation:</p><pre><code class="lang-ada"> WS.Create ("secure.akt", Pass);
</code></pre><h5>Storing</h5><p>Values stored in the wallet are protected by their own encryption keys using AES-256. The encryption key is generated when the value is added to the wallet by using the <code>Add</code> operation.</p><pre><code class="lang-ada"> WS.Add ("Grace Hopper", "If it's a good idea, go ahead and do it.");
</code></pre><p>The <code>Get</code> function allows to retrieve the value. The value is decrypted only when the <code>Get</code> operation is called.</p><pre><code class="lang-ada"> Citation : constant String := WS.Get ("Grace Hopper");
</code></pre><p>The <code>Delete</code> procedure can be used to remove the value. When the value is removed, the encryption key and the data are erased.</p><pre><code class="lang-ada"> WS.Delete ("Grace Hopper");
</code></pre><h3>Getting AKT</h3><p>You can get AKT by using the Ubuntu 18.04 binary packages. You can do this by running:</p><pre><code>wget -O - http://apt.vacs.fr/apt.vacs.fr.gpg.key | sudo apt-key add -
sudo add-apt-repository "deb http://apt.vacs.fr/ubuntu-bionic bionic main"
sudo apt-get install akt
</code></pre><p>For other platforms, you have to get it from the sources. Install the GNAT Ada compiler, either the FSF version or the GNAT GPL version and then, run the following commands:</p><pre><code>git clone --recursive https://github.com/stcarrez/ada-keystore.git
cd ada-keystore
./configure --disable-nls
make build install
</code></pre><p>You can browse the documentation online: <a href="https://ada-keystore.readthedocs.io/en/latest/">Ada Keystore Guide</a>.</p></div> Ada, Java and Python database accessurn:md5:261a4c0a5451f48410ed3bd704b3c9c02018-11-17T14:02:00+00:002018-11-17T14:02:00+00:00Stephane CarrezAdaperformanceJavaPythonMysqlSQLitePostgreSQL
<div class="post-text"><p>The database also has a serious impact on such benchmark and I've measured the following three famous databases:</p><ul><li><a href="https://www.sqlite.org/">SQLite</a></li><li><a href="https://mariadb.org/">MariaDB</a></li><li><a href="https://www.postgresql.org/">PostgreSQL</a></li></ul><p>The purpose of the benchmark is to be able to have a simple comparison between these different databases and different programming languages. For this, a very simple database table is created with only two integer columns one of them being the primary key with auto increment. For example the <a href="https://www.sqlite.org/">SQLite</a> table is created with the following SQL:</p><pre><code>CREATE table test_simple (
id INTEGER PRIMARY KEY AUTOINCREMENT,
value INTEGER
)
</code></pre><p>The database table is filled with a simple <code>INSERT</code> statement which is also benchmarked. The goal is not to demonstrate and show the faster insert method, nor the faster query for a given database or language.</p><h3>Benchmark</h3><p>The SQL benchmarks are simple and they are implemented in the same way for each language so that we can get a rough comparison between languages for a given database. The <code>SELECT</code> query retrieves all the database table rows but it includes a <code>LIMIT</code> to restrict the number of rows returned. The query is executed with different values for the limit so that a simple graph can be drawn. For each database, the SQL query looks like:</p><pre><code>SELECT * FROM test_simple LIMIT 10
</code></pre><p>The SQL statements are executed 10000 times for <code>SELECT</code> queries, 1000 times for <code>INSERT</code> and 100 times for <code>DROP</code>/<code>CREATE</code> statements.</p><p>Each SQL benchmark program generates an XML file that contains the results as well as resource statistics taken from the <code>/proc/self/stat</code> file. An Ada tool is provided to gather the results, prepare the data for plotting and produce an Excel file with the results.</p><h5>Python code</h5><pre><code>
def execute(self):
self.sql = "SELECT * FROM test_simple LIMIT " + str(self.expect_count)
repeat = self.repeat()
db = self.connection()
stmt = db.cursor()
for i in range(0, repeat):
stmt.execute(self.sql)
row_count = 0
for row in stmt:
row_count = row_count + 1
if row_count != self.expect_count:
raise Exception('Invalid result count:' + str(row_count))
stmt.close()
</code></pre><h5>Java code</h5><pre><code>public void execute() throws SQLException {
PreparedStatement stmt
= mConnection.prepareStatement("SELECT * FROM test_simple LIMIT " + mExpectCount);
for (int i = 0; i < mRepeat; i++) {
if (stmt.execute()) {
ResultSet rs = stmt.getResultSet();
int count = 0;
while (rs.next()) {
count++;
}
rs.close();
if (count != mExpectCount) {
throw new SQLException("Invalid result count: " + count);
}
} else {
throw new SQLException("No result");
}
}
stmt.close();
}
</code></pre><h5>Ada code</h5><pre><code>procedure Select_Table_N (Context : in out Context_Type) is
DB : constant ADO.Sessions.Master_Session := Context.Get_Session;
Count : Natural;
Stmt : ADO.Statements.Query_Statement
:= DB.Create_Statement ("SELECT * FROM test_simple LIMIT " & Positive'Image (LIMIT));
begin
for I in 1 .. Context.Repeat loop
Stmt.Execute;
Count := 0;
while Stmt.Has_Elements loop
Count := Count + 1;
Stmt.Next;
end loop;
if Count /= LIMIT then
raise Benchmark_Error with "Invalid result count:" & Natural'Image (Count);
end if;
end loop;
end Select_Table_N;
</code></pre><p>The benchmark were executed on an Intel i7-3770S CPU @3.10Ghz with 8-cores running Ubuntu 16.04 64-bits. The following database versions are used:</p><ul><li>MariaDB 10.0.36</li><li>PostgreSQL 9.5.14</li></ul><h3>Resource usage comparison</h3><p>The first point to note is the fact that both Python and Ada require only one thread to run the SQL benchmark. On its side, the Java VM and database drivers need 20 threads to run.</p><p>The second point is not surprising: Java needs 1000% more memory than Ada and Python uses 59% more memory than Ada. What is measured is the the VM RSS size which means this is really the memory that is physically mapped at a given time.</p><p>The SQLite database requires less resource than others. The result below don't take into account the resource used by the MariaDB and PostgreSQL servers. At that time, the MariaDB server was using 125Mb and the PostgreSQL server was using 31Mb.</p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/sql-memory.png" longdesc="Memory usage" alt="sql-memory.png"></img></div></div></p><h3>Speed comparison</h3><p>Looking at the CPU time used to run the benchmark, Ada appears as a clear winner. The Java PostgreSQL driver appears to be very slow at connecting and disconnecting to the database, and this is the main reason why it is slower than others.</p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/sql-time.png" longdesc="CPU time" alt="sql-time.png"></img></div></div></p><p>It is interesting to note however that both Java and Python provide very good performance results with SQLite database when the number of rows returned by the query is less than 100. With more than 500 rows, Ada becomes faster than others.</p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/sql-sqlite.png" longdesc="SQLite benchmark" alt="sql-sqlite.png"></img></div></div></p><p>With a PostgreSQL database, Ada is always faster even with small result sets.</p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/sql-postgresql.png" longdesc="PostgreSQL benchmark" alt="sql-postgresql.png"></img></div></div></p><p><div class="wiki-img-center"><div class="wiki-img-inner"><img src="/images/Ada/sql-mysql.png" longdesc="MySQL benchmark" alt="sql-mysql.png"></img></div></div></p><h3>Conclusion and references</h3><p>SQLite as an embedded database is used on more than 1 billion of devices as it is included in all smartphones (Android, iOS). It provides very good performances for small databases.</p><p>With client-server model, MariaDB and PostgreSQL are suffering a little when compared to SQLite.</p><p>For bigger databases, Ada provides the best performance and furthermore it appears to be more predictable that other languages (ie, linear curves).</p><p>The Excel result file is available in: <a href="https://github.com/stcarrez/sql-benchmark/raw/master/sql-benchmark-results.xls">sql-benchmark-results.xls</a></p><p>Sources of the benchmarks are available in the following GitHub repository:</p><ul><li><a href="https://github.com/stcarrez/sql-benchmark">https://github.com/stcarrez/sql-benchmark</a></li></ul></div>