3.       SAFAR

3.1         General principles

1.1 

2.1 

In most cases, the development of Arabic NLP applications requires the use of several tools at once, each dealing with a certain level of language. For example, to develop an automatic translator, one approach consists of using a parser as well as a morphological analyzer. Generally, these tools are heterogeneous and raise many SE problems such as interoperability, reusability, portability, etc. Therefore, the ANLP application developer must very often face problems of integration of different technologies, a more difficult maintenance of the system, a larger number of codes or a tedious search of the most appropriate tools. Moreover, researchers are usually in need not only to tools but also to Language Resources (LRs). This complicates the situation even more because researchers will be confronted once again with the same problem regarding using and processing resources.

To overcome the above-mentioned SE issues and to suit the needs of the ANLP community in terms of processing Arabic effectively and providing reusable LRs, we developed SAFAR as a software architecture for Arabic^[1].

As will be detailed in the benchmarking section, SAFAR outperforms existing architectures as one that has been built with the following constraints and principles:

·  Integrate           t only tools and programs but also LRs

·  Structure the architecture to integrate the three types of Arabic, namely MSA, Classical and dialects

·  Respect the Arabic language features in the structure of the architecture

·  Develop tools or LRs when available ones are not satisfactory

·  Provide the architecture to be exploited not only by computer scientists but also by linguists

·  Involve in our team computer scientists, statisticians and linguists

SAFAR is a Java-based framework dedicated to Arabic Natural Language Processing. It brings together all layers of ANLP: LRs, pre-processing, morphology, syntax, semantics and applications. In general, our philosophy is not to develop ourselves all the NLP layers and modules, but to integrate existing ones consistently. Consequently, our approach consists in providing the specification in terms of APIs for each module of our architecture and also provide (if any) implementations of these APIs with tools that have proved to be efficient. However, when modules and LRs are not available, we develop them from scratch inside SAFAR.

3.2         SAFAR architecture

As shown in Figure 1, SAFAR has several layers. Each one is developed as a set of reusable Java APIs that provide services directly usable by other layers in accordance with the relationships modeled with arrows in the figure. SAFAR layers consist of models and interfaces that help standardizing the various aspects shared by tools belonging to the same family.

Figure 1: SAFAR framework general architecture.

·  Morphology: deals with four types of tools namely, stemmers, lemmatizers, morphological analyzers, and generators;

·  Syntax: contains implementations of Arabic syntactic parsers;

·  Semantics: designed to implement tools dealing with semantics;

·  Utilities: includes a set of technical services and pre-processing tools as well as machine and deep learning utilities;

·  Resources: provides services for maintaining, consulting and managing Arabic language resources such as corpora, dictionaries and ontologies;

·  Application: contains high-level applications such as sentiment analysis or Question/Answering systems;

·  Client applications: interacts with all other layers to serve clients via web applications, web services, etc.

3.3         The content

As previously explained, the structure of SAFAR is split in three main packages: MSA, Classical and Dialects. Since Dialects are numerous and exceeds largely the number of Arabic countries, we have been interested so far to integrate only the Moroccan dialect. In addition, each package is split into basic services (split into the tree most known language levels: morphology, syntax and semantics), applications and LRs (split into corpora and lexicon).

3.3.1        MSA

This package is the most populated one. Indeed, for almost two decades the research community spent all their efforts in developing components (tools and resources) for this type of Arabic. The work within SAFAR is either to integrate one or multiple of these components or develop them from scratch.

Once developed/integrated, all SAFAR components become standardized according to several Java interfaces. These interfaces provide a variety of inputs and outputs.

Users have several possibilities when calling methods by specifying appropriate parameters according to their needs. They could also easily create customizable pipelines where the output of one component is the input of another (Jaafar         and Bouzoubaa, 2015). Moreover, other developers can add new implementations of any family components simply by implementing the appropriate interface. All these aspects of SAFAR help solving SE issues especially the interoperability, the reuse and the flexibility of exploitation.

Table 1: MSA tools implemented in SAFAR

Table 1 shows all integrated tools for MSA. We notice that many tools have been integrated within SAFAR such as “BAMA” morphological analyzer, “Stanford Parser”, “Light10” stemmer, etc. These tools have been widely used by the ANLP community and it will be very advantageous to use them within a homogenous and flexible framework. Other tools have been developed from scratch such as “SAFAR stemmer”, “SAFAR POS tagger”, etc. We developed them because: 1) available tools return incorrect results, 2) there are no similar tools within the community, or 3) existing tools cannot be reused in several contexts and environments. To prove this, we indeed made three benchmarking of all similar tools at the stemming (Jaafar and Bouzoubaa, 2016), morphological analysing (Jaafar and Bouzoubaa, 2014) and parsing (Jaafar and Bouzoubaa, 2017) layers. All tools starting with “SAFAR” in the table have been developed from scratch by our research team, while all others have been integrated.

It should be noted that some processing levels have more tools compared to others. For example, the morphological level has many tools compared to the syntactic one. This can be justified by the lack of those types of tools. Therefore, we highly encourage researches to provide tools in such levels, especially in the semantics level.

The column “Per” indicates how many researcher persons have been involved in the development/integration of the corresponding tool. The “Vr” column indicates SAFAR version from which the tool is present.

On another hand, Table 2 shows all integrated resources for MSA. The LRs building process is based on the Arabic language structure. Arabic is a highly structured language respecting a concatenative morphology. The concatenative inflexion denotes that the lemma concatenates to affixes to produce the stem, which in turn concatenates to clitics to yield the word. And according to their features, a lemma is either a verb, a noun or a particle. From this, we identify the basic components taking part in the composition of the Arabic words which are the lemmas (particle, verb and noun), stems and clitics.

Table 2: MSA resources implemented in SAFAR

Thus, SAFAR follows the above Arabic language structure and contains the three basic alphabets (Loukili and Bouzoubaa 2011, Namly et al. 2016), clitics (Namly et al. 2015) and particles lexicon. We also make use of existing and known dictionaries (Contemporary and Interactive). It is worth mentioning that SAFAR contains currently one of the most comprehensive lexicons with more than 7 millions stems and corresponding lemmas (Namly et al. 2019).

Table 3: Moroccan dialect resources and tools implemented in SAFAR