Evolution of artificial intelligence languages, a systematic literature review
Emmanuel Adetiba, Temitope John, Adekunle Akinrinmade, Funmilayo Moninuola, Oladipupo Akintade, Joke Badejo
EEvolution of artificial intelligence languages – a systematic literature review
Emmanuel Adetiba , Temitope M. John , Adekunle A. Akinrinmade , Funmilayo S. Moninuola , Oladipupo O. Akintade , Joke A. Badejo Department of Electrical and Information Engineering, College of Engineering, Covenant University, Ota, Nigeria. HRA, Institute for Systems Science, Durban University of Technology, P.O. Box 1334, Durban, South Africa. Corresponding email: [email protected] Abstract
The field of Artificial Intelligence (AI) has undoubtedly received significant attention in recent years. AI is being adopted to provide solutions to problems in fields such as medicine, engineering, education, government and several other domains. In order to analyze the state of the art of research in the field of AI, we present a systematic literature review focusing on the Evolution of AI programming languages. We followed the systematic literature review method by searching relevant databases like SCOPUS, IEEE Xplore and Google Scholar. EndNote reference manager was used to catalog the relevant extracted papers. Our search returned a total of 6565 documents, whereof 69 studies were retained. Of the 69 retained studies, 15 documents discussed LISP programming language, another 34 discussed PROLOG programming language, the remaining 20 documents were spread between Logic and Object Oriented Programming (LOOP), ARCHLOG, Epistemic Ontology Language with Constraints (EOLC), Python, C++, ADA and JAVA programming languages. This review provides information on the year of implementation, development team, capabilities, limitations and applications of each of the AI programming languages discussed. The information in this review could guide practitioners and researchers in AI to make the right choice of languages to implement their novel AI methods. Keywords Artificial Intelligence; Programming Language; Python; AI; LISP; PROLOG; JAVA; C++; EOLC; ADA 1.
Introduction Artificial intelligence (AI) is concerned with intelligent behaviors in artifacts such as perception, reasoning, learning, communicating and acting in complex environment. AI is concerned about machines possessing the listed characteristics as well as humans can, or even better and faster. The physical symbol system hypothesis states that a physical symbol system has the necessary and sufficient means for general intelligent action. A physical symbol system is a machine like a digital computer that is capable of manipulating symbolic data- adding numbers, rearranging lists of symbols and replacing some symbols by others . Traditionally, computing is used for performing mechanical computations using fixed procedures. This approach implies complex problems would be more difficult to solve. Another shortcoming is that omputers so programmed would have difficulties understanding and adapting to new situations like human’s do. AI is different from this traditional approach in that it requires machines to think and tackle such complex assignments. AI was formally coined by John McCarthy in a workshop conducted by IBM at Dartmouth College in 1956 . When digital computers were first developed in the 1940s and 1950s researchers wrote a number of programs, these programs could play chess, checkers and prove theorems. In the 1960s and 1970s, AI explored various ways to represent problems by developing different search techniques and general heuristics, these enabled development of programs used to solve algebraic word problems and symbolic integration. In the 1970s and 1980s, as a result of more powerful systems, AI programs were used to build expert systems and by 1997 an IBM program named DEEP BLUE defeated the world chess champion, Garry Kasparov. Interest in AI sagged in the late 1950s, it, however, resumed with vigor in the 1980s. Networks of nonlinear elements with adjustable-strength interconnections are now recognized as an important class of non-linear modeling tools. There are now several important applications of AI . AI programming languages are languages capable of implementation of logic , vastly useful for building expert systems and with features for handling relational databases and natural language processing . There are many AI programming languages amongst which are List processor (LISP), French for programming in logic (PROLOG), Python, JAVA, C++ and ADA. LISP is a computer programming language with a long history and a distinctive, fully parenthesized prefix notation. First conceived in 1958. It became the programming language of choice for AI research. It was the basis for many ideas in computer science, including tree data structures, automatic storage management, dynamic typing, conditionals, higher-order functions, recursion, the self-hosting compiler, and the read–eval–print loop . Linked lists are one of LISP's major data structures, and its LISP source code is made of lists. Thus, LISP programs can manipulate source code as a data structure, giving rise to the macro systems that allow programmers to create new syntax or new domain-specific languages embedded in LISP. The idea of PROLOG was first conceived in 1970 and thereafter implemented in 1972 by a group of AI scientists, namely, Alain Colmerauer , Robert Kowalski and Philippe Roussel. It was one of the most popular programming languages since its inception and remains the most popular with a lot of its variants . Prolog is widely used for programming in AI being a general-purpose logic programming language . It differs from other programming languages in that it is intended mainly as a declarative programming language. It has its roots in first-order logic expressed in terms of relations, represented as facts and rules whilst a computation is initiated by running queries over these relations . The language is well suited for different branches of AI, for example logical problems that are randomly selected , expert systems and natural language processing . Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability. It was developed by Guido Van Rossum in the early 1990s . Python's syntax allows programmers to express concepts in fewer lines of code than would be possible in languages such as C. . Python is well adapted for AI tasks , especially in the area of natural language processing. It was conceived towards the end of 1980 but its implementation began in December 1989 as a successor to the ABC language capable of exception handling and interfacing with the Amoeba operating system . By October 2000, Python 2.0 was released with new features which included a full garbage collector and support for Unicode , versions 2.6 and 2.7 followed while Python 3.0 was released on December 3, 2008 . A number of Artificial Intelligence libraries run primarily on the Python Infrastructure; Keras, noted for its user riendliness, modularity and easy extensibility, is a high level neural network API compatible with Python 2.7-3.6 ; Theano allows for the evaluation of mathematical expressions using multi-dimensional arrays efficiently etc. Java was developed in early 1990s by James Gosling from Sun Microsystems . Some of the features of Java is that it can easily be coded, it is highly scalable, making it desirable for AI projects . It is also portable, and can easily be implemented on different platforms since it uses virtual machine technology . C++ was developed in 1979 by Bjarne Stroustrup at Bells Labs and standardized in 1998 as C language extension. Of all the AI programming language, it is the fastest and mostly used by developers in AI projects because of its time-sensitivity and also when speed is of higher priority to improve their project execution time . We reviewed publicly available documents on AI programming languages from SCOPUS, IEEE Xplore and Google scholar. The documents were reviewed to retrieve the year of execution, development team, capabilities and features as well as the limitations and applications of these AI programming languages. 2. Methods In order to provide a systematic review of AI programming languages, we followed the guidelines put forward by Kitchenham . The goal of this systematic review is to access original articles and full length review articles that relates to AI programming languages. The process is detailed below: 2.1 Research Questions The research questions address forthwith are: RQ 1: What is the prevalence of AI programming language publications since 1963? RQ 2: Which of the AI programming languages have received more attention with respect to the volume of research publication been produced? RQ 3: What are the predominant AI programming languages on which recent AI softwares rely? RQ 4: What are the characteristics of AI programming languages which makes them suitable or unsuitable for use across platforms? In addressing RQ 1, we considered the volume of the publications published for each AI programming languages since 1963 each of these programming languages has been described in Table 2 - 11. With respect to RQ 2, we discussed the major AI programming languages; LISP, Prolog, LOOP, ARCHLOG, EOLC, Phyton, C++, ADA and JAVA. We considered the popularity of the languages with respect to the number of publications that have been produced from 1963 to 10 February 2018. To answer RQ 3; we analyze a few recently developed AI software platforms and determined the foundation AI architecture which oversees the operations of these new software. With respect to the characteristics of AI programming languages (RQ 4), we considered the following; capabilities or features, limitations and application of the AI programming language. .2 Search Strategy The search for relevant literature was conducted on the 10 February 2018 using SCOPUS and IEEE Xplore databases, the dates was set to 1963 to 2018. Furthermore, we extracted more related documents from Google Scholar and reference list. Table 1 shows the search terms used for the databases. Table 1. Search terms on Scopus and IEEE Explore (1963 to 2018)
Case Definitions For this research, we signposted our definition of AI based on the definition put forward by Nils Nilsson in his book, “Artificial intelligence is that activity devoted to making machines intelligent, and intelligence is that quality that enables an entity to function appropriately and with foresight in its environment”. This definition would guide our final selection of retained studies in subsequent sections. 2.5 Quality Criteria We ensured that for each of the full text accessed, the AI programming language was explicitly defined and meet the case definition as stated in 2.3. Studies with ambiguous details of the features or capabilities, limitations and applications of AI were excluded from this review. 2.6
Data Extraction Data extraction was conducted by the following reviewers; TJ, AA, FA and OA. The selection of relevant articles was performed by all reviewers. Any disagreement between reviewers on our selection of studies was resolved by the fifth and sixth reviewer EA and JB. From each study, we extracted data on programming language, version, year of execution, development team, capabilities or features, limitations and applications. This paper seeks to present the evolution of AI programming languages, as such, the retained studies are presented in a chronological order and grouped based on the type of the AI programming language. 3
Results 3.4
Systematic Search Our search returned 6,253 documents that relates to AI programming languages; Scopus (5818) and IEEE Xplore (435). Furthermore, a thorough manual search was performed on Google scholar and returned 312 documents. Hence, a total of 6,565 documents were prepared for analysis. Afterwards, 432 duplicated documents were removed. Of the 6,133 documents remaining, 5,947 documents were excluded based on the following criteria; studies that do not relate to Artificial Intelligence programming languages and studies that are reviews, books, book chapters, report, notes, short survey, letter, viewpoints and editorials. A total of 186 full texts and abstracts were assessed for eligibility, thereafter a total of 69 studies were retained for qualitative and quantitative synthesis. The highlighted search procedure is graphically represented in Figure. 1. igure 1. PRIMA flowchart for selected studies 3.5
Study Characteristics The bulk of the retained studies on Artificial Intelligence programming languages can be grouped under the main categories LISP, PROLOG. Two studies in 1986 and 1987 respectively dealt with different variants of Prolog namely TC-Prolog and FProlog. In recent years, we see documentations on new Artificial Intelligence programing language like Logic and Objected-oriented programming language (LOOP) documented by in 2001 which extended Prolog logic programming language with object oriented features while discuss a connection architecture between Prolog and JAVA, Archlog documented in 2006 which can produce high-performance designs without detailed knowledge of hardware development and a framework for designing multiprocessor architectures; Epistemic Ontology Language with Constraints (EOLC) which is used for specifying the epistemic ontology for heterogeneous verification, documented in 2007 by ; McKinley described Python in his 2006 paper , in 2012,2015,2016 and 2017 respectively discussed the use of C++ in Artificial intelligence programming languages while
28, 42, 44-47 discussed the use of JAVA in Artificial intelligence programming languages.
Successes, Challenges And Limitations 3.6.1
Successes of Artificial Intelligence Optimization of AI language has enabled a revolutionary change across many sectors globally. For instance, AI has been deployed across different fields like finance, health, engineering and education and The proliferation of AI languages across these sectors has helped birth new tech businesses . In Education and Research, AI has brought about significant improvement in the quality of delivery of educational resources across the globe, expert systems have been created to provide seamless learning content to students and researchers across boarders . In the health sector, AI has been used to assist in automated data management , development of artificial neural network to assist in rapid patient care . The advancements in AI has provided further dynamics of analysis of face or object recognition techniques for audiovisual. Also in music evolution in composing human like notes .The invasion of cognitive problem-solving skill has brought innovative ideas that is answering engineering questions . 3.6.2 Challenges and Limitations of Artificial Intelligence The main aim of artificial intelligence is to build an intelligent machine that will make life easier for human beings. The machine should be able to think like humans and some intelligence traits added to it. The programmers want to build some emotional quotient into the machines. Expert development is one of the major problems of AI programming language. LISP; a functional language was created as a mathematical notation for computer programmer was developed for lambda calculus which is not part of undergraduate curriculum in higher institutions. This makes it more tedious for beginners to master LISP compare to other object -oriented language like JAVA. The expert community and library capacity is limited due to this difficulty
49, 53-54 . Since AI involves building a machine that is intelligent like human beings, it must also face some challenges like humans. Identifying some of these challenges will minimize the associated risks and at the same time make sure that we take full advantage of this technology ost researchers believe and agree that a super intelligence AI is unlikely to showcase human emotions like love or hate and that therefore, it cannot become intentionally benevolent or malevolent. However, the most likely scenario where it can pose a threat to the society is via autonomous weapons. These are weapons that AI systems are programmed to use to kill. If in the hands of the wrong person, these weapons could easily cause mass casualties. This could even lead to an AI war that would also result in mass causalities . Legal challenges related to AI's application in the financial industry could be related to the consequences of erroneous algorithms and data governance. Erroneous algorithms, due to the lack of appropriate data, can leave a big dent in the profits of an organization by making incorrect and perhaps detrimental predictions. Poor data governance can result in data breaches where customers' PII (personal identifiable information) that acts as a feedstock to an algorithm may get into the hands of hackers and can cause legal challenges for the organization . able
2. Extracted Studies on LISP Artificial Intelligence programming languages Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications LISP Lisp 1.5 John, McCarthy
Ultra slow numerical computation and lack of better representation of block of registers.
It has a higher overhead when compared with other conventional programming languages. This has narrowed its use in AI. It is limited with respect to processing and memory requirements. Its size, unwieldiness, "kitchen sink" design strategy, and general ADAfication is disliked by most of its critics. Slower compare to C++ Used in differential and integral calculus, electrical circuit theory, mathematical logic, game playing and other fields of artificial intelligence. A vastly useful tool for the programming of many Expert Systems. It is programming language of choice of most AI researchers It has been applied in Musical composition and processing Foundational language for all programming languages. Dixon, Brent W Table
3. Extracted Studies on PROLOG Artificial Intelligence programming language Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications PROLOG K. Clark and S. Gregory Technology Research Centre, Department of Engineering Mathematics, University of Bristol, Bristol, BS8 1TR, United Kingdom knowledge acquisition and action execution. This poses a problem because the agent might work in a dynamic environment where unexpected things can happen. Slower compare to C++ Forth-based Prolog is used as basis of the expert system component of an astronaut interface for a series of Spacelab experiments It is a well adapted technology for expert systems design Used in real-time expert system to develop astronaut interface for a series of Spacelab vestibular experiments. Used in the probing of a facility's defenses and to find potential attack paths that meet designated search criteria. Used in electric utility Odette, L. L. Paloski, W. H. list handling essential in representing tree-based data structures and pattern matching .it can also backtrack automatically robotic domain and biological domain The performance Rate is highly improved when connected to Java Ostermager ś kiewicz, G. PROLOG TC-PROLOG Futo, Ivan Papp, Imre Table
4. Extracted Studies on LOOP Artificial Intelligence programming language Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications LOOP (logic and object-oriented programming language) A. Suciu K. Pusztai T. Muresan Z. Simon Table
5. Extracted Studies on ARCHLOG Artificial Intelligence programming language Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications rchlog Fidjeland, A. Luk, W. Table
6. Extracted Studies on EOLC Artificial Intelligence programming language Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications EOLC (Epistemic Ontology Language with Constraints) R. Kumar B. H. Krogh Table
7. Extracted Studies on PYTHON Artificial Intelligence programming languages Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications YTHON McKinley, K. S. (2016) Table
8. Extracted Studies on C++ Artificial Intelligence programming languages Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications C++ Park, J., et al. (2017) Table
9. Extracted Studies on JAVA Artificial Intelligence programming languages rogramming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications JAVA Ostermayer 2014 Java is portable Implementation on different platforms easy because of virtual machine technology Algorithms coding is very easy Java’s Response time is more and less execution speed this makes it slower than C++ It enables the Automatic Speech Recognition (ASR) systems Improved performance when linked with prolog Babu, A. A., et al. (2015) Kurniawan, A., et al. (2015) Mittal, H. and S. D. Mandalika (2015) McKinley, K. S. (2016) Garg, S. and S. Kumar (2017) Raff, E. (2017). Table
10. Extracted Studies on HASKELL Artificial Intelligence programming languages Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications HASKELL Magalhães, J. P. and A. Löh (2015) Table
11. Extracted Studies on ADA Artificial Intelligence programming languages Programming Language Version Authors Year of execution Development team Capabilities/features Limitations Applications
ADA Hattori,Kushima,Wasano High performance and maintainability, list processing facility
Development of large scale AI software Discussion This study was conducted to give a clear understanding and relevance of AI programming language from conception till date. The study confirms that John McCarthy pioneered the concept of AI which has evolved into the implementation of other high level programming languages such as LISP, Prolog, etc. Our findings suggest that the field of AI; a multi-disciplinary field, can be highly enriched if further research is geared towards the development of more syntax and semantic interaction that could produce a robust language understanding systems. We discuss the answers to our research questions in the following sub-sections. 4.1
What is the Prevalence of AI Programming Language Publications Since 1963? AI programming languages have seen significant research interest since 1963. The research area has seen different peaks and valleys in research outputs. Between 1980 and 1986, there was a steep increase of 8 to 242 documents in SCOPUS, a significant reduction between 1987 and 2003 before increasing to 278 in 2004, between 2005 and 2014, a dip occurred in research output and increased back to 275 in 2015. 4.2
Which of the AI Programming Languages have Received more Attention with Respect to the Volume of Research Publication Being Produced? The volume of research outputs focusing on the PROLOG programming language has been rather significant. From Table 3, 34 research publications were reviewed for Prolog in this systematic review, 15 documents were reviewed for the LISP programming language while the remaining 20 documents were reviewed for Logic and Object oriented programming (LOOP), ARCHLOG, Epistemic Ontology Language with Constraints (EOLC), Phyton, C++, ADA and JAVA programming languages making it a total of 69 reviewed articles. 4.3
What are the Predominant AI Programming Languages on which Recent AI Software’s Rely? A number of AI libraries have been written based on some of the reviewed foundational AI programming languages; some of these programs are listed in Table 12. As illustrated in the Table, C/C++ has enjoyed the highest level of patronage (N=7) by developers of modern AI libraries. This is closely followed by Python (N=6). The reasons for the high adoption of these languages may be attributed to their unique strengths compared with other reviewed languages as earlier illustrated in Tables 7 and 8. Table 12. Details of Modern AI Libraries and their Foundational AI Programming Languages S/N Modern AI Libraries Initial Release data Original Author AI programming language platform 1 ALICE 1995 Joseph Weizenbaum Java 2 OpenNN 2003 International Center for Numerical Methods in Engineering(CIMNE) C++ 3 OpenCog C++, Python TensorFlow 2015 Google Brain Team Phyton, C++, CUDA 5 Siri 2011 Apple Objective C 6 Neural Designer Arteinics C++ 7 Keras 2015 Python 8 Scikit-learn 2007 David Cournapeau Python, Cython, C, C++ 9 Pandas 2017 Wes McKinney Python 10 SciPy 2017 Travis Oliphant, Pearu Peterson, Eric Jones Python Fortran, C, C++ 4.4
What are the Characteristics of AI Programming Languages which make them Suitable or Unsuitable for use across Pplatforms? One of the most desirable qualities of AI programming languages is speed. PROLOG and LISP are slower compared to C/C++ and Python. An additional ability to support object oriented principles is desirable. Sometimes it may be necessary to trade between quick response time and execution speed. Other desirable qualities are portability and ease of coding. The combination of C/C++ with Python in developing most of the trending AI libraries (such as TensorFlow and Keras) as illustrated in Table 12 may be born out of the need to complement the weakness of one language with the strength of the other and vice-versa. ś kiewicz, G., Prolog-Scripted Tactics Negotiation and Coordinated Team Actions for Counter-Strike Game Bots. IEEE Transactions on Computational Intelligence and AI in Games 2016, 8 (1), 82-88. 92. Nickles, M., A tool for probabilistic reasoning based on logic programming and first-order theories under stable model semantics. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016; Vol. 10021 LNAI, pp 369-384. 93. Morozov, A. A.; Sushkova, O. S.; Polupanov, A. F., Towards the distributed logic programming of intelligent visual surveillance applications. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017; Vol. 10062 LNAI, pp 42-53. 94. Zhang, S.; Khandelwal, P.; Stone, P. In Dynamically constructed (PO)MDPs for adaptive robot planning, 31st AAAI Conference on Artificial Intelligence, AAAI 2017, 2017; pp 3855-3862. 95. Postma, G. J.; Vandeginste, B. G. M.; van Halen, C. J. G.; Kateman, G., COMPARISON OF LISP AND PROLOG. TrAC - Trends in Analytical Chemistry 1987, 6 (1), 2-5. 96. Postma, G. J.; Vandeginste, B. G. M.; van Halen, C. J. G.; Kateman, G., Implementation of a teaching program for IR spectrometry in Lisp and Prolog. Trends in Analytical Chemistry 1987, 6 (2), 27-30. 97. Falk, H., AI techniques enter the realm of conventional languages. Electronic Systems Technology and Design/Computer Design's 1988, 27 (19), 45-49. 8. Magalhães, J. P.; Löh, A., Hierarchy in generic programming libraries. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015; Vol. 9129, pp 93-112. 99. Hattori, F.; Kushima, K.; Wasano, T. In comparison OF LISP, PROLOG, AND ADA programming productivity in AI area, Proceedings - International Conference on Software Engineering, 1985; pp 285-291.kiewicz, G., Prolog-Scripted Tactics Negotiation and Coordinated Team Actions for Counter-Strike Game Bots. IEEE Transactions on Computational Intelligence and AI in Games 2016, 8 (1), 82-88. 92. Nickles, M., A tool for probabilistic reasoning based on logic programming and first-order theories under stable model semantics. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016; Vol. 10021 LNAI, pp 369-384. 93. Morozov, A. A.; Sushkova, O. S.; Polupanov, A. F., Towards the distributed logic programming of intelligent visual surveillance applications. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2017; Vol. 10062 LNAI, pp 42-53. 94. Zhang, S.; Khandelwal, P.; Stone, P. In Dynamically constructed (PO)MDPs for adaptive robot planning, 31st AAAI Conference on Artificial Intelligence, AAAI 2017, 2017; pp 3855-3862. 95. Postma, G. J.; Vandeginste, B. G. M.; van Halen, C. J. G.; Kateman, G., COMPARISON OF LISP AND PROLOG. TrAC - Trends in Analytical Chemistry 1987, 6 (1), 2-5. 96. Postma, G. J.; Vandeginste, B. G. M.; van Halen, C. J. G.; Kateman, G., Implementation of a teaching program for IR spectrometry in Lisp and Prolog. Trends in Analytical Chemistry 1987, 6 (2), 27-30. 97. Falk, H., AI techniques enter the realm of conventional languages. Electronic Systems Technology and Design/Computer Design's 1988, 27 (19), 45-49. 8. Magalhães, J. P.; Löh, A., Hierarchy in generic programming libraries. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2015; Vol. 9129, pp 93-112. 99. Hattori, F.; Kushima, K.; Wasano, T. In comparison OF LISP, PROLOG, AND ADA programming productivity in AI area, Proceedings - International Conference on Software Engineering, 1985; pp 285-291.