A design of human-like robust AI machines in object identification
AA design of human-like robust AI machines in objectidentification
Bao-Gang Hu and Wei-Ming Dong
1, 2 NLPR, Institute of Automation, Chinese Academy of Science, 100091, China University of Chinese Academy of Sciences, Beijing, 100091, China * [email protected] ABSTRACT
This is a perspective paper inspired from the study of Turing Test proposed by A.M. Turing (23 June 1912 - 7 June 1954) in1950. Following one important implication of Turing Test for enabling a machine with a human-like behavior or performance,we define human-like robustness (HLR) for AI machines. The objective of the new definition aims to enforce AI machines withHLR, including to evaluate them in terms of HLR. A specific task is discussed only on object identification, because it is the mostcommon task for every person in daily life. Similar to the perspective, or design, position by Turing, we provide a solution of howto achieve HLR AI machines without constructing them and conducting real experiments. The solution should consists of threeimportant features in the machines. The first feature of HLR machines is to utilize common sense from humans for realizing acausal inference. The second feature is to make a decision from a semantic space for having interpretations to the decision. Thethird feature is to include a “human-in-the-loop” setting for advancing HLR machines. We show an “identification game” usingproposed design of HLR machines. The present paper shows an attempt to learn and explore further from Turing Test towardsthe design of human-like AI machines. Introduction
In recent years, the tremendous success of deep learning (DL) has advanced artificial intelligence (AI) for widerapplications . However, some weaknesses are appeared in applications, such as black box lacking interpretationsabout the outcomes of DL or AI machines . One typical example showing the weaknesses is about identifying apanda animal (Figure 1) . In an image with a panda, a DL machine was able to identify it correctly with 57.7%confidence. However, after adding a certain degree of noise, the machine identified the noised image with a gibbonanimal having 99.3% confidence, and failed to give interpretations for the new decision. If asking any person tosee the noised image, one can still tell the animal correctly. This example shows that the existing AI machinesdo not take human-like knowledge in their identifications. For overcoming this difficulty, we consider to apply theprinciple behind Turing Test in the design of AI machines. Turing Test was proposed based on the question “ Canmachine think ”, and then it was transformed into an evaluation problem for a human evaluator to judge an identityfrom an “ imitation game ”. Hence, Turing Test is actually a design for evaluating a machine in terms of human-likeintelligence. Inspired by this idea, we will go further to propose a design for enforcing a machine with human-likerobustness which is defined in the next section.
Definitions of robustness and two goals
In this paper we take most terms directly in a wider sense. For example, we consider machine to be computer,system, agent, model, etc. Only some core terms are defined, such as.Definition 1:
Robustness is a desirable property of a living or a machine in action correctly under any type ofdisturbances.Definition 2:
Human robustness ( HR ) refers to a class of robustness gained by humans.Definition 3: Machine robustness ( MR ) refers to a class of robustness gained by machines.Definition 4: Human-like robustness ( HLR ) is a class of MR shared in a union with HR.Figure 2 shows the relations among the three specific sets (or classes) in robustness. Within a set of robustness,there still exist more subsets, such as for other living things . When humans shows HR in telling a panda fromthe noised image in Figure 1, a machine can demonstrate its MR in watermark detections from a copyright image.Note that all robustness sets are dynamic in evolutions, and HLR is defined in relation to the two goals of AImachines stated by Hu and Qu :“ Engineering goal: To create and use intelligent tools in helping humans maximumly for good. igure 1.
A demonstration of the “brittleness” of DL machines in object identification . Left: Original imageidentified as panda. Middle: The noise added on the original image. Right: Noised image identified as gibbon. Figure 2.
An Euler diagram of three specific sets in robustness.
Scientific goal: To gain knowledge, better in depth, about humans themselves and other worlds. ”The proposal of HLR shares the same goals above. When the future machines should evolve to include HRas much as possible in their MR, or to increase their HLR, we also emphasize its evolution on “ helping humansmaximumly for good ”, particularly for humanity . Three critical features of HLR machines
This paper follows the similar approach by Turing on proposing an “ initial game ” , which was a design of fulfillingan evaluation task in AI studies. We will propose a design for human-like robust AI machines, or HLR machines.Three critical features are suggested as high-level guidelines in the design of HLR machines below.Feature 1: To utilize common sense from humans for realizing a causal inference.Feature 2: To make a decision from semantic similarity for having interpretations to the decision.Feature 3: To include a “ human-in-the-loop ” setting for advancing HLR machines.For clarifying some core terms used in the features above, we provide their definitions below.Definition 5: Common sense is a class of empirical knowledge for humans used in our daily life.Definition 6:
Causal inference is a class of inference that establishes cause-and-effect relationships.Definition 7:
Semantic similarity is a likeness metric between two semantic objects.Definition 8:
Human-in-the-loop ( HITL ) is a setting in a loop at which a human operator is able to insert apriori into an AI machine and to determine the output of the machine.For justifying the features in HLR machines above, we take Figure 1 as an example for discussions. It is knownthat humans apply common sense in their daily life reasoning . For example, after seeing Figure 1, one mayidentify the object first and then explain it by the following saying.Saying 1: “
I saw two eyes in the image, the form of the eyes is the mostly distinguished characteristics for apanda, then I identify it to be a panda ”.Although the common sense may be varied with different persons, humans do utilize them in their decisionmaking and process them mostly from a causal inference . In Saying 1, a specific cause-and-effect relationship igure 3.
Schematic interpretation about hybrid-augmented intelligence .is based on a set of key words “ eyes, distinguished characteristics, panda ”. Unfortunately, most of the existing DLmachines miss the utilization of common sense and a mechanism for a causal inference in such object identifications.For Feature 2, we stress that the explicit knowledge in human brain is mostly represented by a form of semanticrepresentation . Generally, semantic representations are considered to be a form of high-level knowledge , suchas common sense. Saying 1 implies that humans apply semantic similarity in decision and provide interpretationsabout the decision. Hence, Feature 2 guarantees high-level knowledge with more semantic meanings in the outcomeof HLR machines, rather than the only label information gained from the conventional AI machines.For Feature 3, HITL is necessary for training in HLR machines, but may be not for testing or using. A humanoperator can be either a modeler or a user of machine, even for a group of operators. Two basic tasks are specifiedfor the operator, that is, to provide instruction (including a priori knowledge) into the machine and to decide thefinal output from the AI machine. We use the term setting to reflect a physical place for humans located withinsuch feedback loop. A schematic design of HLR models
The main idea of HLR machines is also inspired by a so called “ hybrid-augmented intelligence (HAI)” proposed byPan and Zheng et al respectively, for integrating both human intelligence (HI) and AI. Figure 3 illustrates animportant working format of advancing intelligent level of HAI machines. We present the following mathematicdescription about HAI in a dynamic form: HAI t + = HI t ∪ AI t ≥ HAI t , (1)where the subscript t represents the set at time t (= , , , ... ) . If viewing them in terms of robustness sets, we canobserve that the union of HR and MR will be enlarged by their integrations in the iterations of knowledge updating.For realizing HLR to be enlarged at the same time, we propose a schematic design of HLR models in Figure 4. Theterm schematic indicates that the design is still at a very primary level. One can see that HLR models include bothmachine and human operator. We present several remarks below for explanations about the design in the contextof object identification shown in Figure 1.Remark 1: An operator can use common sense, like Saying 1, as a priori to insert AI machine, so that Feature1 will be satisfied.Remark 2: AI machine will be able to transform a priori knowledge from natural language into a structuredrepresentation, generate novel data, extract a posteriori knowledge, and make a decision from semantic similarity,so that Features 1 and 2 will be satisfied.Remark 3: An AI machine output (MO) is a reference for humans which will include both data and extractedknowledge (or a posteriori knowledge).Remark 4: After receiving MO, an human operator will examine or modify it to make a final decision as a finaloutput (FO), so that Features 3 will be satisfied. We stress a final decision from humans for the reason that thegoal “ for good ” will be technically possible for HLR models.Remark 5: HLR models are a special class of HAI machines following an iteration procedure in Equation 1.Therefore, HLR will be increased by adding more knowledge, both implicit and explicit, via iterations. igure 4. Schematic design of HLR models including an AI machine and a human operator. A machine output(MO) is set as a reference and a final output (FO) as a decision of HLR models. Both outputs include data andextracted knowledge (or a posteriori knowledge), but may be different. A setting of “human-in-the-loop” willensure a human operator to control and adjust AI machine via instruction (or a priori ) feedback. (a)
Example 1 (b)
Example 2
Figure 5.
Two image examples in an “identification game”. Human operator can guess two given images (croppedfrom the right image in Figure 1). An “identification game” in using HLR models
Similar to the “ imitation game ” in Turing Test which was related to some imagined dialogues between the two ofthree players, we also show an “ identification game ” in using HLR models. In the proposed “ identification game ”,we suppose AI machine in HLR models to be an advanced machine (M), satisfying Features 1 and 2 for dealing withany type of knowledge, such as abstracting or extracting knowledge. M is capable of fulfilling many tasks basedon the instruction of an human operator (H). We further suppose that, from the request of H, input data (D) issufficiently supplied to M and M is able to preprocess D or generate novel D for M’s using. For example, H caninstruct M to make an image example shown in Figure 5 (a), and can provide the following saying from his/herjudgment.Saying 2: “ I still can identify it to be a panda because I can see one ear and one eye of the panda. ”H will provide such a priori for M to make a training with more data. If for the given example M is able tooutput a correct answer satisfying Feature 2, we can say that M carries HLR in a certain degree.The game can keep going like this, such as on Example 2 in Figure 5. At this stage, H will provide the followingsaying:Saying 3: “
I cannot tell what it is about if I do not know the original image. ”We still can test M for seeing its performance. If M can provide a correct answer as well as a meaningful inter-pretation, we can say that M will have a higher level of HLR over that of the operator in such object identification.The examples show that HLR models are trained at high-level, yet explicit, knowledge on both input and output.The input can be natural language, graph, or sound data, following the similar position of Turing. Note that, afterthe proposal of “ imitation game ”, Turing commented that: “ the question and answer method seems to be suitablefor introducing almost any one of the fields of human endeavour that we wish to include. ” To support this positionfurther, we use two more examples from cartoon graphs in Figure 6. One may call them “happy cat” and “saddog”, respectively. Human identify them not only in terms of animal kinds, but also of their emotional aspects.In using MLR machines, a human operator can input common sense in training the machines for achieving suchhuman identifications in a wider sense. After a training stage, H can still decide an HLR machine working in aform of:FO = MO , and ( A priori ) t + = ( A posteriori ) t , (2) Example 3 (b)
Example 4
Figure 6.
Two cartoon examples in an “identification game”. One may call them “happy cat” and “sad dog”,respectively. Human operator can input common sense in training MLR machines for achieving such humanidentifications.in which the HLR machine will fulfill tasks without any human interaction for real-time applications, such as forcontrolling a fast process. In some situations, the feedback loop is removed in testing and one will get only a staticHLR model without knowledge updating.One may argue that HLR is an operator dependent. If an incorrect piece of common sense is used, its HLRmodels may not show the increasing evolution described by Equation 1. In this study, we suggest readers considerEquation 1 in a statistical sense over human populations. Hence, a correctness of Equation 1 relies on a conditionthat human knowledge is increasingly evolved, which forms a basis of common sense for humans.
Final remarks
In this paper, we propose a design of HLR machines in the context of object identification, which is much inspiredby Turing Test proposed 70 years ago. When Turing aimed at the evaluating the machines in terms of intelligence,we advance such idea on enabling machines in terms of human-like robustness. We wish the design will provide atechnical solution for reaching the two goals of AI machines. We summarize two detailed contributions as well astwo limitations below about the design.Contribution 1: We define the three classes of robustness for distinguishing them in both linguistic and mathe-matic senses. The novel definitions will help us in building HLR AI machines, which is significantly different withthe existing solutions from the low-level robustness in most studies.Contribution 2: The design of HLR AI machines advances the idea of Turing Test by including both enablingand evaluating intelligence over machines in terms of HLR. We propose the three features as the design guidelinesfor realizing HLR AI machines so that the idea of HAI is advanced further.Limitation 1: The design is very primary and staying at the high-level aspects of the models. Neither methods noralgorithms are given about the implementations of the models. We suggest readers refer to the related papers, suchas the methods in common sense representations , semantic similarity (or distance) , and assessmentof robustness .Limitation 2: The intelligence in the design is limited only within a concept of robustness. The example is givenonly on a visual robustness. The human robustness covers a large spectrum in applications, such as, perception,control, decision making, etc.More issues exists about HLR AI machines, such as “ to originate anything ” for such kind of machines. Ourposition is to learn from Turing by proposing a novel idea first, and putting implementations or related issues infuture studies. We hope that the proposal of the design is able to bring a new study space for us to explore.In a final remark, when we express the deep respect to Alan Turing for establishing a link of humans andmachines in terms of intelligence, we stress the term “ HUMANITY ” in designs and applications of human-like AImachines at the same time, and express the great respect to Confucius ( 孔子 ) (551 –
479 BC) in proposing humanitygoal ( 仁者愛人 , ren or humaneness) for humans before two thousand years ago which we can never ignore inour future development. (Dated: December 16, 2020) eferences Turing, A. M. Computing machinery and intelligence-am turing.
Mind , 433–460 (1950). LeCun, Y., Bengio, Y. & Hinton, G. Deep learning.
Nature , 436–444 (2015). Schmidhuber, J. Deep learning in neural networks: An overview.
Neural Networks , 85–117 (2015). Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretablemodels instead.
Nat. Mach. Intell. , 206–215 (2019). Schramowski, P. et al.
Making deep neural networks right for the right scientific reasons by interacting withtheir explanations.
Nat. Mach. Intell. , 476–486 (2020). Goodfellow, I. J., Shlens, J. & Szegedy, C. Explaining and harnessing adversarial examples. arXiv preprintarXiv:1412.6572 (2014). Hernández-Orallo, J. & Dowe, D. L. Measuring universal intelligence: Towards an anytime intelligence test.
Artif. Intell. , 1508–1539 (2010). Zeng, W. & Liu, B. A statistical watermark detection technique without using original images for resolvingrightful ownerships of digital images.
IEEE Transactions on Image Process. , 1534–1548 (1999). Hu, B.-G. & Qu, H.-B. Generalized constraints as a new mathematical problem in artificial intelligence: Areview and perspective. arXiv preprint arXiv:2011.06156 (2020).
Tu, W.
Humanity and self-cultivation: Essays in Confucian thought (Cheng & Tsui, 1998).
Wang, W. & Siau, K. Artificial intelligence, machine learning, automation, robotics, future of work and futureof humanity: a review and research agenda.
J. Database Manag. (JDM) , 61–79 (2019). Minsky, M.
The emotion machine: Commonsense thinking, artificial intelligence, and the future of the humanmind (Simon and Schuster, 2007).
McCarthy, J. Applications of circumscription to formalizing common-sense knowledge.
Artif. intelligence ,89–116 (1986). Mueller, E. T.
Commonsense reasoning: an event calculus based approach (Morgan Kaufmann, 2014).
Rubin, D. B. Comment: Neyman (1923) and causal inference in experiments and observational studies.
Stat.Sci. , 472–480 (1990). Pearl, J.
Causality (Cambridge university press, 2009).
Shams, L. & Beierholm, U. R. Causal inference in perception.
Trends Cogn. Sci. , 425–432 (2010). Patterson, K., Nestor, P. J. & Rogers, T. T. Where do you know what you know? the representation ofsemantic knowledge in the human brain.
Nat. reviews neuroscience , 976–987 (2007). Suchanek, F. M., Kasneci, G. & Weikum, G. Yago: a core of semantic knowledge. In
Proceedings of the 16thInternational Conference on World Wide Web , 697–706 (2007).
Pan, Y. Heading toward artificial intelligence 2.0.
Engineering , 409–413 (2016). Zheng, N.-N. et al.
Hybrid-augmented intelligence: collaboration and cognition.
Front. Inf. Technol. & Electron.Eng. , 153–179 (2017). Liu, H. & Singh, P. Conceptneta practical commonsense reasoning tool-kit.
BT Technol. J. , 211–226(2004). Davis, E. & Marcus, G. Commonsense reasoning and commonsense knowledge in artificial intelligence.
Com-mun. ACM , 92–103 (2015). Budanitsky, A. & Hirst, G. Semantic distance in wordnet: An experimental, application-oriented evaluation offive measures.
Lord, P. W., Stevens, R. D., Brass, A. & Goble, C. A. Investigating semantic similarity measures across thegene ontology: the relationship between sequence and annotation.
Bioinformatics , 1275–1283 (2003). Mikolov, T., Chen, K., Corrado, G. & Dean, J. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).
Baker, J. W., Schubert, M. & Faber, M. H. On the assessment of robustness.
Struct. Saf. , 253–267 (2008). Ackerly, B. A. Is liberalism the only way toward democracy? confucianism and democracy.
Polit. Theory ,547–576 (2005). Hu, B.-G. The greatest contribution of confucius: Proposal of humanity goal for human development(in chinese).
Newsletters Critical Think. Innov. Educ.