Exploring generation of a genetic robot's personality ...

Viewer
Transcript

Data & Knowledge Engineering 70 (2011) 923–954

Contents lists available at ScienceDirect

Data & Knowledge Engineering j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / d a t a k

Exploring generation of a genetic robot's personality through neural and evolutionary means Kang-Hee Lee ⁎ Global School of Media, Soongsil University, 511 Sando-dong, Dongjak-gu, Seoul, 156-743, Republic of Korea

a r t i c l e

i n f o

Article history: Received 6 February 2010 Received in revised form 3 June 2011 Accepted 9 June 2011 Available online 25 June 2011 Keywords: Optimization Genetic robot Artiﬁcial creature Software robot Artiﬁcial life Emotional robot Robot genome Robot personality Neural network Evolutionary algorithm Mobile phone

a b s t r a c t This paper proposes a way to generate a robot genome that contributes to defining the personality of a software robot or an artificial life in a mobile phone. The personality should be both complex and feature-rich, but still plausible by human standards for an emotional life form. However, it becomes increasingly difficult and time-consuming to ensure reliability, variability and consistency for the robot's personality while manually initializing values for the individual genes. To overcome this difficulty, this paper proposes a neural network algorithm for a genetic robot's personality (NNGRP) and an upgraded version of a previously introduced evolutionary algorithm for a genetic robot's personality (EAGRP). The robot genomes for heterogeneous personalities are demonstrably generated via the NNGRP and the EAGRP and compared. The implementation is embedded into genetic robots in a mobile phone to verify the feasibility and effectiveness of each algorithm. © 2011 Elsevier B.V. All rights reserved.

1. Introduction Since “The Origin of Species” by Charles Darwin in 1859, the concepts of gene and its evolution have become widely accepted around the world. Richard Dawkins claimed that “We and other animals are machines created by our genes [1]” and also emphasized the importance of genes by comparing the evolutionary design which is “blind” to the purpose of the animal to the design by a designer who speciﬁes the job to be performed [2]. Sloman considered the gene as a very powerful thing with its implicit goals and the way it functions [3]. Similarly, Roll stated that “Genes can specify the goal for action [4].” Thus, the genes are considered as key components in deﬁning a creature's personality and the essence of this research should be on the genome of various types of artiﬁcial creatures such as pet-type, humanoid-type, or head-type, which can be implemented in either a real hardware robot or a simulated software robot (sobot) [5–10]. Considering the user interactions, the personality of the artiﬁcial creatures needs to be taken into consideration, as the personality is crucial in building a believable emotional creature. First, having a diverse personality is important because “Personality is the engine of behavior [11],” “Emotions are the regulators of behaviors [12],” and “Emotions are integral to human perception and strongly connected with goal achieving or failure [13].” It can be encoded as an inherited trait, which decides the behavior based on an internal state in response to the stimulus. It is characterized by the Big Five personality dimensions [14,15]. This allows for the creation of diverse personalities for the agent, e.g., allowing it to express highly agreeable and at the other end of the scale, highly antagonistic characteristics. This approach is also parallel to modeling emotions with multidimensional logic to

⁎ Tel./fax: + 82 2 828 7270. E-mail address: [email protected]. 0169-023X/$ – see front matter © 2011 Elsevier B.V. All rights reserved. doi:10.1016/j.datak.2011.06.002

924

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

solve habitual contradictory-emotions problems [13]. Second, the personality can be universally applied to the diverse architectures of artiﬁcial creatures from basic rule-based reaction-machines (instinct-machines or infant emotions) to complex critic-selector based machines (adult-machines or cascaded emotions) [12]. This paper uses the emotional creature, AnyRobot, as a good testbed. Because it also adopts a hybrid behavior-selection mechanism employing both types of machines. The inherent behavior logic is a kind of rule-based mechanism while the main behavior selection uses the internal state-correlated probabilistic mechanism in Fig. 1. In spite of the importance of innate personality in deciding the behavior, the concretization of diverse personality has not been investigated to any great extent in previous research ventures, compared to that of artiﬁcial intelligence that is accumulated through experience and knowledge as posterior information during its lifetime by means of learning or interaction with human. A general approach to doing this concretization would be to identify generative mechanism that might have the power to compose a variety of particular states and behaviors simply by varying a few parameters [11]. It is necessary in producing a really complex, multifunction design [16] and genetic encoding and algorithm for modularity and reusability can serve as an engine of consistency and coherence economically [3]. Evolutionary techniques such as neural networks or genetic algorithms [17] are being strongly recommended to encode the features (genotype) that would lead to ‘emotional behavior’ (phenotype) and generate different emotional systems with an architecture that was totally hand-coded and therefore very difﬁcult to tune [18]. The reinforcement learning such as Q-learning [19] may improve the hand-coded gene-generation strategies. Newer optimization combined with the fuzzy approximations [20] may boost the gene-generation speed and complement its convergence rate. Besides, evolutionary mechanism is widely used to study how learning and evolution interact, and to model ecosystems, immune system, cognitive systems, and social systems [21]. Motivated by engineered concepts of genes above, this paper proposes a genetic robot deﬁned as an artiﬁcial creature which has its own genome that is composed of multiple artiﬁcial chromosomes, and its gene-generative mechanism. Each chromosome in a robot genome consists of many genes that contribute to deﬁning the robot's personality. They are also crucial factors in determining its internal state and external behaviors at any moment in time. The structure employing the robot genome provides primary advantages for artiﬁcial reproduction, reusability among robots, and the ability to evolve. The large number of genes also allows for a highly complex system; however, it becomes increasingly difﬁcult and time-consuming to ensure reliability, variability and consistency for the robot's personality while manually initializing values for the individual genes. To overcome this difﬁculty, neural network algorithm for a genetic robot's personality (NNGRP) and an upgraded version of evolutionary algorithm for a genetic robot's personality (EAGRP) [17] are proposed, which generate the robot genome as their output. NNGRP and EAGRP are novel algorithms to identify the generative mechanism that characterizes a variety of internal states and their concomitant behaviors by simply varying a few parameters through a graphical user interface (GUI). They evolve a gene pool that customizes the robot's genome so that it closely matches a simpliﬁed set of features desired by the user. They do this using several new techniques. They act on a two-dimensional individual upon which a masking method is used to derive a plausible individual given the restricted preference settings desired by the user. As the reproduction scheme of two-dimensional genomes, NNGRP uses the neural network training algorithm while EAGRP uses the evolutionary algorithm. Finally, the evaluation procedures for individuals are carried out in a virtual environment using tailored perception scenarios. The genetic robot is validated by implanting the robot genomes generated by NNGRP and EAGRP into the internal state module of a virtual creature. As

(a) Pet-type and robot-type

(b) Internal architecture Sobot AnyRobot Sensors Vision

Sound

Tactile

Gyro

IR

Timer

Perception module Symbolizer

Attention selector

Reward/penalty signal

Symbol vector

Voice learner

Preference learner

Virtual environment

Motivation Sensor value

Inherent behavior selector

Homeostasis

Learning module

Emotion

Curiosity

Fatigue

Happiness

Intimacy

Hunger

Sadness

Monotony

Drowsiness

Anger

Internal state module

Masking

Behavior module

Urgent flag

Behavior

Behavior selector Behavior

End signal

Actuator

Fig. 1. Genetic robot, AnyRobot.

Motor module

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

925

a result, both algorithms shows the feasibility well. NNGRP shows the fast training speed and high convergence rate, while EAGRP shows the superior variability within consistency and the plausibility with comparable intensities. Therefore the robot genome generated by EAGRP is implanted into AnyRobot in a real mobile phone. As well as emotional reactions and motivational states, the experimental results show the consistency in observable behaviors with such reactions and state successfully. The remaining structure of this paper is outlined as follows: The next section brieﬂy introduces an artiﬁcial creature, AnyRobot as a genetic robot and its internal architecture. The new concept of robot genome is proposed and its prerequisites are described. Sections 3 and 4 propose the basic structure of NNGRP and EAGRP. Their procedures are described from a theoretical viewpoint and the experiments are also carried out to demonstrate its performance. The empirical and theoretical analyses follow to investigate the characteristics of EAGRP with comparative advantages in Section 5. Concluding remarks follow in Section 6. 2. Genetic robot and its prerequisites This section introduces an artiﬁcial creature, AnyRobot [22,23] and the novel concepts of the artiﬁcial chromosome, the robot genome and the genetic robot are proposed through AnyRobot in a mobile phone. 2.1. Artiﬁcial creature, AnyRobot An artiﬁcial creature is deﬁned as an agent which behaves autonomously driven by its own motivations and emotions. It must also be able to interact with humans and its environment in real time. AnyRobot is designed to fulﬁll the requirements for an artiﬁcial creature or a software robot (sobot). It represents itself visually on the screen as a dog or a robot and may interact with real humans based on stimuli via keypads on the mobile phone. The internal architecture is composed of four primary modules such as perception module, internal state module, behavior selection module, learning module, and motor module. All the modules are embodied in AnyRobot. It is developed in a mobile S/W development platform WIPI 2.0 and works well on a mobile phone, SPH-B3600 or above, which is made by Samsung Electronics. Fig. 1(a). is a photograph of mobile phones showing two types of AnyRobot, in a real mobile 3D environment. 2.1.1. Perception module The perception module can recognize and assess the environment and subsequently send the information to the internal state module. AnyRobot has several virtual sensors for light, sound, temperature, touch, vision, gyro and time. AnyRobot can perceive 47 stimulus (perceptions) and a set of all the perception groups is A = Apo ; Aob ; Abt ; Aph ; Aso ; Aof ; Aba where Apo is the perception group related to posture, Aob obstacle, Abt brightness/temperature, Aph pat/hit, Aso sound, and Aof object/face, and Aba battery as shown in Table 1. 2.1.2. Internal state module The internal state module deﬁnes the creature's internal state with the motivation unit, the homeostasis unit and the emotion unit. In AnyRobot, motivation is composed of six states: curiosity, intimacy, monotony, avoidance, greed and the desire to control. Homeostasis includes three states: fatigue, hunger and drowsiness. Emotion includes ﬁve states: happiness, sadness, anger, fear and neutral. In general, the number of internal states depends on an artiﬁcial creature's architecture. Each internal state is updated by its own weights which connect the stimulus vector to itself and are also represented as a vector. For instance, motivation vector M is deﬁned as T

M ðt Þ = ½m1 ðt Þ; m2 ðt Þ; ⋯; m6 ðt Þ

ð1Þ

where mk(t), k = 1, 2, ⋯, 6, is kth state in the internal state module and the number of states in motivation is 6. Each motivation state is updated by n o T M mk ðt + 1Þ = mk ðt Þ + λk ðmk −mk ðt ÞÞ + S ⋅Wk ðt Þ

ð2Þ

where S is the stimulus vector, WkM is a weight matrix connecting S to kth state in the internal stats module, mk is the mean value of kth state, and λk is the kth state gain. Update equations are deﬁned similarly for the homeostasis unit using state vector H(t) and weight matrix WkH, k = 7, 8, 9, and also the emotion unit using state vector E(t) and weight matrix WkE, k = 10, 11, ⋯, 14. 2.1.3. Behavior selection module The behavior selection module is used to choose a proper behavior based on AnyRobot's internal state and indirectly on the stimulus vector. When there is no command input from a user, various behaviors can be selected probabilistically by introducing a voting mechanism where each behavior has its own voting value. The procedure of the algorithm is as follows: 1. 2. 3. 4.

Determine the temporal voting vector, Vtemp using M and H. Calculate voting vector V by masking Vtemp using attention, command and emotion masks. Calculate a behavior selection probability, p(b), using V. Select a proper behavior b with p(b) among various behaviors in Table 2.

926

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954 Table 1 List of perception groups and perceptions. Group

Index

Perception

Posture (Apo)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

POWER_ON SHAKEN Dandled Shocked Lifted Fallen Correct_posture Obstacle_exist Distance_near Distance_mid Distance_far Sudden_appear Sudden_disappear Cliff Sudden_brightness Sudden_darkness Glaring Normal Dark Very_dark Mild Hot Cold Head_patted Head_hit Body_patted Body_hit Sound_noisy Sound_normal Sound_calm Sudden_loud Sudden_calm Voice Voice_good Voice_bad Voice_hello_or_bye Bject1_detected Object1_close Object2_detected Object2_close Object3_detected Object3_close Face_detected Face_close Battery_low Battery_normal Battery_full

Obstacle (Aob)

Brightness/temperature (Abt)

Pat/Hit (Aph)

Sound (Aso)

Object/face (Aof)

Battery (Aba)

Each step is described in detail as follows: 1. For AnyRobot, there are 6 motivation states and 3 homeostasis states and the temporal voting vector is calculated as follows: T T T Vtemp = M DM + H DH = ½vt1 ; vt2 ; ⋯; vtz

ð3Þ

where z represents the number of behaviors. vtr, r = 1, ⋯, z, is the temporal voting value, and 6 × z matrix DM, 3 × z matrix DH are the behavioral weight matrices connecting motivation and homeostasis to behaviors, respectively. 2. Three masking matrices assist AnyRobot in selecting more appropriate behaviors by masking the most unusual behaviors. An attention masking matrix Q a(a) is obtained by the attention stimulus, a. Each attention stimulus has its own masking value and the matrix is deﬁned as diagonal matrix with diagonal entries q1a(a), ⋯, qza(a) where z is the number of behaviors, qra(⋅), r = 1, ⋯, z, is the masking value, and 0 ≤ qra(⋅) ≤ 1. Command and emotion masking matrices, Q v(c) and Q e(e), where c is the voice command and e is the dominant emotion, are deﬁned in the same way. From these three masking matrices and the temporal voting vector, the behavior selector obtains a ﬁnal voting vector as follows: T

T

a

v

e

V = Vtemp Q ðaÞQ ðcÞQ ðeÞ = ½v1 ; v2 ; ⋯; vz

ð4Þ

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

927

Table 2 List of behavior groups and behaviors by grouping method 1. Group

Index

Behavior

Stop

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68

Stop Sit Crouch Lie Walk_forward Run_forward Walk_left Run_left Turn_left Walk_right Run_right Turn_right Hurrah Dance_with_arms Mouth_open Shake_arms Head_up Shake_head Hit_head Roar Turn_and_ignore Look_away Hit_ground Head_down Move_backward Resist Hide_head Flinch Cover_eyes Scratch_head Hide_face Shamble Shake_arm_and_leg Once_again Hello Pardon Cover_head Refuse Approve Handshake Eyecontact Glad Complain_for_food Stretch Sleep Snooze Yawn Rest Rumble Recharge Belch Hiccup Fart Piss Cough Tremble Be_ill Faint Look_around Search_looking_around Search_wandering Ride Mate Chase Approach Kick Touch Observe

Move

Happiness

Anger

Sadness Fear

Shame

Monotony Positive response to user

Physiology

Search

With target

(continued on next page)

928

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954 Table 2 (continued) Group

Index

Behavior

With target

69 70 71 72 73 74 75 76 77

Photography Record Collect Move Give_me_ball Hug_me Show_me_face Show_PDA Show_message

Control objects

Information

where vr, r = 1, 2, ⋯, z, is the rth behavior's voting value. By using masking matrices, emotion can be taken into consideration on selecting a behavior because the emotion masking matrix Q e(e) is equal to the behavioral weight matrix connecting emotion to behaviors, DE. 3. The selection probability p(bk) of a behavior, bk, k = 1, 2, ⋯, z, is calculated from the voting values as follows: pðbk Þ = vk

z

∑ ðvr Þ

ð5Þ

r=1

4. By using the proportional selection mechanism based on the probability, the behavior selector can show diverse behaviors. In addition to behaviors, AnyRobot can have ﬁve facial expressions such as happiness, sadness, anger, fear and neutral based on the dominant emotion. 2.1.4. Learning module The learning module in AnyRobot is composed of two distinct units, the preference and voice learning units. Using these AnyRobot may be trained in the same manner as a real pet would be trained [24,25]. 2.1.5. Motor module The motor module incorporates virtual actuators to execute the selected behavior in the virtual 3D environment. 2.2. Artiﬁcial chromosomal set: robot genome A genetic robot is deﬁned as an artiﬁcial creature or a robot which has its own robot genome. The concept of the genetic robot is veriﬁed by implanting the robot into AnyRobot. Given a set of c artiﬁcial chromosomes, the kth artiﬁcial chromosome, ck, consists of three gene vectors: the F-gene vector, xkF, the I-gene vector, xkI, and the B-gene vector, xkB, representing the fundamental, internal state and behavior genes respectively. Each vector is deﬁned with length w, y and z accordingly. The ﬁnal composition of the chromosome ck is given by h i F I B T ck = xk ; xk ; xk :

ð6Þ

A robot genome, g, represents an artiﬁcial chromosomal set with genetic codes determining a robot's personality and is deﬁned as g = ½ c1

j

c2

j

⋯

j

cc ;

ð7Þ

where c is the number of artiﬁcial chromosomes in the robot genome. AnyRobot has fourteen artiﬁcial chromosomes through which its traits may be passed on to its offspring. Fig. 2 shows the 14 chromosomes, where ﬁrst six ones c1 −c6 are related to motivation: curiosity (c1 ), intimacy (c2 ), monotony (c3 ), avoidance (c4 ), greed (c5 ), and desire to control (c6 ), the next three ones c7 −c9 to homeostasis: fatigue (c7 ), drowsiness (c8 ), and hunger (c9 ), and the next ones c10 −c14 to emotion: happiness (c10 ), sadness (c11 ), anger (c12 ), fear (c13 ), and neutral (c14 ). The genes in Fig. 2 are originally represented by real numbers: values of F-genes range from 1 to 500, I-genes from − 500 to 500, and B-genes from 1 to 1000. These genes are normalized to brightness values from 0 to 255, which are expressed as blackand-white rectangles. The darker the color is, the higher its value is. In addition to the positive normalization, I-genes may have negative values and be normalized as red-and-black rectangles in the same manner. Each chromosome has 5 fundamental (F-genes), 47 internal state (I-genes) and 77 behavioral (B-genes) genes. In total, AnyRobot has 1806 genes. F-genes represent fundamental characteristics of AnyRobot, including genetic information such as volatility, initial and mean values, mk in Eq. (2), and the decay rate of each internal state. Volatility determines whether the internal state is volatile or non-volatile since operating point in time. mk is the value to which the internal state converges without

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

C1

C2

C3

C4

C5

C6

C7

C8

929

C9 C10 C11 C12 C13 C14

F-GENES

I-GENES

B-GENES

Fig. 2. Robot genome of AnyRobot.

any stimuli. A high value means high desire to enhance corresponding internal state. F-genes can also include sex, life span, color and so on to deﬁne its fundamental nature. I-genes includes genetic codes representing its internal preference by setting the weights of WM k ðt Þ in Eq. (2). These genes shape a variety of the internal state affected by stimuli and have information of whether the stimulus satisﬁes, ampliﬁes, or has nothing to do with the internal state. B-genes includes genetic codes related to output behavior by setting the weights of DM and DH in Eq. (3), and Q e in Eq. (4). These genes are responsible for behavior selection, its frequency, and its activation level based on the internal state. They also include masking information which prevents AnyRobot from doing unnecessary emotional expression and behaviors. 2.3. User preference A user's preference for a genetic robot's personality is obtained via a graphical user interface (GUI) as shown in Fig. 3. GUI is developed in Visual C++ 6.0, which is possible to control each gain between 0 and 100 by using slider buttons. Gains of I-genes (each internal state) and B-genes (its relevant behavior selection) are denoted by Ψ, where " Ψ=

ψI1

ψI2

⋯

ψIc

ψB1

ψB2

⋯ ψBc

# ð8Þ

where ψkI is deﬁned as the gain of I-genes of the kth artiﬁcial chromosome, and ψkB is deﬁned similarly for the B-genes. 2.4. Gene masking This paper proposes the gene masking in order to epitomize a real creature with a genetic robot [2,11]. Given a genetic robot with own robot genome, the gene masking is deﬁned as the masking required for the robot genome in order to generate plausible internal states and external behaviors. Then the robot genome should be an appropriate personality for its context related to society, culture and history. The gene masking process is divided into F-genes masking (F-masking) for the fundamental characteristics, I-genes masking (I-masking) for the internal stat module and the B-genes masking (B-masking) for the behavior selection module. For AnyRobot, I-masking and B-masking values may be designed individually or by perception group through a GUI depending on a user's desire (refer to Tables 1 and 2). Tables 4, 5, 6, 7, and 8 show these masks, where ‘C,’ ‘I,’ ‘M,’ ‘A,’ ‘G,’ ‘C,’ represent motivation groups: curiosity, intimacy, monotony, avoidance, greed, desire to control, ‘F,’ ‘D,’ ‘H,’ homeostasis groups: fatigue, drowsiness, hunger, and ‘H,’ ‘S,’ ‘A,’ ‘F,’ ‘N,’ emotion groups: happiness, sadness, anger, fear, neutral. Each masking value may be designed individually or by perception group (refer to Table 1). The original goal of this research is to reduce the difﬁculty and detail invalid in manually setting. As shown in Fig. 3, a user can use the default mask values or change each mask value easily by clicking either ‘I-Mask’ or ‘B-mask’ button according to his/her

930

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Fig. 3. GUI window for setting the preference values of each internal state (Agreeable personality). If one clicks either ‘I-Mask’ or ‘B-mask’ button, the gene mask values can also be changed by a user's preference.

Table 3 List of behavior groups and behaviors by grouping method 2. Internal state

Index

Group

Behavior

Motivation

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Curiosity Intimacy Monotony Avoidance Greed Control Fatigue Drowsiness Hunger Happiness Sadness Anger Fear Neutral

Approach, eyecontact, observe, touch Touch, approach, show_me_face, hug_me, lean Shake_arm_and_leg, snooze, look_around, stretch Turn_and_ignore, look_away, move_backward, cover_eyes Search_looking_around, search_wandering, collect, roar, chase Show_me_face, hug_me, give_me_ball, once_again, Refuse, rest, crouch, recharge, faint, head_down Yawn, sleep, snooze, lie, rest Complain_for_food, rumble, search_wandering Hurrah, dance_with_arms, mouth_open, shake_arms, shake_head Hit_ground, head_down, be_ill, hug_me, weep Shake_head, hit_head, roar, turn_and_ignore, look_away Resist, hide_head, tremble, faint, move_backward Stop, lie, look_around

Target

Index

Group

Behavior

Objects (3 balls)

15 16 17 18 19 20 21 22 23 24 25 26

Like Neutral Dislike Like Neutral Dislike FriendlyNeutral FriendlyHit FriendlyPat UnfriendlyNeutral UnfriendlyHit UnfriendlyPat

Kick, touch, chase, jump_to_object Observe, touch, move Look_away, turn_and_ignore, get_rid_of_it Glad, mate, handshake, hurrah Observe, eyecontact, hello Hit_head, turn_and_ignore, roar, ﬁght Show_me_face, hug_me, give_me_ball, complain_for_food Cover_head, head_down Once_again, rub, shake_arms, head_up Complain_for_food, give_me_ball, watch_out, laugh_at Cover_head, roar, hit_ground Stop, ﬂinch, head_up

Homeostasis

Emotion

Sobots

User

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

931

Table 4 F-mask. Group

C

I

M

A

G

C

F

D

H

H

S

A

F

N

Volatility Initial Mean Decay Trajectory

1 0 0 1 0

1 0 0 1 0

1 0 1 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 0 1 0

1 0 1 1 0

Table 5 I-mask in group. Group

C

I

M

A

G

C

F

D

H

H

S

A

F

N

Posture Obstacle Brightness/temperature Pat/hit Sound Object/face Battery

0 0 0 0 1 1 0

1 0 0 1 1 1 0

1 0 0 1 1 1 0

1 1 0 1 1 1 0

0 0 0 0 0 1 0

0 0 0 1 0 1 0

0 0 0 0 0 0 1

1 0 1 1 1 0 1

0 0 0 0 0 0 1

1 0 0 1 0 1 0

1 0 0 1 0 1 0

1 0 0 1 0 1 0

1 0 1 1 1 1 0

1 0 0 1 1 0 0

preference. For instance, a user may wish that his/her genetic robot's curiosity only change under the inﬂuence of perceived events related to posture, pat/hit and sound but to ignore events from the other groups. In this case he/she chooses the desired perception groups in the graphical user interface (GUI) for positive masking and the remaining groups for zero masking. B-masking may also be applied individually or by the groups shown in Table 2. Similarly, applying these preferences to the strongly correlated groups is a simple, but importantly intuitive feature necessary for designing the robot's personality. F-masking is required so that both NNGRP and EAGRP may focus on target traits consistently by ﬁxing unconcerned characteristics on zero as shown in Table 6 since a genetic robot's traits are highly correlated with all the F-genes. I-masking is required so that all kinds of perception-based internal state values may increase or decrease appropriately for its genome. Given I-masking matrices mIk and I-genes vectors xIk , the I-masking process ΘIm xI for the masking of the I-genes in a robot genome is deﬁned as I I Θm x = mI1 abs xI1

⋯ mIc abs xIc

ð9Þ

where abs(xkI) is the resultant vector after the absolute value of each element is taken and c is the number of artiﬁcial chromosomes. By I-masking process, all the I-genes xI in a robot genome are replaced by their masked equivalents for the NNGRP and EAGRP. Each entry in the I-masking vector has one of three masking values −1, 0, or +1, which represent negative masking, zero masking, and positive masking, respectively. Positive masking increases the value of the perceived event, while negative masking decrease the value of the relevant internal state. Zero masking prevents the value of the relevant internal state from changing.

Table 6 I-negative-mask in individual. Group C I M A G C F D H H S A F N

Internal state-negative-mask in individual SHAKEN, HEAD, BODY_HIT SHOCKED, HEAD, BODY_PATTED, OBJECT1, 2, 3, FACE_DETECTED

HEAD, BODY_HIT, VOICE_BAD SHAKEN, SHOCKED, LIFTED, FALLEN, SUDDEN_BRIGHTNESS, GLARING, HEAD, BODY_HIT, SUDDEN_LOUD, VOICE, FACE_DETECTED SHAKEN, SHOCKED, LIFTED, HEAD, BODY_HIT, VOICE_BAD, OBJECT3_DETECTED, CLOSE DANDLED, HEAD, BODY_PATTED, OBJECT1, 2, FACE_DETECTED, CLOSE DANDLED, HEAD, BODY_PATTED, OBJECT1, 2, FACE_DETECTED, CLOSE DANDLED, HEAD, BODY_PATTED, OBJECT1, 2, FACE_DETECTED, CLOSE

932

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Table 7 I-zero-mask in individual. Group

Internal state-zero-mask in individual

C I M A G C F D H H, S, A, F N COMMON

OBJECT1, 2, 3_CLOSE OBJECT1, 2, 3, FACE_DETECTED, CLOSE HEAD, BODY_HIT, OBJECT1, 2, 3, FACE_CLOSE HEAD, BODY_PATTED, VOICE, VOICE_GOOD, BAD_HELLO_OR_BYE, OBJECT1, 2, FACE_DETECTED, CLOSE OBJECT1, 2, 3_CLOSE VOICE, VOICE_HELLO_OR_BYE, OBJECT1, 2, 3_DETECTED, CLOSE BATTERY_FULL BATTERY_FULL BATTERY_FULL SHOCKED, LIFTED, FALLEN, HEAD, BODY_PATTED CORRECT_POSTURE, NORMAL, MILD, DISTANCE_FAR, SOUND_NORMAL, SOUND_CALM, BATTERY_NORMAL

B-masking is required so that a genetic robot may select more appropriate behaviors given a speciﬁed internal state and perceived event. Given B-masking matrices mBk consisting of B-masking vectors and B-gene vectors xBk , the B-masking process ΘBm xB for masking of the B-genes in a robot genome is deﬁned as B B Θm x = mB1 xB1

B B ⋯ mc xc :

ð10Þ

In the same manner as I genes, all B-genes are replaced by their masked equivalents for NNGRP and EAGRP. Elements of the Bmasking vector take values 0 or 1, zero masking and positive masking. Given a perceived event, positive masking increases the voting value of the behavior relevant to the perceived event. Zero masking prevents the behavior from being selected. 2.5. Perception scenario A perception scenario (PS) is used to trial the robot genome at every evaluation. A PS is designed using stimulus from its environment to observe the creature's internal states and external behavior. This manner in which the stimuli may be applied is customizable and is formalized as follows: each step in the PS is characterized by an event. For the user, the event represents a stimulus applied to the artiﬁcial creature. For the creature, it is a perceived event. Based on this formalization, given an artiﬁcial creature, a perception scenario is deﬁned as the permutation of its perceivable information with the perception time. Each event is characterized by the parameters A; Γ; V; rmin ; rmax ; ts ; tp ; Ts ; tr Þ

ð11Þ

where A = fA1 ; A2 ; ⋯; An g is the set of all groups representing similar perceived events and n is the number of perception groups. ΓðAÞ = fAF1 ; AF2 ; ⋯; AFm g for m ≤ n is a restricted subset of the groups in A. This is used in a scenario for which a user does not wish to or cannot feasibly expose the robot to a complete set of stimuli. V = fVF1 ; VF2 ; ⋯; VFm g are the set of voting values used to prioritize the random occurrence of events associated with each AFi ∈ΓðAÞ. Events occur at discrete time intervals with variable time step, ts ∈ [rmin, rmax]. tp is the current time, Ts is the duration length of the scenario which is called the perception scenario time and tr is the reset time to initialize all the variables of a genetic robot in a simulation.

Table 8 B-mask in group. Group

C

I

M

A

G

C

F

D

H

H

S

A

F

N

Stop Move Happiness Anger Sadness Fear Shame Monotony Positive response to user Physiology Search With target Control Information

0 0 0 0 0 0 0 0 0 0 1 1 0 0

0 0 0 0 0 0 0 0 1 0 1 1 0 0

1 0 0 0 0 0 0 1 0 1 1 0 0 0

0 1 0 1 1 1 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 1 1 0 0

0 0 0 0 0 0 0 0 0 0 0 0 1 1

0 0 0 0 0 0 0 1 0 1 0 0 0 0

1 0 0 0 0 0 0 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0 0 1 0 0 0 0

1 1 1 0 0 0 0 0 1 0 1 1 1 0

1 1 0 1 1 1 1 1 1 0 0 0 1 0

1 1 0 1 1 1 1 1 0 0 1 0 1 0

1 1 0 1 1 1 1 1 0 0 1 0 1 0

1 1 0 1 1 1 1 1 1 1 1 1 1 0

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

933

time and tr is the reset time to initialize all the variables of a genetic robot in a simulation. 0.0

BEGIN

42.4

RESET

0.6

SHAKEN

46.0

SUDDEN_DISAPPEAR

89.4 RESET

15.5

SHAKEN

61.3

DISTANCE_FAR

101.5 VOICE_HELLO_OR_BYE

18.9

DANDLED

81.0

DISTANCE_MID

111.4 OBJECT2_DETECTED

23.1

FALLEN

85.4

SUDDEN_APPEAR

117.2 FACE_DETECTED

31.4

FALLEN

87.4

DISTANCE_NEAR

140.5 VOICE_BAD

38.2

FALLEN

89.6 VOICE_GOOD

150.0 END

Fig. 4. An example of a perception scenario.

Fig. 4 shows a perception scenario. ts ∈ [0.1 s, 10 s] and Ts = 150 s were used in each piece of scenario. ΓðAÞ = Apo and V = f0:5g, were used from 0.0 s to 38.2 s, ΓðAÞ = fAob g and V = f0:5g from 46.0 s to 87.4 s, and ΓðAÞ = Aso ; Aof and V = f0:7; 0:3g from 89.6 s to 150 s. trs were 42.4 s and 89.4 s. 2.6. Personality model and its key principles This paper uses the following key principles to make plausible or believable emotional agents [26,27]: • Consistency: the generated behaviors have to some consistency in emotional reactions and motivational states, as well as in the observable behaviors associated with such reactions and states. The followings are major sources of emotional and behavioral consistency. - Psychological constraint: There must be some ‘psychological constraints’ that limit the responses that are produced between emotions, motivations, and behaviors, as people usually predict with reasonable accuracy how other individuals will respond to and behave in certain kinds of situations. - Trait: People only get into emotional states when they ‘care’ about something. Traits are chronic propensities to get into corresponding emotional states, which means that if we have a way of representing and creating internal states that correspond to emotions, we can capture many trait. • Variability: There must be variability within consistency as usually, people do not always react in the same way in the same kind of situation. • Coherency: Believability entails not only that emotions, motivations, and actions ﬁt together in a meaningful and intelligible way at the local level, but also that they cohere at a more global level—across different kinds of situations, and over quite long time periods. • Intensity: Similar situations elicit emotions of comparable intensity. For example, emotionally volatile people explode with the slightest provocation while their placid counterparts remain unmoved. In order to produce consistent internal states and behaviors we will focus on developing traits that do not live in isolation but cluster together. Consequently the trait space can be characterized in terms of a small number of factors. We usually expect similar, correlated traits for personalities that exhibit speciﬁc traits. For instance, a friendly person will also be thought of as generous or warm. McCrae and Costa classiﬁed individual traits in the trait space as belonging to one of ﬁve primary groups as follows [15]: • • • • •

Extroverted (as opposed to introverted): sociable, warm, and talkative Agreeable (as opposed to antagonistic): forgiving, good-natured, and softhearted Conscientious (as opposed to negligent): hard working, well organized, and reliable Openness (as opposed to closed) Neuroticism (as opposed to emotional stability)

NNGRP and EAGRP are designed based on the above principles, where initialization, perception scenarios, and masking as well as individual representation are designed in consideration of consistency, variability and coherency. In particular, ﬁtness functions and optimization processes are inspired by emotional intensity and trait groups. 2.7. Problem deﬁnition This section deﬁnes trait spaces for the genetic robot. The perception scenario is designed for generating genetic robot's personality. Among big ﬁve trait personality models in Section 2.6, the personalities of both an agreeable genetic robot and an antagonistic genetic robot are engineered to demonstrate the feasibility of NNGRP and EAGRP. The agreeable personality assumes strong intimacy and happiness, and weak greed, desire to control, avoidance, anger and fear. On the contrary, the antagonistic

934

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Fig. 5. Perception scenarios for experiments.

personality assumes weak intimacy and happiness, and strong greed, desire to control, avoidance, anger and fear. The procedure for generating and testing RGs by NNGRP and EAGRP is applied in the following manner: First, perception scenarios for NNGRP and EAGRP are prepared as shown in Fig. 5. One PS in Fig. 5(a). is for generating robot genomes by NNGRP and EAGRP and the other PS in Fig. 5(b). is for testing RGs generated by NNGRP and EAGRP. The parameters, rmin = 0.1 s, rmax = 10 s, Ts = 500 s, ΓðAÞ = Apo ; Aob ; Abt ; Aph ; Aso ; Aof ; Aba and V = f0:5; 0:5; 0:5; 0:7; 0:5; 0:7; 0:5g, were used. The perception scenarios at the top of the ﬁgure represent the indexes of 47 types of perceptions for Ts. The histograms represent the frequencies of each perception. The pie charts represent the percentages in which each perception group is used. Second, the user sets the gains ψkI of each internal state and ψkB of relevant behavior for the genetic robot's personality by the user's preference through the GUI (refer to Eq. (8) and Table 9), where each gain varies between 0 and 100. The number of I-genes or perceptions, y = 47, the number of B-genes or behaviors, z = 77, and k = 1, 2, ⋯, 14. Refer to the indexes of Tables 2 and 3.

Table 9 Preference gains for agreeable and antagonistic personalities. Internal state

Agreeable personality

Antagonistic personality

Upper

Lower

k

I-gain ψkI

B-gain ψkB

I-gain ψkI

B-gain ψkB

Motivation

Curiosity Intimacy Monotony Avoidance Greed Control Fatigue Drowsiness Hunger Happiness Sadness Anger Fear Neutral

1 2 3 4 5 6 7 8 9 10 11 12 13 14

50 80 50 20 20 10 20 20 20 80 50 20 20 50

50 80 50 20 20 10 20 20 20 80 50 20 20 50

20 20 50 80 80 70 20 20 20 20 50 80 80 20

20 20 50 80 80 70 20 20 20 20 50 80 80 20

Homeostasis

Emotion

Bold data mean high preference gains for dominant characteristics of each personality.

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

935

3. Neural network algorithm for a genetic robot's personality (NNGRP) This section proposes the neural network algorithm for a genetic robot's personality (NNGRP). It produces and trains a genetic robot's internal state module (GRISM) in order to construct a personality based on the neural network according to a user's preference. The motivation for this research was to implement an algorithm with a capability to perform in run-time on systems without the time or computational requirements necessary for evolutionary generation. The experimental results verify the feasibility and effectiveness of the proposed GRISM and NNGRP. 3.1. Internal state module (ISM) The submodules as shown in Fig. 6 are required to implement the internal state module (ISM) for input perceptions as follows: ﬁrst, the neural net module for I-gene mask (NNIM) which outputs I-gene mask values of each internal state (refer to Section 3.1.1), second, the neural net module for I-gene magnitude based on many adaptive linear neurons (NNIMAL) which outputs I-gene magnitude (refer to Section 3.1.2, and third, the merging module which completes the ISM by integrating both modular neural net modules. 3.1.1. Neural network for I-gene mask Neural network module for I-gene mask in Fig. 6 is based on the gene masking in Section 2.4, which adopts a general perception layered structure. There are p kinds of perception and c number of internal states. The purpose of neural network module for I-gene mask (NNIM) is to send out I-gene mask vector corresponding to qth perception when the qth perception index comes into the input layer. I-gene mask vector consists of diagonal elements of mIqk in Eq. (9), where y is the number of I-genes in an artiﬁcial I chromosome, and mqk , q = 1, 2, ⋯, y, is − 1, 0, or 1. These parameters are same as Eq. (17), which states robot genome structure of genetic robot. For example, if the 9th perception is inputted to the input layer of NNIM, each neuron in the output layer I I I outputs each element of the vector [− 1 0 0 1 0 0 0 0 0 1 − 1 1 1 1], which corresponds to the 9th row [m91 , m92 , ⋯, m9c ] in the I I-gene mask matrix mk .

Internal state module Neural net module for I-gene mask Merging module Hidden layer1

Hidden layer2

Output layer

Input layer

Each Internal state value

Perception

Input layer

Perception layer

Adaline layer

Neural net module for I-gene magnitude

Fig. 6. Internal state module which is composed of modular neural nets.

936

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Table 10 Speciﬁcation of parameters for training NNIM.

Network

Training

Training data

Parameter

Value

Number of input neurons Number of output neurons Number of hidden neurons Training method Maximum number of epochs Performance goal Learning rate Minimum performance gradient Network inputs Network targets

1 14 10 × 14 Levenberg –Marquardt (LM) 1000 0.0004 0.05 10− 18 [1 2 ⋯ y], y = 48, the number of perceptions y × c-sized I-gene mask matrix, c = 14, the number of internal states (chromosomes in a RG)

The speciﬁcation of the four-layer feed-forward neural network for training the NNIM is as described in Table 10. Fig. 7 shows the training process of NNIM by Levenberg–Marquardt algorithm [28,29], which is a popular alternative to the Gauss–Newton method of ﬁnding the minimum of a function that is a sum of squares of nonlinear functions, F ðxÞ =

1 m 2 ∑ ½f ðxÞ : 2 i=1 i

ð12Þ

Let the Jacobian of fi(x) be denoted Ji(x), then the Levenberg–Marquardt method searches in the direction given by the solution p to the equations T

T

Jk Jk + λIÞpk = −Jk fk ;

ð13Þ

where λk are nonnegative scalars and I is the identity matrix. The method has the nice property that, for some scalar Δ related to λk, the vector pk is the solution of the constrained subproblem of minimizing ||Jkp + fk||22/2 subject to ||p||2 ≤ Δ[30]. The results of trained network by using NNIM and LM method are shown in Fig. 8. When 48 perceptions are given to the input layer in sequence, the output layer sends out its corresponding I-gene mask vector. These are the result of converting from merging module where each element value between −0.1 to 0.1, −2 to −0.9, and 0.9 to 2 are changed to 0, −1, and − 2 respectively. Consequently, the plot of I-gene mask matrix is equivalent to the result. 3.1.2. Neural network for I-gene magnitude based on many adaptive linear neurons The purpose of neural network for I-gene magnitude based on many adaptive linear neurons (NNIMAL) is to send out c number of internal state of I-gene magnitude value, which falls into each perception, when the arbitrary perception index comes into the input layer within p types of perception. As shown in Fig. 6, NNIMAL is composed of the perception layer, the Adaline layer, and the connecting weights between both layer. The perception layer has y perception neurons and the Adaline layer has c Adalines. The connecting weights are wqk, q = 1, ⋯, I y, k = 1, ⋯, c. wqk in many adaptive linear neuron (Madaline) structure and xqk , which is the qth I-gene of the kth artiﬁcial chromosome, in Eq. (6) are exactly matches as 1 to 1 ratio. When the arbitrary indexed q perception comes into the input layer, the corresponding values are changed to 1 × y-sized

perception vector, x = x1 ; x2 ; ⋯; xq ; ⋯; xy . Its initial elements are set to 0, but the qth bit is set to 1. This motivates a stimulation of

Fig. 7. Neural net module for I-gene mask trained by Levenberg–Marquardt method.

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(a) 2D I-gene mask

937

(b) 3D I-gene mask

1

5

0.8 1

10

I-mask value

15

I-gene

20 25 30 35

0.6 0.4

0.5

0.2 0 0 -0.5

-0.2 -0.4

-1 15

40

-0.6 60

10 40

45

5 2

4

6

8

10

12

14

Chromosome

Chromosome

20 0

0

-0.8 -1

I-gene

Fig. 8. Outputs of trained NNIM for 48 perceptions.

qth perception neuron in the perception layer. Similar to Eq. (14), the output of each Adaline in Adaline layer, δwk, is a sum of linear combination of individual perception vector, which is multiplied by its weight. y

δwk = ∑ wqk × xq

ð14Þ

q=1

Consequently, δwk is the magnitude of variation in each internal state within the merging module and this value is always positive. 3.1.3. Merging module The merging module sends out the current status of internal state as an output. Refer to Eqs. (15) and (16). 3.2. Genetic robot's internal state module (GRISM) GRISM consists of modular neural nets in Fig. 6 and receives input at every sampling time. Perception and internal state have y and c number of types respectively. q and k is the perception index and the index of each internal state where q = 1, 2, ⋯, y and k = 1, 2, ⋯, c. In this paper, the value of c is set to 14 and the value of y is set to 47. The GRISM is performed at every sampling time in the following manner: Step 1. Receiving perception vector, x Step 2. Recognizing the qth perception from x in the order from 1 I I Step 3. Saving [mq1 , ⋯, mqc ], the output of the qth I-gene mask from NNIM, and [δwq1, ⋯, δwqc], the output of the qth I-gene magnitude, in each internal state from NNIMAL to buffer. Step 4. Repeating Step 2 to Step 4 until the value q equals y. Step 5. Summarizing the results from Step 1 to Step 5 and lastly, the output from buffer are transformed into the following form: 2

3 y I ∑ m δw q1 q1 6 7 2 6 q=1 7 I 3 Δw1 6 7 6 7 6 7 ⋮ 6 7 6 7 ⋮ 6 y 7 6 7 6 7 I I 6 7 ∑ mqk δwqk 7 Δw = 6 ΔwIk 7 = 6 6 7: 6 7 q=1 6 7 6 7 6 7 ⋮5 4 6 7 ⋮ 6 7 I 6 y 7 Δwc 4 5 I ∑ mqc δwqc

ð15Þ

q=1

Step 6. Updating each internal state by Eq. (16) o I ik ðt + 1Þ = ik ðt Þ + βk ðmk −mk ðt ÞÞ + γ⋅Δwk ðt Þ

ð16Þ

where mk is the mean value of kth internal state, and βk is the decay value of the kth internal state, which decreases the internal state per every sampling time. Step 7. In the next sampling time, go back to Step 1 and repeat the process.

938

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Procedure NNGRP begin e

0

i)

input according to a user’s preference, up

ii)

initialize W (e) and I (e) of GRISM

iii)

evaluate I (e)

iv)

while (not termination-condition) do begin e

v)

e+ 1

evaluate I (e)

vi)

update W (e) end

end Fig. 9. Procedure NN for genetic robot's personality.

3.3. Basic structure of NNGRP NNGRP uses the perception scenario from Fig. 5(a). as input data. The user deﬁned preference vector has been employed as target data. GRISM is trained by Adaline's training algorithm (delta rule) with the generated ISM. This results in the GRISM acquiring weightings for the genetic robot's personality according to the result of the NNGRP. Fig. 9 illustrate the algorithm that NNGRP uses. The overall structure of the NNGRP can be explained in detail as follows: i) A user's preference is received as a percentage value. e means the training epoch. ii) Initialize W which is the set of y × c weights, and I = [i1, ⋯, ic] which is the set of c internal states. iii), v) each internal state is evaluated to give a measure of percentages of each internal state in a perception scenario (Fig. 10). ipk is the percentage of kth cumulated internal state in a perception scenario and upk is the preference gain of the kth internal state of a user where upk ∈up, k = 1, ⋯, c. iv) Steps v) and vi) are repeated until the termination condition is reached, which uses errorsum ≤ 3. vi) As shown in Fig. 11 by using the data rule of Adaline training method, wqks, q = 1, ⋯, y which are output values of NNIMAL are updated, where η is the learning rate.

Procedure evaluate(I) begin t

0

while (t ≤ Tscn ) do begin t

t + t samp

update each ik (t )in I by equation (16) cumulate each ik (t ) end calculate the percentage, ipk (t ), of each cumulated internal state end

Fig. 10. Procedure evaluate.

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

939

Procedure update(W) begin k

0

while(k ≤ c ) do begin k

k+1

error k wk

upk wk +

error sum

ip k

errork error sum + |error k |

end error sum

error sum /c

end

Fig. 11. Procedure update.

3.4. Experiments This section veriﬁes the feasibility and plausibility of the NNGRP. To do this, the NNGRP generates robot genomes for the personalities of both an agreeable robot and an antagonistic robot through perception scenario 1 (PS1). Then the generated robot genomes are implanted into AnyRobots A and B and perception scenario 2 (PS2) is applied to validate the results of the training process. The results shows well-designed personalities related to internal states that are consistent with expectations. 3.4.1. Generation of robot genome by NNGRP Fig. 12 shows the training processes of NNGRP for both agreeable and antagonistic personalities, which the perception scenario 1 in Fig. 5(a) is applied. Figs. 13 and 14 compare the agreeable RG A with the antagonistic RG B generated by NNGRP. As described in Section 4.6.1, the trained RGs of each personality have different structures and appropriately represent their own characteristics. Figs. 15 and 16 show the training results of NNGRP for both agreeable and antagonistic personalities. These ﬁgures, especially Figs. 15(d) and 16(d), show that NNGRP appropriately optimized both agreeable and antagonistic personalities according to each user's preference. NNGRP has the advantage that it has fast training speed as shown in Fig. 12 and high convergence rate as shown in Figs. 15 and 16.

(a) Agreeable personality

(b) Antagonistic personality 5 4.8

Training-Blue Goal-Black

Training-Blue Goal-Black

5.5

5

4.5

4

3.5

4.6 4.4 4.2 4 3.8 3.6 3.4 3.2 3

3 0

1

2

3

4

5

6

7

8

9

10

0

1

Epoch

2

3

4

5

6

Epoch Fig. 12. Training process of NNGRP.

7

8

9

10

11

940

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(a) 2D I-gene

(b) 3D I-gene 150

5 200

10

150

I-mask value

15

I-gene

20 25 30

100

100 50

50

0

-50

35

0

-100 15

40

60

10 40

45

5 2

4

6

8

10

12

14

0

Chromosome

Chromosome

-50

20 0

I-gene

Fig. 13. Agreeable RG A which is composed of weights generated by NNGRP.

3.4.2. Implantation of robot genomes into genetic robot Figs. 17 and 18 show the simulation results of both agreeable and antagonistic personalities by NNGRP for perception scenario 2 in Fig. 5(b). These results represent variability, consistency and plausibility of both personalities well. 4. Evolutionary algorithm for a genetic robot's personality (EAGRP) EAGRP uses a novel individual representation for the individual that was introduced in the previous section, ‘robot genome.’ The algorithm uses the chromosomes contained with the genome and its own evolutionary techniques for production of a genetic robot's personality. 4.1. Basic structure of EAGRP EAGRP is an evolutionary probabilistic algorithm which maintains a population of individuals named robot genomes with the form of 2-dimensional matrix, P ðt Þ = gt1 ; gt2 ; ⋯; gtn at generation t, where n is the size of the population. Fig. 19 illustrates the overall structure of EAGRP in the following manner: i) The robot genomes and EA gains are initialized with the assistance of a graphical user interface (GUI). (Section 2.3) ii) and viii) Each robot genome gti is masked by I-masking and B-masking based on the Eliza-Meme scheme in order to generate plausible internal states and external behaviors. Masked robot genomes replace the original robot genomes. (Section 2.4)

(a) 2D I-gene

(b) 3D I-gene

5 150 200

10

150

I-mask value

15

I-gene

20 25 30

100 100 50 50 0 -50 0

35

-100 15

40

60

10 40

45

5 2

4

6

8

Chromosome

10

12

14

Chromosome

20 0

0

Fig. 14. Antagonistic RG B which is composed of weights generated by NNGRP.

I-gene

-50

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(b) Homeostasis 1000

500 00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

500

500 0 0 1000 500 00 1000

Drowsiness

0 00 1000

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

500

0 1000

500 0 0 1000

50

100

150

200

50

100

150

200

250

300

350

400

450

500

250

300

350

400

450

500

500 0

0

Hunger

Control

Greed Avoidance Monotony Intimacy Curiosity

(a) Motivation

941

500

0

Time (sec)

Time (sec)

(c) Emotion

(d) User’s preference and percentage of

500 0 0 1000

Fear

100

150

200

250

300

350

400

450

500

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

Emotion

60 40 20 0

1

2

3

4

5

6

Motivation

100 50

100

150

200

250

300

350

400

450

7

8

9

10

11

12

13

14

50

100

150

200

250

300

350

400

450

Homeostasis

Emotion

500

500

80 60 40 20

500 0

Homeostasis

Internal state

500 00 1000

Motivation

80

500 0 0 1000

Neutral

50

Preference gain

100 1000

Percentage

Anger

Sadness Happiness

possession

0

50

100

150

200

250

300

Time (sec)

350

400

450

500

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Internal state

Fig. 15. Internal states of the agreeable genetic robot A after the training of NNGRP.

iii) and ix) Each robot genome gti is evaluated to give a measure of its ﬁtness. (Sections 2.4 and 2.5) iv) and x) The best solution is then selected among the solutions P(t), and stored in bðt Þ which is the best solution and robot genome for generation t. v) Steps vi)–x) are repeated until the termination condition is reached. This paper uses the maximum generation as the termination condition. vi) In the while loop, a new population (iteration t + 1) is formed by selecting the ﬁttest robot genomes in that generation. F vii) Some members of the new population undergo transformations by means of the crossover operators, Θχ , ΘχI and ΘχB , F I B and the mutation operators Θμ, Θμ and Θμ , to form new solutions. (Section 2.5)

4.2. Individual representation As mentioned earlier in Section 2.2, EAGRP uses a new individual representation, called a robot genome codifying the internal architecture of a genetic robot. A robot genome is a 2-dimensional structure composed of multiple chromosomes. Each chromosome is responsible for an internal state unit of a genetic robot and includes three types of genes such as F-genes, I-genes, and B-genes.

942

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(b) Homeostasis 1000

500 00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

500

500 0 0 1000 500 00 1000

Drowsiness

0

00 1000

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

500

0 1000

500 0 0 1000

50

100

150

200

50

100

150

200

250

300

350

400

450

500

250

300

350

400

450

500

500 0

0

Hunger

Control

Greed Avoidance Monotony Intimacy Curiosity

(a) Motivation

500

0

Time (sec)

Time (sec)

(c) Emotion

(d) User’s preference and percentage of

0 1000

Anger Fear

0

50

100

150

200

250

300

350

400

450

500

500 0

0

50

100

150

200

250

300

350

400

450

500

0 0 1000

50

100

150

200

250

300

350

400

450

500

Preference gain

500

1000

Homeostasis

Emotion

80 60 40 20 0

1

2

500

0

50

100

150

200

250

300

350

400

450

500

1000

4

5

6 7 8 9 Internal state

10

11

Homeostasis

12

13

14

Emotion

80 60 40 20

500 0

3

Motivation

100

500 0

Neutral

Motivation

100 1000

Percentage

Sadness Happiness

possession

0 0

50

100

150

200

250

300

350

400

450

500

1

2

3

4

Time (sec)

5

6 7 8 9 Internal state

10

11

12

13

14

Fig. 16. Internal states of the antagonistic genetic robot B after the training of NNGRP.

Given a population of robot genomes P ðt Þ = gt1 ; gt2 ; ⋯; gtn at generation t, where n is the size of the population, gti is a robot genome deﬁned as follows: 2

3 xFt i 6 7 6 7 h i 6 It 7 t t t t gi = ci1 ci2 ⋯cic = 6 xi 7 6 7 4 5 Bt xi 2 3 2 Ft Ft Ft x xi1 ⋯ xFt ic 6 xi1 ⋯ ic 7 6 6 7 6 6 It It 7 6 It It 7 =6 x ⋯ xic 6 xi1 ⋯ xic 7 = 6 6 i1 6 7 4 4 Bt 5 xi1 ⋯ Bt xBt ⋯ xBt i1 ic x

3 7 7 7 7 7 5

ic

where c is the number of artiﬁcial chromosomes in a robot genome, and i = 1, 2, ⋯, n.

ð17Þ

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(b) Homeostasis 1000

500 00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

500

500 0 0 1000 500 00 1000

Drowsiness

0

00 1000

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

500

0 1000

500 0 0 1000

50

100

150

200

50

100

150

200

250

300

350

400

450

500

250

300

350

400

450

500

500 0

0

Hunger

Control

Greed Avoidance Monotony Intimacy Curiosity

(a) Motivation

943

500

0

Time (sec)

Time (sec)

(c) Emotion

(d) User’s preference and percentage of

0

0

Fear

50

100

150

200

250

300

350

400

450

500

1000 500 00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Preference gain

500

Homeostasis

Emotion

80 60 40 20 0

1

2

500 0 0 1000

Neutral

Motivation

100 1000

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

4

5

6 7 8 9 Internal state

10

11

Homeostasis

12

13

14

Emotion

80 60 40 20

500 0

3

Motivation

100

Percentage

Anger

Sadness Happiness

possession

0 0

50

100

150

200

250

300

350

Time (sec)

400

450

500

1

2

3

4

5

6 7 8 9 Internal state

10

11

12

13

14

Fig. 17. Internal states of the agreeable genetic robot A by NNGRP for PS2.

4.3. Initialization according to a user's preference F0 I0 B0 In the step of ‘initialize P(t)’, xpk , xqk , xrk , p = 1, 2, ⋯, w, q = 1, 2, ⋯, y, r = 1, 2, ⋯, z, k = 1, 2, ⋯, c, of all g0i = gti j t = 0 , i = 1, 2, ⋯, n, 0 0 0 where P ð0Þ = g1 ; g2 ; ⋯; gn at generation t = 0, are initialized according to a user's preference obtained via a graphical user interface (GUI). w, y, and z are the total number of F-genes, I-genes, and B-genes in an artiﬁcial chromosome. A population of genomes is randomly initialized in the following ranges: I-genes [−500, 500] and B-genes [1, 1000]. Each F0 F-gene, xwk , of all g0i = gti j t = 0 is initialized individually, since the format and the scale of genes are distinctly different each other. F0 F0 F0 F0 F0 x1k , x2k , x3k , x4k , and x5k represents volatility for genes inheritance, initial value, mean value, decay rate, and operating time for F0 F0 homeostasis in order. In particular x2k , x3k ∈ U[0, Fmax] (uniformly distributed random variables on [0, Fmax]) where Fmax is the F0 upper bound for the F-genes. In contrast the decay rate must for practical purposes be scaled differently. For AnyRobot, x4k ∈ U[10, 0.1Fmax] where 100 b Fmax ≤ 500. This is to ensure the decay rate is never zero as this would make divergence of internal states.

4.4. Fitness function and its candidates The procedure of ‘evaluation’ has four steps as follows. Step 1: a robot genome equal to an individual is imported to a genetic robot's architecture, Step 2: perception scenarios apply a variety of random stimuli in a ﬁnite time to the genetic robot in a virtual environment, Step 3: the genetic robot's internal states and relevant behaviors vary in response to stimuli in a perception scenario based on the robot genome, and Step 4: a ﬁtness function is evaluated, which is composed of the genetic robot's internal and external outputs.

944

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(b) Homeostasis 1000

500 00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

500

500 0 0 1000 500 00 1000

Drowsiness

0

00 1000

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

500

0 1000

500 0 0 1000

50

100

150

200

50

100

150

200

250

300

350

400

450

500

250

300

350

400

450

500

Hunger

Control

Greed Avoidance Monotony Intimacy Curiosity

(a) Motivation

500

500 0

0

0

Time (sec)

Time (sec)

(d) User’s preference and percentage of

(c) Emotion

possession Motivation

0

0

50

100

150

200

250

300

350

400

450

500

00 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

1000 500

0 0 1000

50

100

150

200

250

300

350

400

450

500

Fear

40 20 0

1

2

3

4

5

6 7 8 9 Internal state

Motivation

10

11

Homeostasis

12

13

14

Emotion

80 60 40 20

500 0

Emotion

60

100

500

Homeostasis

80

500 0 0 1000

Neutral

Preference gain

500

Percentage

Anger

Sadness Happiness

100 1000

0 0

50

100

150

200

250

300

350

400

450

500

Time (sec)

1

2

3

4

5

6 7 8 9 Internal state

10

11

12

13

14

Fig. 18. Internal states of the antagonistic genetic robot B by NNGRP for PS2.

A genetic robot, AnyRobot can have a variety of ﬁtness functions. Possible candidates for the ﬁtness function in EAGRP are classiﬁed into two categories: ﬁtness candidates related to internal states and ﬁtness candidates related to external behaviors. 4.4.1. Fitness candidates evaluating internal states Fitness candidates related to internal states can adopt two kinds of ﬁtness functions. Given c internal states, IT ðt; gÞ = ½α1 ðt; gÞ; ⋯; αc ðt; gÞ, one evaluates the percentage of possession of each internal state and the other the win-count of each internal state in a perception scenario. The ﬁrst, the percentages of possession of the kth internal state in a perception scenario ΦIpk ð jΔT; gÞ, is deﬁned as I

Φpk ð jΔT; gÞ =

!,

Ts = ΔT

∑ αk ð jΔT; gÞ

j=1

I

Φp ð jΔT; gÞ;

ð18Þ

with the sums of the percentages of possession of all internal states

I

Ts = ΔT

Φp ð jΔT; gÞ = ∑

j=1

c

∑ αk ð jΔT; gÞ

ð19Þ

k=1

where ΦIpk ðt; gÞ is the percentage of possession of the kth internal state of a genetic robot depending on the value k, Ts the duration of a perception scenario, ΔT the sampling time of a perception scenario for each k = 1, ⋯, c. For AnyRobot, MT ðt; gÞ =

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

945

Procedure EAGRP begin t

0

i)

initializeP (t) according to a user’s preference

ii)

maskP (t)

iii)

evaluate P (t)

iv)

store the best solution b(t) among P(t)

v)

while (not termination-condition) do begin t

t +1

vi)

selectP (t) from P (t − 1)

vii)

alter P (t)

viii)

mask P (t)

ix)

evaluate P (t)

x)

store the best solution b(t ) among b(t−1) and P(t ) end

end

Fig. 19. Procedure EA for genetic robot's personality.

½m1 ðt; gÞ; ⋯; m6 ðt; gÞ = ½α1 ðt; gÞ; ⋯; α6 ðt; gÞ i n E q. ( 1 ) , HT ðt; gÞ = ½h7 ðt; gÞ; ⋯; h9 ðt; gÞ = ½α7 ðt; gÞ; ⋯; α9 ðt; gÞ, a n d ET ðt; gÞ = ½e10 ðt; gÞ; ⋯; e14 ðt; gÞ = ½α10 ðt; gÞ; ⋯; α14 ðt; gÞ. The second, the win-count of the kth internal state in a perception scenario ΦM wk ð jΔT; gÞ, is deﬁned as Ts = ΔT h i M T Φwk ð jΔT; gÞ = ∑ Wink M ð jΔT; gÞ ;

ð20Þ

j=1

where Wink ½x is 1 if the kth element in x is maximum and 0 otherwise for each k = 1, ⋯, 6. Similarly, ΦHwk ð jΔT; gÞ and ΦEwk ð jΔT; gÞ are deﬁned in the same manner. 4.4.2. Fitness candidates evaluating behaviors Fitness candidates related to behaviors can adopt three kinds of ﬁtness functions. Given a set of behaviors BT = ½b1 ; b2 ; ⋯; bz , one examines the frequency of each behavior, another the frequency of each behavior group, and the other is the frequency of each facial expression. The ﬁrst, the frequency of the rth behavior in B in a perception scenario is deﬁned as ΦBfr ð jΔT; gÞ, where the data set consists of nB = ∑zr = 1 ΦBfr observations, with the behavior br appearing ΦfrB times for each r = 1, 2, ⋯, z.

The second, given a set of behavior groups BTg = β1 ; β2 ; ⋯; βg , the percentage of the frequency of the kth behavior group in Bg in a perception scenario is deﬁned as

BG

BG

Φfk ð jΔT; gÞ = fk ð jΔT; gÞ = nBG ð jΔT; gÞ;

ð21Þ

BG where the data set consists of nBG = ∑gk = 1 ΦBG fk ð jΔT; gÞ observations, with the behavior group βk appearing fk ð jΔT; gÞ times for each for k = 1, 2, ⋯, g. As shown in Table 3, the behavior groups are classiﬁed on the basis of how each behavior group is closely related to each internal state, where each behavior has the advantage of good consistency with each internal state.

The third, given a set of facial expressions FT = π1 ; π2 ; ⋯; πf 1 , the frequency of the kth facial expression in F in a perception FE scenario is deﬁned as Φfk ð jΔT; gÞ, where the data set consists of nFE = ∑fk1= 1 ΦFE fk ð jΔT; gÞ observations, with the behavior group ϕk appearing fkFE ð jΔT; gÞ times for each for k = 1, 2, ⋯, f1.

946

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

4.5. Crossover and mutation The crossover operator Θχ is divided into the F-crossover operator, ΘχF , I-crossover operator, ΘχI , and B-crossover operator, ΘχB . These operators are performed only between parental genes of the same kind, length, and chromosomal order. For example, there F Ft are two parental robot genomes gt1 and gt2 . In case of Θχ , xFt 1k can crossover only via x 2k of the same F-genes with the length of 5 F I B genes. Based on this philosophy, Θχ, Θχ and Θχ operate together between two arbitrary parents at crossover rate pχ, where two offspring are generated from two parents (one-point crossover was used in this paper). In addition, each chromosome can have a different evolutionary convergence rate depending on the order of chromosomes in a genome. In the same manner, the mutation operator Θμ is divided into operators ΘμF, ΘμI , and ΘμB and the same method outlined above may be applied. 4.6. Experiments This section veriﬁes the feasibility and the possibility of EAGRP. First, EAGRP generates robot genomes for the personalities of both an agreeable robot and an antagonistic robot through the PS1. Second, the generated RGs are implanted into AnyRobots A and B and the PS2 are applied. The results shows well-designed personalities related to internal states. 4.6.1. Generation of robot genomes by EAGRP The ﬁtness function is deﬁned as "

14

Φð jΔT; gÞ = C−ρ ∑

k=1

I 1 = φk

14 I I B B BG φk −Φpk ð jΔT; gÞ + ∑ 1 = φk φk −Φfk ð jΔT; gÞ

# ð22Þ

k=1

with the normalized gain φkI of ψkI and the normalized gain φkB of ψ B, deﬁned by , I φk

=

I ψk

14

∑

l=1

, I ψl ;

B φk

=

B ψk

14

B

∑ ψl

ð23Þ

l=1

where ΦIpk ð jΔT; gÞ, is the percentages of possession of the kth internal state of a genetic robot in Eq. (18), ΦBG fk ð jΔT; gÞ is the percentage of the frequency of the kth behavior group in Bg in Eq. (21). C is a constant and ρ the scaling factor for percentage. I (1/φkI) are the kth I-penalty weight for boosting the convergence rate of Φpk on φkI. (1/φkB) is the kth B-penalty weight for boosting BG the convergence rate of Φfk on φkB. The parameter settings of EAGRP were applied equally in both cases of agreeable and antagonistic personalities. The population size was 10 and the number of generations was 1000. The crossover and mutation rates for the I- and B-genes were set to (0.1, 0.05) and (0.2, 0.05), respectively. F-crossover and F-mutation rates were set to 0.0, for the 3rd gene, mean value and the 4th gene, the decay value are the critical factors in F-genes that inﬂuence the evolved personality. Namely, either remarkably small decay value or big mean value makes the corresponding internal state divergent in a PS during EAGRP process. If F-genes undergo Fcrossover and F-mutation repeatedly, the corresponding RGs will get the smallest decay value but lose consistency, variability, and intensity of the preferred personality. Figs. 20(a) and 21(a) illustrates the process of generating a robot genome for an agreeable personality, RG A. This resulted in chromosomes for intimacy and happiness with strong I-genes and B-genes while chromosomes for avoidance, greed, desire to

(a) Agreeable

(b) Antagonistic

5

x 10

3.8

Fitness

Fitness

3.5

x 10

3.4 3.3

5

3.6

3.4

3.2 3.1

3.2

3 3

2.9 2.8

2.8

2.7

Best Average

2.6 2.5

0

100

200

300

400

500

600

700

800

900

Best Average

2.6

1000

2.4

0

100

200

Generation Fig. 20. Best ﬁtness and average ﬁtness of EAGRP.

300

400

500

600

700

800

900

1000

Generation

(b) F-genes

(c) I-genes

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14

Gene value

350 400

300

300

250

200

200

100

150 100

0 1 15

2

50

10

3 5

4

F-gene

5 0

0

Chromosome

Fig. 21. A generated RG A: agreeable.

(d) B-genes

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(a) Agreeable RG A

947

948

(b) F-genes

(c) I-genes

C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14

450 400

500

350

Gene value

400

300

300

250

200

200 100 150 0 1

100 2

15 3

10 4

F-gene

5 5 0

50 0

Chromosome

Fig. 22. A generated RG B: antagonistic.

(d) B-genes

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(a) Antagonistic RG B

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

949

control, anger and fear have weak I-genes and B-genes. Despite providing an intuitive starting point, these settings didn't always guarantee a viable set of emotive responses for an agreeable personality in a complex test environment. They do however, provide an invaluable starting point for optimizing the robot genome for a more consistent personality with EAGRP. The progress of the EAGRP is shown in Fig. 20(a), where a steady improvement in performance is quantitatively demonstrated. Fig. 21(a) illustrates the robot genome after EAGRP, indicating signiﬁcant changes to the robot genome's structure. Similarly Figs. 20(b) and 22(a) illustrate the same process of EAGRP for a robot genome that has been initialized for an antagonistic personality. Figs. 21(b), (c), (d) and 22(b), (c), (d) are F-genes, I-genes, and B-genes of the RG A and the RG B. Figs. 21(a) and 22(b) show that the mean values (the 3rd F-genes) and the decay values (the 4th F-genes) reﬂect a user's preference. The 4th F-gene values of c2 and c10 are very low in Fig. 21(b). Similarly, the 4th F-gene values of c4 , c5 , c6 , c12 , and c13 are very low in Fig. 22(b). The F-genes of the internal state with high preference have high mean value and low decay rate. They are generated and restricted by the initialization procedure of EAGRP, since the preferred internal states diverge and lose their own personality and variability if the decay rate is excessively high. A user's preference, the applied PS, and EAGRP affect the magnitudes of negative I-genes values, as shown in Figs. 21(c) and 22(c). These characteristics enable generated personalities to have variability as much as possible.

4.6.2. Implantation of robot genomes into genetic robot This section veriﬁes the concept of genetic robot by implanting the agreeable RG A and the antagonistic RG B generated by EAGRP into two genetic robots, AnyRobots A and B. Figs. 23 and 24 show the results related to percentages of internal states in the simulations to which the agreeable RG A and the antagonistic RG B generated by EAGRP are applied.

(b) Homeostasis 15

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

50

100

150

200

250

300

350

400

450

500

500 0 0 1000

5 0

100 0 0 1000

10

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

30

Drowsiness

0 0 200

20 10 0

50

100

150

200

250

300

350

400

450

500 30

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

500 0 0

50

100

150

200

250

300

350

400

450

Hunger

Control

Greed

Avoidance Monotony Intimacy

Curiosity

(a) Motivation

20 10 0

500

Time (sec)

Time (sec)

Fear

100

150

200

250

300

350

400

450

500

Percentage

50

500 0 0 1000

Homeostasis

Emotion

80

500 0 0 1000

Motivation

100

1000

60 40 20

50

100

150

200

250

300

350

400

450

500

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Internal state 500 0 0 1000

Neutral

(d) Percentage of each internal state

100

150

200

250

300

350

400

450

Emotion

4000

50

100

150

200

250

300

350

400

450

500

3000 2000 1000

500 0 0

Homeostasis

500

500 0 0 1000

Motivation

5000 50

Win−count

Anger

Sadness

Happiness

(c) Emotion

0 50

100

150

200

250

300

350

400

450

500

1

2

3

4

5

Time (sec)

Fig. 23. Internal states of the agreeable genetic robot A with the RG A.

6

7

8

Internal state

9

10

11

12

13

14

950

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(b) Homeostasis 15

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

50

100

150

200

250

300

350

400

450

500

Fatigue

1000

500

50

100

150

200

250

300

350

400

450

500

500 0 0 1000

5

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

0

50

100

150

200

250

300

350

400

450

500

30

100 0 0 1000

10

0

Drowsiness

0 0 200

20 10 0

50

100

150

200

250

300

350

400

450

500 30

500 0 0 1000

50

100

150

200

250

300

350

400

450

500

500 0 0

50

100

150

200

250

300

350

400

450

Hunger

Control

Greed

Avoidance Monotony Intimacy

Curiosity

(a) Motivation

20 10 0

500

Time (sec)

Time (sec)

100

500

80

0 0 1000

Fear

50

100

150

200

250

300

350

400

450

500

500

Percentage

1000

Motivation

Homeostasis

Emotion

60 40 20

0 0 1000

50

100

150

200

250

300

350

400

450

500

0

1

2

3

4

5

6

7

100

150

200

250

300

350

400

450

9

10

11

12

13

14

Homeostasis

Emotion

500 4000

50

100

150

200

250

300

350

400

450

500

3000 2000 1000

500 0 0

Motivation

5000 50

500 0 0 1000

8

Internal state

500 0 0 1000

Neutral

(d) Percentage of each internal state

Win−count

Anger

Sadness

Happiness

(c) Emotion

0 50

100

150

200

250

Time (sec)

300

350

400

450

500

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Internal state

Fig. 24. Internal states of the antagonistic genetic robot B with the RG B.

Fig. 23(a) shows that curiosity and intimacy states have a wider distribution than avoidance, greed, and desire to control in motivation for the PS time, 500 s. Fig. 23(c) shows that happiness state has the widest distribution among emotions. Fig. 23(d) shows the percentages and the win-counts of each internal state. The horizontal axis represents the index of 14 internal states and the vertical axis represents the percentages of possession of each internal state, which are calculated individually by motivation, homeostasis, and emotion. The maximum count of the win-counts is 5000 since the sampling time is 0.1 s and each win-count is sampled out at every sampling time for the PS time. If several internal state have same value at every sampling time, their wincounts increases simultaneously. Referring to percentages of possession at the top of the ﬁgure, the 1st, 2nd, and 10th internal states have high percentages of possession which indicate strong curiosity and intimacy in motivation and happiness in emotion. On the other hand, the 4th, 5th, 6th, 12th and 13th internal states have respectively low percentages of possession which indicate weak avoidance, greed, and desire to control in motivation, and anger and fear in emotion. Win-count results at the bottom of the ﬁgure show similar tendency, which validates this possibility as a ﬁtness suitable for EAGRP. Fig. 24(a) shows that avoidance, greed, and desire to control states have wider distribution than curiosity and intimacy states in motivation for the PS time. Fig. 24(c) shows that sadness, anger, and fear states have wider distribution than happiness in emotion. Fig. 24(d) shows the percentages and the win-counts of each internal state. Referring to the percentages of possession at the top of the ﬁgure, the 4th, 5th, 6th, 11th, 12th and 13th internal states have high percentages of possession which mean strong avoidance, greed and desire to control in motivation, and sorrow, anger, and fear in emotion, while the 1st, 2nd, 3rd, 10th and 14th internal states have respectively low percentages of possession which indicate weak curiosity, intimacy, and monotony in motivation, and happiness and neutral in emotion. Besides, the robot genome (individuals) is continuously optimized by evolutionary process. Even a much simpler model can be worked equally well. The result of homeostasis shows such a characteristics.

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

951

5. Evaluation of the algorithm 5.1. Veriﬁcation related to external behaviors Figs. 25 and 26 show the results related to facial expressions, genetic robot's movements, and behaviors in the simulations to which the agreeable RG A and the antagonistic RG B generated by EAGRP are applied. Fig. 25(a). shows the indexes of behaviors and facial expressions of the agreeable genetic robot A for the PS time. Behaviors are indexed sequentially based on Table 2. The indexes of 5 facial expressions, neutral, happiness, sorrow, anger, and fear are set to 0, 1, 2, 3, and 4. In Fig. 25(a), there are much more facial expressions of happiness than other kinds of facial expressions, which reﬂect the agreeable personality. Fig. 25(b). the trajectories of a face, objects 1, 2, 3, and the genetic robot (AnyRobot) A or Sobot A for the PS time. If the genetic robot A is in front of a user's face, objects 1, 2, or 3, it reacts to them and moves by taking various behaviors as shown in Tables 2 and 3. Those reactions can be considered together in Fig. 25(c). which shows the frequencies of individual behaviors, group behaviors by the

(a) Behavior and face

(b) Trajectories 30 Face Object1 Object2 Object3 Sobot

20 10

y

0 -10 -20 -30 -40 -30

-20

-10

0

10

20

30

x

(c) Frequencies of individual behaviors, group behaviors by the grouping method 1 in Table II, and group behaviors by the grouping method 2 in Table III 50

Frequency

40 30 20 10 0

140

0 1. Stop

10

20

30

40

Behavior

2. Move

4. Anger 6. Fear 8. Monotony 3. Happiness 5. Sadness 7. Shame 9. Positive to user

50

60

10. Physiology

11. Search

70 12. With target

80

14. Information 13. Control

Frequency

120 100 80 60 40 20 0

0

10

20

30

40

50

60

70

80

Behavior group 1. Motivation

100

2. Homeostasis

3. Emotion

4. Objects

5. Sobots

6. User

Frequency

80 60 40 20 0

0

5

10

15

20

Behavior group

Fig. 25. External behaviors of the agreeable genetic robot A with RG A.

25

30

952

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

(a) Behavior and face

(b) Trajectories 30 Face Object1 Object2 Object3 Sobot

20

y

10

0

-10

-20

-30 -30

-20

-10

0 x

10

20

30

(c) Frequencies of individual behaviors, group behaviors by the grouping method 1 in Table II, and group behaviors by the grouping method 2 in Table III 50

Frequency

40 30 20 10 0

0

10

Frequency

150

20

30

40

Behavior

2. Move

4. Anger 6. Fear 8. Monotony 3. Happiness 5. Sadness 7. Shame 9. Positive to user

1. Stop

50

60

10. Physiology

70 12. With target

11. Search

80

14. Information 13. Control

100

50

0

0

10

20

30

40

50

60

70

80

Behavior group 1. Motivation

100

2. Homeostasis

3. Emotion

4. Objects

5. Sobots

6. User

Frequency

80 60 40 20 0

0

5

10

15

20

25

30

Behavior group

Fig. 26. External behaviors of the antagonistic genetic robot B with RG B.

grouping method 1 in Table 2, and group behaviors by the grouping method 2 in Table 3. The behaviors at the top of the ﬁgure are matched directly to the behavior groups by the method 1 at the middle. The behavior groups by the method 2 at the bottom are indexed sequentially by Table 3. The graph at the middle represents that the frequencies of the behaviors belonging to the groups such as ‘4: anger,’ ‘5: sadness,’ ‘6: fear,’ and ‘13: desire to control’ are very low. High frequencies of the group ‘9: positive to user’ represent that the genetic robot A is intimate with a user and high frequencies of the group ‘12: with target’ represent that the genetic robot A took actions for a face and objects 1 and 2 with strong intimacy as shown in Fig. 25(b). The graph at the bottom represents that the frequencies of the behaviors belonging to the groups such as ‘1: Motivation-curiosity,’ ‘intimacy,’ ‘4: ObjectsLike,’ ‘5: Sobots-Like,’ ‘6: User-FriendlyNeutral’ are high. Consequently, the analysis based on these two grouping methods shows that the generated RG A reﬂects the agreeable personality well. Fig. 26(a) shows the indexes of behaviors and facial expressions of the antagonistic genetic robot B for the PS time. In the ﬁgure, there are much more facial expressions of sorrow, anger, and fear than facial expressions of happiness, which reﬂect the antagonistic personality. Fig. 26(b) the trajectories of a face, objects 1, 2, 3, and the genetic robot B or Sobot B for the PS time. Those reactions can be considered together in Fig. 26(c). The graph at the middle represents that the frequencies of the behaviors belonging to the groups such as ‘4: anger,’ ‘5: sadness,’ ‘6: fear,’ and ‘13: desire to control’ are very high respectively compared to

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

Curiosity Intimacy Drowsiness Hunger Look_around 11. Call 12. Sit down 13. No 14. Good 15. Let us go 16. Pat

Current command Internal state Current Behavior Command

953

Happiness Sadness Anger Fear Hurrah 61. Shake 62. Tab 63. Surprise 64. Lift 65. Hit

Fig. 27. Two types of genetic robots mounted on the SPH-B3600. This ﬁgure illustrates the result of two genetically evolved personalities. The ﬁrst represents an antagonistic personality type, while the second is an agreeable type. The differences are primarily reﬂected in their internal state and their behaviors.

the frequencies of the behaviors in Fig. 25(c). Low frequencies of the groups ‘9: positive to user’ and ‘12: with target group’ reﬂect the antagonistic personality of the RG B for objects 1, 2 and a face. Considering this result together with Fig. 26(b), much movement of the Sobot B can be thought of being resulted from frequent sorrow, fear, etc. The graph at the bottom represents that the frequencies of the groups such as ‘1: Motivation-avoidance, greed,’ ‘3: Emotion-Anger, Fear, Surprise,’ ‘4: Objects-Dislike,’ ‘5: Sobots-Dislike,’ ‘6: User-UnfriendlyHit, UnfriendlyPat groups’ are very high. These experimental results prove that the RG B generated by EAGRP reﬂects the antagonistic personality well. All the results show the consistency in emotional reactions and motivational states as well as in the observable behaviors associated with such reactions and state. Genetic Robots do not always react in the same way in the same kind of perception and internal states, which means the variability with in consistency. Finally, all the behavioral outputs and internal state outputs shows the plausibility with comparable intensities. 5.2. Evaluation of the algorithm This section veriﬁes the robustness of EAGRP with respect to two key points. 5.2.1. Evolutionary stability The algorithm was tested repeatedly for each personality. In 100% of cases, the ﬁtness function increased in value and demonstrated convergence. The population variance in all samples was on average 6.9% of the population mean and was never greater than 11% of the population. This indicates that the algorithm population was not unstable, but also sufﬁciently able to evolve. 5.2.2. Human plausibility In one experiment, thirty randomly selected humans were requested to interact with both agreeable and antagonistically generated pet types (refer to Fig. 27) for 2 min. Upon conclusion, they gave a mark between 0 (low) and 10 (high) for the following questions applied to both agreeable and antagonistic types (agreeable type, antagonistic type): 1–0. 1–1. 1–2. 1–3.

How plausible is the personality? (8.9, 9.35) Are the facial expressions believable? (8.7, 8.95) Is the internal state performing in an expected manner? (8.4, 8.7) Are the behaviors appearing properly? (8.3, 7.9)

All responses averaged between 7.9 and 9.35. These statistical results provide a conﬁdence in the truth that the RGs A and B generated by EAGRP reﬂect their personalities well. 6. Conclusions This paper revised the robot genome as the basic building block for an artiﬁcial life form. Using these building blocks, evolutionary tools and mechanisms have been developed upon this concept that enables the genetic robot with a richness and complexity in personality. These tools include the proposed NNGRP and extended EAGRP. Both tools demonstrated good performance in generating the robot genome with a personality that was both complex and feature-rich and still plausible by human standards for an emotional life form. EAGRP showed a comparative advantage especially in terms of variability and intensity. Consequently, it is necessary for gene-generative algorithms such as that presented here to be more robust and diverse in order for them to be applicable in a wide variety of situations. However NNGRP has the advantage that it has high convergence rate and fast training speed. This lends itself to the possibility of being applied simply and practically in various runtime instances where computation power is not sufﬁcient for evolutionary processes. Another possible future direction is to consider the passing of the robot genome between ubiquitous robots. This data may easily move within a network and connect to other systems without any time or geographical limitations. This concept is

954

K.-H. Lee / Data & Knowledge Engineering 70 (2011) 923–954

fundamental to the idea of the ubiquitous and networked robot. The robot genome and its masking vectors for evolving speciﬁc types of robot personality may be exchanged and evolved as genes and memes [1,31] in the user community of robots on the mobile platform, the real robot platform, or the heterogeneous platform. Acknowledgement This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (grant number: 2011-0015064). References [1] R. Dawkins, The Selﬁsh Gene, The Oxford Publication Press, 1976. [2] R. Dawkins, The Blind Watchmaker, Longman, Harlow, 1986. [3] A. Sloman, Emotions in Humans and Artifacts, MIT Press, Cambridge, MA, 2002, pp. 145–146, chapter Designing Emotions for Activity Selection in Aunomous Agnets. [4] E. Roll, Emotions in Humans and Artifacts, MIT Press, Cambridge, MA, 2002, pp. 17–23, chapter A Theory of Emotion, Its Functions, and Its Adaptive Value. [5] B.M. Blumberg, Old tricks, new dog: Ethology and interactive creatures, Ph.D. dissertation, MIT, Cambridge, MA, 1996. [6] R. Arkin, M. Fujita, T. Takagi, R. Hasekawa, An ehtological and emotional basis for human –robot interaction, Robot. Auton. Syst. 42 (2003) 191–201. [7] H. Miwa, T. Umetsu, A. Takanishi, H. Takanobu, Robot personality based on the equation of emotion deﬁned in the 3D mental space, Proc. of the IEEE International Conference on Robotics and Automation, 2001, pp. 2602–2607. [8] C. Breazeal, Designing Sociable Robots, MIT Press, Cambridge, MA, 2002. [9] C. Elliott, J. Brzezinski, Autonomous agents as synthetic characters, AI Mag. 19 (2) (1998) 13–30. [10] J.-H. Kim, K.-H. Lee, Y.-D. Kim, The origin of artiﬁcial species: Genetic robot, Int. J. Control. Autom. Syst. 3 (4) (2005) 564–570. [11] A. Ortony, Emotions in Humans and Artifacts, MIT Press, Cambridge, MA, 2002, pp. 189–203, chapter On Making Believable Emotional Agents Believable. [12] M. Minsky, The Emotion Machine: Commonsense Thinking, Artiﬁcial Intelligence, and the Future of the Human Mind, Simon & Schuster, Nov 2007. [13] C. Gershenson, Modelling emotions with multidimensional logic, Proc. of the 18th International Conference of the North American Fuzzy Information Processing Society (NAFIPS '99), 1999, pp. 42–46, New York City, NY. [14] P.T. Costa, R.R. McCrae, The NEO Personality Inventory Manual, Psychological Assessment Resources, Odessa, FL, 1985. [15] R.R. McCrae, P.T. Costa, Validation of a ﬁve-factor model of personality across instruments and observers, J. Pers. Soc. Psychol. 52 (1987) 81–90. [16] C. Breazeal, Function meets style: Insights from emotion theory applied to HRI, IEEE Trans. Syst. Man Cybern. C 34 (May 2004) 187–194. [17] J.-H. Kim, C.-H. Lee, K.-H. Lee, Evolutionary generative process for an artiﬁcial creature's personality, IEEE Trans. Syst. Man Cybern. C 39 (3) (2009) 331–342. [18] L.D. Cañamero, Emotions in Humans and Artifacts, MIT Press, Cambridge, MA, 2002, pp. 132–139, chapter Designing Emotions for Activity Selection in Aunomous Agnets. [19] Zhengxing Huang, W.M.P. van der Aalst, Xudong Lu, Huilong Duan, Reinforcement learning based resource allocation in business process management, Data Knowl. Eng. 70 (1) (January 2011) 127–145, doi:10.1016/j.datak.2010.09.002, http://dx.doi.org/10.1016/j.datak.2010.09.002. [20] Yi Cheng, The incremental method for fast computing the rough fuzzy approximations, Data Knowl. Eng. 70 (1) (January 2011) 84–100, doi:10.1016/j. datak.2010.08.005, http://dx.doi.org/10.1016/j.datak.2010.08.005. [21] M. Mitchell, Genetic algorithms and artiﬁcial life, Artif. Life 1 (3) (1994) 267–289. [22] K.-H. Lee, K.-C. Kim, H.-S. Shim, J.-K. Kim, J.-C. Lee, Ubiquitous robot S/W platform: AnyRobot Studio 1.0, The Workshop on Ubiquitous Robotic Space Design and Applications, Proc. of the IEEE International Conference on Intelligent Robots and Systems, San Diego, USA, Nov. 2007, 2007. [23] K.-H. Lee, H.-S. Shim, W.-S. Han, K.-C. Kim, K.-C. Park, Ubiquitous robot S/W platform and its application: AnyRobot Studio and AnyKids Service, Proc. of the 17 IEEE International Symposium on Robot and Human Interactive Communication, Munich, Germany, Aug. 2008, 2008. [24] Y.-D. Kim, Y.-J. Kim, J.-H. Kim, Behavior selection and learning for synthetic character, Proc. of the IEEE Congress on Evolutionary Computation, 2004, pp. 898–903. [25] K.-H. Lee, “Evolutionary Algorithm for Genetic Robot's Personality,” Ph.D. thesis, 2006. [26] A. Stern, Emotions in Humans and Artifacts, MIT Press, Cambridge, MA, 2002, p. 353, chapter Creating Emotional Relationships with Virtual Characters. [27] A. Ortony, On making believable emotional agents believable, in: R. Trappl, P. Petta, S. Payr (Eds.), Emotions in Humans and Artifacts, The MIT Press, Cambridge, 2002, pp. 189–191. [28] K. Levenberg, A method for the solution of certain problems in least squares, Q. Appl. Math. 2 (1944) 164–168. [29] D. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, SIAM J. Appl. Math. 11 (1963) 431–441. [30] P.R. Gill, W. Murray, M.H. Wright, The Levenberg –Marquardt Method, Section 4.7.3 in Practical Optimization, Academic Press, London, 1981, pp. 136–137. [31] Alexandre Evﬁmievski, Ronald Fagin, David Woodruff, Epistemic privacy, J. ACM 58 (1) (December 2010) 45, doi:10.1145/1870103.1870105 Article 2, http:// doi.acm.org/10.1145/1870103.1870105. Kang-Hee Lee received the B.S., M.S., and Ph.D degrees in electrical engineering and computer science from Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea, in 1999, 2001, and 2006, respectively. Since 2006, he has been a Senior Engineer in Digital Media & Communication Research Center, Samsung Electronics Company, Ltd., Korea. He has been a dispatched researcher in the Robotics Institute, Carnegie Mellon University in 2008. Since moving to Soongsil University in 2009, he is with the Department of Global School of Media, Soongsil University, Seoul, Korea. His current research interests include the areas of ubiquitous robotics, evolutionary robotics, media robotics, cognitive task planning system, and knowledge-based reasoning system.

A Second-Generation Genetic Linkage Map of Tilapia ...

The molecular genetic architecture of human personality

Exploring the role of personality in the relationship ...

Walking Trajectory Generation for Humanoid Robots ...

Evolution of Millon's Personality Prototypes