Agent Design Consistency Checking via Planning Nitin Yadav, John Thangarajah, and Sebastian Sardina University of Melbourne and RMIT University th In 26 International Joint Conference on Artificial Intelligence, Melbourne, Australia, 2017. The Agent Design Consistency Checking Problem

Consistency Checking via Planning: Acyclic Designs Types: goal (g), plan (p), step (s) Actions: Predicates :action handleGoal :par (?g - goal ?p • Requirements

- plan) :pre active-goal(g)∧can-handle(p,g) ∧ ¬canProgress :eff ¬active-goal(g) ∧ active-plan(p)

Scenario Goal Overview •

next ?s ?s step-goal ?s ?g child-goal ?child ?parent :action generateGoal goal-and ?g :par (?p - plan ?g - goal) goal-or ?g :pre active-plan(p) ∧ sub-goal(g,p) ∧ ¬goal-generated(g,p) Agent Detailed Design ∧ ¬canProgress :eff goal-generated(g,p)∧active-goal(g) can-handle ?p ?g ∧ ¬ accounted(g)))) sub-goal ?g ?p :action progressStep :par (?s - step ?g - goal) :pre current(s) ∧ met-goal(g) ∧ step-goal(s,g) ∧ ¬accounted(g) :eff ¬ current(s) ∧ ∀x (next(s,x)→current(x)) ∧ ∀x (met-goal(x)→accounted(x)

• Tracking active-goal ?g active-plan ?p goal-generated ?g ?p current ?s accounted ?g

Agent Detailed Design How to automatically check whether an agent design is consistent with the requirements (scenario + goal overview) of the system? Problem Definition

Derived Predicates: met-goal ?g - goal:(∃p ( active-plan(p) ∧ can-handle (p,g))) ∨ (goal-or(g) ∧ (∃x (met-goal (x) ∧child-goal(x,g))))∨ (goal-and(g)∧ (¬(∃x (¬met-goal(x) ∧ child-goal(x,g))))) canProgress:∃s,g(current(s)∧met-goal(g)∧step-goal(s,g)∧¬ accounted(g))

Requirement Traces: A trace realizes a specification if it is a concatenation of sequences achieving each scenario step.

Extensions: Cyclic Designs & Repair Hints

τ1 =PurchaseRequest-CheckInventory-GetPayment-GetPayment-SendItem

Cyclic Designs: A pending-goal function to keep track of active goals

τ2 =PurchaseRequest-CheckInventory-PaymentRequest-ProcessPaymentGetPayment-SendItem τ3 =PurchaseRequest-CheckInventory-PaymentRequest-ProcessPaymentPaymentRequest-ProcessPayment-SendItem Design Runs:A design run is an interleaving of various plan runs’ such that each posted goal is handled by at most one plan. τ1 =PurchaseRequest-ProcessPurchaseOrderPlan-GetPayment-SendItem τ2 =PurchaseRequest-ProcessPurchaseOrderPlan-SendItem-GetPayment τ3 =SendItem-SendGoodsPlan τ2 ||τ3 =PurchaseRequest-ProcessPurchaseOrderPlan-SendItem-SendItemSendGoodsPlan-GetPayment

Is there a run of given agent design that is consistent with the requirements?

:action handleGoal :par (?g - goal ?p - plan) :pre: pending-goal(g)>= 1 ∧ can-handle(p,g) ∧ ¬active-plan(p) :eff (pending-goal(g)=-1) ∧ active-plan(p) :action executePlan :para (?p - plan) :pre active-plan(p) :eff ¬accounted(g)∧∀x(sub-goal(x,p)→pending-goal(x)=+1∧¬active-plan(p))

Repair Hints via two actions: - skipStep: Skip the current scenario step - swapStep: Swap two scenario steps

Empirical Evaluation Coverage 3

g/p : 2

4

p/g : 2

Previous Approaches & Our Contributions • Trace based approach [Abushark et al., 2014, 2015]:

3

– Algorithmic. – Scales poorly with increase in the size of agent design. – Unknown if the approach is complete.

4

• Model checking based approach [Yadav et al., 2016]: FastDownward McMAS NuSMV Percentage of instances completed in 10 minutes.

– Based on formal semantics (solution via reduction to ATL). – Scales better than trace based approach (but not that efficient). – Sound and complete.

Time comparison (2-2-8) 512.00

Proposed Planning-based Approach

2. Handles flexible goal hierarchies. 3. Provides debugging hints to the designer.

Time in sec (log)

1. Allows detailed agent designs to have cycles.

128.00 32.00 8.00 2.00 0.50

0.05

0

10

20

30

40

Case number

4. Yields efficient & scalable approach (tested on 16K+ random instances).

Scenario Length

2

4

6

8

Technique

fd

nmv

mcm

50