Active Behavior Recognition in Beyond Visual Range Air Combat Ron Alford1

Hayley Borck2

Justin Karneeb3

David W. Aha4

1 ASEE/NRL Postdoctoral Fellow [email protected] 2 Knexus Research Corporation [email protected] 3 Knexus Research Corporation [email protected] 4 U.S.

Naval Research Laboratory [email protected]

May 31st, 2015 Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

1 / 17

Beyond Visual Range Air Combat

100km $300 million planes

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

2 / 17

Beyond Visual Range Air Combat

100km $300 million planes $10 million sensor package Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

2 / 17

Beyond Visual Range Air Combat

100km $300 million planes $10 million sensor package $1 million $ missiles Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

2 / 17

Adding UAV wingmen to the mix

The Promise: More platforms per pilot Better strategies

Reduced pilot risk Retain (most) human judgment The Caveat: Pilot is already cognitively burdened UAV needs to respond (or act) intelligently

Source: Dassault Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

3 / 17

Some obstacles to intelligent behavior

Partial-observability Continuous action space Multi-agent (non-zero-sum)

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

4 / 17

Some obstacles to intelligent behavior

Partial-observability Full-observability Continuous action space Discrete action space Multi-agent (non-zero-sum) Single-agent with fixed (unknown) opponent

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

4 / 17

Behavior Recognition (Assumptions) Aggressive Assumptions: Finite set of predictive agent models Used in training recognizer Used to predict future states

Safety-Aggressive

Agents use fixed polices React to history of observations Not rational nor optimal

Behavior Recognition (generic): Inputs: Agent models History of observations

Passive

Oblivious

Output: A probability distribution over the models. Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

5 / 17

Acting depends on behavior recognition Almost all actions in air combat are dependent (or relative) to other agents. Safety-Aggressive vs. Aggressive

Safety-Aggressive vs. Oblivious

Safety-Aggressive vs. Safety-Aggressive

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

6 / 17

Behavior recognition depends on acting Our actions determine what we observe. Fly 90 Fly 0 Safety-Aggressive Aggressive

Aggressive Oblivious

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

7 / 17

Case-based Behavior Recognition

The rough algorithm: During training:

φ

Run a number of randomized trials Project states to a feature space Record short histories of features and their associated models as cases

During recognition:

θ

d

Retrieve cases with similar histories Treat the relative frequency of agent models as a probability distribution

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

8 / 17

How acting influences Case-based Behavior Recognition



 A 0.48  SA 0.48  Ob 0.04

 

 A 0.03  SA 0.03  Ob 0.94

Alford, Borck, Karneeb and Aha

 A 0.33  SA 0.33  Ob 0.33 A Aggressive SA Safety-Aggressive Ob Oblivious

Active Behavior Recognition in BVR Combat

May 31st, 2015

9 / 17

How acting influences Case-based Behavior Recognition

A Aggressive SA Safety-Aggressive Ob Oblivious



 A 0.03  SA 0.94  Ob 0.03

 

 A 0.48  SA 0.04  Ob 0.48

Alford, Borck, Karneeb and Aha

 A 0.33  SA 0.33  Ob 0.33

Active Behavior Recognition in BVR Combat

May 31st, 2015

10 / 17

How acting influences Case-based Behavior Recognition

Acting and Behavior Recognition: Head-long flight disambiguates Safety-Aggressive Perpendicular flight disambiguates Oblivious Need both to make a confident prediction

Similar to a POMDP Move with uncertainty about other agents’ behaviors Our actions give evidence about that behavior POMDPs are hard

Approximate as an MDP

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

11 / 17

Planning Domain Plan over histories of observations Observations divided into 60 second epochs Actions: Four discrete actions Four possible outcomes (agent models) Probability dependent on behavior recognizer and current history

Use flight simulator (AFSIM) applying action to a history Purpose: Maximize a utility function over finite horizon

Alford, Borck, Karneeb and Aha

Fly 0◦

Fly 60◦

Fly 90◦

Fly 180◦

Active Behavior Recognition in BVR Combat

May 31st, 2015

12 / 17

Sample-based planning (PROST) 

A  SA Ob Fly 0

A,0.33

 0.33 0.33  0.33

Fly 90

Ob,0.33

Fly 0



A  SA Ob

 0.48 0.04  0.48

Fly 90

A,0.48

Ob,0.48 

A  SA Ob

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

 0.02 0.02  0.96

May 31st, 2015

13 / 17

Meta-goal reasoning We require a utility function! Possible mission success functions: Number of “kills” Air space denied Reconnaissance Diversion

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

14 / 17

Meta-goal reasoning We require a utility function! Possible mission success functions: Number of “kills” Air space denied Reconnaissance Diversion

Road blocks Roll-outs (simulation) are slow Success functions are often discontinuous

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

14 / 17

Meta-goal reasoning We require a utility function! Possible mission success functions: Number of “kills” Air space denied Reconnaissance Diversion

Road blocks Roll-outs (simulation) are slow Success functions are often discontinuous

Instead: Average confidence in most likely model Confidence is generally smooth Emphasize the role of planning in resolving recognition ambiguity

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

14 / 17

Experimental Setup Safety-Aggressive Four different observer behaviors running the behavior recognizer: Safety-Aggressive Passive Random Active behavior recognition planner

Passive

Random

Evaluation metric: Confidence in correct behavior over time.

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

15 / 17

Recognition Results Behavior Probability Over Time - All Behaviors Behavior Probability

1 0.9 0.8

Active Planner

0.7

Random Baseline

0.6

Passive Baseline

0.5

Safety Aggression Baseline 480

460

440

420

400

380

360

340

320

300

280

260

240

220

200

180

160

140

120

80

100

60

40

0.4 Time in Simulation (Seconds)

Both Safety-Aggressive and Passive fail to disambiguate between two behaviors Random eventually distinguishes between all behaviors Planning gets good (>90%) recognition scores faster Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

16 / 17

Conclusion / Future Work

Behavior Recognition and Acting: Probabilistic recognition pairs well with probabilistic planning Need faster roll-outs to persue mission success Discrete states, actions, and policies

Game theoretic play (regret minimization) When do we need a separate behavior recognition component?

Alford, Borck, Karneeb and Aha

Active Behavior Recognition in BVR Combat

May 31st, 2015

17 / 17