Formal Consistency Checking over Specifications in Natural Languages Rongjie Yan1, Chih-Hong Cheng2, Yesheng Chai1,3 1 State Key Laboratory of Computer Science, Institute of Software, China 2 Industrial Software Technologies, ABB Corporate Research, Germany 3 School of Computer Science & Technology, Soochow University, China
It’s all about engineering efficiency • Specifications concretize vague ideas how a system or a component should behave, often represented in the form of natural languages. • Early validation and verification (V&V) can reveal potential problems early • Validation – is the specification describing what we want?
13-Mar-15
http://www.bowdoin.edu/~disrael/what-the-customer-really-needed/what-the-customer-really-needed.jpg
1
Summary: What do we do in this work? • SpecCC – a tool to synthesize control components from specifications in natural language • Reports “inconsistent” when no implementation is possible
• Benefits: early V&V via • Checking inconsistencies in specification • Observing the behavior of the synthesized components, to infer if one needs to modify the specification
• How? • Heuristic translation natural language spec to formal spec • Algorithmic synthesis from formal spec to implementation (covered by [CAV’14]) 13-Mar-15
2
Outline • Background • Maintaining consistencies between natural language and formal language • Maintaining consistencies between formal language and implementability • Implementation and evaluation • Conclusion
13-Mar-15
3
Background • Natural languages: • a rich diversity of structures – we work on structured English • semantic ambiguities
• Linear temporal logic: • a temporal logic, • ϕ::=p | ~ϕ | ϕ∨ϕ | Xϕ | Fϕ | Gϕ |ϕUϕ • expressing properties of paths in a computation tree. neXt in Future 13-Mar-15
Xϕ
Fϕ
Globally
ϕ ϕ
Until
Gϕ ϕ φUϕ
φ
ϕ φ
ϕ
ϕ
φ
LTL looks intimidating at first but they are natural in essence!
ϕ
ϕ 4
Maintaining consistencies between natural and formal languages A structured English Lexical and syntactic parsing Syntax tree Basic syntax processing
Rule matching LTL formula
Semantic reasoning
Massaging the formula for meaningful & efficient synthesis
Reduced LTL formulas Time abstraction Formula ready for control synthesis engine 13-Mar-15
Input and output partition
Reduced LTL formulas 5
Lexical and syntactic parsing • Dependency relation extraction -> Rule matching -> Formula generation Example: When auto-control mode is entered, eventually the cuff will be inflated.
implication F
inflate_cuff
enter_auto-control_mode
G (enter_auto-control_mode -> F inflate_cuff) 13-Mar-15
6
Rationale behind post-processing • Semantic reasoning: understand “off == !on” • Time abstraction: avoid using 100neXt operators in synthesis • I/O partition: An LTL formula has no I/O defined! LTL formula
Semantic reasoning
Massaging the formula for meaningful & efficient synthesis
Reduced LTL formulas Time abstraction Formula ready for control synthesis engine 13-Mar-15
Input and output partition
Reduced LTL formulas 7
Semantic reasoning – antonym If pulse wave or arterial line is available, and cuff is selected, corroboration is triggered. Antonym subject antonym candidates
candidate
Select a candidate
Look for its antonyms
wordset antonym candidates corresponding antonyms
Intersection of two sets
Empty?
Yes
No antonym exists
No Antonyms exist for the given candidate 13-Mar-15
8
Semantic reasoning: example • If pulse wave or arterial line is available, and cuff is selected, corroboration is triggered • If pulse wave and arterial line are unavailable, and cuff is selected, and blood pressure is not valid, next manual mode is started. pulse wave
wordset
available unavailable
antonym candidates
antonyms
available
unavailable
unavailable
13-Mar-15
available
available_pulse_wave
available_pulse_wave
unavailable_pulse_wave
~ available_pulse_wave 9
Time counting and abstraction • Number of time unit is convert to Next operators • Θ={θ0,…, θn} be a set of numbers of successive Next operators, θi’ be the number after abstraction, where θi=θi’x d + Δi • Greatest common divisor based reduction • d is the greatest common divisor of Θ and Δi=0 • Error bound constrained abstraction • minimize , minimize
13-Mar-15
To reduce the complexity of LTL formulas
To reduce the error bound
10
Time counting and abstraction Req-08 If Air Ok signal remains low, auto-control mode is terminated in 3 seconds. Req-28 If a valid pressure is unavailable for 180 seconds, manual mode should be triggered. Req-42 When auto-control mode is running, and the arterial line or pulse wave or cuff is lost, an alarm should sound in 60 seconds. According to the three requirements, Θ={3, 180, 60} After reduction: Θ’={1,60,20}, where d = 3 •Req-08 G(~Air_ok_signal -> (terminate_auto-control_mode || X terminate_auto-control_mode )). •Req-28 • We still need to monitor 60 neXt! • ~ avail && X ~ avail && XX ~ avail ….. XXXXXXXXX…X ~ avail 13-Mar-15
11
Time counting and abstraction Req-08 If Air Ok signal remains low, auto-control mode is terminated in 3 seconds. Req-28 If a valid pressure is unavailable for 180 seconds, manual mode should be triggered. Req-42 When auto-control mode is running, and the arterial line or pulse wave or cuff is lost, an alarm should sound in 60 seconds.
According to the three requirements, Θ={3, 180, 60} After abstraction with error bound 5: Θ’={0,3,1}, where d = 60, Δ0=3, Δ1=0, Δ2=0 •Req-08 G(~Air_ok_signal -> terminate_auto-control_mode). •Req-28 G ((~ avail && X ~ avail && XX ~ avail && XXX ~ avail ) -> manual_mode) No error in this case (only strengthening and loose sampling): • • •
Error appears in „sense for 3 seconds then decide“ -> we decide before waiting Error appears in „actuate 3 seconds from now“ -> we actuate immediately The computed error is an estimate without differentiating for, in, and from now
13-Mar-15
12
Input and output variable partition • Automatically • Individual formula partition according to the positions of Implication and Until operators • Conflict resolution between the sets of inputs and sets of outputs of all formulas • Manually • Adjustment and confirmation G((available_pulse_wave || available_arterial_line) && select_cuff -> trigger_corroboration) Input: {available_pulse_wave, available_arterial_line, select_cuff} 13-Mar-15
Output: {trigger_corroboration} 13
Maintaining consistencies between formal language and implentability • Consistency checking by synthesizing the set of generated LTL formulas • The existence of a controller shows the consistency • Failure may locate the inconsistency • Heuristic refinement over inconsistent specification • Locate the pair of inconsistent requirements • Adjust the existing input/output variable partition • Modify the requirements 13-Mar-15
14
Implementation
13-Mar-15
15
Evaluation-1 Name
CARA
13-Mar-15
No.
Specification
Num. of formulas
Num. of Input
Num. of Output
Time(s)
0
Working mode and switching
30
22
28
34
1
Pump Monitor
20
9
14
2
2.1.1
BPM: cuff detector
14
13
12
1
2.1.2
BPM: AL detector
15
11
14
2
2.1.3
BPM: pulse wave detector
14
9
12
1
2.2.1
BPM: initial auto control
16
14
15
1
2.2.2
BPM: first corroboration
19
11
16
29
2.2.3
BPM: valid ctrl blood pressure
13
11
10
2
2.2.4
BPM: cuff source handler
11
9
10
2
2.2.5
BPM: arterial line blood pressure
16
9
13
1
2.2.6
BPM: arterial line corroboration
12
8
13
1
2.2.7
BPM: pulse wave handler
20
10
21
23
3.1
(PA) Model ctrl algorithm
9
15
11
3
3.2
(PA) Polling algorithm
56
12
20
11
16
Evaluation-2 Name
TELE
Robot
No.
Specification
Num. of formulas
Num. of Input
Num. of Output
Time(s)
1
Shopping
29
11
24
8
2
Article processing
17
3
13
1
3
On-line reservation
6
3
4
1
4
Information
15
8
14
1
5
Local bulletin board
17
7
16
1
1
A robot with 4 rooms
9
2
5
1
2
A robot with 9 rooms
14
2
10
1
3
Two robots with 5 rooms
25
2
11
7
The execution time is only about checking consistency, not about printing the strategy • We use compositional methods and conjucnt sub-strategies in BDD (printout strategy = printout BDD) 13-Mar-15
17
Conclusion • Set a bridge between system designers and formal methods for correctness-by-construction • Scalability is obtained from G4LTL-ST with compositional synthesis
• Combine syntax and semantic analysis in natural language processing • Antonym reasoning to reduce the number of propositions • Support time expression in specification • Semi-automatic input/output variable extraction
• Guided refinement over inconsistent specification 13-Mar-15
18
13-Mar-15
19
