versat: A Verified Modern SAT Solver Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy Computer Science, The University of Iowa, USA

VMCAI 2012 January 23, 2012

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

SAT Solver Verification Motivation I

Theoretically: simple specification, sophisticated implementation =⇒ “an interesting challenge”

I

Practically: SAT solvers are used in formal verification / A.I. =⇒ “increases the trust level”

Approaches to Verified SAT Solving I I

Verify the code Verify the certificate using a trusted checker: I I

SAT instance: a model that is found by the solver (easy) UNSAT instance: a refutational proof The certified track of SAT competition proof ≈ execution trace

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Verifying the Code: Related Works S. Lescuyer (2008) [Coq] Recursive definition (the original DPLL)

N. Shankar (2011) [PVS] Modern DPLL: conflict analysis, clause learning, backjumping

F. Mari´c (2009) [Isabelle] Modern DPLL: conflict analysis, clause learning, backjumping Also implemented the two-literal watch lists

Summary I

Used model theoretic specification: ∃M.M  Φ

I

Proved sound and complete

I

Inefficient implementation at low-level

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

versat: a Verified SAT Solver Goal: A verified real-world SAT Solver I

implemented modern SAT techniques

I

low-level optimized using efficient data structure

I

Verified all the way down to machine words and bits

Focus on performance & productivity I

Statically verified to produce sound UNSAT answers

I

SAT certificates are checked at run-time (low overhead)

I

No completeness(termination) proof

I

Performance is more important than guarantee of termination

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

The Guru Programming Language Guru is a functional programming language with:

Dependent type system (for verification)

Resource type system (for efficient code generation)

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

The Guru Programming Language Guru is a functional programming language with:

Dependent type system (for verification) I

inductive datatypes (induction on first order variables)

I

general recursion (reasoning about partial functions)

I

first order formula types with equality predicate

I

provable equality over operational semantics

Resource type system (for efficient code generation)

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

The Guru Programming Language Guru is a functional programming language with:

Dependent type system (for verification) I

inductive datatypes (induction on first order variables)

I

general recursion (reasoning about partial functions)

I

first order formula types with equality predicate

I

provable equality over operational semantics

Resource type system (for efficient code generation) I

configurable resource management policies:

I

reference counting (default, automatic)

I

linear typing (mutable data structures, annotations)

I

arrays with constant time access

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Specification: Soundness of UNSAT answer

Statement of Unsatisfiability I

Model Theoretically: “∀M.M 2 Φ” or “Φ  ⊥”

I

Proof Theoretically: “Φ ` ⊥”

I

They are all equivalent for propositional logic

I

Solver returns UNSAT when the empty clause is deduced

Verification Strategy I

Verify the deduction steps follow the proof rules

I

Isolate conflict analysis, where deductions are performed

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Specification: Inference System (the pf type) The pf type encodes “`res ” (a weaker system than “`”) Define lit := word Define clause := Define formula := Inductive pf : Fun(F : formula)(C:clause).type := pf_asm : Fun(F : formula)(C:clause) (u : { (member C F eq_clause) = tt }). | pf_res : Fun(F : formula)(C1 C2 Cr : clause)(l:lit) (d1 : ) (d2 : ) (u : { (is_resolvent Cr C1 C2 l) = tt }). I

type represents the judgement F ` C

I

A value of represents a proof of F ` C

I

Term constructors represents the inference rules

I

is resolvent tests if Cr is a resolvent of C1 and C2

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Specification: The type of solve function Inductive answer : Fun(F:formula).type := sat : Fun(spec F:formula). | unsat : Fun(spec F:formula)(spec p:). Define solve : Fun(F:formula). := ...

I

(nil lit) is the empty list of literals (the empty clause)

I

spec (specificational) arguments are only for type checking

I

So, proofs are not generated at run-time

I

G URU makes sure that spec arguments are only dependent on the invariants (always computable)

I

solve function should contain implementation and proof (internal verification)

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Specification: Summary

I

Encodes the propositional logic

I

The trusted core of versat The rest of versat is actual implementation and proof

I

I

I I

to be checked and certified by the G URU compiler

Size: 259 lines of G URU code (small and straightforward) Includes a parser for DIMACS benchmark format I I

a trusted interpretation of string as formula 145 lines (out of 259 lines)!

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Implemented Features A core set of modern features

Engineering: I

Two-literal Watch Lists

I

Conflict Analysis + Fast Resolution

I

Backjumping (Non-chronological Backtracking)

Heuristics: I

Decision Heuristics (Scoring variable activities)

I

Clause Learning

Summary: I

9884 lines of G URU code and proofs

I

Proved 247 lemmas

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Efficient Representation of Clauses aclause type: array-based clause and invariants Inductive aclause : Fun(nv:word)(F:formula).type := mk_aclause : Fun(spec nv:word)(spec F:formula) (spec n:word)(l:) (u1:{ (array_in_bounds nv l) = tt }) (spec c:clause)(spec pf_c:) (u2:{ c = (to_cl l) }) . I

keep specification simple, implementation efficient

I

aclause stores a clause in the array

I

array in bounds: all variable numbers are within bounds and the array is null-terminated

I

to cl interprets a null-terminated array as a list

I

the interpretation of array is valid in F

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (1/7)

C ∨ ¯l

D∨l ./ l C∨D

I

Problem: duplicate literals

I

Solution: a look-up table

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

var .. .

status

v .. .

Not/Pos/Neg

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (2/7)

D1 ∨ l1 D2 ∨ l2

C ∨ l¯1

./

./

l1

l2

Dn ∨ ln

...

./ ln

I l1 . . . ln are assigned after the last decision literal I C 0 will have only one literal assigned after the last

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

C0

decision

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (3/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯ 145 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis: C

D

l

¯45 ¯ 23

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (3/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯ 145 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis: C

D

l

¯45 ¯ 23 ¯41 ¯ 23

¯45 1

5

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (3/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯ 145 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis: C

D

l

¯45 ¯ 23 ¯41 ¯ 23 ¯ ¯ 231

¯45 1 ¯4 ¯ 3

5 ¯ 4

I

Time complexity of removal depends on the length of C

I

Literals being resolved are assigned after the last decision

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (4/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯45 1 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis (Old & Improved): C

D

l

C1

C2

¯45 ¯ 23 ¯ ¯ 2341 ¯1 ¯ 23

¯45 1 ¯4 ¯ 3

5 ¯ 4

2

¯45 ¯ 3

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

D

l

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (4/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯45 1 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis (Old & Improved): C

D

l

C1

C2

D

l

¯45 ¯ 23 ¯ ¯ 2341 ¯1 ¯ 23

¯45 1 ¯4 ¯ 3

5 ¯ 4

2 ¯ 21

¯45 ¯ 3 ¯ 34

¯45 1

5

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (4/7) An Example Conflict: Clauses ¯4 ¯ 3 ¯45 1 ¯45 ¯ 23 .. .

¯ 3d , 4, ¯ 5 Assignment Sequence: 1d , 2, ¯45 ¯ =⇒ conflicting with 2 3

Analysis (Old & Improved): C

D

l

C1

C2

D

l

¯45 ¯ 23 ¯ ¯ 2341 ¯1 ¯ 23

¯45 1 ¯4 ¯ 3

5 ¯ 4

2 ¯ 21 ¯ 21

¯45 ¯ 3 ¯ 34 ¯ 3

¯45 1 ¯4 ¯ 3

5 ¯ 4

I

Time complexity of removal depends on the length of C2

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (5/7)

Data Structure: I

For duplication removal & faster remove operation C0 Di ∨ li

C

C0

./ li

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (6/7) Data Structure (even better): I

For duplication removal & constant time remove operation C0 D i ∨ li

C

C0

./ li

I

C2 is not calculated at run-time & C2L tracks the length

I

Removal is a constant time operation!

I

At the end, C2 has only one literal (C2L = 1)

I

At the end, C2 can be deduced

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conflict Analysis with Fast Resolution (7/7) Invariants:

C0 D i ∨ li

C

C0

X li

(u1:{ (u2:{ (u3:{ (u4:{ (u5:{

(all_lits_are_assigned T (append C1 C2)) = tt }) (cl_has_all_vars (append C1 C2) T) = tt }) (cl_set_at_prev_levels dl dls C1) = tt }) C2L = (length C2) }) (cl_unique C2) = tt })

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Example Theorem: Clearing the Look-up Table

Define cl_has_all_vars_implies_clear_vars_like_new : Forall (nv:word) (T:) (C:clause) (u:{ (cl_valid nv C) = tt }) (r:{ (cl_has_all_vars C T) = tt }) .{ (clear_vars T C) = (array_new nv UN) }

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Results: versat vs. proof checking The Certified Track benchmarks of SAT Competition 2007 I

16 benchmarks (believed to be UNSAT)

I

System: Intel Core 2 Duo 2.40GHz w/ 3GB of memory

I

One hour timeout for solving and checking, individually

Systems versat picosat + RUP picosat + TraceCheck

#Solved 6 14 14

#Certified 6 4 12

Trusted Base: I

versat: G URU compiler + 259 lines of G URU code

I

checker3 (RUP checker): 1,538 lines of C code

I

tracecheck (TraceCheck checker): 2,989 lines of C code + boolforce library (minisat-2.2.0 is ≈2,500 lines of C++)

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Results: versat vs. State-of-the-art Solvers

SAT Race 2008 Test Set 1 I

50 benchmarks

I

System: Intel Xeon X5650 2.67GHz w/ 12GB of memory

I

900 seconds timeout for solving

Systems versat picosat-936 minisat-2.2.0

#Solved 19 46 47

#Timeout 31 4 3

#Error/Wrong 0 0 0

Note: versat solved velev-live-sat-1.0-03 (78MB size, 224,920 variables, 3,596,474 clauses)

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver

Conclusion versat is a modern SAT solver verified in G URU I

UNSAT-soundness is verified statically

I

Can solve and certify realistic formulas

I

Comparable with the current proof checking technology

I

Source code is available at http://cs.uiowa.edu/~duoe/

I

Standalone certified C code is also available

Future Work: I

Add more features: CC Minimization, Restarting, Preprocessing

I

Implement other related tools: verified/efficient RUP proof checker

Duckki Oe, Aaron Stump, Corey Oliver, Kevin Clancy

versat: A Verified Modern SAT Solver