TextInfer ⋅ July 30, 2011

Representing and resolving ambiguities in ontology-based question answering Christina Unger & Philipp Cimiano Semantic Computing Group, CITEC, Bielefeld University

1 / 34

Outline Ambiguities in ontology-based interpretation

Representing and resolving ambiguities Enumeration Underspecification Ontological reasoning

Results and conclusion

Outline

2 / 34

Outline Ambiguities in ontology-based interpretation

Representing and resolving ambiguities Enumeration Underspecification Ontological reasoning

Results and conclusion

Ambiguities in ontology-based interpretation

3 / 34

Ambiguities

Ambiguities comprise all cases in which natural language expressions have more than one meaning, due to: ▸

structural properties (e.g. modifier/PP attachment sites) ▸ ▸

▸

Put the box on the table by the window in the kitchen. We saw old elves and wizards.

alternative lexical meanings ▸

bank

Ambiguities in ontology-based interpretation

3 / 34

Ontology-based interpretation

Goal: Map natural language expressions (e.g. questions) to formal representations aligned to the ontology (e.g. SPARQL queries). ▸

Which city has the most inhabitants?

▸ PREFIX geo:

SELECT ?c WHERE { ?c rdf:type geo:city . ?c geo:population ?p . } ORDER BY DESC ?p LIMIT 1

Ambiguities in ontology-based interpretation

4 / 34

Ontology-based interpretation

The meaning of a lexical expression is the ontology concept that this expression verbalizes. ▸

Which city has the most inhabitants? ▸ city → geo:city ▸ has inhabitants → geo:population

Ambiguities in ontology-based interpretation

5 / 34

Ontology-based interpretation

Main challenge: Map natural language expressions to corresponding ontology concepts. This mapping needs not be one-to-one. ▸

different expressions can refer to the same ontology concept ▸ flows through → geo:flowsThrough ▸ traverses → geo:flowsThrough

▸

one expression can refer to different ontology concepts ▸ New York → geo:new_york, geo:new_york_city ▸ has → geo:flowsThrough, geo:inState

Ambiguities in ontology-based interpretation

6 / 34

Ambiguities in ontology-based interpretation

Ambiguities in the context of ontology-based interpretation comprise all cases in which a natural language expression cannot be mapped uniquely to an ontology concept.

Ambiguities in ontology-based interpretation

7 / 34

Non-overlapping alternatives ▸

What is the area of New York? ▸ (New York state) → geo:new_york ▸ (New York city) → geo:new_york_city

▸

What is the biggest city? ▸

▸

SELECT ?c WHERE { ?c a geo:city . ?c geo:population ?n . } ORDER BY DESC ?n LIMIT 1 SELECT ?c WHERE { ?c a geo:city . ?c geo:area ?n . } ORDER BY DESC ?n LIMIT 1

Ambiguities in ontology-based interpretation

8 / 34

Overlapping alternatives

▸

Give me all films starring Jeff Bridges. ▸ ▸

▸

only inculding leading roles also including supporting roles

Which cities have more than two million inhabitants? ▸ ▸



Ambiguities in ontology-based interpretation

9 / 34

Context-dependency ▸

Which state has the most rivers? ▸

▸

SELECT COUNT(?s) AS ?n WHERE { ?s a geo:state . ?x a geo:river . ?x geo:flowsThrough ?s. } ORDER BY DESC ?n LIMIT 1

Which state has the most cities? ▸

SELECT COUNT(?s) AS ?n WHERE { ?s a geo:state . ?x a geo:city . ?x geo:inState ?s. } ORDER BY DESC ?n LIMIT 1

Due to sortal restrictions, only one of the alternatives is admissible. Ambiguities in ontology-based interpretation

10 / 34

Ambiguities are pervasive. ▸

QALD-1 training questions for DBpedia ▸

▸

At least 16 % contain expressions that do not have a unique ontological correspondent.

880 user questions for GeoBase ▸ ▸

1278 occurences of light expressions: is/are, has/have, with, in, of 151 ocurrences of context-dependent expressions: big, small, major

Ambiguities in ontology-based interpretation

11 / 34

Outline Ambiguities in ontology-based interpretation

Representing and resolving ambiguities Enumeration Underspecification Ontological reasoning

Results and conclusion

Representing and resolving ambiguities

12 / 34

Representing ambiguities

▸

Enumeration: constructing a different semantic representation and query for every meaning alternative

▸

Underspecification: constructing only one underspecified representation that subsumes all different interpretations

Representing and resolving ambiguities

12 / 34

Enumeration Constructing a different meaning representation for every possible interpretation

Representing and resolving ambiguities

Enumeration

13 / 34

Enumeration ▸

How big is New York? ▸

▸

▸

▸

SELECT ?a WHERE { geo:new_york_city geo:area ?a . } SELECT ?p WHERE { geo:new_york_city geo:population ?p.} SELECT ?a WHERE { geo:new_york geo:area ?a . } SELECT ?p WHERE { geo:new_york geo:population ?p . }

▸

two lexical entries for big, one referring to geo:area and one referring to geo:population

▸

two lexical entries for New York, one referring to geo:new_york and one referring to geo:new_york_city

Representing and resolving ambiguities

Enumeration

14 / 34

Enumeration

▸

Which state has the most rivers? Which state has the most cities?

▸

two lexical entries for has, one referring to geo:flowsThrough and one referring to geo:inState

▸

Problem: geo:flowsThrough is only possible if the relevant argument is a river, geo:inState is only relevant if the relevant argument is a city

▸

Solution: In order not to derive inconsistent interpretations, we need to capture sortal restrictions.

Representing and resolving ambiguities

Enumeration

15 / 34

Adding sortal restrictions Lexical entries are enriched with sortal restrictions vˆclass. S DP1 ↓

VP V has

geo:flowsThrough (y, x)

DP2 ↓

(DP1 , x), (DP2 , y) yˆgeo:river

S DP1 ↓

VP V has

geo:inState (y, x)

DP2 ↓

Representing and resolving ambiguities

(DP1 , x), (DP2 , y) yˆgeo:city

Enumeration

16 / 34

Adding sortal restrictions When translating semantic representations into a formal query, sortal restrictions are added as a condition. ▸

Which state has the most rivers?

▸ SELECT COUNT(?y) as ?c WHERE {

?x a geo:state . ?y a geo:river . ?y geo:flowsThrough ?x . ?y a geo:river . } ORDER BY ?c DESC LIMIT 1 ▸ SELECT COUNT(?y) as ?c WHERE {

?x a geo:state . ?y a geo:river . ?y geo:inState ?x . ?y a geo:city . } ORDER BY ?c DESC LIMIT 1 Representing and resolving ambiguities

Enumeration

17 / 34

Example: major N ADJ major

N↓

geo:population (x, p) p > 150 000

(N, x) xˆgeo:city N ADJ major

N↓

geo:population (x, p) p > 10 000 000

(N, x) xˆgeo:state

Representing and resolving ambiguities

Enumeration

18 / 34

Example: major

▸

Give me all major cities.

▸ SELECT ?x WHERE {

?x a geo:city . ?x geo:population ?p . ?x a geo:city . FILTER ( ?p > 150000 ) } ▸ SELECT ?x WHERE {

?x a geo:city . ?x geo:population ?p . ?x a geo:state . FILTER ( ?p > 10000000 ) }

Representing and resolving ambiguities

Enumeration

19 / 34

Problems with enumeration The enumeration strategy relies on a conflict that automatically filters out unwanted interpretations by constructing a query that returns no result. ▸

Extensionality No way to distinguish between queries that return no result due to an inconsistency introduced by a sortal restriction, and queries that return no result, because there is no result (e.g. Which states border Hawaii?).

▸

Number of constructed interpretations User questions easily lead to 20 or 30 different possible interpretations at least.

Representing and resolving ambiguities

Enumeration

20 / 34

Underspecification Constructing one underspecified representation that subsumes all different interpretations

Representing and resolving ambiguities

Underspecification

21 / 34

Adding metavariables

Semantic representations are enriched with metavariables and metavariable specifications that list all possible instantiations of a metavariable given certain sortal restrictions.

Representing and resolving ambiguities

Underspecification

22 / 34

Example: biggest N ADJ biggest

N↓

y

P (x, y) max(y) (N, x) P → geo:area (x = geo:city ⊔ geo:state) ∣ geo:population (x = geo:city ⊔ geo:state) ∣ geo:height (x = geo:mountain)

Representing and resolving ambiguities

Underspecification

23 / 34

Semantically light expressions

For semantically light expressions, we assume the most underspecified representation: a metavariable without a metavariable specification.

Representing and resolving ambiguities

Underspecification

24 / 34

Example: has

S DP1 ↓

VP V has

DP2 ↓

P (y, x) (DP1 , x), (DP2 , y)

Representing and resolving ambiguities

Underspecification

25 / 34

Example: final underspecified representation ▸ ▸

Which state has the biggest city? x, y, z geo:state(y) geo:city(x) P (x, y) Q (x, z) max(z)

Q → geo:area (x = geo:city ⊔ geo:state) ∣ geo:population (x = geo:city ⊔ geo:state) ∣ geo:height (x = geo:mountain)

Representing and resolving ambiguities

Underspecification

26 / 34

The interpretation process thus constructs one underspecified representation that has to be specified in order to yield a specific query that can be evaluated w.r.t. a knowledge base.

Representing and resolving ambiguities

Underspecification

27 / 34

Ontological reasoning Integrating a reasoner in order to resolve metavariables and thereby construct exactly those interpretations that are possible and consistent

Representing and resolving ambiguities

Ontological reasoning

28 / 34

Resolving Q x, y, z geo:state(y) geo:city(x) P (x, y) Q (x, z), max(z)

Q → geo:area (x = geo:city ⊔ geo:state) ∣ geo:population (x = geo:city ⊔ geo:state) ∣ geo:height (x = geo:mountain) Check satisfiability of the intersection of x's type information with the sortal restrictions: ▸ geo:city ⊓ (geo:city ⊔ geo:state)

true

▸ geo:city ⊓ geo:mountain

false

Representing and resolving ambiguities

Ontological reasoning

29 / 34

Resolving Q

x, y, z geo:state(y) geo:city(x) P (x, y) Q (x, z) max(z)

Q → geo:area (x = geo:city ⊔ geo:state) ∣ geo:population (x = geo:city ⊔ geo:state)

Representing and resolving ambiguities

Ontological reasoning

30 / 34

Resolving Q

x, y, z

x, y, z

geo:state(y) geo:city(x) P (x, y) geo:area (x, z) max(z)

geo:state(y) geo:city(x) P (x, y) geo:population (x, z) max(z)

Representing and resolving ambiguities

Ontological reasoning

30 / 34

Example: Resolving P

x, y, z

x, y, z

geo:state(y) geo:city(x) P (x, y) geo:area (x, z) max(z)

geo:state(y) geo:city(x) P (x, y) geo:population (x, z) max(z)

The ontology is searched for a relation that admits geo:city (or a superclass) as domain and geo:state (or a superclass) as range. ▸ geo:inState Representing and resolving ambiguities

Ontological reasoning

31 / 34

Example: Resolving P

x, y, z

x, y, z

geo:state(y) geo:city(x) geo:inState (x, y) geo:area (x, z) max(z)

geo:state(y) geo:city(x) geo:inState (x, y) geo:population (x, z) max(z)

The ontology is searched for a relation that admits geo:city (or a superclass) as domain and geo:state (or a superclass) as range. ▸ geo:inState Representing and resolving ambiguities

Ontological reasoning

31 / 34

Outline Ambiguities in ontology-based interpretation

Representing and resolving ambiguities Enumeration Underspecification Ontological reasoning

Results and conclusion

Results and conclusion

32 / 34

Results

Of the 880 GeoBase user questions, Pythia can handle 624.

Enumeration Reasoning

Results and conclusion

Total # queries 3180 2100

Avg. # queries 5.1 3.4 (-44%)

Max. # queries 96 24 (-75%)

32 / 34

Distribution (improved results) .

.242

.111 .87 .51

.

.3 .1

Results and conclusion

.2

.3

.4

.5

.21 .6

. 0 .7

.4 .8

. 0 .9

.4 .10

.

33 / 34

Conclusion

Ambiguous expressions are everywhere. But disambiguation strategies ▸

allow to discard interpretations that are not consistent in the context and therefore should be filtered out

▸

allow to significantly reduce the number of constructed queries: the average number of queries per question can be reduced by 44%, the maximum number of queries per question can be reduced even by 75%

Results and conclusion

34 / 34