What is meaning?

Going Beyond Shallow Semantics Martha Palmer University of Colorado June 23, 2011 RELMS 2011 Portland, Oregon 1

How do we sprinkle atmospheric dust over our sentences?  

Where are we now? Where should we go next? 



Inference, probably probabilistic…

How do we get there?

3

… just piling up words, one after the other, won't do much of anything until something else has been added. That something is named quite precisely by Anthony Burgess in this sentence from his novel Enderby Outside (1968):  And the words slide into the slots ordained by syntax, and glitter as with atmospheric dust with those impurities which we call meaning. Stanley Fish, How to Write a Sentence: And How To Read One,p.2 From Mark Liberman’s Language Log, June 14, 2011 2

Where we are now: Explicit Semantic Dependencies

The county coroner says he urged Saint Rita's to move its patients.

4

1

Where we are now: Explicit semantic dependencies The county coroner says he urged Saint Rita's to move its patients.

Where should we go next? 



county coroner

The county coroner says he urged Saint Rita's to move its patients.

say he

urge

Detecting similar concepts Recovering implicit arguments

 

Saint Rita’s

Question Answering, Machine Translation, Information Extraction, Textual Entailment,…

move its patients

5

Where should we go next?

The eventual devastation threatened their lives. Did the flooding put their lives in danger?

8

Where should we go next?

Examples from Daniel Marcu, GALE PI mtg, 2011  



Recovering implicit predicates Between Munich and LA you need less than 11 hours by plane. From Munich to LA it does not take more than 11 hours by plane.



Types of Inference   

Semantic equivalence, class membership, etc Recovering implicit arguments Recovering implicit predicates

Question Answering, Machine Translation, Information Extraction, Textual Entailment,… 7

8

2

Is that all? 

Of course not



But these are next steps that are feasible now

How do we sprinkle atmospheric dust over our sentences? 

9

Where we are now - DETAILS 

DARPA-GALE, OntoNotes 5.0 

   

 

BBN, Brandeis, Colorado, Penn Multilayer structure Three languages: English, Arabic, Chinese Several Genres (@ ≥ 200K ): NW, BN, BC, WT Parallel data, E/C, E/A PropBank frame coverage for rare verbs Recent PropBank extensions

 

Where are we now? Where should we go next? How do we get there?

10

Multilayer Design  The literal meaning of sentences – A frame-based representation of predicates and their arguments – Referring expressions and the textual phrases they refer to – Coarse-grained word sense tags for most polysemous verbs  Does this lay a foundation for inference? Text Treebank Names

PropBank

Co-reference

Word Sense V

OntoNotes Annotated Text 11

12

3

Included in OntoNotes 5.0: Extensions to PropBank 

Original annotation coverage: 





English Noun and LVC annotation 



PropBank: verbs; past participle adjectival modifiers NomBank: relational and eventive nouns. light verbs, other predicative adjectives, …

“…[yourARG0] [decisionREL] [to say look I don't want to go through this anymoreARG1]”



Example within an LVC: Make a decision 

13

Extensions to PropBank, cont.

Roleset: Arg0: decider, Arg1: decision…



Substantial gap 

Example Noun: Decision

“…[the PresidentARG0] [madeREL-LVB] the [fundamentally correctARGM-ADJ] [decisionREL] [to get on offenseARG1]” 14

Arabic Noun and LVC annotation •Example noun





Eventive nouns play an important role with respect to event relations and event coreference; so do predicative adjectives Current OntoNotes PropBanking now includes  



Eventive nouns Light verbs (LVC) with the nominal as the true predicate Predicative adjectives (primarily for Arabic) 15

‫ قرار‬decision:

Roleset: Arg0 (decider), Arg1(decision)

‫صدر أمس مزيد من المواقف المنددة بقرار الحكومة الكندية فرض حظر‬ ‫على حزب هللا‬ More statements were released condemning [the Canadian government's ARG0] [decisionREL] [to impose a ban on Hezbollah ARG1]

•Example within an LVC ‫اتخذ القرار‬to take a decision: ‫اتخذ القرار بشن العملية‬ [He Arg0] [took REL-LVB] [the decisionREL] [to launch the operation Arg1] 16

4

Verb Frames Coverage By Language

PropBank Verb Frames Coverage 100%

 



The set of verbs is open But the distribution is highly skewed For English, the 1000 most frequent lemmas cover 95% of the verbs in running text. 

Graphs show counts over English Web data containing 150 M verbs.

99%

98%

Projected Final Count

Estimated Coverage in Running Text

English

5,100

99%

Chinese

18,200

96%

Language

97%

96%

Arabic

5,250*

99%

95%

94% 1000

2000

3000

4000

5000

6000

7000

* This covers all the verbs and most of the predicative adjectives/nouns in ATB.

8000

How do the PropBank verb frames relate to Word Senses?

17

18

Sense Hierarchy 

PropBank Framesets – ITA >90% coarse grained distinctions 20 Senseval2 verbs w/ > 1 Frameset Maxent WSD system, 73.5% baseline, 90%



Empirical Validation – Human Judges the 90% solution (1700 verbs)

Sense Groups (Senseval-2/OntoNotes) - ITA 89% Intermediate level (includes Verbnet/some FrameNet) – SVM, 86.9%



19

WordNet – ITA 73% fine grained distinctions, 64%

Chen, Dligach & Palmer, ICSC 2007

20

5

OntoNotes Status for Sense Tags    

  

More than 2,500 verbs grouped Average ITA per verbs = 89% http://verbs.colorado.edu/html_groupings/ More than 150,000 instances annotated for 2000+ verbs WSJ, Brown, ECTB, EBN, EBC, WebText Training and Testing Links to VerbNet, FrameNet, PropBank, WordNet

These were some of the pieces We’ve reviewed



 

PropBanking coverage Sense tagging approach

And mentioned



 



Treebanking Coreference annotation

Now let’s put them together…

21

22

Details of “Where we are now”

Now: Explicit semantic dependencies

From a Broadcast Conversation story about Hurricane Katrina: The county coroner says he urged Saint Rita's to move its patients. (TOP (S (NP-SBJ (DT The) (NN county) (NN coroner)) (VP (VBZ says) (SBAR (-NONE- 0) (S (NP-SBJ (PRP he)) (VP (VBD urged) (NP-1 (NNP Saint) (NNP Rita) (POS 's)) (S (NP-SBJ (-NONE- *PRO*-1)) (VP (TO to) (VP (VB move) (NP (PRP$ its) (NNS patients))))))))) (. /.))) 23

The county coroner says he urged Saint Rita's *PRO* to move its patients. ARGO: PER: county coroner PB: say.01 ARG0: PER: he

ARG1: PB: urge.01 move-v.1: change position move-v.2: intentionally act, decide move-v.3: affect, impress move-v.4: change residence or employment move-v.5: sell, dispose of move-v.6: socially or professionally interact move-v.7: make intrusive advances on

ARG1: ORG: Saint Rita’s ARG0: PRO ARG2: PB: move.01 ARG1: PER: its patients

24

6

How do we sprinkle atmospheric dust over our sentences?

Where should we go next? 



Detecting similar concepts Recovering implicit arguments Palmer, et. al, ACL-86, Gerber & Chai, ACL-2010

  

Where are we now? Where should we go next? How do we get there?

The county coroner says he urged Saint Rita's to move its patients. 



25

Where should we go next? 

Semantic link 



Every path from back door to yard was covered by a grape-arbor, and every yard had fruit trees. Where are the grape arbors located?

26

Where should we go next? 

Recovering Implicit Arguments [Gerber & Chai, 2010]

[Arg0 The two companies] [REL1 produce] [Arg1 market pulp, containerboard and white paper]. The goods could be manufactured closer to customers, saving [REL2 shipping] costs. 

27

The eventual devastation [of Saint Rita’s] threatened their lives. Did the flooding put the patients’ lives in danger?

Used VerbNet for subcategorization frames 28

7

How do we sprinkle atmospheric dust over our sentences?

VerbNet – Karin Kipper Schuler 

Class entries: 

  

Where are we now? Where should we go next? How do we get there? VERBNET?

 



Verb entries:  

29

VerbNet: send-11.1 

Roles

Argument roles for ship

One Frame:NP V NP PP.destination



 



(Members: 11, Frames: 5) includes “ship”





  

Refer to a set of classes (different senses) each class member linked to WN synset(s) and FrameNet frames

30

Agent [+animate | +organization] Theme [+concrete] Source [+location] Destination [+animate | [+location & -region]]



Capture generalizations about verb behavior Organized hierarchically Members have common semantic elements, semantic roles, syntactic frames, predicates

 

Agent [+animate | +organization] Theme [+concrete] Source [+location] Destination [+animate | [+location & -region]]

example "Nora sent the book to London." syntax Agent V Theme {to} Destination semantics motion(during(E), Theme) location(end(E), Theme, Destination) cause(Agent, E) 31

32

8

Hidden Axioms

Hidden Axioms REVEALED!

[Companies] shipped [goods] to [customers].  THEMATIC ROLES: AGENT V THEME SOURCE DESTINATION  SEMANTICS





 



[Companies] shipped [goods] to [customers].  THEMATIC ROLES: AGENT V THEME SOURCE DESTINATION  SEMANTICS

CAUSE(AGENT,E) MOTION(DURING(E), THEME), LOCATION(END(E), THEME, DESTINATION),

 



33

Where should we go next?

34

VerbNet – cover, fill-9.8 class 



Semantic similarity 



Every path from back door to yard was covered by a grape-arbor, and every yard had fruit trees. Where are the grape arbors located?

CAUSE(Companies, E) MOTION(DURING(E), goods), LOCATION(END(E), goods, customers),

WordNet Senses: …, cover(1,2, 22, 26),…, staff(1),



Thematic Roles: Agent [+animate] Theme [+concrete], Destination [+location, +region]



Frames with Semantic Roles “The employees staffed the store" “ The grape arbors covered every path" Theme V Destination

location(E,Theme,Destination) location(E,grape_arbor,path)

35

36

9

Where should we go next? 



VerbNet – move, roll 51.3.1 Class

Detecting similar concepts Recovering implicit arguments





Palmer, et. al, ACL-86, Gerber & Chai, ACL-2010

The county coroner says he urged Saint Rita's to move its patients. 



WordNet Senses: …, move(1,2, 3), Thematic Roles: Agent [+Intentional Control] Theme [+concrete], Location [+concrete]

Agent V Theme

The eventual devastation [of Saint Rita’s] threatened their lives. Did the flooding put the patients’ lives in danger?



Frames with Semantic Roles “[Saint Rita’s] …. to move [their patients].

motion(during(E), their patients) cause(Saint Rita’s, E) 37

38

Semantic links 

[threaten, endanger] 



WordNet synsets

[endanger, “put in danger”]  

PropBank light verb construction annotation Noun predicates, preposition predicates*

SEMLINK 

 



*Ken Litkowski working with FrameNet 

Extended VerbNet 5,391 lexemes (91% PB) Type-type mapping PB/VN, VN/FN Semi-automatic mapping of PropBank instances to VerbNet classes and thematic roles, hand-corrected. (now FrameNet) VerbNet class tagging as automatic WSD Brown, Dligach, Palmer, IWCS 2011

[cover fill9.8, location] 

Sense tagging, VerbNet semantic predicates 39



Run SRL, map Arg2 to VerbNet roles, Brown performance improves Yi, Loper, Palmer, NAACL07 40

10

Mapping from PropBank to VerbNet (similar mapping for PB-FrameNet) Frameset id = ship.01 Arg0

Sense = ship Sender

VerbNet class = Send -11.1 Agent/Sender*

Arg1

Package

Theme

Arg2

Recipient

Arg3

Source

Destination/ *Goal OR Recipient Source

*FrameNet Labels41

Baker, Fillmore, & Lowe, COLING/ACL-98 Fillmore & Baker, WordNetWKSHP, 2001

How do we sprinkle atmospheric dust over our sentences?   

42

Limitations to Lexical Resources 

How do we sprinkle atmospheric dust over our sentences?

GIZA++ finds almost 80% of parallel predicate pairs in Gold Standard parallel Chinese/English PropBanks. Alignment Ch.pred ↔ En.pred

GIZA++ 53.1%

Where we are now? Where should we go next? How do we get there? SEMLINK?

Human Annotator



66.3%

 

Where we are now? Where should we go next? How do we get there?

Percentage of aligned predicates on 200 random Sentences in the Xinhua Corpus

1/3 of the predicates have no mapping in the other language. 43

44

11

Where should we go next?  



Recovering implicit predicates Between Munich and LA you need less than 11 hours by plane. From Munich to LA it does not take more than 11 hours by plane.

Where should we go next? [Between Munich and LA ARGM-ADV] [you ARG0] [need REL] [less than 11 hours by plane ARG1]. 

45

Where should we go next? 

TO FLY [Between Munich and LA ARGM-ADV] [you ARG0] [need REL] [less than 11 hours by plane ARG1].

46

Constructions allow us to 

 

TO FLY [ [elided verb ] [From Munich ARGM-DIR] [to Los Angeles ARGM-GOL] ARG0] , [ it] does [not ARGM-NEG] [take REL-2.take10] [more than eleven hours by plane ARG1] .

47

[ [elided verb ] [From Munich ARGM-DIR] [to Los Angeles ARGM-GOL] ARG0] , [ it] does [not ARGM-NEG] [take REL-2.take10] [more than eleven hours by plane ARG1] .



Recognize a path prepositional phrase, and that it necessarily goes with a “MOTION” event If we have a MOTION event we can associate the plane with it as a vehicle Or just the plane itself can suggest a motion event… 48

12

Pandora’s box?  



Which constructions? Which semantic predicates should they be associated with, give rise to? How to determine?

How do we sprinkle atmospheric dust over our sentences?   

Where we are now? Where should we go next? How do we get there? SEMLINK + Constructions + Statistics

13:40 Claire Bonial, Susan Windisch Brown, Jena D. Hwang, Christopher Parisien, Martha Palmer and Suzanne Stevenson: Incorporating Coercive Constructions into a Verb Lexicon 49

Acknowledgments 



We gratefully acknowledge the support of the National Science Foundation Grants for , Consistent Criteria for Word Sense Disambiguation, Robust Semantic Parsing, Richer Representations for Machine Translation, and DARPA-GALE via a subcontract from BBN. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. 51

50

And thanks to 





Postdocs: Paul Kingsbury, Dan Gildea, Nianwen Xue, Students: Hoa Dang, Tom Morton, Karin Kipper Schuler, Jinying Chen, Szu-Ting Yi, Edward Loper, Susan Brown, Dmitriy Dligach, Jena Hwang, Will Corvey, Claire Bonial, Jin-ho Choi, Lee Becker, Shumin Wu Collaborators: Christiane Fellbaum, Suzanne Stevenson 52

13