Equip Tourist with Knowledge Mined from Travelogues Qiang Hao, Rui Cai, Changhu Wang, Rong Xiao, Jiang-Ming yang, Yanwei Pang, Lei Zhang

Agenda I. Motivation & Challenges II. Location-Topic Model III. Applications a. b. c.

Destination Recommendation Destination Summarization Travelogue Enrichment

IV. Demo & Prototype

4/27/2010

Microsoft Research, Asia

2

I. Motivation & Challenges 4/27/2010

Microsoft Research, Asia

3

Travel is Always a Hot Topic • Online travel planning becomes popular – Transportation & accommodation

More requirements? How does it look?

Where to go?

4/27/2010

What to see?

Microsoft Research, Asia

4

Travel Related UGC Data • People would like to share travel experience – Discussion boards, Weblogs, photo sharing websites

Visual 4/27/2010

Text Microsoft Research, Asia

5

Understand Travelogue Tourists’ Needs

Where

Limitations of Travelogues

Lack of destination recommendation

Destinations:

• Relevant to a travel Survey manyintention travelogues • Similar to somewhere familiar Characteristics of destinations:

What

• Attractions to visit Summarize many travelogues • Activities to do • Styles to experience

How 4/27/2010

• Appearance of landmarks Search images manually • Illustration of text Microsoft Research, Asia

Lack of destination comparison Noisy & unstructured Partial viewpoints Lack of visual descriptions 6

Understand Travelogue – An Example We went back to the hotel, packed our bags and took the shuttle back to the airport for our flight from Oahu to Kauai… Since we had a few hours to kill before our big sunset dinner cruise, we went and checked out Spouting Horn near Poipu Beach. It's a pretty cool thing so see because as water rushes under a lava shelf, it spurts through a small opening at the surface. A very unique sight! …. After a few photos, we continued on our drive. What a scenic ride! We drove along the eastern coastline and up to Kilauea, Princeville and Hanalei...just beautiful. We stopped along the way for lots of pictures, one place had at least a dozen chickens in the parking lot...

4/27/2010

Microsoft Research, Asia

Oahu, Kauai, Poipu Beach, Kilauea, Princeville … locations

surface water … lava ... sunset coastline cruise ... local topics

hotel, shuttle, bag ... airport, flight ...

picture, photo ... global topics 7

To Better Leverage Travelogues • An appropriate space in which we can – – – –

keep location-representative information characterize multiple aspects of travel destinations measure the similarity of two places even measure the similarity of an arbitrary term to a place casino

B

summit

C Dive

A 4/27/2010

Microsoft Research, Asia

beach 8

What We have Done … • Extract location-representative knowledge from travelogues • Enable several application scenarios.

Travelogues

Location Extraction

Destination Recommendation

Travelogue Modeling

Destination Summarization User

LocationRepresentative Knowledge

Travelogue Enrichment

Knowledge Mining

4/27/2010

Microsoft Research, Asia

Applications

9

II. Location-Topic Model 4/27/2010

Microsoft Research, Asia

10

Travelogue Decomposition • How was a travelogue produced? Our assumptions are: – Terms are from location related or unrelated – Location related terms are from multiple aspects of a place

𝑝 𝑤 𝑑 = 𝜆×

4/27/2010

×

𝐿

𝑇 𝑙𝑜𝑐

𝑙=1

𝑧=1

+ (1-λ) ×

𝑝 𝑤 𝑧 𝑝 𝑧 𝑙 𝑝 𝑙 𝑑 + 1−𝜆 × (I)

Microsoft Research, Asia

×

Global Topic

×

Term

λ×

Global Topic

Document

Location

Location

=

Local Topic

Local Topic

Term

Term

Document

Document

𝑇𝑔𝑙 𝑧′ = 1

𝑝 𝑤 𝑧′ 𝑝 𝑧′ 𝑑 (II)

11

Location-Topic Model • LDA-like model – Gibbs sampling for parameter estimation casino Location-related

χd,s Switcher

γ

β l

ξd,s

πd,s Sd

summit

L

x

φloc

z

ηloc

beach

T loc

w Nd

α

φgl D

ηgl

T gl

Global-related 4/27/2010

Microsoft Research, Asia

12

Data Collection • Two travelogue corpora Language

Number of documents

Number of unique locations

Web sites

English

100K

18K in the U.S.

Live Spaces, TravelPod, IgoUgo, TravelBlog, …

Chinese

94K

32K in China

Live Spaces, Ctrip, …

• Data Processing – Location detection & disambiguation – Stemming – Stop-word removal

4/27/2010

Microsoft Research, Asia

13

Illustrative Topics • 100 local topics + 50 global topics • Top terms of several example topics Local #62

Local #66

Local #69

dive

casino

mountain

snorkel

gamble

peak

airport

best

hotel

fish

play

rocky

fly

fun

bed

aquarium

slot

snow

plane

beautiful

inn

sea

table

high

check

enjoy

breakfast

boat

machine

feet

bag

wonderful

bathroom

whale

game

lake

air

love

night

reef

card

summit

travel

amaze

door

…

…

…

…

…

…

4/27/2010

Global #8 Global #19 Global #26 Local #62 flight great

Microsoft Research, Asia

room

14

III. Applications 4/27/2010

Microsoft Research, Asia

15

A. Destination Recommendation  “I quite enjoyed the trip to Honolulu last year. Is there any other destination with similar style?” 𝐿𝑜𝑐𝑆𝑖𝑚 𝑙1 , 𝑙2 = exp⁡{−𝜏D𝐽𝑆 𝜓𝑙1 𝜓𝑙2 }

• Collect 36 top destinations suggested by TripAdvisor for four travel intentions Kauai

LT Model

Aspen

TF-IDF

Williamsburg

Stowe

North_Conway

Vail

Taos

Washington

Big_Bear_Lake

Myrtle_Beach Oahu

Maui

Lake_Tahoe

Daytona_Beach Taos

Big_Bear_Lake North_Conway

Cocoa_Beach

Lake_Placid Breckenridge

Lake_Tahoe Aspen

San_Diego

Park_City Vail

Williamsburg New_York

Fort_Lauderdale

Savannah

Savannah

Fort_Lauderdale Boston

Stowe

Park_City

Lake_Placid Breckenridge

Miami

Virginia_Beach

Cocoa_Beach

Oahu Philadelphia

San_Francisco Shreveport

Biloxi

Biloxi

Atlantic_City

San_Antonio Reno Atlantic_City

Reno

Charleston

Maui Virginia_Beach

New_Orleans San_Francisco

Philadelphia

Charleston

Las_Vegas

New_Orleans Boston San_Diego

San_Antonio

Tunica

Shreveport

Kauai

Miami New_York

Las_Vegas

4/27/2010

Microsoft Research, Asia

Tunica

Myrtle_Beach

Daytona_Beach

16

A. Destination Recommendation (2)  “I plan to go skiing next month. Could you recommend some destinations good for skiing?” 𝑅𝑎𝑛𝑘𝑆𝑜𝑟𝑒𝑤𝑞 𝑙 ⁡~ − D𝐾𝐿 𝛿𝑤𝑞 𝜓𝑙 Query

beach casino family history skiing 4/27/2010

#Ground#Matches at top K TopMethods 10 recommended destinations Query truth K=5 K=10 K=15 K=20 Myrtle Beach, Maui, Miami, Santa Monica, Destin, Hilton Head Island, baseline 1 4 7 9 Virginia beach Beach,35Daytona Beach, Key West, San Diego LT model 4 9 12 13 Las Vegas, Atlantic City, Lake Tahoe,2Biloxi, Reno, Deadwood, baseline 2 3 3New Orleans, casino 6 Detroit, Tunica, New York City LT model 4 5 5 5 baseline 4 6 8 D.C., 11 Orlando, Las 38 Vegas, New York City, Washington, New Orleans, family LT model 3 Francisco, 5 Walt8Disney11 Charleston, Myrtle Beach, Chicago, San World baseline 4 6 8 8 history New Orleans, 12 Charleston, Williamsburg, Washington, D.C., New York City, LT model 5 8 9 10 Chicago, Las Vegas, Philadelphia, San Francisco, San Antonio baseline 2 4 4 6 skiing 20 Lake Tahoe, Park City,LT South Lake Tahoe, Jackson Hole, model 3 5 10 Vail,12Breckenridge, Winter Park, Salt Lake City, Beaver Creek, Steamboat Springs Microsoft Research, Asia

17

B. Destination Summarization  “What are the most representative things in San Francisco? Can you tell me with a few words or sentences?”

• Representative tag generation 𝑝 𝑤 𝑙𝑞

𝑤=1:𝑊

• Representative snippet (sentence-based) for each tag 𝑆𝑐𝑜𝑟𝑒𝑙𝑞 ,𝑤𝑞 𝑠 = 𝐺𝑒𝑜𝑅𝑒𝑙𝑒𝑙𝑞 𝑠 × 𝑆𝑒𝑚𝑅𝑒𝑙𝑒𝑤𝑞 𝑠 𝐺𝑒𝑜𝑅𝑒𝑙𝑒𝑙𝑞 𝑠 = # 𝑙𝑞 ⁡appears⁡in⁡ℒ𝑠

𝑆𝑒𝑚𝑅𝑒𝑙𝑒𝑤𝑞 𝑠 = 4/27/2010

𝑤∈𝒲𝑠

ℒ𝑠

𝑇𝑒𝑟𝑚𝑆𝑖𝑚 𝑤𝑞 , 𝑤 ⁡ log 1 + 𝒲𝑠

Microsoft Research, Asia

18

B. Destination Summarization (2) Destination Anchorage Boston Chicago Las Vegas Los Angeles Maui New York City Orlando San Francisco Washington, D.C. Query Alaska + wildlife Las Vegas + casino

Top 10 representative tags bear, moose, alaskan, glacier, fish, cruise, salmon, wildlife, trail, mountain fenway, whale, historic, sox, cape, england, red, history, revere, church

michigan, institute, field, lake, museum, cta, tower, loop, windy, cub strip, casino, show, hotel, bellagio, gamble, fountain, venetian, mgm, slot hollywood, star, studio, universal, movie, boulevard, theatre, china, getty, sunset island, beach, snorkel, whale, ocean, luau, volcano, dive, fish, surf subway, broadway, brooklyn, zero, avenue, island, yorker, manhattan, village, greenwich disney, park, universal, resort, world, theme, studio, kingdom, magic, epcot bay, cable, alcatraz, chinatown, wharf, bridge, prison, bart, fisherman, pier museum, memorial, monument, national, metro, capitol, war, smithsonian, lincoln, president

Top snippets 1 2 3 1 2 3

Waikiki 1 Beach 2 + beach 4/27/2010 3

This time, the bus stopped at the Alaska Wildlife Conservation Center, where we saw musk oxen, bison, brown bears, black bears, 5.00 eagles, owls, foxes, reindeers, and a few other animals. Baseline LT-Model 4.17 at the Alaska Wildlife Along the way we stopped Recreation Center to check on the musk ox, Moose,4.05 Brown and Black Bears, and 4.09 4.03 3.97 3.78 many other Arctic animals. 3.67 3.64 4.00 The rest of my week in Alaska was filled with more wildlife sightings, including numerous brown bears and cubs, caribou, and a rare black wolf in Denali; and a moose with a calf and several wild trumpeter swans. With the ringing of slot machines it finally hit me, I was rolling into a Las Vegas casino! Sitting at the first row of slot machines we 3.00 experienced one of the most amazing aspects of common casino etiquette. Being more inclined to stick my hard earned cash in a money market fund than a slot machine, Las Vegas and casinos have never held 2.00 much allure for me, but I have to admit that the Las Vegas Strip with all the big casinos seemed pretty glamorous and exciting. We have played in the Casinos of Las Vegas! They want you to stay in the Casinos so they have waitresses coming around giving you free drinks and all you have to do is tip her $1 or $2 dollars and she just keeps coming back. 1.00 famous Waikiki Beach is very beautiful and is packed with swimmers from the early hours, it has lovely clean sand but the beach The geographic semantic comprehensiveness overall satisfaction is getting smaller by relevance the year and is already at relevance the base of a couple of hotels! The dreaded global warming! Since Waikiki Beach faces southwest, we were able to enjoy an extraordinary sunset right from the beach in front of our hotel, then we retired to our room where we spent the evening sitting out on the lanai, having some cocktails, and listening to the surf. Research, Asia and stay for the beautiful sunset. 19 If you are near the Waikiki Beach area, enjoy Microsoft the day at the beach, relax,

C. Travelogue Enrichment  “Where did Jack visit when he was in New York City? And how do those places look like??”

• Characterize a location l in a document 𝑢𝑙,𝑤

𝒖𝑙 = 𝑢𝑙,1 , … , 𝑢𝑙,𝑊 , where = # 𝑤⁡𝑎𝑝𝑝𝑒𝑎𝑟𝑠⁡in⁡𝑑 × 𝑝 𝑙 𝑤 , 𝑤 = 1, … , 𝑊

• Characterize an image r 𝒗𝑟 = 𝑣𝑟,1 , … , 𝑣𝑟,𝑊 , where 𝑣𝑟,𝑤 =

𝑡∈𝒯𝑟

𝑇𝑒𝑟𝑚𝑆𝑖𝑚 𝑡, 𝑤 ,

⁡𝑤 = 1, … , 𝑊

• Rank Images 𝑆𝑐𝑜𝑟𝑒𝒖𝑙 𝑟 = 𝒖𝑙 , 𝒗𝑟 𝑘

𝑘−1

𝑢𝑙,𝑤 = 𝑢𝑙,𝑤 4/27/2010

1 ∙ , log 1 + 𝒯𝑟 × 𝑒𝑥𝑝 −𝜏 ∙ 𝑣𝑟𝑘 ,𝑤

Microsoft Research, Asia

𝑟 ∈ ℛ𝑙

Keep Image Diversity 20

C. Travelogue Enrichment (2) This was our first trip to Hawaii, let alone Maui. The beaches, activities, types of accommodations, and restaurants make it a great choice for a first visit to the islands. 1) The beaches! There are so many all over the island, and all different types: white, black, even red. Large, busy, and with amenities and activities, or small, 5.00 private, and rustic (no facilities). 2) The activities! Go 4.23 snorkeling, diving, surfing, parasailing, fishing, 3.77 golfing, hiking up an old volcano, biking down the 4.00 volcano, four-wheeling on unpaved, virtually vacant dirt roads through old lava flows, driving on narrow, curvy, crowded roads through tropical forests, and helicopter 3.00 rides around the island. 3) The restaurants! There are so many fine-dining choices with all types of menus, as well as sandwich shops and the more familiar chains.

Baseline

LT-Model

4.07

4.05

3.92

ocean, 3.22 life, blue, sea, brown,3.18 travel, vacation, mountain, 3.27 green, beach, water, animal, cold, tourism, island, coral, hawaii, sand, marine, volcano, hawaii, islands, underwater, turtle, shell, nationalpark, paradise, diving, maui, snorkeling, pacific, horizon, maui, reef, creature, flipper haleakala, crater, summit, …

sky, beach, water, clouds, hawaii, sand, surf, maui, palmtrees

2.00

Lobster, lobster everywhere, and very reasonably 1.00 priced. The city was a GREAT place to find seafood. We geographic went to a Red Sox game, which was greatrelevance fun and we also went to Cape Cod, and Plymouth. This was a highly recommended vacation that we all enjoyed. Take a whale watching tour or visit the aquarium and cruise around the harbor. You can see Old Iron Sides, where the English lost their tea. We also recommend The Ghost and Graves tour which was fun, and a way to get a feel for Boston while getting some local history.

4/27/2010

semantic relevance

diversity

park, new, york, red, boston, d50, nikon, baseball, stadium, sox, fenway, pitcher, yankees, ballpark, ...

Microsoft Research, Asia

overall satisfaction

blue, boston, aquarium, lobster

ocean, boston, island, harbor, boat, whale

21

IV. Demo & Prototype 4/27/2010

Microsoft Research, Asia

22

Internal Demo: TravelScope

4/27/2010

Microsoft Research, Asia

23

Online Prototype: TravelGuide http://travel.msra.cn

Destination Recommendation

Destination Summarization

Travelogue, Q&A, route recommendation 4/27/2010

Microsoft Research, Asia

24

Thanks & Have a Nice Trip! 4/27/2010

Microsoft Research, Asia

25