Google Image Swirl: A Large-Scale Content-Based Image Browsing System Yushi Jing, Henry A. Rowley, Charles Rosenberg, Jingbin Wang, Marius Pasca, Yi Liu, Ming Zhao, Michele Covell {jing,har,chuck,jingbinw,mars,yliu,mingzhao,covell}@google.com

Google Inc., Mountain View, California, United States of America

Abstract We demonstrate the first large-scale image browsing system applied to 200,000 popular queries which utilizes image content to organize image search results. Given a query, the system extracts image content features such as color, shape, local features, face signatures and metadata from up to 1000 image results, and hierarchically clusters them to form an exemplar tree. A dynamic web-based user interface allows the user to navigate this hierarchy, allowing fast and interactive browsing. The exemplars of each cluster provide a comprehensive visual overview of the query results, and allow the user to quickly navigate to the images of interest.

1. Exemplar Tree This work organizes image search results into an exemplar tree. We begin organizing up to 1000 results of an image search query by building a pairwise similarity matrix among these images. The similarity computation is based on a combination of image features such as color, texture, local features, face signatures, and the metadata associated with the images. We then perform clustering to partition the search results into hierarchical clusters, each associated with a representative, or exemplar, image. The hierarchical clusters for each query are pre-computed for this demonstration. For more detail, see [3].

Figure 1. Radial layout of cluster hierarchy.

2. User Interface After hierarchical clustering has been performed, the results of an image search query are organized in the structure of a tree. A number of options exist for how to present such a tree to the user. Beyond the typical layered diagram

Figure 2. Expanding sub-clusters selected by user.

used to illustrate tree data data structures, there are many options in the literature, including hyperbolic geometry to better utilize space [4], and a variety of approaches based on tree-maps [1]. In this demonstration, we used radial layout in which each layer of the tree is arranged radially around its parent, see Figure 1. When the user selects a branch of the tree to explore, it is separated from the parent and expanded, while the parent is shrunk to make space, see Figure 2. The rearrangement is animated to allow the user to follow the change without getting lost.

3. Future Work We hope to expand this demonstration to provide the users with text annotations on the clusters, to allow browsing to related queries and to incorporate other media types. We also hope to incorporate a semantic ontology such as WordNet [2] into the generation of the exemplar tree.

References [1] B. Bederson, B. Shneiderman, and M. Wattenberg. Ordered and quantum treemaps: Making effective use of 2D space to display hierarchies. ACM Transactions on Graphics, 21(4):833–854, 2002. [2] C. Fellbaum, editor. WordNet: An electronic lexical database. MIT Press, 1998. [3] Y. Jing, H. A. Rowley, C. Rosenberg, J. Wang, and M. Covell. Visualizing web images via google image swirl. In NIPS Workshop on Statistical Machine Learning for Visual Analytics, 2009. [4] J. Lamping and R. Rao. Laying out and visualizing large trees using a hyperbolic space. In Proceedings of the 7th annual ACM symposium on User interface software and technology, pages 13–14. ACM New York, NY, USA, 1994.

Google Image Swirl: A Large-Scale Content ... - Research at Google

used to illustrate tree data data structures, there are many options in the literature, ... Visualizing web images via google image swirl. In NIPS. Workshop on ...

794KB Sizes 1 Downloads 382 Views

Recommend Documents

Google Image Swirl: A Large-Scale Content ... - Research at Google
{jing,har,chuck,jingbinw,mars,yliu,mingzhao,covell}@google.com. Google Inc., Mountain View, ... 2. User Interface. After hierarchical clustering has been performed, the re- sults of an image search query are organized in the struc- ture of a tree. A

Google Image Swirl - Research at Google
Web image retrieval systems, such as Google or Bing image search, present ... ods used to build such system and shares the findings from. 2-years worth of user ...

Content Fingerprinting Using Wavelets - Research at Google
Abstract. In this paper, we introduce Waveprint, a novel method for ..... The simplest way to combine evidence is a simple voting scheme that .... (from this point on, we shall call the system with these ..... Conference on Very Large Data Bases,.

Web-scale Image Annotation - Research at Google
models to explain the co-occurence relationship between image features and ... co-occurrence relationship between the two modalities. ..... screen*frontal apple.

A New Baseline for Image Annotation - Research at Google
indexing and retrieval architecture of Web image search engines for ..... cloud, grass, ... set has arisen from an experiment in collaborative human computing—.

Example-based Image Compression - Research at Google
Index Terms— Image compression, Texture analysis. 1. ..... 1The JPEG2000 encoder we used for this comparison was Kakadu Soft- ware version 6.0 [10]. (a).

Optimal Content Placement for a Large-Scale ... - Research at Google
CONTENT and network service providers are facing an explosive growth in ... a 1-h 4 K video takes up about 20 GB of disk [2], and today's. VoD providers are ...

A CAPTCHA Based On Image Orientation - Research at Google
Apr 20, 2009 - another domain for CAPTCHA generation beyond character obfuscation. ... With an increasing number of free services on the internet, we ..... 100. 200. 300. 400. 500. Figure 8: An image with large distribution of orientations.

Improving Access to Web Content at Google - Research at Google
Mar 12, 2008 - No Javascript. • Supports older and newer browsers alike. Lynx anyone? • Access keys; section headers. • Labels, filters, multi-account support ... my screen- reading application, this site is completely accessible for people wit

The W3C Web Content Accessibility Guidelines - Research at Google
[2], became a W3C recommendation in December 2008. WCAG 2.0 was ... ally possible to make static HTML websites WCAG 1.0 AA conforming without.

Public vs. Publicized: Content Use Trends and ... - Research at Google
social connections (e.g. Facebook personalized sites [8]) or users with similar ... the walls of a service or a large social network) to sup- port personalization ...

Video2Text: Learning to Annotate Video Content - Research at Google
learning framework for learning celebrity facial signatures from images in the .... In each iteration, we add .... reggae, soundtrack, uplifting, electronic trance. 4.

Learning to Rank with Joint Word-Image ... - Research at Google
notation that can scale to learn from such data. This includes: (i) .... tors, which is expensive for large Y . .... computing fi(x) for each i ∈ Y as the WARP loss does.

Image Saliency: From Intrinsic to Extrinsic Context - Research at Google
sic saliency map (d) in the local context of a dictionary of image patches but also an extrinsic saliency map (f) in the ... notated image and video data available on-line, for ac- curate saliency estimation. The rest of the ... of a center-surround

Large-Scale Content-Based Audio Retrieval ... - Research at Google
Oct 31, 2008 - Permission to make digital or hard copies of all or part of this work for ... Text queries are also natural for retrieval of speech data, ...... bad disk x.

CAMP: Content-Agnostic Malware Protection - Research at Google
Chrome requested between eight to ten million reputation re- quests a day. .... or compromised web sites that may infect users with malware. Browsers integrate ...

Indirect Content Privacy Surveys: Measuring ... - Research at Google
We present a design for indirect surveys and test the design's use as (1) a means to ... concerned about company/government access to health records. [39]. Almost as ... their purchase histories on web sites like Blippy [4], are will- ing to trade ..

Full Resolution Image Compression with ... - Research at Google
This paper presents a set of full-resolution lossy image compression ..... Computing z1 does not require any masked convolution since the codes of the previous.

Semi-Supervised Hashing for Scalable Image ... - Research at Google
Large scale image search has recently attracted consid- erable attention due to easy availability of huge amounts of data. Several hashing methods have been ...

Image Reconstruction in the Gigavision Camera - Research at Google
to estimate the light intensity through binary observations. We model the ... standard memory chip technology is fast and has low cost. Different from the ..... The input image is 'building.bmp' .... counting imaging and its application. Advances in