Google: history, products and its secrets Ricardo de Paula Castro [
[email protected]] - Date: 2008-12-28
History This article intends as main purpose describe a bit more about Google’s products and applications, secrets and its power search engine. But it is impossible don’t mention the Google’s history and how it was founded. The Google Inc. is an American company that was co-founded by Larry Page and Sergey Bin while Ph.D. students from Stanford University. The company was firstly incorporated as a privately held company on September, 1998. Google's initial public offering took place on August, 2004, raising US$1.67 billion, making it worth US$23 billion. It is really impressive to any businessman or starter students with brilliant ideas. The name Google has originated from a misspelling of “Googol” that means to 10100, the number represented by a 1 followed by one-hundred zeros. In a brief the Google Inc. is specialized in internet search and online advertisement that beyond these two topics intends much more that search and advertisement. Google applications are well-known web services which its web search is the main factor of Google success. Following Nielsen (2007) the Google web search is basically the most search engine used by internet users with 53.6% of market share, ahead of Yahoo with 19.9% and Live Search with 12.9% (1). Might one reason of success comes from of manner how the search engines works which indexes billion or more of web pages daily or better, all internet contents, amazing. Google has also employed the web search technology into other search services, including Image Search, Google News. The price comparison site Google Product Search, the interactive usenet archive Google Groups, Google Maps, Picasa, Gmail and its features like spam-filtering technology and capability also to search emails. Another interesting multimedia application is the YouTube that was bought by Google for US$1.65 billion in stock in 2006. Google has also bought some others small internet companies considered strategic in its business like Double-click, Postini, Pyra Labs, Orkut and Jot Spots.
Products Google Earth is very famous interactive application mapping program powered by satellite and aerial imagery that covers the vast majority of the planet. Google Earth is generally considered to be remarkably accurate and extremely detailed. Many major cities in the planet have such detailed images that one can zoom in close enough to see vehicles and pedestrians clearly. Consequently, there have been some concerns about national security implications in despite of the images has been not updated constantly. Google has many others products through the Google Labs not released yet due it are still being tested for use by general public. One good differential on Google Search is regarding to logic engine based on Boolean Logic created by mathematician Britain George Boole. Therefore the Google engine allows finding words, texts and so on using logic values conditioned to: • • •
The value must be true or false The value must not be true and false at same time If true, it is defined as 1 and if false it is defined as 0(zero)
The Boolean logic is based on: AND – The Google engine will search terms and text based on conditions true only regarding to terms insert in search field. For instance, take the text “Debian AND Linux” to search does the Google engine search evidences only related to “Debian” AND “Linux” at same document, page, file and so on. As we know there are a lot of Linux distributions beyond to Debian but only evidences for Debian will be showed. My comment in this specific example is the term Linux has a lot of evidences related to and may will show evidences related to others distributions as well that require you change the filter for that using only Debian as example. OR – In that case the engine will search evidences considering only one term and or all terms wanted. For instance, take the text Debian and Linux used in previous example. At this time will type “debian or Linux”. Now the engine will show results related to Debian and Linux that could be not in the same document, page, file or any kind of midia wanted. NOT – This operator can be useful like a filter. In Google engine it is used as a signal ( - ) on terms and text that you don’t want to show. Still considering the example used above, “Debian Linux”, we can use the operator NOT if we want to search evidences in Linux but not related to Debian. In that case we will type “Linux – Debian” 1-
The rank is based on information released by RankAbove.com for 2007.
INVERTED COMMAS (“ “) – In despite of Google engine does search terms that have two or more words, inverted commas can be useful to filter some terms that are really need in a complete expression. For instance, when finding you name in Google using inverted commas, Google will only show evidences with the complete name and not pieces of them. In spite of Boolean logic is very useful, it does not correspond to 2% of Google engine features. The advanced search on Google does really good to find something with 99,99% of success. Looking on images, it is possible to search images on Google using to “Image Advanced Search” as filter to get what is really needed. So is possible to get advanced search images with content types, size, file type, domains and safe search. Below in figure 1, is possible to see these features.
Figura 1: Google Advanced Image Search
The Google Advanced Search is of course applicable to texts, terms, files and so on. In that way is possible to do an advanced search in texts with following terms: • • • • •
Idioms Format file Domains Books Codes
The figure 2 below show the Google Advanced Search that is possible to see these features.
Figura 2: Google Advanced Search
The Google Calendar another service available by Google to make your live easier. There is possible to find dates using Julian calendar or Gregorian calendar that is used nowadays. The Microsoft Office has a good feature that makes possible to open files extension supported like xls, docx, ppt, accdb and so on directly direc from Google Search results.. For instance, when searching a “filetype:xls”, all the results showed can be possible to open on Excel tool using the web link copied in Google. For that only do a copy in Google page address searched and paste the address on Excel function at “fx= web address” . The result is like open a file within your computer. Regarding to advertisers I am quite convinced that the Internet is the main way to sell your idea, products and solutions making money without or to pay for that when required.. So Google AdWords is a powerful way to people when they are actively looking for information about your products and services online, and send targeted visitors directly to what you are offering.. It is based on cost-per-click cost pricing that make easy to control your costs and you only pay when people click on your ad. The Google AdSense ense is a good feature that make possible you earn arn more revenue from your website, while providing visitors with a more rewarding online Figura 3:: Google search results and its experience and automatically delivers text and image AdWords ads that are precisely targeted to your site and your site content—ads so well-matched, matched, in fact, that your readers adding Google Web Search to your site. If you get interested, will actually find them useful when you add please get more information at Google Advertising Program through http://www.google.com/intl/en/ads/ And about to Search Engine; Is Google a good web search engine? Yes, it is really good web search engine, perhaps much more than needed, mainly to hackers and crackers. For instance when using special character “ * “ make you possible to find any information that it does might comple complete with the word inserted in Google Search. Let to see some example: •
•
•
“*@domain.com”: this example can be easily used by a hacker to search everything regarding to a web company based on its domain. For instance, the “*@kaizen.com.br” showed 1450 results that hat is possible to many articles, forums, letters, issues and so on that this company have had with its customers. Might a hacker could take some advantage to steal confidential information to be sent to a competitor. “*numbers”: this example can be easil easilyy used by a cracker to find bank accounts or credit cards numbers based on some numbers that do default by banks. Of course that any number inserted to be search with “*“ will show a uncalculated numbers of results. For that, a kid cracker could easily use to Google Advanced Search to filter the results wanted to find confidential information about people like its ID, home address, bank account and credit card budget.. Another way to filter the information is using “filetype” that you want to search. For instance tance is difficult to believe how many people leave your personal data in excel spreadsheet like ID, home address, personal number, bank account and credit cards numbers in files like spreadsheet spreadsheet.. So in that way a kid cracker only need to insert “8240 1340 * filetype:xls” supposing the 8 first number are regarding to credit card number most used in the world. “inurl:file”: let us take a hack sample to explain it. Some web editors like MS FrontPag FrontPage allow add some forms into website in order to collect feedback from users. It is default be stored all this information in path called_private\form_results.txt on the tree created by FrontPage. The file “private” usually has ha all profile and its permissions correctly set in order to get limit access to data base, but sometimes or most of them, them, people forget to set this file to set not correctly.. In that way does the web site vulnerable to any attack by hackers et al. To see yourself, type in Google Search the following text: inurl:”form_results.txt” and see the results.
Another good tool is the Gmail is a free POP3 and IMAP webmail service service provided by Google. In the United Kingdom and Germany it is officially called Google Mail. With an initial storage capacity of 1 GB, Gmail drastically increased the webmail standard for free storage from the 2-4MB 2 4MB its competitors offered at that time. The service currently offers over 7250 MB of continually increasing (at a rate of 353.9 KB per day) free storage with a additional dditional storage ranging from 10 GB to 400 GB available for $20 to $500 (US) per year.
Gmail has a search-oriented interface and a "conversation view" similar to an Internet forum. Software developers know Gmail for its use of the Ajax (3) programming technique that runs on the Google GFE/1.3 server which runs on Linux Another good application is Google Video; it is a free video sharing website and also a video service from Google that allows anyone to upload video clips to Google's web servers as well as make their own media available free of charge; some videos are also offered for sale through the Google Video Store. Uploaded videos were saved as a .gvi files under the "Google Videos" folder in "My Videos" and reports of the video(s) details are logged and stored in the user account. The report sorts and lists the number of times that each of the users videos have been viewed and downloaded within a specific time frame. These range from the previous day, week, month or the entire time that the videos have been there for. Totals are calculated and displayed and the information can be downloaded into a spreadsheet format or printed out. Competing services include iFilm, Metacafe, Veoh, blip.tv, IKlipz.com, and Outloud.tv. In October, 2006 Google agreed to buy former competitor YouTube for $1.65 billion in stock, but YouTube will remain a separate service under its own identity for the near future, though Google Video searches include YouTube results as well. Google announced in June, 2007 that the Google Video search results would begin to include videos discovered by their search crawlers on other hosting services, in addition to YouTube's and their own uploads. Search result links now open a frameset with a Google Video header at the top, and the original player page below it, similar to the way the Google Images search results are presented. Now we came to Google Desktop (2) is desktop search software made by Google for Mac OS X, Linux, and Microsoft Windows. The program allows text searches of a user's e-mails, computer files, music, photos, chats, Web pages viewed, and other "Google Gadgets." Google Desktop have the following features:
23-
•
File indexing: After initially installing Google Desktop, the software completes an indexing of all the files in the computer. And after the initial indexing is completed, the software continues to index files as needed. Users can start searching for files immediately after installing the program. After performing searches, results can also be returned in an Internet browser on the Google Desktop Home Page much like the results for Google Web searches.
•
Sidebar: Screenshot of gadgets. Google Desktop running on Microsoft Windows Vista. A prominent feature of Google Desktop is the Sidebar, which holds several common Gadgets and resides off to one side of the desktop. The Sidebar is available with the Microsoft Windows version of Google Desktop only. The Sidebar comes pre-installed with the following gadgets: Email - a panel which lets one view one's Gmail messages. Scratch Pad - here one can store random notes; they are saved automatically Photos - displays a slideshow of photos from the "My Pictures" folder (address can be changed) News - shows the latest headlines from Google News, and how long ago they were written. The News panel is personalized depending on the type of news you read. Weather - shows the current weather for a location specified by the user. Web Clips - shows recent posts from RSS news feeds. Google Talk - If Google Talk is installed, double clicking the window title will dock it to one's sidebar.
•
Quick Find: When searching in the sidebar, deskbar or floating deskbar, Google Desktop displays a "Quick Find" window. This window is filled with 6 (by default) of the most relevant results from one's computer. These results update as one types so that one can get to what one wants on one's computer without having to open another browser window.
•
Deskbars: Deskbars are boxes which enable one to type in a search query directly from one's desktop. Web results will open in a browser window and selected computer results will be displayed in the "Quick Find" box (see above). A Deskbar can either be a fixed deskbar, which sits in one's Windows Taskbar, or a Floating Deskbar, which one may position anywhere one wants on one's desktop.
Google Desktop and its features were available in December 2008. More info can be found at http://desktop.google.com/pt/BR/. AJAX Programming is a group of interrelated web development techniques used for creating interactive web applications or rich internet applications.
•
Email indexing: Google Desktop includes plugins that allow one to index and search through the contents of local Microsoft Outlook, IBM Lotus Notes, and Mozilla Thunderbird email databases, outside of the client applications' built built-in search functions. For Lotus Notes, only local databases are indexed for searching. Google Desktop's email indexing feature is also integrated with Google's web web-based based email service, Gmail; it can index and search the email messages in one's Gmail account.
•
Gadgets & plug-ins: Desktop gadgets are interactive mini-applications mini applications that can be placed anywhere on the user's desktop – or docked in the Sidebar – to show new email, weather, photos, and personalized news. Google offers a gallery of pre-built pre gadgets for download on the official website. For developers, Google offers an SDK and an official blog for anyone who wants to write gadgets or plug-ins plug ins for Google Desktop. An automated system creates a developer hierarchy called the "Google Desktop Hall of Fame", where programmers can advance based on their gadgets' number and popularity.
And about to Google Picasa ? It is a software application for organizing and editing digital photos originally created by Idealab and now owned by Google. "Picasa" is a blend of the name of Spanish painter Pablo Picasso, the phrase mi casa for "my house," and "pic" for pictures. In July 2004, Google acquired Picasa and began offering Picasa for free download.
Figura 4: Google SketchUp
Here we have a tool might no so much known by lazes or common users, the Google SketchUp is a 3D modeling program designed for architects, civil engineers, filmmakers, game developers, and related professions. It also includes features to facilitate the placement of models models in Google Earth. It was designed to be more intuitive, flexible, and easier to use than other 3D CAD programs. It is marketed as an easy-to-use use conceptual tool with a simple interface. A well known feature in the SketchUp World is the 3D Warehouse. User Users can use their Google Accounts to upload models, and then browse the 3D Warehouse for many components and models. Some of its key features and uses include: • • • • •
A "smart" drawing cursor (inference) system that allows users to draw 3D objects using a 2D screen and mouse. Simple massing study capability via "push-pull". "push "Follow Me," which creates 3D forms by extruding 2D surfaces along predetermined paths. Ability to animate camera and sun movements. Interoperability with Google Earth
There is recent product uct application released as Google Chrome. It is a free web b browser developed by Google and based on the WebKit layout engine and application framework. It was first released as a beta version for Microsoft Windows on September, September 2008. The name is derived from rom the graphical user interface frame, or "chrome", of web browsers. The public stable stable release was in December December, 2008. The GrandCentral is another free Google Google-owned Internet service that uses VoIP to link custo customers' phone numbers together. The service is currently in private beta, and no new invitations have been sent out since March, 2007. Whenever the customer's GrandCentral number is called, the chosen phone(s) ring, allowing the customer to answer his or her chosen phone. The phones can be chosen based on time of day; for example, the home phone can be turned off during business hours, and based on groups (Family, Friends, Work, Web Button, Others) or personalized based on an address book. Google have others products that might were not listed here. The purpose of Google products listed in this article was to pointed out the most relevant and promissory products.
Secrets May many people think that Google secret is saved in code of Google Search Engine since it was the kick off for Google Empire. For sure it is part of success, but it does not represent 1% of total. The secret is saved on Google infrastructure in the world, its operability, stability, high performance and mainly how people handle and manage all Google projects or work for that. In that way I can say that Google Platform is the main “spine” of the company. Google Platform requires large computational resources in order to provide their service. The resources are classified by network topology, index, servers’ types, hardware and software.
The Network topology does some people estimate that Google maintains over 450,000 servers, arranged in racks located in clusters in cities around the world, with major centers in Mountain View, California; Council Bluffs, Iowa; Herndon, Virginia; Lenoir, North Carolina; Atlanta, Georgia; Dublin, Ireland; Saint-Ghislain, Belgium; Zürich, Switzerland; Tokyo, Japan; Beijing, China; and new facilities constructed in The Dalles, Oregon. In 2009 Google is planning one of its first sites in the upper midwest to open in Council Bluffs, Iowa close to abundant wind power resources for fulfilling green energy objectives and proximate to fiber optic communications links. When an attempt to connect to Google is made, DNS servers resolve www.google.com to multiple IP addresses, which acts as a first level of load balancing by directing clients to different Google clusters. (When a domain name resolves to multiple IP addresses, typical implementation of clients is to use the first IP address for communication; the order of IP addresses provided by DNS servers for a domain name is typically done using Round Robin policy.)(4) Each Google cluster has thousands of servers, and upon connection to a cluster further load balancing is performed by hardware in the cluster, in order to send the queries to the least loaded web server. This makes Google one of the biggest and most complex content delivery networks.
Figura 5: Example of Google Servers Load Balancing
Racks are custom-made and contain 40 to 80 servers (20 to 40 1U servers on either side), while new servers are 2U Rackmount Figura 6: Example of Google systems. Each rack has a switch. Servers are connected via a 100 Servers Rack Mbit/s Ethernet link to the local switch. Switches are connected to core gigabit switch using one or two gigabit uplinks. Main index: since queries are composed of words, an inverted index of documents is required. Such an index allows obtaining a list of documents by a query word. The index is very large due to the number of documents stored in the servers. Server types: Google's server infrastructure is divided in several types, each assigned to a different purpose:
Google load balancers take the client request and forward it to one of the Google Web Servers via Squid proxy servers. Squid proxy servers take the client request from load balancers and return the result if present in local cache otherwise forward it to Google Web Server. Google web servers coordinate the execution of queries sent by users, then format the result into an HTML page. The execution consists of sending queries to index servers, merging the results, computing their rank, retrieving a summary for each hit (using the document server), asking for suggestions from the spelling servers, and finally getting a list of advertisements from the ad server. Data-gathering servers are permanently dedicated to spidering the Web. Google's web crawler is known as GoogleBot. They update the index and document databases and apply Google's algorithms to assign ranks to pages.
Each index server contains a set of index shards. They return a list of document IDs ("docid"), such that documents corresponding to a certain docid contain the query word. These servers need less disk space, but suffer the greatest CPU workload. Document servers store documents. Each document is stored on dozens of document servers. When performing a search, a document server returns a summary for the document based on query words. They can also fetch the complete document when asked. These servers need more disk space. Ad servers manage advertisements offered by services like AdWords and AdSense. Spelling servers make suggestions about the spelling of queries.
Server and its hardware: Servers are commodity-class x86 PCs running customized versions of Linux. The goal is to purchase CPU generations that offer the best performance per dollar, not absolute performance. Estimates of the power required for over 450,000 servers range upwards of 20 megawatts, which cost on the order of US$2 million per month in electricity charges. Specifications: Upwards of 15,000 servers ranging from 533 MHz Intel Celeron to dual 1.4 GHz Intel Pentium III (as of 2003[update]). A 2005 estimate by Paul Strassmann has 200,000 servers, while unspecified sources claimed this number to be upwards of 450,000 in 2006.[3] One or more 80GB hard disks per server (2003) 2–4 GB of memory per machine (2004) The exact size and whereabouts of the data centers Google uses are unknown, and official figures remain intentionally vague. In a 2000 estimate, Google's server farm consisted of 6000 processors, 12,000 common IDE disks (2 per machine, and one processor per machine), at four sites: two in Silicon Valley, California and two in Virginia. Each site had an OC-48 (2488 Mbit/s) (5) internet connection and an OC-12 (622 Mbit/s) connection to other Google sites. The connections are eventually routed down to 4 x 1 Gbit/s lines connecting up to 64 racks, each rack holding 80 machines and two Ethernet switches. The servers run custom server software called Google Web Server. One of the largest Google data centers is located in the town of The Dalles, Oregon, on the Columbia River, approximately 80 miles from Portland. Codenamed "Project 02", the new complex is approximately the size of two football fields, with cooling towers four stories high. The site was chosen to take advantage of inexpensive hydroelectric power, and to tap into the region's large surplus of fiber optic cable, a remnant of the dot-com boom. A blueprint (6) of the site has appeared in print. In a 2008 book,[15] the reporter Randall Stross wrote: "Google's executives have gone to extraordinary lengths to keep the company's hardware hidden from view. The facilities are not open to tours, not even to members of the press." He wrote this based on interviews with staff members and his experience of visiting the company. Servers and its software: Most of the software stack that Google uses on their servers was developed in-house. Google has acknowledged that Python has played an important role from the beginning, and that it continues to do so as the system grows and evolves. The software that runs the Google infrastructure includes: Google File System BigTable Chubby lock service MapReduce and Sawzall programming language Protocol buffers Server operation: Most operations are read-only. When an update is required, queries are redirected to other servers, so as to simplify consistency issues. Queries are divided into subqueries, where those sub-queries may be sent to different ducts in parallel, thus reducing the latency time.
5-
6-
OC-xx is an Optical Carrier levels describe a range of digital signals that can be carried on SONET fiber optic network. The number in the Optical Carrier level is directly proportional to the data rate of the bit stream carried by the digital signal. A blueprint is a type of paper-based reproduction usually of a technical drawing, documenting an architecture or an engineering design. More generally, the term "blueprint" has come to be used to refer to any detailed plan.
Conclusion Looking at some ranks on web, we can find Google has 31% of the online advertising market revenue, and 56% of the market share for online searches. It is still expecting Google business growing up more and more. Following Jeremy Article (2007), Google spends 20%-25% of its resources pursuing new ventures. Google has recently bought the acquisition of DoubleClick after a fierce fight with Microsoft. The DoubleClick is the number 1 online ad agency. Also Google has come out with its own productivity software suite including word processors, spreadsheets, database software, presentation software, instant messenging and video conference software to compete with Microsoft in the corporate arena. Microsoft charges about $500 a year for its suite per user, whereas Google will only charge $50. Now that Google has entered in software market playing in many strategic areas like Mobile Phone (Google Mobile), the main mobile companies are now negotiating how Google and its market share can be part of their solutions. Recently the HTC and T-Mobile made a deal with Google to sell mobiles with Google OS that make possible features from iPhone and BlackBerry in one unique cell phone. Google has also constantly expanding its platform through massive server farms and fiber-optic capacity. Google is also currently working on Artificial Intelligence systems, an example of that is its spell checking system, its language translator, and eventually a search engine that almost knows what you're looking for as soon as you start looking for it. Looking to Google financials, it seems that one of the largest technology companies and Internet world is also going to be affected by the financial crisis seen in 2008 and without forecast to end. Google is one of them, because although the largest source of financing that has its advertising services (Adsense / Adwords), another of its contributory sources that had so far was the investment of many people through their actions listed on the U.S. technology index Nasdaq named. With the problems many of their stock shares have been sold, so that the company has had to deal with many of these payments because nobody wanted done with them. This situation has occurred since the beginning of IPO in Google, people could buy stock, but never lose control if the company has repurchased and those that were sold or held for the remaining shares as, or distributed as part of the payment to their employees. As of this poor financial situation and business Google was raised new trade agreements in order to save their economic situation: the treaty raised to film companies for the distribution of films by Youtube or who tried to do with Yahoo for sharing both of advertising platforms such as rights of many of its advertisers A union that would have generated a very positive synergy for Google, doing that could increase their incomes by up to 32%; this operation was truncated, and they are starting to run on rumors that 10,000 of the 25,000 workers that Google has a direct or indirect worldwide are off ‘to’ and offset a change in company stock. This measure has a double meaning, first cutting large amount of fixed costs (salaries of such employees) and second, a financial boost by the arrival of large number of shareholders that they would be 10,000 employees. This second action if it could be important for the company that has seen only this year and as you can see in the upper area has fallen from a rate of 600 to less and $ 300 per share. In the words of the co-founder of Google, Mr. Sergey Bin: “It is undeniable that the number of workers is too high for the current financial situation” Judging from last quarter's call and recent "what me worry" comments about the economy relative to Google, I expect Google CEO Eric Schmidt to maintain his "It's Always Sunny in Silicon Valley" means. But he could surprise us all by adding a cautionary note about a hiring freeze or some such conservative measure. It's probably too soon, but I expect more conservative actions from Google in early 2009 as many companies has been doing as well
REFERENCE: 1. CARMONA, Tadeu. Segredos do Google. 2º Edição. Digicel Books, 2006, 128p. 2. 3.
NIELSEN, NetRatings. RankAbove: Industry-specific search engine optimization. Available at: http://rankabove.com/p/top-10-search-providers-august-2007/
4.
WIKIPEDIA. Google Search. Available at: http://en.wikipedia.org/wiki/Google_search
5.
WIKIPEDIA. Google Desktop. Available at: http://en.wikipedia.org/wiki/Google_Desktop
6.
C JEREMY. The Google Future. Available at: http://team1co.blogspot.com/2007/04/future-ofgoogle.html
7.
HOF, ROB: Is Google too powerful. Available at: http://www.businessweek.com/the_thread/techbeat/archives/2007/03/is_google_too_p.html?cha n=search
