Here!Comes!Big!Data:!Perspectives!from!Indian!Enterprises! !
Executive)Summary)
Contents' In!this!Document!...............................................................................................................................................!1! Big!Data!Defined!................................................................................................................................................!1! Big!Data!Technology!..........................................................................................................................................!2! Key!Findings!of!the!Study!..................................................................................................................................!3! Closing!Remarks!..............................................................................................................................................!10! !
In'this'Document' ! The present document provides the key findings of the study "Here Comes Big Data: Perspectives from Indian Enterprises" commissioned by NetApp to IDC. The study examines the Big Data technology and services market in India, providing total addressable market (TAM) for the period from 2011 to 2014, and presenting end user insights on the current and future adoption levels of such solutions by Indian organizations in the large segment (with >1000 employees) across verticals such as BFSI, Media & Entertainment, Oil & Gas, Telecommunications, Government, and Retail and Wholesale.
Big'Data'Defined' IDC defines Big Data technologies as a new generation of technologies and architectures designed to extract value economically from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, and/or analysis. This definition encompasses hardware and software that integrates, organizes, manages, analyzes, and presents data that is characterized by the "four Vs": ! Volume ! Variety ! Velocity ! Value While the first three attributes of Big Data are defined in terms of technical characteristics, the fourth attribute, value, is defined by the perceived value of the data and the technology to any given organization. These are shown through a schematic representation below:
!
!
!! Figure 1 – The Big Data 4 Vs Source: IDC, 2012
!
Big'Data'Technology! Big Data technology is represented by a stack of four layers, Infrastructure, Data Organization and Management, Analysis and Discovery, and Decision Support and Automation Interface.
!
Figure 2 – Big Data Technology Source: IDC, 2012 ! Infrastructure includes the Big Data portions of: ! External storage systems purchases by enterprises and cloud service providers and direct purchases of HDDs by select large cloud service providers (It also includes supporting storage software for device, data replication, and data protection of Big Data storage assets. Internal storage installed directly on servers is included in the server segment, not the storage segment of the market sizing.) !
Server (including internal storage, memory, network cards) and supporting system software as well as spending for self-built servers by large cloud service providers
!
Datacenter networking infrastructure used in support of Big Data server and storage infrastructure (Specifically, this forecast models spending based on IDC's research into the following markets: Ethernet switches, Fiber Channel switches, InfiniBand switches, and application delivery.)
Software portion of the Big Data Technology includes: ! Data organization and management software, including parallel and distributed file systems with global namespace, highly scalable (size and structure) relational database, key-value pair (KVP) data store, graph database, XML databases, object-oriented databases, dynamic application data stores and caches, data integration, event-driven middleware, and others !
Analytics and discovery software, including search engines used for Big Data applications, data mining, text mining, rich media analysis, data visualization, and others
!
Decision support and automation software including business process or industry-specific applications such as for Web click stream analysis, fraud detection, logistics optimization, and others
Services includes business consulting, business process outsourcing, plus IT project-based services, IT outsourcing, and IT support, and training services related to Big Data implementations.
! !
'
!
Key'Findings'of'the'Study' ! !
The intelligent economy produces a constant stream of data that is being monitored and analyzed. IDC estimates that in 2011, the amount of information created and replicated surpassed 1.8ZB (1.6 trillion gigabytes). Social interactions, mobile devices, facilities, equipment, R&D, simulations, and physical infrastructure all contribute to the flow. In aggregate, this is what is called Big Data.
!
There has been a tremendous explosion of Information amongst the Indian enterprises over the last few years. According to the present survey, 40% of Indian organizations (across verticals like BFSI, Media & Entertainment, Telecommunications, and Government) at present have more than 100 TB data currently.
!
! More than 1000 TB 501TB-1000TB 251-500TB 101-250TB 51-100TB Less than 50TB 0
10
20 30 40 % of organizations
50
! Figure 3 – Data Explosion within Indian organizations Base: 299 Source: IDC, 2012
! !
!
60
!
! !
Over the last 2-3 years, with the increased technology adoption, and emergence of newer form factors, along with the rise of enterprise mobility, cloud, and social media, the data growth has been tremendous. About a third of Indian organizations covered in the survey have indicated more than 60% year-on-year (y-o-y) data growth.
! ! Data Growth in 2011 vis-àvis 2010
33
35
14
17
Less than 20% 20-40%
40-60%
>60%
!
!
Figure 4 – Data Growth within Indian organizations over the last 2 years Base: 299 Source: IDC, 2012
! ! !
There is a high prevalence of unstructured data in Indian organizations; with more than a third of Indian enterprises having mostly unstructured data.
!
18
39
42
Unstructured Structured Almost equal
!
!
Figure 5 – Prevalence of type of Data (Structured/Unstructured) amongst Indian organizations Base: 54 Source: IDC, 2012
! !
Business Documents, configuration logs, pdf documents, and video/audio/multimedia are the major data types outside databases in a major chunk of organizations.
!
!
!
!
Figure 6 – Prevalence of type of Data (Structured/Unstructured) amongst Indian organizations Base: 54 Source: IDC, 2012
! ! !
An overwhelming majority of Indian organizations believe that they are effective in handling large and complex data sets.
! Very!effective!
50%!!
Somewhat!effective!
47%!!
Don’t!know/Not!sure!
3%!!
Table 1 – How effective is the organization in managing large data sets? Base: 38 Source: IDC, 2012
! !
However, they still face umpteen number of data management challenges, the most prominent of them being delay in information making due to lack of a uniform view of data, security issues, and escalating data management issues.
!
1
Delay in information making because of too much data (47%)
2 Security Issues(45%)
3
Escalating data management costs (45%)
4
Huge strain on ICT infrastructure(e.g. Slowdown on IT Systems) (42%)
5
Lack of a uniform view of data (39%)
6 Manpower Crunch (37%)
!
!
Figure 7 – Major Data Management Challenges Base: 54 Source: IDC, 2012
! !
A major chunk of organizations do not have a formal Big Data Strategy yet. More than a half of the organizations do not distinguish between data and Big Data. Big Data management still figures as a part of the overall data management strategy for a major chunk of enterprises.
!
5 42
53
Yes
No
Don't know/Not Sure
!
!
! Figure 8 – Does your organization distinguish "data" from "big data," using distinct tools and management approaches for higher volume, complexity and dynamic data processing? Base: 54 Source: IDC, 2012
! ! !
! !
The Big Data Solutions market in India is currently at a nascent stage, with only 5% of the organizations in the large and very large segment (with more than 1000 employees) having embraced this technology. " IDC feels that this is a point of influx, and the market would experience an exponential growth in the next 2-3 years. " By 2014, IDC expects the market penetration to reach 18.4% of the organizations in the large and very large segment across different verticals.
!
A majority of the Indian organizations (52%) do not feel the need for implementing Big Data solutions yet. Around a third of the organizations (30%) feel that they lack information on the potential of Big Data and the available Big Data solutions.
! ! Lack of internal IT skills to manage such projects
29
Don’t see any competitive advantage in deploying such solutions
38
Low awareness levels/confusion about available Big Data & Solutions
30
High maintenance cost
5
High Capex / High implementation cost / Initial cost
20
Don’t see a real need for implementing Big Data Solutions
52 0
10
20 30 40 % of organizations
!
50
60
!
Figure 9 – Reasons for non-adoption of Big Data Solutions in India Base: 54 Source: IDC, 2012 !
The Big Data Solutions Market stood at US$ 58.4 million in 2011, with IT Services and Software contributing the major chunk of the overall Big Data Solutions market currently.
!
IDC expects the Big Data solutions market to grow to $153.1 million in 2014. This represents a compound annual growth rate (CAGR) of 37.8% for the period 2011-2014. " Storage would command the fastest CAGR growth (67.1% CAGR) as compared to other segments for the period 2011-2014.
!
Major IT vendors will increasingly offer both database solutions and configurations supporting Big Data, including Hadoop (and other MapReduce architectures) plus graph databases and Big Analytics. This will come through differentiation and enhancement of the products as well as through acquisitions.
!
Big Data appliances that include integrated software and hardware will expand rapidly, partly as a response to alleviate the deficit in expert IT skills associated with optimizing hardware and software to work together for Big Data workloads. These Big Data appliances will be deployed both on-premises and in the cloud.
! !
Advanced analytics will gain in prominence, which will increase the number of analytic applications that incorporate predictive models.
!
IDC has divided verticals in the following four quadrants based on the current and future adoption of Big Data Solutions: Ignore, Monetize, Strategize, and Develop. " Develop - Verticals such as BFSI and Oil & Gas are low hanging fruits and those that cannot be ignored by the Big Data Technology Vendors. " Strategize - Verticals such as Healthcare, Telecommunications, Process Manufacturing, and Government are those that are beginning to witness the benefits of implementing Big Data solutions, and where the mainstream adoption of these solutions holds tremendous value to the organizations in these verticals. Big Data Solution vendors should focus and develop suitable strategies to tap into this segment.
High
! Strategize
Capitalize
Potential for0Big0Data0Solutions
Government (inc0 R&D0institutions) Telecom Retail0&0 Wholesale Discrete0Manufacturing IT0&0ITeS
Low
Travel0&0 Transportation
Monetize
Ignore Low
Current'Adoption'of'Big'Data'Solution
Figure 10 – Vertical Assessment Source: IDC, 2012
!
Media0&0 Entertainment
Oil0&0Gas
Healthcare Process0Manufacturing
BFSI
!
High
!
! !
Possible benefits to be accrued from deploying big data solutions include scalability, possible reductions in IT cost, and a centralized view that can be used to search, browse, navigate, analyze and visualize business critical information. ! Possible reductions in IT cost (83%)
! Ability to use lowcommodity hardware (67%)
! Standardization of procedures and service (67%)
! Scalability (83%)
2
3
1
4 5
6
! Organization of data into a fabric that can be searched, browsed, navigated, analyzed and visualized (50%)
! Alignment of customer expectations with IT capabilities (33%)
!
!
!
Figure 11 – Possible benefits from Big Data technology implementations Base: 15 Source: IDC, 2012!
!
Open-source tools are being used or considered by a major chunk of users and intenders; however, main-stream adoption for Big Data projects could take some time.
!
Public Cloud Infrastructure or Storage is being thought of a viable alternative for dealing with Big Data.
!
The power to invest in Big Data Solutions rests with the CIOs.
! CIO / Top IT Executive
37
Owner / President / CEO / MD
34
Centralized IT Department
21
Centralized Board
3
Relevant IT user Department
3
0
5
10
15 20 25 % of organizations
30
35
Figure 12 – Decision making authority for Big Data technology implementations Base: 54 Source: IDC, 2012 !
40
!
Vendor Brand and Technology Leadership are the major criteria for the selection of a Big Data Solution provider.
!
Functionality of the Product
33
Technology Leadership
53
Vendor Brand/Reputation
53
0
10
20
30 % of organizations
40
!
50
60
!
Figure 13 – Criteria for selection of Big Data technology vendors Base: 54 Source: IDC, 2012
Closing'Remarks' ! Big Data technology and services as defined in this study will be the next essential capability. It is emerging as one of the pillars of the 3rd ICT platform and a foundation for the intelligent economy. Organizations should recognize the value of fact based decision making, and leverage the potential of Big Data for both cost reduction and business expansion. There needs to be a thorough assessment of the data management capabilities of the organization before embarking upon a Big Data Project. It is also important to develop an IT infrastructure strategy, as a part of the centralized approach to Big Data implementations that optimizes the server, storage, and network resources. The Big Data market brings to the table a plethora of opportunities for not only the providers of analytical solutions, but also for a host of other providers- IT consulting, hardware, and services like system integration, and infrastructure management services. This is indicative of the fact that revenue opportunities exist at all levels of the Big Data technology stack as well as in services. Big Data technology vendors need to articulate the value proposition by connecting technology capabilities to business problems or opportunities. Big Data technology is not an end in itself. Recognize the value of Big Data to drive employee and customer decisions and actions.
) For any queries, please contact: Sandeep Kumar Sharma, Senior Market Analyst, IDC Email:
[email protected] Contact Number: 988830780
