Indian government initiative for Big Data

Written by Indraneel Ghose, Embassy of Switzerland in India

In 2012, the size of the big data analytics market was estimated globally at US$ 8 billion and it was expected to grow at a compounded annual growth rate (CAGR) of 45 per cent to reach US$ 25 billion in 2015. The market for India for the same period was estimated at US$ 200 million, growing at a CAGR of 83% to reach a figure of US$ 1 billion this year. These were the finding of a study commissioned by NASSCOM, the Indian IT industry body in 2012. However, the Indian market touched revenues of US$ 1 billion in 2014 and this is expected to grow to US$ 2.3 billion by the year 2018. Not all of this is export driven as the domestic market, estimated at US$ 163 million in 2014 is expected to grow to US$ 375 million by 2018. Big data analytics accounted for some 29,000 jobs in India in 2014, of which 5000 jobs were focussed on the domestic market. So big data and data analytics presents a big opportunity for IT companies in India and Bangalore is quickly becoming a hub for data science in India, as reported earlier in this blog. However, not all big data and data analytics activity in India is industry driven. The government has been a driving force in creating platforms and large datasets which will require the acquisition and manipulation of massive amounts of data. Some of these are to provide identification documents to citizens who would not other-wise have access to them easily, or to provide e-governance platforms which would simplify contacts with governmental agencies. Some of these initiatives are described below.


Digital India is an initiative of Government of India to integrate the government departments and the people of India. It aims at ensuring the government services are made available to citizens electronically by reducing paperwork. source:

Governmental initiatives in big data collection and usage

aadhaar logo

Aadhaar: Digital identity for digital India

People belonging to marginalized sections of society in India often do not have a valid proof of identity. As a result, they miss out on availing social benefits provided by the government. To overcome this Indian government launched a scheme to issue a unique 12-digit number, termed ‘Aadhaar’ (meaning ‘foundation’ or ‘support’) to every resident of India. It is an identification that a person can carry for a life time and potentially use with any service provider. Aadhaar is the world’s largest ID platform. Since the first set of Aadhaar numbers were issued in September 2010, 883,644,522 identification numbers have been issued (as of 15 July 2015; some 20 million enrolments take place per month). It is also the largest biometric programme in the world, as biometric data of each person is recorded and stored. This unique identification is now being used by various Government agencies to ensure that services and subsidies are made available only to the people to whom they are targeted and preventing leakages in the delivery mechanisms. To learn more visit


Digital Locker: the first step to a paper-free india source:

DigiLocker provides a personal storage space in the cloud to Indian citizens. Organizations that are registered with DigiLocker can push electronic copies of documents and certificates (e.g. driving license, Voter ID, School certificates) directly into citizens’ lockers. Citizens can also upload scanned copies of their legacy documents in their accounts. These legacy documents can be electronically signed using the eSign facility provided in DigiLocker. A citizen can share these electronic certificates online with various agencies while applying for the services provided by them, without having to provide paper copies. Since its soft launch on 10 February 2015 (the official launch took place on 1st July 2015), over 825,000 users have registered to use the digital lockers, with over 1.5 million documents being uploaded. Over the last 30 days an average of 22,000 documents has been uploaded every day, with the highest done day number being 47,640. To learn more visit

National Scholarship Portal Source:

National Scholarship Portal

The portal, officially launched on 1 July 2015, is one stop solution for end to end scholarship process right from submission of student application, verification, sanction and disbursal to end beneficiary for all the scholarships provided by the Government of India. The application process for students will be simplified as there is be a common application form for all scholarships and registration is online. Based on eligibility criteria, the system itself suggests the schemes for which a student is eligible. Transparency will be increased as duplicate applications will be eliminated and the successful applicants will have the scholarship amounts will be credited to the bank accounts for the beneficiaries. The platform is scalable and configurable. To date 11 scholarship schemes. 950 institutions and 155,000 students have registered on the portal. Almost 30,000 applications for scholarships have been made. To learn more visit


The Indian government launched its crowdsourcing platform MyGov in July 2014 to provide a citizen-centric platform which would enable citizens to connect with the Government and contribute towards good governance. The MyGov platform, which started with discussion around 7 themes has 215 of them today, with over half a million contributions. There are more than 1 million registered users.  A large professional data analytics team works behind the scenes to process and filter the data and key points emerging from the debates on the site and helps in gauging the popular mood about particular issues from social media sites. A global consultancy firm is helping in mining the data. Suggestions have been generated on various policy challenges such as expenditure reforms, job creation, energy conservation, skill development and government initiatives such as Clean India, Digital India and Clean Ganga. The suggestions generated on the platform have been sent to different ministries which have been asked to take them into consideration in their policy formulations. The first year of experience of the platform has been positive as it is action-oriented and it is expected that the number of users will increase dramatically in the near future. To learn more visit

Data related policy and initiatives

National data Sharing and Accessibility Policy



Since 2012, India has a policy of making all non-sensitive data generated through public funds by all agencies of the Indian government and which could be used for meeting scientific, economic and developmental goals accessible to all Indian citizens. Through the Open Government Data Platform government ministries and departments must publish datasets, documents, services, tools, and applications for public use. However, interoperability of these data sets is currently a problem. To learn more visit

The Big Data Initiative of the Government of India

Researchers from India are active, amongst others, in the fields of astrophysics, materials science, earth and atmospheric observations, energy, computational biology, bioinformatics, cognitive science, statistics etc., which generate a lot of data. These challenges require the development of advanced algorithms, visualization techniques, data streaming methodologies and analytics.

Keeping in mind the momentum that big data analytics is gaining in India, the need to build a sustainable eco-system that brings in a strong partnership across the industry players, government, and academia. With this objective, the Indian government has launched a Big Data Initiative¸ with the following aims:

  • promoting and fostering big data science, technology and applications in India and developing core generic technologies, tools and algorithms for wider applications in the government
  • understanding the present status of the industry in terms of market size, different players providing services across sectors/ functions, opportunities, SWOT of industry, policy framework (if any), present skill levels available etc.
  • carrying out market landscape surveys to assess the future opportunities and demand for skill levels in next 10 years
  • carrying out gap analysis in terms of skills levels and policy framework
  • evolving a strategic road map and action plan clearly defining of roles of various stakeholders – government, industry, academia, industry associations and others with clear timelines and outcome for the next 10 years.

To learn more visit

The institutional connection

With the growth of the big data analytics in India, research and teaching institutions in India are also joining the bandwagon by offering courses. Business analytics has been routinely taught in management schools for some time now. Technology schools have started offering post graduate courses in data sciences. Through the Big Data Initiative mentioned above, Centres of Excellence will be created, to which academics, government agencies, industry and other stakeholders will have access.

Privacy issues

Protection of sensitive personal data and information of individuals, including passwords, financial information such as bank account or credit card details, physical, physiological and mental health condition, sexual orientation, medical records and history and biometric information is provided for in the Information Technology Act of 2000 and the Information Technology (Reasonable Security Practices and Procedures and Sensitive Personal Data or Information) Rules 2011.

While the IT Act and the IT Rules regulate the collection and use of sensitive personal information, the government has proposed to enact a specific legislation on privacy (Privacy Bill) which will override the IT Rules. The Privacy Bill recognises an individual’s right to privacy and provides that it cannot be infringed except in certain circumstances, such as for protection of India’s sovereignty or integrity, national security, prevention of commission of crime and public order. Unauthorised collection, processing, storage and disclosure of personal information will be treated as infringement of privacy under the Privacy Bill. However, this bill, which was first drafted in 2013, has not yet been presented in parliament.

The last word

The debate on big data is lively. See for example an article that appeared in the Hindu (a leading newspaper) entitled Missing the Big Picture on Big Data. The author proposes that the big data revolution that is on aims at straight jacketing everybody into a flow of data, which can then be processed for optimal value extraction (read monetisation). His thesis is that while dominant consensus right now is overwhelmingly positive, if we delve deeper, the use of big data and what it would entail for the future of human lives will unravel a problematic picture. The biggest casualty would be privacy, but the graver threat would be a digital replay of colonial era exploitation, with data replacing mineral resources and raw materials as the source of value. He asks the question if analytics can find solutions to humanity’s problems and responds by saying yes, but not to the problems that human beings choose not to address. He concludes by saying that big data is no different from gold — it is firstly, and ultimately, a commodity.

You can access the full article at


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s