The Big Data revolution


Financier Worldwide Magazine

April 2016 Issue

April 2016 Issue

You’ve heard about it in the news, no doubt seen an increasing number of people offering to ‘analyse’ it for you and you’re more than likely responsible for generating some of it; but what is Big Data and what should it mean for you and your business?

Big Data storage and Big Data analytics are business practices that have arisen in recent years. They utilise huge quantities of business intelligence, statistics and real-time information gathering to identify trends and patterns in order to create the opportunity for a variety of business and organisational improvements, including greater production efficiency, more targeted marketing and cleaner, leaner processes – ultimately to give your business an edge over the competition.

Following a study carried out by US consultancy group Bain & Co, it was assessed that over 400 US companies that had adopted Big Data analytic processes had gained a significant lead over the rest of the corporate world. It was also established that practically any business that makes, grows and sells anything can use Big Data analytics to create efficiencies and improve business outputs through the ability to access and draw-upon or ‘mine’ the huge volumes of valuable information held within Big Data repositories. These practices have also created opportunities for companies looking to connect a different set of dots – identifying potential new customers, spotting fraud or cyber crime in its early stages, or improving products and services.

Unsurprisingly, and to an extent, controversially, one of the biggest users of Big Data analytics is the US National Security Agency (NSA) which is collecting and analysing unfathomable quantities of data in the fight against crime and terrorism across the world. Ultimately, whatever your stance on civil liberties, the technologies underpinning the NSA’s data collection and analysis programmes are working; a number of notable arrests and counter terrorist operations have cited Big Data analytics as supporting conventional military and law enforcement activity in the pursuit of global stability. The NSA’s primary data storage facility in Bluffdale, Utah is capable of holding a ‘yottabyte’ of data – that’s one thousand trillion gigabytes and is the single largest repository of data by a single organisation anywhere in the world.

So, is Big Data a good thing? It is clearly being used to good effect by a diverse group of agencies and organisations in pursuit of their strategic and corporate objectives. But as with all things that bring rewards, there are risks associated.

With regards to Big Data storage, the old adage of ‘keeping all of your eggs in one basket’ certainly springs to mind. The potential impact upon an organisation that is holding such large amounts of operational data or personal sensitive data may be all the more significant and damaging should a disruptive event occur. Arguably, organisations that hold huge quantities of data make themselves a more attractive target for a wide range of threat actors as well as increasing the need for more robust, and costly, business continuity and disaster recovery measures.

Equally, the ability to account for and maintain the quality, accuracy and integrity of Big Data is another major consideration, as well as knowing and understanding what is held and where it is at any given time. Is it possible to avoid duplication of data or using an obsolete version? How about being able to identify when something has gone wrong? When dealing with such large amounts of data, the task of carrying out all of these basic management tasks is increased exponentially. Therefore, the time and resources required as well as risk, is increased.

One of the biggest concerns facing organisations in both the public and private sectors, however, is maintaining compliance with legislative and regulatory requirements over the storage and use of Big Data. Current data protection legislation states very clearly that information held by an organisation must be done so to fulfil a defined purpose. Appropriate security measures should be put in place to protect that information and the information held no longer than required to achieve the defined purpose.

Holding data without a defined and fully justified reason, holding that data beyond the point at which it has been used to achieve that purpose and a failure to provide adequate protection for that data could all constitute a breach of data protection legislation that may result in a financial penalty, currently set at £500,000. This fine does not take into account additional reputational damage and the impact upon future income. This issue is compounded both by the large amounts of information held within Big Data repositories, as well as the imminent arrival of new European Union Data Protection rules that will almost certainly mean that organisations must ensure all information is used and secured appropriately. Failure to do so could result in even higher fines that could, in the near future, include a percentage of global turnover.

So what can be done to ensure that the risks to organisations using Big Data storage and analytics are mitigated and the potential rewards from such practices maximised?

The first and most important thing to consider is ‘if you don’t need it, don’t keep it’. The burden resulting from storing information that has no defined purpose or adds no value to the organisation can be significant, both in terms of financial cost as well as time and resources. This should be considered against the risks mentioned above and the increased impact and consequences that may result from those organisations storing large amounts of data. Organisations need to understand the legislation and should produce documented evidence in the form of policies and procedures to outline the rationale behind the use of this information; this should include a retention and disposal schedule, detailing what information should be retained and for how long.

Responsibility for Big Data should be apportioned to an appropriate person or department within the organisation who will be responsible for that data, as well as identifying and managing any risks to that data. This ownership should form part of a wider governance framework where responsibility and accountability for data, big or otherwise, is properly allocated.

Consideration must be given to the areas of incident management, disaster recovery and business continuity. Appropriate and proportionate measures should be applied to Big Data storage and analytic processes to ensure that the organisation is able to provide a suitable response in the event of something going wrong. These measures should also be tested on a periodic basis through tabletop exercises or full disruption tests in order to assess their effectiveness and adequacy.

If any of your Big Data processes are outsourced to a third party, it is worth considering what measures the provider has in place to protect your data and that appropriate assurance activities are carried out, providing evidence of due diligence in how the data is used and protected. One major consideration at the moment is the geographical location of data storage facilities. In a world of cloud computing and data hosting, organisations are not always fully aware of where their data is actually physically stored. There are many legal ramifications relating to the storage of data in certain countries and consideration should at least be given to ensuring that if your data is stored outside of the UK or EU, then your provider is fully compliant with appropriate data protection legislation.

The use of Big Data has heralded an age of innovation and an introduction to new and exciting business practices that have brought many tangible benefits to organisations across the world. If the use of such practices would benefit your organisation, then go for it. However, be mindful of the risks associated with it and be sure to apply appropriate controls and measures to protect the data and ultimately your organisation from the potential pitfalls that are associated with this way of doing business.


Paul Oughton is a security consultant at Advent IM. He can be contacted on +44 (0)121 559 6699 or by email:

© Financier Worldwide

©2001-2016 Financier Worldwide Ltd. All rights reserved.