dark data November 2017

Why telcos need to shine a light on ‘dark data’

Businesses in the telecommunications sector have more reason than most to stand up to the challenge presented by so-called ‘dark data’ sooner rather than later.

Telcos, like many major enterprises, are already sitting on a goldmine of data that is yet to be analysed for any business or competitive intelligence. To illustrate the point, market intelligence provider International Data Corporation estimates that up to 90 percent of all company-generated data is ‘dark’.

However, while failing to tap into potentially vital data insights in areas such as user preferences, location and time could have held back telcos from fulfilling their potential in 2017, in the coming years, such complacency could have a far more serious impact.

Dark data opportunity

Research and advisory firm Gartner first defined the term ‘dark data’, describing it as “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes”.

In other words, the data is within reach, but the tools required to index the data efficiently and provide a fuller picture to aid business decisions are simply not part of the day-to-day operations.

“Dark data is rapidly growing,” says Christopher Tozzi, the author of ‘A History of the Free and Open Source Software Revolution’ and an expert on data management.

“More and more devices, software-defined infrastructure layers and so on mean more machine data, which is a leading source of dark data.

“The big opportunity in dark data is that by definition it is data you already collect but don't use. So, getting value out of it is not particularly difficult or costly.

“In most cases, it just means integrating it into the analytics workflows that you probably already have in place for other data. In this respect, leveraging dark data is much easier than building an analytics stream from scratch.”

Integrating dark data

GeoSpock allows for the straightforward integration of previously dark data into analytics workflows, indexing location and time-based data with a system built for an era of ‘extreme data’, rather than merely ‘big data’.

However, although the tools exist today to handle this extreme data, the uncomfortable truth for industries like telecoms is that, by the end of this decade, unless proactive steps have been taken to put the right infrastructure in place, companies keen to shine a light on dark data will be fighting a losing battle.

Just looking at the established core business of a telco, global mobile data traffic is expected to increase sevenfold between 2016 and 2021, according to technology conglomerate Cisco. At a compound annual growth rate of 46 percent, by the end of the five-year period, a staggering 48.3 exabytes will be gobbled up every single month. One exabyte could hold all of the information contained in the United States’ Library of Congress in Washington, D.C.... 3,000 times over.

However, the diversifying nature of telecoms – not only providing fixed-line, mobile and internet services, but also branching out into growth areas like OTT broadcasting, Augmented Reality and Virtual Reality – adds a fresh dimension to the challenge and opportunity.

Extreme data

"Increased data growth over the past decade has created an unstructured data nightmare. It’s not just the cost to store it. Huge volumes of dark data make it harder to find what is useful and may mean we miss business opportunities.”
- Alan Daley, Research Director | Gartner

The industry is on the front line when it comes to innovation, but there will also be a simultaneous danger of being swamped unless the right provisions are in place to handle rocketing levels of data.

VR and AR traffic will increase 20-fold in the five years up to 2021, while the amount of video-on-demand traffic will nearly double to the equivalent of 7.2 billion DVDs per month over the same period. It would take more than five million years to watch the amount of video that will cross global IP networks each month in 2021.

Telcos will be supporting these unavoidable consumer trends. That is why, by 2020, global annual spending on telecommunications is expected to reach nearly $1.5 trillion.

However, managing the ‘extreme data’ associated with telco services and successfully converting dark data will be an immense challenge.

In 2021, global consumer IP traffic is expected to reach 232,655 petabytes per month – a huge increase from just under 100,000 petabytes this year. One petabyte is equivalent to 4,000 digital photos per day… of your entire life.

If telcos, for example, are only making full use of 10 percent of the data at their disposal currently, it is inevitable that the percentage will nosedive if the necessary support infrastructure is not in place.

Security compliance

Gartner estimates that through to 2021 more than 80 percent of organizations will fail to develop a consolidated data security policy across silos, leading to potential non-compliance when the floodgates are opened to extreme data.

“There is an important security angle worth discussing, which I think bears particular relevance for media and telecommunications,” Tozzi adds. “If you have dark data sitting around, it's an easy target for attackers, especially because if you are not using the data, you are probably also not monitoring it closely.

“So I think there's a security imperative, as well as a business-value one, for making better use of dark data.

“Whatever processes you build to handle dark data need to be able to scale, because the amount of dark data that the typical organization faces is likely to grow at a rapid pace. Scalability means having highly-automated processes in place.”

Whether the perspective is security, compliance, cost-saving or expanding and exploiting revenue streams, telecommunications companies will need to be equipped with the right tools to exploit the opportunities of dark data – or risk being eclipsed by the competition.

Read more about how GeoSpock can help you to lift the lid on dark data and stay ahead of the game or how we work with businesses in the telecommunications sector.





Back to GeoSpock Blog