Big Data and Business Intelligence

Data has always been at the heart of successful businesses. The amount of data created has been increasing year-on-year; there’s now a phenomenal amount of data being created every day.

It’s my belief that we in the UK have the skills, expertise and passion to build the tools that will make sense of this ‘big data’ now and in the future.

Unlike traditional data, big data is made up of data sets so large that they can’t be comprehended by humans or made sense of with traditional software.

This sheer volume means it takes specialised software to analyse it and find trends, patterns, and connections.

Back to Table of Contents

Striking a balance

The government must strike the right balance between mitigating risk and encouraging innovation. This will help the UK on the path to leading the world.

We should be a country empowered and driven by this vast resource of information, not intimidated by or frightened of it.

For this to happen, the government needs to do six things:

Provide robust guidance on proportionate and relevant data privacy;
Standardise and allow access to taxpayer-funded data;
Educate the public on the benefits versus risks of big data;
Educate entrepreneurs on exploiting it responsibly and with care;
Continue to incentivise research and development, and
Continue to create opportunities for people to develop skills.

It’s up to the government to provide entrepreneurs with frameworks, both legal and technical, to securely and legally access and use this information to benefit people.

In doing so, everyone needs to approach the matter sensitively, striking a balance between risk and innovation.

Back to Table of Contents

Accessibility of data

In order for entrepreneurs to build businesses out of big data, it has to be accessible, and in a usable format.

Tried-and-tested frameworks exist for enabling access to taxpayer-funded big data, such as the one used by the DVLA.

A vetting process and licensing fee model means not only is this data secure, but it pays its own way. However, we must also protect the ability of entrepreneurs to profit from data.

The cost of accessing data must not be a barrier to entry.

By applying the DVLA model across all public sector data, charges will come down as levels of access increase.

Back to Table of Contents

Perception of risk

There’s a changing perception of risk, both in building apps around big data, and in sharing data with others.

Young people are more open, and share their data more freely, but who also expect more transparency from those who hold it.

Those who build apps may also be less risk-averse, while the laws around how data is collected, handled, secured and stored remain the same.

There is therefore a role for the government as educator, ensuring the public and entrepreneurs are educated on the legalities and realities of data privacy and sovereignty when engaging big data.

Back to Table of Contents

Informing the public

Firstly, the public must be well-informed on what is happening to their data, and what advantages they will enjoy as a result.

There are powerful voices in the private sector with whom government can work to provide expert reassurance of the security of their personal information, and the benefits of opening it up in a secure way.

Secondly, the government must ensure that people are well-informed on the legislation that protects personal data – both the general public, and those working on apps.

Attitudes to risk are changing with each generation, and in general younger people are taking a much more open stance as to what they consider to be private compared to older generations.

While this is a problem for the general public, for someone creating an app it could be disastrous.

It should be part of the government’s role to help businesses stay on the right side of regulation, in order that trust is maintained and everyone’s data is safe.

Back to Table of Contents

Creating opportunities from Big Data

The UK government and devolved administrations are already creating opportunities for businesses around big data, by requiring public sector bodies to up their game, based on the big data they collect.

We were involved in a very successful project with the Scottish Housing Regulator.

We built them a system to gather and analyse performance data for every landlord across the country.

That data is now accessible to social housing tenants, who can use it to become far better informed and empowers them to seek better standards.

We want to see that kind of revolutionary access to data across the board, enabled by systems developed by UK companies.

For something that would have an immediate impact, adding a well-defined ‘Big Data’ option to the qualification criteria for R&D tax credits will smooth the path for entrepreneurial businesses to start creating innovative, competitive solutions.

Longer-term, the government’s continued investment in technology apprenticeships will continue to pay dividends.

Young people can learn so much through hands-on experience, and with the right support they will go on to do great things themselves.

Apprenticeships within forward-thinking companies will create tomorrow’s big data entrepreneurs.

Back to Table of Contents

Towards a ‘Golden Record’

So where should the government look to start? It is worth aiming high, with the goal of creating a “golden record” for every citizen – not just within each public sector organisation, but across all organisations.

Work is already underway in this area amongst Scottish Local Authorities working in collaboration with key Government Agencies.

In Estonia, the government has embraced data-driven democracy. Citizens receive a universal ID, which they can view on their mobile device.

At any time, they can use it to check their own health records and prescriptions, pay for public transport, and check how their kids are doing in school.

They can even use it to vote. Because it has the proper safeguards in place, Estonians don’t live in a Big Brother society, but in one that is easier to navigate and that gets more done.

Back to Table of Contents

A day-to-day revolution

When EPOS and barcode technology was introduced to supermarkets, it revolutionised the way they operated. Goods could be ordered just in time to meet demand.

Stores across the country now combine EPOS information with weather data, seasonal trends and other factors to improve productivity and profitability.

Tesco capitalised on the shopping habits of millions with their revolutionary Clubcard scheme, which uses the big data processing algorithms to suggest items to customers whilst giving them unprecedented insight into purchasing trends when aligning the data to events, weather, bank holidays and so forth.

The growth of big data could revolutionise every aspect of life for individuals in the way EPOS was a game-changer for commerce.

Handled sensitively, and with the right privacy protections in place, I believe it will change all our lives for the better.

The application of big data could, for example, lead to the end of stories where a local authority mistakenly contacts a recently-deceased person, causing grief to their loved ones.

Back to Table of Contents

Just the beginning

We are at the start of the story for big data. We could have access to undreamed-of levels of information, offering opportunities to make everyone’s lives easier.

The government’s role is now to nurture entrepreneurship and innovation, educate the public, and put legislative and technical frameworks in place to exploit, manage and protect data.

Back to Table of Contents

Where Big Data meets Business Intelligence

What if we told you we could predict the best choices for your business, but rather than the mists of a crystal ball, it’s your big data that can drive this.

What is Business Intelligence? How can you use it? Where would you even start?

“Business Intelligence (BI) is an umbrella term that includes the applications, infrastructure and tools, and best practices that enable access to and analysis of information to improve and optimize decisions and performance” – Gartner Inc

Back to Table of Contents

Half a billion DVDs of data a day

According to Eric Schmidt, CEO of Google, five exabytes of data was created between the dawn of time and 2003 – enough to fill more than a billion DVDs.

That amount is now created every two days. Imagine what you could do if you were able to drill down into this data to better understand how we work? The world would be your oyster!

Business Intelligence tools aren’t a magic crystal ball that will tell you the future. But they will give you a better grasp of your business data, and help you start to forecast what the future will bring.

Back to Table of Contents

Making your data dynamic

Let’s begin by consigning the idea of static data reporting to the past. Imagine a more flexible and agile way of evaluating your data.

One that doesn’t need IT support or data analyst boffins to run queries and return complex reports. Imagine those reports update themselves from live data, and are fully-interactive and shareable.

Finally, eliminate the idea that this will take hours of your time, at your office, on your PC. There are now BI tools that let you report on rich, dynamic data, drawing on facts, figures and even social media insights.

Microsoft Office 365 has opened up huge reporting possibilities using your data through its BI toolkit.

This data doesn’t have to be stored centrally; the tools work together with your existing Microsoft suite to reveal data you may not even have been aware of.

Dive into SharePoint, critique your dashboards in Dynamics CRM and package all this together with Microsoft Power BI to get visually-engaging, interactive, up-to-date analytics that can be sourced and managed from anywhere and on any device.

Back to Table of Contents

Self-service reporting – anywhere, on any device

Imagine sitting on the train heading in for a board meeting. You really want to impress the board by getting an accurate analysis of your data. Can you do it?

You’re stuck with the old tools. You don’t know how to run queries so you’ll need to speak with IT for that.

In addition you only have your tablet, so you’ll need to wait till you’re back in the office to be on your secure desktop PC.

But imagine you could do it there and then. Office 365 would let you securely access your data from anywhere on just about any device. And query writing?

Just type in what you need to know – in plain English – and let Power BI generate the queries for you.

Back to Table of Contents

Big Data and Business Intelligence Awards

Our business intelligence system for the Scottish Housing Regulator recently took two accolades at the Holyrood Connect Awards, for Scotland’s most effective use of data and analytics in the public sector. Could we do the same for you?

The Scottish Housing Regulator picks up two awards for excellence in public sector IT for their social landlord portal.

Last night we were celebrating as our customer, the Scottish Housing Regulator (SHR), walked away with two accolades at the annual Connect Awards. Run by Holyrood Magazine, the awards celebrate excellence in Scottish public sector IT.

We worked with SHR to create their new Social Landlord Portal system, which is driving improvements in social housing by giving citizens access to better data across the sector.

SHR took home the Citizen Award, for creating an engaging, public-centric service, and the Insight Award, for the most effective use of data and analytics.

The Insight Award recognises the huge impact of the powerful self-service business intelligence tools we implemented for SHR.

These have revolutionised the way their staff work, and the information available to the public.

On top of the public investment already in place, we invested an additional £50k of our own to help SHR replace a legacy Oracle solution with brand new Microsoft Power BI capabilities.

Big Data and Business Intelligence Awards

It’s great to see the investment paying off in such a big way – not just with these awards, but with the sweeping changes the new portal is causing.

The awards recognise the fact that the solution we built with SHR is empowering citizens, improving conditions for tenants, and delivering better value for the public purse.

Back to Table of Contents