Data Mining

Vast quantities of data are generated every day by devices and websites, by Government and probably by your own organisation. These data hold many of the answers to challenging problems facing businesses and public sector organisations. But how can you usefully extract information and intelligence from such vast and dynamic data sets?

So-called Big Data is big business. From the public to the private sector, organisations are benefitting from the advent of sophisticated new techniques to help harness this valuable resource.

Data mining techniques are used where traditional statistical methods begin to struggle with the quantity of data available. It might be the sheer scale of the numbers of observations, high numbers of variables, or the speed at which the data needs to be processed. Data mining techniques enable you to extract the information these data hold to uncover hidden patterns, market trends, customer preferences and other critical business information.

Data mining tools include:

• Outlier detection to identify fraud or contaminants.
• Association rule learning to find relationships between variables and create personalised recommendations.
• Cluster analysis to identify groups of customers with similar buying behaviour or preferences.
• Classification analysis to assign new customers to distinct groups and target marketing appropriately.
• Neural networks to identify complex patterns and produce robust predictions.
• Decision trees to generate the optimal strategy for any situation.

It’s often the computational tools that unlock the real potential in big data. Big Data analytics was born with the advent of cloud-based supercomputing: platforms such as Hadoop which stores large datasets across distributed clusters and MapReduce which coordinates, combines and processes data from multiple sources. Without these tools and the expertise to set them up and use them, your chances of extracting useful insight from Big Data are greatly diminished.