{"id":2177,"date":"2016-10-03T15:38:10","date_gmt":"2016-10-03T22:38:10","guid":{"rendered":"https:\/\/www.springboard.com\/?p=2177"},"modified":"2024-06-03T07:53:09","modified_gmt":"2024-06-03T14:53:09","slug":"data-mining-python-tutorial","status":"publish","type":"post","link":"https:\/\/www.springboard.com\/blog\/data-science\/data-mining-python-tutorial\/","title":{"rendered":"Data Mining in Python: A Guide"},"content":{"rendered":"\n
\n
\n
\n
\n
\n

Data mining and algorithms<\/span><\/h2>\n

Data mining is t<\/span>he process of discovering predictive information from the analysis of large databases. For a data scientist, data mining can be a vague and daunting task – it requires a diverse set of skills and knowledge of many data mining techniques to take raw data and successfully get insights from it.  You’ll want to understand the foundations of statistics and different programming languages<\/a> that can help you with data mining at scale.  <\/span><\/p>\n

This guide will provide an example-filled introduction to data mining using Python, one of the most widely used data mining tools – from cleaning and data organization to applying machine learning algorithms. First, let\u2019s get a better understanding of data mining and how it is accomplished.<\/span><\/p>\n

A data mining definition <\/b><\/p>\n

The desired outcome from data mining is to create a model from a given data set that can have its insights generalized to similar data sets. A real-world example of a successful data mining application can be seen in <\/span>automatic fraud detection from banks and credit institutions.<\/span><\/p>\n

Your bank likely has a policy to alert you if they detect any suspicious activity on your account – such as repeated ATM withdrawals or large purchases in a state outside of your registered residence. How does this relate to data mining? Data scientists created this system by applying algorithms to classify and predict whether a transaction is fraudulent by comparing it against a historical pattern of fraudulent and non-fraudulent charges. The model \u201cknows\u201d that if you live in San Diego, California, it\u2019s highly likely that the thousand dollar purchases charged to a scarcely populated Russian province were not legitimate.<\/span><\/p>\n

That is just one of a number of the powerful applications of data mining. Other applications of data mining include genomic sequencing, social network analysis, or crime imaging – but the most common use case is for analyzing aspects of the consumer life cycle. Companies use data mining to discover consumer preferences, classify different consumers based on their purchasing activity, and determine what makes for a well-paying customer – information that can have profound effects on improving revenue streams and cutting costs.<\/span><\/p>\n

If you’re struggling to find good data sets to begin your analysis, we’ve compiled 19 free data sets for your first data science project<\/a>.<\/p>\n

What are some data mining techniques?<\/b><\/p>\n

There are multiple ways to build predictive models from data sets, and a data scientist should understand the concepts behind these techniques, as well as how to use code to produce similar models and visualizations. These techniques include:<\/span><\/p>\n