{"id":12940,"date":"2021-10-08T11:52:13","date_gmt":"2021-10-08T18:52:13","guid":{"rendered":"https:\/\/www.springboard.com\/?p=12940"},"modified":"2023-06-29T02:15:33","modified_gmt":"2023-06-29T09:15:33","slug":"decision-tree-implementation-in-python","status":"publish","type":"post","link":"https:\/\/www.springboard.com\/blog\/data-science\/decision-tree-implementation-in-python\/","title":{"rendered":"Decision Tree Implementation in Python with Example"},"content":{"rendered":"\n

A decision tree is a simple representation for classifying examples. It is a supervised machine learning technique where the data is continuously split according to a certain parameter. Decision tree analysis can help solve both classification & regression problems. The decision tree algorithm breaks down a dataset into smaller subsets; while during the same time, an associated decision tree is incrementally developed. A decision tree consists of nodes (that test for the value of a certain attribute), edges\/branch (that correspond to the outcome of a test and connect to the next node or leaf) & leaf nodes (the terminal nodes that predict the outcome) that makes it a complete structure. In this blog post, we are going to learn about the decision tree implementation in Python, using the scikit learn Package. <\/p>\n\n\n\n

\"Decision
Source: Javatpoint<\/a><\/figcaption><\/figure>\n\n\n\n

For our analysis, we have chosen a very relevant, and unique dataset which is applicable in the field of medical sciences, that will help predict whether or not a patient has diabetes, based on the variables captured in the dataset (more datasets here). This information has been sourced from the National Institute of Diabetes, Digestive and Kidney Diseases and includes predictor variables like a patient\u2019s BMI, pregnancy details, insulin level, age, etc. Let\u2019s dig right into solving this problem using a decision tree algorithm for classification.<\/p>\n\n\n\n

Decision Tree Implementation in Python<\/strong><\/h2>\n\n\n\n

As for any data analytics problem, we start by cleaning the dataset and eliminating all the null and missing values from the data. In this case, we are not dealing with erroneous data which saves us this step. <\/p>\n\n\n\n

1. We import the required libraries for our decision tree analysis & pull in the required data<\/p>\n\n\n\n

# Load libraries
import pandas as pd
from sklearn.tree import DecisionTreeClassifier # Import Decision Tree Classifier
from sklearn.model_selection import train_test_split # Import train_test_split function
from sklearn import metrics #Import scikit-learn metrics module for accuracy calculation
col_names = ['pregnant', 'glucose', 'bp', 'skin', 'insulin', 'bmi', 'pedigree', 'age', 'label']
# load dataset
pima = pd.read_csv(\"pima-indians-diabetes.csv\", header=None, names=col_names)<\/pre>\n\n\n\n
Let\u2019s check out what the first few rows of this dataset look like<\/p>\n\n\n\n
pima.head()<\/pre>\n\n\n\n
2. After loading the data, we understand the structure & variables, determine the target & feature variables (dependent & independent variables respectively) <\/p>\n\n\n\n
#split dataset in features and target variable
feature_cols = ['pregnant', 'insulin', 'bmi', 'age','glucose','bp','pedigree']
X = pima[feature_cols] # Features
y = pima.label # Target variable<\/pre>\n\n\n\n
3. Let\u2019s divide the data into training & testing sets in the ratio of 70:30. <\/p>\n\n\n\n
# Split dataset into training set and test set
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1) # 70% training and 30% test<\/pre>\n\n\n\n
As a standard practice, you may follow 70:30 to 80:20 as needed. <\/p>\n\n\n\n
4. Performing The decision tree analysis using scikit learn <\/p>\n\n\n\n
# Create Decision Tree classifier object
clf = DecisionTreeClassifier()
# Train Decision Tree Classifier
clf = clf.fit(X_train,y_train)
#Predict the response for test dataset
y_pred = clf.predict(X_test)<\/p>\n\n\n\n
5. But we should estimate how accurately the classifier predicts the outcome. The accuracy is computed by comparing actual test set values and predicted values.<\/p>\n\n\n\n
# Model Accuracy, how often is the classifier correct?print(\"Accuracy:\",metrics.accuracy_score(y_test, y_pred))

Accuracy: 0.6753246753246753<\/pre>\n\n\n\n
Looks like our decision tree algorithm has an accuracy of 67.53%. A value this high is usually considered good. <\/p>\n\n\n\n
6. Now that we have created a decision tree, let’s see what it looks like when we visualise it<\/p>\n\n\n\n
The Scikit-learn’s export_graphviz function can help visualise the decision tree. We can use this on our Jupyter notebooks. In case you are not using Jupyter, you may want to look at installing the following libraries:<\/p>\n\n\n\n