What is a decision tree in machine learning?<\/h2>\n\n\n\n
If you are an aspiring data scientist<\/a> and involving with machine learning, decision trees may help you produce clearly interpretable results and choose the best feasible option. Let\u2019s start by explaining decision trees. Let\u2019s start by explaining decision trees. Decision trees are a class of machine learning models that can be thought of as a sequence of \u201cif\u201d statements to apply to an input to determine the prediction.<\/p>\n\n\n\n In greater rigor, a decision tree incrementally constructs vertices within a tree that represent a certain \u201cif\u201d statement and has children vertices connected to the parent by edges representing the possible outcomes of the parent vertex if condition (in decision tree lingo, this is referred to as the cut).<\/p>\n\n\n\n Eventually, after some sequence of \u201cif\u201d statements, a tree vertice will have no children but hold a prediction value instead. Decision trees can learn the \u201cif\u201d conditions and eventual prediction, but they notoriously overfit the training data. To prevent overfitting, oftentimes decision trees are purposefully underfit and cleverly combined to reach the right balance of bias and variance.<\/p>\n\n\n\n Now we\u2019ll explore random forests, the brainchild of Leo Breiman<\/a>. Random forests are a type of ensemble learning or a collection of so-called \u201cweak learner\u201d models whose predictions are combined into a single prediction.<\/p>\n\n\n\n In the case of random forests, the collection is made up of many decision trees. Random forests are considered \u201crandom\u201d because each tree is trained using a random subset of the training data (referred to as bagging in more general ensemble models), and random subsets of the input features (coined feature bagging in ensemble model speak), to obtain diverse trees.<\/p>\n\n\n\n Bagging decreases the high variance and tendency of a weak learner model to overfit a dataset. For random forests, both types of bagging are necessary. Without both types of bagging, many of the trees could create similar \u201cif\u201d conditions and essentially highly correlated trees.<\/p>\n\n\n\n Instead of bagging and creating many weak learner models to prevent overfitting, often, an ensemble model may use a so-called boosting technique to train a strong learner using a sequence of weaker learners.<\/p>\n\n\n\n In the case of decision trees, the weaker learners are underfit trees that are strengthened by increasing the number of \u201cif\u201d conditions in each subsequent model.<\/p>\n\n\n\n XGBoost<\/a>, CatBoost<\/a>, and LightGBM<\/a> have emerged as the most optimized boosting techniques for gradient-boosted tree algorithms. Scikit-learn also has generic implementations<\/a> of random forests and gradient-boosted tree algorithms, but with fewer optimizations and customization options than XGBoost, CatBoost, or LightGBM, and is often better suited for research than production environments.<\/p>\n\n\n\n Each of XGBoost, CatBoost, and LightGBM have their own frameworks, distinguished by how the decision tree cuts are added iteratively.<\/p>\n\n\n![\"decision](\"https:\/\/www.springboard.com\/library\/static\/3b71ee0fb317d36e7e20f7a2b7c390dd\/c0388\/screen-shot-2020-12-01-at-3.24.06-pm.png\" "\\"decision")
<\/a><\/figure>\n\n\n\nWhat are random forests in machine learning?<\/h2>\n\n\n\n
Get To Know Other\tData Science Students<\/h4>
![\"Esme](\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1629203193\/Student%20Success\/Esme_Gaisford_125x125.png\")
<\/a>![\"Ginny](\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1629203192\/Student%20Success\/Ginny_Zhu_125x125.png\")
<\/a>![\"Leoman](\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1629203191\/Student%20Success\/Leoman_Momoh_125x125.png\")
<\/a>