{"id":1943,"date":"2016-07-05T14:56:36","date_gmt":"2016-07-05T21:56:36","guid":{"rendered":"https:\/\/www.springboard.com\/?p=1943"},"modified":"2023-07-11T20:40:22","modified_gmt":"2023-07-12T03:40:22","slug":"probability-bayes-theorem-data-science","status":"publish","type":"post","link":"https:\/\/www.springboard.com\/blog\/data-science\/probability-bayes-theorem-data-science\/","title":{"rendered":"Bayes Theorem, Probability, Logic, and Data"},"content":{"rendered":"\n

Bayes Theorem and Probability<\/h2>\n\n\n\n

When we think of probability we often think of flipping coins, rolling dice, and roulette wheels. Maybe it’s this association with gambling that leads us to think statistics has mystical powers. As soon as statisticians start talking, they mumble strange things about “Bayes Theorem”, “p-values” and “chi-squared”. Uttering these words, they get excited or dismayed as though these were signs from beyond.<\/p>\n\n\n\n

\"bayes<\/a><\/figure>\n\n\n\n

A more useful understanding of probability is a <\/span>probability as an extension of logic<\/span><\/i>. Classical logic forces us to constrain our beliefs to absolutes. Something can either be true or false, but we have no way of expressing uncertainty. Probability allows us to extend the rules of logic to reason about an uncertain world.<\/span><\/p>\n\n\n\n

Probability as Logic<\/b><\/h3>\n\n\n\n

The mapping from Boolean logic to probability is remarkably simple. In Boolean logic, we have <\/span>true<\/span><\/i> and <\/span>false<\/span><\/i> values. In probability, we have a continuous range of values from 0 to 1. The extremes of 1 and 0 map to <\/span>true<\/span><\/i> and <\/span>false<\/span><\/i> in Boolean logic. All the other possible values express different beliefs between absolute truth and falsity.<\/span><\/p>\n\n\n\n

Boolean logic has three operations that are used to combine <\/span>true<\/span><\/i> and <\/span>false<\/span><\/i> values: <\/span>and<\/span><\/i>, <\/span>or<\/span><\/i> and <\/span>not<\/span><\/i>. We can quite easily map these logical rules to probabilistic rules.<\/span><\/p>\n\n\n\n

“A or B” is the sum of two probabilities, <\/span>P(A)+P(B)<\/strong>.<\/span><\/p>\n\n\n\n

“A and B” is the product of two probabilities, <\/span>P(A)\u22c5P(B)<\/strong>.<\/span><\/p>\n\n\n\n

“not A” is just <\/span>(1-P(A))<\/strong>.<\/span><\/p>\n\n\n\n

Given these simple rules, we can use probability just like we do traditional logic. We all know that classical logic often fails to explain everyday common sense. This is because the world is full of uncertainties. It turns out that probabilistic logic is just the formalization of how we often reason about everyday problems. As an example let’s take a look at a scenario that reflects everyday reasoning. We’ll see that probability allows us to model the way we think quite well.<\/span><\/p>\n\n\n\n

Reasoning about Bacon<\/b><\/h3>\n\n\n\n

It’s Friday evening and you think to yourself “It’s been a long week, I’ll make myself a nice breakfast in the morning”. You know you have eggs because you bought some yesterday. But you don’t remember if you have bacon. It’s 9 pm and you also don’t remember if the grocery store is still open on Friday that late.<\/span><\/p>\n\n\n\n

The question is “should I go to the grocery store?” We can model the answer to this question with the variable Store. There are two major facts we’d like to know to decide what to do: “Do I have bacon?” (we’ll name this fact Bacon) and “is the grocery store open?” (this fact will be Open).<\/span><\/p>\n\n\n

Get To Know Other\tData Science Students<\/h4>
\"Mikiko<\/a>

Mikiko Bazeley<\/p>

ML Engineer at MailChimp<\/p><\/div>

<\/div>

Read Story<\/a><\/p><\/div><\/div>

\"Diana<\/a>

Diana Xie<\/p>

Machine Learning Engineer at IQVIA<\/p><\/div>

Read Story<\/a><\/p><\/div><\/div>

\"Jonah<\/a>

Jonah Winninghoff<\/p>

Statistician at Rochester Institute Of Technology<\/p><\/div>

Read Story<\/a><\/p><\/div><\/div><\/div><\/div>\n\n\n\n

Traditional Logic at Home<\/b><\/h3>\n\n\n\n

Suppose you’re at home and you think to yourself “Hmmm… I wonder if we need more bacon?”. You can look in the fridge and see that “no, sadly there is no bacon”. You can also easily look up the grocery store’s phone number and call them. You find out that the grocery store is open!<\/span><\/p>\n\n\n\n

Now we have a nice traditional logic problem:<\/span><\/p>\n\n\n\n

Bacon = False Open = True<\/span><\/em><\/p>\n\n\n\n

(not Bacon) AND (Open) => Store<\/span><\/em><\/p>\n\n\n\n

(not False) AND (True) => True<\/span><\/em><\/p>\n\n\n\n

All the facts we need to solve our logic puzzle are right before us. What an easy and simple world! But it’s Friday night, you’re out with your friend getting drinks to celebrate a busy week! You can’t look in your fridge and it\u2019s too loud to call the grocery store. Now we have a probabilistic logic problem!<\/span><\/p>\n\n\n\n

Logic of Uncertainty<\/b><\/h3>\n\n\n\n

To reason about bacon in a bar,<\/span> we need to use probabilistic logic and Bayes Theorem. You obviously can’t check your refrigerator. But you do remember shopping earlier in the week to get eggs. Sadly, you also remember thinking: “I probably don’t need bacon this week”. Also, you’re pretty sure that you went grocery shopping last Monday at 9:30 pm so it seems likely that the grocery store is still open.<\/span><\/p>\n\n\n\n

You can simply assign some values to our logic based on our certainty in these facts. Since we remember that we foolishly decided to not get bacon, it’s very unlikely that you have any.<\/span><\/p>\n\n\n\n

P(Bacon)=0.01<\/span><\/em><\/p>\n\n\n\n

You don’t assign <\/span>P(Bacon)=0<\/span> because you really don’t remember if you did change your mind later. It is possible you got bacon, but very unlikely.<\/span><\/p>\n\n\n\n

You definitely remember it was late when you went shopping, probably around 9:30. It could have been 8:30, and so you are not completely sure. We can model this uncertainty as well:<\/span><\/p>\n\n\n\n

P(Open)=0.96<\/span><\/em><\/p>\n\n\n\n

The probabilistic equivalent of our logical statement from before is:<\/span><\/p>\n\n\n\n

P(Store)=(1-P(Bacon))\u22c5P(Open)\u22480.95<\/span><\/em><\/p>\n\n\n\n

Wow 0.95 certainty! That’s like having a p-value of < 0.05! Surely we’ll go to the store!<\/span><\/p>\n\n\n\n

Except it’s 9 pm on a Friday and your friend says “I’ll buy the next round”. It would be really sad to go to the store to get bacon, only to find either it was closed or you already had bacon at home. You decide that even though the probability is reasonably high that you should go to the store, the little uncertainty makes it not worth your time.<\/span><\/p>\n\n\n\n

One of the benefits of living in an uncertain world is that we can assign value to our knowledge. Sometimes being just a little more sure than not makes something worth it. Other times taking a huge risk requires near-absolute certainty. Why risk certain beer for uncertain bacon?<\/span><\/p>\n\n\n\n

\"bayes<\/a><\/figure>\n\n\n\n

Probabilistic Logic is Common Sense<\/b><\/h3>\n\n\n\n

Ignoring the mathematics, the probabilistic example is much closer to how with think every day. We have some data, often in the form of beliefs and memories, then we use that to make decisions. Decisions often have a cost. We weigh the potential risks and rewards of actions with our certainty in the results. <\/span><\/p>\n\n\n\n

Even when we make “stupid” decisions we typically are thinking probabilistically with a modified version of Bayes Theorem. The trouble is we usually don’t do the math. Even with fuzzy guesses, there are many cases where we would be surprised with how uncertain we really are. <\/span><\/p>\n\n\n\n

This is the key feature that makes probabilistic thinking in data science different from using Bayes Theorem in everyday life.<\/strong> In data science<\/a> we do have numbers, often backed by data. Even in the cases when we don’t have data we can use quantitative values to model what we know. If our lead designer is certain a new page design will improve the number of signups we can model that opinion. Because everything has a value we can see how different information and different beliefs affect our conclusions.<\/span><\/p>\n\n\n\n

Bayesian thinking is a <\/span>better way to model how to make decisions on a day-to-day<\/b> basis. It can help you win <\/span>poker hands<\/span><\/a>, and help you become a <\/span>better thinker<\/span><\/a> (or maybe be a better data scientist<\/a>). We just have to learn this powerful new tool and how it works with existing data science methods<\/a> to apply it. <\/span><\/p>\n\n\n\n

Back to the Bar<\/b><\/h3>\n\n\n\n

Your friend notices the look on your face and asks ” You seem really stressed! What’s wrong? “. You reply that you’ve been thinking about this probability problem. You explain the bacon and all your facts. Finally you tell your friend that you’re not going because why risk it when you’re both having fun. Your friend laughs and exclaims “Yeah, but that grocery store is open until midnight on Friday, I’m nearly positive.” Now we have a different model of the P(Store). P(Bacon) remains the same but P(Open) has gone way up:<\/span><\/p>\n\n\n\n

P(Store)=(1-P(Bacon))*P(Open)=0.99\u22c50.9999\u22480.9999<\/span><\/em><\/p>\n\n\n\n

Not only are you more certain, but the cost of going has decreased! You’re definitely going to get bacon and have that beer! Hooray for Bayes Theorem<\/strong>! <\/span><\/p>\n\n\n\n

New information from our friend has convinced us that we should go to the store. In the first example, we saw how probability can model common sense. In this case, we get some insight into how common sense can influence how we should think when using probability to make choices.<\/span><\/p>\n\n\n\n

Probability as Reasoning<\/b><\/h3>\n\n\n\n

While probability can help us understand reasoning, reasoning should inform how we think about probability. The heart of statistics, especially <\/span>Bayesian statistics<\/span><\/a>, is the formalization of natural reasoning to understand the data we have. <\/span><\/p>\n\n\n\n

This new trend of thinking impacts a debate nearly every expert in statistics is having. Each one has an opinion about the pros and cons of p-values. <\/span>Andrew Gelman<\/span><\/a>, <\/span>the 538 blog<\/span><\/a> and even <\/span>John Oliver<\/span><\/a> have all weighed in on the issue. But at it’s heart the issue is not about p-values. The important thing is that no single test, or statistical values is a substitute for reasoning–not even Bayes Theorem.\u00a0<\/span><\/p>\n\n\n\n

When we test a hypothesis what we are doing is making an argument. In our bar scenario, we are arguing about whether or not to go to the store. It’s easy to see how by modifying our risk and increasing our certainty our friend can convince us to stay for a beer and get bacon. Far too often, we don’t hold the same standards to scientific publications, or our own hypothesis tests in our data science work.<\/p>\n\n\n\n

A p-value should never convince you that something is true. Changing a p-value to Bayes’ Factor or a Posterior probability shouldn’t either. The entire argument presented by a combination of probabilistic reasoning and facts should argue the case to you. If you aren’t convinced then more data is needed<\/strong>. Whether at work or in a paper, <\/span>statistics are always about making an argument about the way the world works. <\/b><\/p>\n\n\n\n

This is a guest post by Will Kurt, one of our data science mentors<\/a> for our Foundations of Data Science<\/a> workshop. He blogs frequently about probability and Bayes Theorem at his blog Count Bayesie<\/a>.\u00a0<\/em><\/p>\n\n\n\n

Since you\u2019re here\u2026<\/strong>Are you interested in this career track? Investigate with our free guide to what a data professional actually<\/em> does<\/a>. When you\u2019re ready to build a CV that will make hiring managers melt, join our Data Science Bootcamp<\/a> which will help you land a job or your tuition back!<\/p>\n","protected":false},"excerpt":{"rendered":"

Bayes Theorem and Probability When we think of probability we often think of flipping coins, rolling dice, and roulette wheels. Maybe it’s this association with gambling that leads us to think statistics has mystical powers. As soon as statisticians start talking, they mumble strange things about “Bayes Theorem”, “p-values” and “chi-squared”. Uttering these words, they […]<\/p>\n","protected":false},"author":10,"featured_media":1948,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_eb_attr":"","_eb_data_table":"","footnotes":""},"categories":[67],"tags":[],"marketing_tags":[],"class_list":{"0":"post-1943","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-data-science"},"acf":[],"_links":{"self":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/1943"}],"collection":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/comments?post=1943"}],"version-history":[{"count":3,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/1943\/revisions"}],"predecessor-version":[{"id":48094,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/1943\/revisions\/48094"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/media\/1948"}],"wp:attachment":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/media?parent=1943"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/categories?post=1943"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/tags?post=1943"},{"taxonomy":"marketing_tags","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/marketing_tags?post=1943"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}