{"id":24303,"date":"2021-06-14T17:32:00","date_gmt":"2021-06-15T00:32:00","guid":{"rendered":"https:\/\/www.springboard.com\/blog\/?p=24303"},"modified":"2023-06-25T20:36:02","modified_gmt":"2023-06-26T03:36:02","slug":"recurrent-neural-networks","status":"publish","type":"post","link":"https:\/\/www.springboard.com\/blog\/data-science\/recurrent-neural-networks\/","title":{"rendered":"What are Recurrent Neural Networks?"},"content":{"rendered":"\n<p>Whether you want to build an automated image captioning tool or a machine-based translation tool, the first question you should ask is &#8212; What are recurrent neural networks. Recurrent neural networks (RNNs) give an interesting twist to the simple neural networks by remembering the past data. While simple neural networks just give fixed-sized input and outputs, RNN memorise the previous inputs in order to give sequential data as output.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Understanding What are Recurrent Neural Networks<\/h2>\n\n\n\n<p>A recurrent neural network or RNN helps process sequences like sentences, daily stock prices, or even sensor measurements. RNN is a set of algorithms which helps in processing sequences by retaining the memory (or state) of the previous value in the sequence.<\/p>\n\n\n\n<p>Recurrent mainly means that the output of the current time step actually becomes the input for the very next time step. That means, at every element, the model not only considers the current input but also remembers all the previous elements.<br><\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"563\" height=\"328\" src=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks.png\" alt=\"recurrent neural networks\" class=\"wp-image-46182\" srcset=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks.png 563w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-400x233.png 400w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-380x221.png 380w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-380x221.png 420w\" sizes=\"(max-width: 563px) 100vw, 563px\" \/><figcaption class=\"wp-element-caption\">Image Credit: Pintrest<\/figcaption><\/figure>\n\n\n\n<p>Recurrent neural networks are designed to mimic the way human brains think while processing and sequencing. For instance, we consider the entire sentence before speaking or writing instead of just considering the words we want to use.&nbsp;Similarly, the memory of RNNs allows it to learn more about long-term dependencies which in turn allows it to take the entire context of the sequence into account before making the next prediction.<\/p>\n\n\n\n<p>For building RNNs, you can use Keras, the neural network library which is built in Python. Although there are many other neural network libraries that can be used for RNN, Keras is often preferred because of its fast development time, ease of use, and support for Python. With Keras, you can also easily build <a href=\"https:\/\/www.springboard.com\/blog\/data-science\/predictive-text-generation\/\" target=\"_blank\" data-type=\"URL\" data-id=\"https:\/\/www.springboard.com\/blog\/data-science\/predictive-text-generation\/\" rel=\"noreferrer noopener\">text generation and predictive models using RNN<\/a>.<\/p>\n\n\n\n<p>The advantages of using RNN structure include:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Processing input of any length<\/li>\n\n\n\n<li>The model size does not increase with the size of the input<\/li>\n\n\n\n<li>All the computations take historical data into account<\/li>\n\n\n\n<li>All the weights are shared across time<\/li>\n<\/ul>\n\n\n\n<p>There are also some disadvantages of using recurrent neural network including:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>It takes a long time for computations as compared to simple neural networks<\/li>\n\n\n\n<li>It can be difficult to access information a long time ago<\/li>\n\n\n\n<li>It\u2019s not possible to consider any future input for the current state<\/li>\n<\/ul>\n\n\n<div class=\"bg-leaf-50 p-4 my-3\"><h4 class=\"fw-bold text-center\">Get To Know Other\tData Science Students<\/h4><div class=\"row row-cols-1 row-cols-lg-3\"><div class=\"col\"><div class=\"card success-story-card h-100 d-flex justify-content-between mb-0\"><div class=\"flex-grow-1 text-center\"><a class=\"d-inline-block rounded-circle\" href=\"\/success\/rane-najera-wynne\" style=\"width:125px;height:125px;overflow:hidden\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1659153158\/Student%20Success\/Rane_Najera_Wynne.jpg\" alt=\"Rane Najera-Wynne\" style=\"object-fit:contain;max-width:170px;height:125px\" \/><\/a><p class=\"fw-bold mb-0\">Rane Najera-Wynne<\/p><p class=\"text-muted lh-1\">Data Steward\/data Analyst at BRIDGE<\/p><\/div><div class=\"w-100 d-block d-md-none mt-3\"><\/div><p class=\"mb-0 mx-auto text-center\"><a class=\"btn btn-primary mx-auto\" href=\"\/success\/rane-najera-wynne\">Read Story<\/a><\/p><\/div><\/div><div class=\"col d-none d-md-block\"><div class=\"card success-story-card h-100 d-flex justify-content-between mb-0\"><div class=\"flex-grow-1 text-center\"><a class=\"d-inline-block rounded-circle\" href=\"\/success\/isabel-van-zijl\" style=\"width:125px;height:125px;overflow:hidden\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1629203192\/Student%20Success\/Isabel_van_Zijl_125x125.png\" alt=\"Isabel Van Zijl\" style=\"object-fit:contain;max-width:170px;height:125px\" \/><\/a><p class=\"fw-bold mb-0\">Isabel Van Zijl<\/p><p class=\"text-muted lh-1\">Lead Data Analyst at Kinship<\/p><\/div><p class=\"mb-0 mx-auto text-center\"><a class=\"btn btn-primary mx-auto\" href=\"\/success\/isabel-van-zijl\">Read Story<\/a><\/p><\/div><\/div><div class=\"col d-none d-md-block\"><div class=\"card success-story-card h-100 d-flex justify-content-between mb-0\"><div class=\"flex-grow-1 text-center\"><a class=\"d-inline-block rounded-circle\" href=\"\/success\/nick-lenczewski\" style=\"width:125px;height:125px;overflow:hidden\"><img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/res.cloudinary.com\/springboard-images\/image\/upload\/v1667235351\/Student%20Success\/Nick_Lenczewski.jpg\" alt=\"Nick Lenczewski\" style=\"object-fit:contain;max-width:170px;height:125px\" \/><\/a><p class=\"fw-bold mb-0\">Nick Lenczewski<\/p><p class=\"text-muted lh-1\">Data Scientist at Ovative Group<\/p><\/div><p class=\"mb-0 mx-auto text-center\"><a class=\"btn btn-primary mx-auto\" href=\"\/success\/nick-lenczewski\">Read Story<\/a><\/p><\/div><\/div><\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">How RNNS Differ from Simple Neural Networks<\/h2>\n\n\n\n<p>A simple forward feed <a href=\"https:\/\/www.springboard.com\/blog\/data-science\/beginners-guide-neural-network-in-python-scikit-learn-0-18\/\" target=\"_blank\" data-type=\"URL\" data-id=\"https:\/\/www.springboard.com\/blog\/data-science\/beginners-guide-neural-network-in-python-scikit-learn-0-18\/\" rel=\"noreferrer noopener\">neural network<\/a> is good at learning about patterns between the present set of inputs and outputs. For instance, a neural network can help identify whether an image has a dog in it or not. But then, if you input an image of a cat after an image of a dog, it won\u2019t have any memory or context about the previous dog image. It will classify the new image with the cat in it independently. As a result, simple neural networks are not suitable for sequences which require previous memory or context to make future predictions like predicting stock prices based on the past data.&nbsp;After all, you can\u2019t just arbitrarily predict stock prices. You have to take into account the previous stock values and market patterns to make accurate predictions.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1200\" height=\"663\" src=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-1200x663.png\" alt=\"Recurrent Neural Networks and Feed-Forward Neural Network\" class=\"wp-image-46184\" srcset=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-1200x663.png 1200w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-400x221.png 400w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-768x424.png 768w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-1536x849.png 1536w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-380x210.png 380w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-700x387.png 700w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network.png 1600w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/recurrent-neural-networks-and-feed-forward-neural-network-380x210.png 420w\" sizes=\"(max-width: 1200px) 100vw, 1200px\" \/><figcaption class=\"wp-element-caption\">Image Credit: nerdcoder<\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">The Architecture of a Basic Recurrent Neural Network<\/h2>\n\n\n\n<p>There are four main elements in an RNN architecture &#8212; Input, hidden state, weights, and output.<\/p>\n\n\n\n<figure class=\"wp-block-image aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"920\" height=\"248\" src=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network.png\" alt=\"The Architecture of a Basic Recurrent Neural Network\" class=\"wp-image-46186\" srcset=\"https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network.png 920w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network-400x108.png 400w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network-768x207.png 768w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network-380x102.png 380w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network-700x189.png 700w, https:\/\/www.springboard.com\/blog\/wp-content\/uploads\/2021\/06\/the-architecture-of-a-basic-recurrent-neural-network-380x102.png 420w\" sizes=\"(max-width: 920px) 100vw, 920px\" \/><figcaption class=\"wp-element-caption\">Image Credit: <a href=\"http:\/\/www.deeplearningbook.org\/contents\/rnn.html\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\"deeplearningbook (opens in a new tab)\">deeplearningbook<\/a><\/figcaption><\/figure>\n\n\n\n<p><strong>Input:<\/strong> This is the initial input for the network<\/p>\n\n\n\n<p><strong>Hidden state:<\/strong> This acts as the main memory of the network. It is calculated based on previous hidden state and the current input<\/p>\n\n\n\n<p><strong>Weights:<\/strong> The input to hidden connections are directly parameterised through a weight matrix. It also takes into consideration the hidden to hidden recurrent connections and hidden to output connections.<\/p>\n\n\n\n<p><strong>Output:<\/strong> This is the final output of the network<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Different Types of Recurrent Neural Networks<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">1. One to many<\/h3>\n\n\n\n<p>These networks have fixed size information as input and give a sequence of data as output. For example, a captioning algorithm which takes an image as the input and gives a sentence caption as the output with a sequence of words.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">2. Many to one<\/h3>\n\n\n\n<p>These networks take a sequence of information as input and give a fixed size output. For example, a sentiment analysis algorithm that inputs a sentence and then classifies it based on whether it has a positive or negative sentiment.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">3. Many to many<\/h3>\n\n\n\n<p>It takes a sequence of information as the input and gives another sequence as output. For example, a machine translation where recurrent neural network takes a sentence in one language and translates it into a different language.<\/p>\n\n\n\n<p>Now that you know what a recurrent neural network is, you will know that an RNN algorithm is working behind the scenes the next time you see a speech to text application or a machine-based translation tool.<\/p>\n\n\n\n<p><em>For further reading, <a href=\"https:\/\/www.springboard.com\/blog\/data-science\/data-science-definition\/\" data-type=\"post\" data-id=\"2291\">learn more about data science here<\/a> and <a href=\"https:\/\/www.springboard.com\/blog\/data-science\/what-does-a-data-scientist-do\/\" data-type=\"post\" data-id=\"24427\">find out what a data scientist does<\/a>.<\/em><\/p>\n\n\n\n<p class=\"rm has-background\" style=\"background-color:#efeff6\"><strong>Since you\u2019re here\u2026<br><\/strong>Curious about a career in data science? Experiment with our <a rel=\"noreferrer noopener\" href=\"https:\/\/www.springboard.com\/resources\/guides\/data-science-process\/\" target=\"_blank\">free data science learning path<\/a>, or join our <a rel=\"noreferrer noopener\" href=\"https:\/\/www.springboard.com\/courses\/data-science-career-track\/\" target=\"_blank\">Data Science Bootcamp<\/a>, where you\u2019ll get your tuition back if you don&#8217;t land a job after graduating. We\u2019re confident because our courses work \u2013 check out our <a rel=\"noreferrer noopener\" href=\"https:\/\/www.springboard.com\/success\/\" target=\"_blank\">student success stories<\/a> to get inspired.<\/p>\n\n\n\n<p><br><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Whether you want to build an automated image captioning tool or a machine-based translation tool, the first question you should ask is &#8212; What are recurrent neural networks. Recurrent neural networks (RNNs) give an interesting twist to the simple neural networks by remembering the past data. While simple neural networks just give fixed-sized input and [&hellip;]<\/p>\n","protected":false},"author":100,"featured_media":24304,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_eb_attr":"","_eb_data_table":"","footnotes":""},"categories":[67],"tags":[],"marketing_tags":[],"class_list":{"0":"post-24303","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-data-science"},"acf":[],"_links":{"self":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/24303"}],"collection":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/users\/100"}],"replies":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/comments?post=24303"}],"version-history":[{"count":3,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/24303\/revisions"}],"predecessor-version":[{"id":46189,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/posts\/24303\/revisions\/46189"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/media\/24304"}],"wp:attachment":[{"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/media?parent=24303"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/categories?post=24303"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/tags?post=24303"},{"taxonomy":"marketing_tags","embeddable":true,"href":"https:\/\/www.springboard.com\/blog\/wp-json\/wp\/v2\/marketing_tags?post=24303"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}