{"id":10349,"date":"2023-09-27T12:12:25","date_gmt":"2023-09-27T19:12:25","guid":{"rendered":"https:\/\/www.springboard.com\/?p=10349"},"modified":"2023-10-06T05:22:50","modified_gmt":"2023-10-06T12:22:50","slug":"machine-learning-gpt-3-open-ai","status":"publish","type":"post","link":"https:\/\/www.springboard.com\/blog\/data-science\/machine-learning-gpt-3-open-ai\/","title":{"rendered":"OpenAI GPT-3: Everything You Need to Know [Updated]"},"content":{"rendered":"\n

OpenAI\u2019s latest model has gone viral again. Much like its predecessor, there is no stopping to the buzz that OpenAI\u2019s latest model GPT-3 is creating around the internet. As experts praise the model for its intuitive capabilities which range from writing articles to generating code, many experts including the founder of OpenAI have called out the hype \u201cway too much\u201d. The timing of the release lines up OpenAI\u2019s new business model of commercialising its AI through API. Unarguably, OpenAI GPT-3 draws the attention of data science enthusiasts towards the idea of artificial general intelligence taking shape. In this blog, we present the various aspects of GPT-3, from the details of building blocks to the performance of the model and the possibilities of what all it can do.<\/p>\n\n\n\n

OpenAI GPT-3: What is GPT-3?<\/h2>\n\n\n\n

Generative Pre-trained Transformer 3<\/em> (GPT-3<\/strong>) is a language model that leverages deep learning to generate human-like text (output). Not only can it produce text, but it can also generate code, stories, poems, etc. For these capabilities and reasons, it has become such a hot topic in the area of natural language processing (NLP- – an essential sub-branch of data science<\/a>). <\/p>\n\n\n\n

GPT-3 was introduced by Open AI earlier in May 2020 as a successor to their previous language model (LM) GPT-2. It is considered to be better and bigger than GPT-2. In fact, with around 175 Billion trainable parameters, OpenAI GPT-3\u2019s full version is the largest model trained so far when compared to other language models. This 72-page research paper<\/a>** describes in great detail the features, capabilities, performance and limitations of the model. <\/p>\n\n\n\n

**We will refer to the research paper as \u201cthe paper\u201d in our blog at multiple places. <\/em><\/p>\n\n\n\n

In this blog, we have included some illustrations directly from the paper and focused on some key highlights so as to save you from having to read through the paper. <\/p>\n\n\n\n

GPT-3 is a very large language model. Given some input text, it can probabilistically determine what tokens from a known vocabulary will come next. <\/em>Before we go ahead and see what makes GPT-3 so special, lets first understand what is a language model? <\/p>\n\n\n\n

What are Language Models (LMs)?<\/h2>\n\n\n\n

Simply put, language models are statistical tools to predict the next word(s) in a sequence. In other words, language models are probability distribution over a sequence of words. Language models have many applications like:<\/p>\n\n\n\n