CS25 I Stanford Seminar - Transformers in Language: The development of GPT Models including GPT3

Course Feature
  • Cost
    Free
  • Provider
    Youtube
  • Certificate
    Paid Certification
  • Language
    English
  • Start Date
    On-Demand
  • Learners
    No Information
  • Duration
    1.00
  • Instructor
    /
Next Course
2.5
8 Ratings
This seminar explores the development of GPT models, beginning with the 3-Gram Model proposed by Shannon in 1951. It then examines the use of Recurrent Neural Nets, Big LSTM, and Transformer models, before discussing GPT-2 and GPT-3, two of the most advanced models. GPT-2 is a Big Transformer, while GPT-3 is a Very Big Transformer.
Show All
Course Overview

❗The content presented here is sourced directly from Youtube platform. For comprehensive course details, including enrollment information, simply click on the 'Go to class' link on our website.

Updated in [February 21st, 2023]

What does this course tell?
(Please note that the following overview content is from the original platform)


Introduction.
3-Gram Model (Shannon 1951).
Recurrent Neural Nets (Sutskever et al 2011).
Big LSTM (Jozefowicz et al 2016).
Transformer (Llu and Saleh et al 2018).
GPT-2: Big Transformer (Radford et al 2019).
GPT-3: Very Big Transformer (Brown et al 2019).
GPT-3: Can Humans Detect Generated News Articles?.
Why Unsupervised Learning?.
Is there a Big Trove of Unlabeled Data?.
Why Use Autoregressive Generative Models for Unsupervised Learnin.
Unsupervised Sentiment Neuron (Radford et al 2017).
Radford et al 2018).
Zero-Shot Reading Comprehension.
GPT-2: Zero-Shot Translation.
Language Model Metalearning.
GPT-3: Few Shot Arithmetic.
GPT-3: Few Shot Word Unscrambling.
GPT-3: General Few Shot Learning.
IGPT (Chen et al 2020): Can we apply GPT to images?.
IGPT: Completions.
IGPT: Feature Learning.
Isn't Code Just Another Modality?.
The HumanEval Dataset.
The Pass @ K Metric.
Codex: Training Details.
An Easy Human Eval Problem (
[email protected]
-0.9).
A Medium HumanEval Problem (
[email protected]
-0.17).
A Hard HumanEval Problem (
[email protected]
-0.005).
Calibrating Sampling Temperature for
[email protected]
The Unreasonable Effectiveness of Sampling.
Can We Approximate Sampling Against an Oracle?.
Main Figure.
Limitations.
Conclusion.
Acknowledgements.


We consider the value of this course from multiple aspects, and finally summarize it for you from three aspects: personal skills, career development, and further study:
(Kindly be aware that our content is optimized by AI tools while also undergoing moderation carefully from our editorial staff.)
This course introduces the concept of unsupervised learning as a means of natural language processing. It covers different algorithms for building generative models for text and language understanding, including 3-gram models, recurrent neural nets, big LSTM, the Transformer, GPT-2, GPT-3, the Unsupervised Sentiment Neuron, and Zero-Shot Reading Comprehension. It also explores the possibility of applying GPT-3 to images through IGPT, examining the HumanEval dataset and the
[email protected]
metric as a measure of human evaluation.
Possible Development Paths:
Learners of this course can develop their skills in natural language processing and unsupervised learning. They can use the algorithms and techniques learned in this course to build generative models for text and language understanding, and apply GPT-3 to images.
Learning Suggestions:
Learners of this course should also consider related topics such as machine learning, deep learning, natural language processing, and computer vision. They should also explore other datasets and metrics for measuring the performance of their models. Additionally, they should practice applying the algorithms and techniques learned in this course to real-world problems.

Show All
Recommended Courses
free ai-companions-social-impact-how-tech-like-chatgpt-virtual-companions-and-robots-will-change-us-3313
AI Companions: Social Impact - How tech like ChatGPT virtual companions and robots will change us
3.0
Youtube 8 learners
Learn More
This course will explore the social impact of AI companions such as ChatGPT virtual companions and robots. We will look at real world examples, as well as fictional examples such as Joi from Blade Runner 2049, NS5 from I, Robot, Ava from Ex Machina, and Samantha from HER. We will discuss the potential problems of addiction, exploitation, and authentic human connection. We will also look at the potential benefits of AI companions such as lack of judgment, infinite patience, and a super intelligent ally. Finally, we will draw conclusions about the implications of AI companions on our society.
free i-got-chatgpt-to-build-me-an-entire-marketing-campaign-3314
I got ChatGPT to build me an entire marketing campaign
2.0
Youtube 304,856 learners
Learn More
Learn how to create an entire marketing campaign from start to finish with ChatGPT! In this video, you'll see how one person used ChatGPT to create ads, social media posts, blog posts, email sequences, SMS messages, and a sales page. Plus, you'll get FREE access to "The One-Page Marketing Cheatsheet" to help you get better results faster than ever before. Don't miss out on this amazing opportunity - click now to get started!
free chatgpt-pour-les-debutants-3315
ChatGPT pour les debutants
4.5
Udemy 401,000 learners
Learn More
ChatGPT est une technologie qui permet aux débutants de comprendre le fonctionnement des conversations en ligne. Avec ChatGPT, les utilisateurs peuvent apprendre à utiliser cette technologie dans leur vie quotidienne et décupler leur productivité au travail. Les débutants peuvent également améliorer leur utilisation de ChatGPT en apprenant à mieux comprendre et à mieux utiliser cette technologie.
free gpt-4-launched-new-chatgpt-is-here-gpt-4-all-features-details-explained-3316
GPT 4 Launched New ChatGPT Is Here GPT 4 All Features & Details Explained
3.0
Youtube 41,564 learners
Learn More
Are you ready to take your content game to the next level? Learn how to use Chat GPT Prompt Engineering to pass AI detection as 100% human. This tutorial will show you how to craft natural language that can even trick the smartest AI algorithms. Get ready to beat AI detection and create content that's so human-like, you'll be scratching your head in disbelief.
Favorites (0)
Favorites
0 favorite option

You have no favorites

Name delet
arrow Click Allow to get free CS25 I Stanford Seminar - Transformers in Language: The development of GPT Models including GPT3 courses!