generative pre-trained transformer
Sign in to saveAlso known as GPT, generative pretrained transformer
type of large language model
~21 min read
Encyclopedic overview
Original GPT model A generative pre-trained transformer (GPT) is a type of large language model (LLM) that is widely used in generative artificial intelligence chatbots. GPTs are based on a deep learning architecture called the transformer. They are pre-trained on large datasets of unlabeled content, and able to generate novel content.
OpenAI was the first to apply generative pre-training to the transformer architecture, introducing the GPT-1 model in 2018. The company has since released many bigger GPT models. The chatbot ChatGPT, released in late 2022 (using GPT-3.5), was followed by many competitor chatbots using their own generative pre-trained transformers to generate text, such as Gemini, DeepSeek and Claude.
Excerpted from Wikipedia’s “generative pre-trained transformer” article, available under the CC BY-SA 4.0 licence.