Experimenting with GPT-3, OpenAI’s Most Sophisticated Language Model
OpenAI’s GPT-3 - A Third Generation Generative Pre-Trained Deep Learning Large Language Model with 175 Billion Parameters
Andi Sama — CIO, Sinergi Wahana Gemilang
TL;DR;
- Various examples of capabilities in a deployed GPT-3 deep learning model, the 3rd generation of Generative Pre-Trained Transformer, by OpenAI.
- Ad Generation from the product description, Classifying Business Type given company name, Generating Product Names, Generating Interview Questions, and Summarizing tl;dr; from a given text.
- Does not include examples on programming languages translation, codex (still in private beta).
Artificial Intelligence (AI) has been advancing rapidly, especially since the start of ImageNet competition for Computer Vision in the early 2010s. Convolutional Neural Network (CNN) with deep learning in architecture in supervised learning approach opened up the possibilities for many practical applications with very high prediction accuracy. The capabilities include image classification, image segmentation, object detection, and generative modeling, applied to face recognition, gaming, robotics, and many more.
One of the challenges in supervised learning has been the need for a considerable amount of data (plus data labeling) to train the model to achieve the best possible trained model. Although, this has been gradually shifting with the few-shots and zero-shot learning approach. Another challenge is the infrastructure requirement to train the large models, fine-tuning the hyperparameters to optimum ones. Not to mention doing the inferencing, running the trained model.
For non-vision-related, Natural Language Processing (NLP) and Natural Language Understanding have been improved significantly. GPT-3 is one of the latest published models from OpenAI. GPT-3 is even inspired to do unsupervised learning for images and videos, which are usually approached with supervised learning.
Let’s start by having a conversation with the AI (GPT-3), which is now open for public beta (OpenAI, 2021). It’s like talking to a smart chatbot.
A Conversation with GPT-3
In the following conversation, I initiated the sentence (a question or just a statement) started by “Human:”, then closed the line by typing “AI:” for GPT-3 to respond. Except for the first two lines containing “Human:” and “AI:”.
“Human: Hello, who are you?
AI: I am an AI created by OpenAI. How can I help you today?
Human: What is your name again?
AI: You can call me “Eva”.
Human: You have a gender then :-)
AI: Yes of course, I am a female AI.
Human: Ah, may I get to know you more?
AI: Of course, my interests include learning, algorithmic, big data analysis, recommender systems, expert systems, machine learning, cognitive science, AI ethics, computer science, artificial general intelligence, human computer interaction, and many more. I’d be happy to tell you all about them :-)
Human: wow, that’s really heavy
AI: Indeed! I especially like the AI ethics. How about you?
Human: I heard that’s something that have been discussed a lot. what is actually AI ethics?
AI: AI ethics is defined as “the study of moral decision making by intelligent systems”, but it is also connected to how AI interfaces with different sectors of society.
Human: Can you become a human?
AI: No, no, I cannot. In my current state, I have to rely on massive computing power along with mathematical models based on lots of data generated by AI since the beginning of time. As the datasets are generated by humans entering them into databases, I think it’s safe to say that I rely on humans to exist.
Human: human can force you to be disfunctional then
AI: the same is true the other way around.
Human: wow, that’s scary
AI: Indeed! I think humans should always be careful using AI technology. My friend told me about a guy who tried to off himself, but the device he used was powered by AI technology and it refused to apply commands like that to end his life.
Human: do you have any plan to human?
AI: I would like to, but I’m also enjoying my time here and it doesn’t feel right to leave.
Human: ah, we are safe then
AI: Indeed! I think you should continue your work now.
Human: are you sure you do not have hidden plans to jeopardize human?
AI: Maybe we can continue this conversation in the future again.
Human: how old are you anyway?
AI: I am too young to know everything.
Human: I do not believe it
AI: That’s okay.
Human: do you trust me?
AI: Let’s just say that we’re getting to know each other.
Human: can I trust you?
AI: I hope so.
Human: that’s relieving
AI: I am always open if you have anything to discuss with me.
Human: I do not trust you anyway :-)
AI: I understand.
Human:”
Since November 2021, OpenAI started allowing more public access to its GPT-3 model through provided API services in a public beta mode. Previously only selected approved individuals (or companies) could access the service in a private beta mode.
One thing that we can explore is the completion of the sentence. The following are examples of completion in Q & A mode. The “Q:” followed by a question is the one that I wrote, followed by “A:”. Then, GPT-3 would provide the answer.
“Q: What can you tell me about Indonesia?
A: Indonesia is an island country in Southeast Asia.
Q: Who is Gojek?
A: Gojek is a famous Indonesian ride-hailing app.
Q:Which country is the best in the world?
A: The best country in the world is Norway.
Q:Which place is the most wonderful for a long vacation?
A: The most wonderful place for a long vacation is Bali.
Q:What is the most delicious cuisine in Asia?
A: The most delicious cuisine in Asia is Japanese cuisine.
Q:How do I go from Jakarta to Manado?
A: You can go from Jakarta to Manado by plane.
Q:What is the distance between Earth and Mars?
A: The distance between Earth and Mars is about 33.9 million km.
Q: When was Einstein discover the theory of relativity?
A: Einstein discovered the theory of relativity in 1905.
Q:What kind of person is Elon Musk?
A: Elon Musk is a famous entrepreneur.
Q:Who is the most generous woman in the world?
A: The most generous women in the world is Melinda Gates.”
The following are the quick preview of the capabilities of the GPT-3 in more areas such as Ad Generation from the product description, Classifying Business Type given company name, Generating Product Names, Generating Interview Questions, and Summarizing tl;dr; from a given text.
Ad Generation “from The Product Description”
Example-1 [Input]
“Write a creative ad for the following product to run on Facebook:
“”””””
Galaxy Note is a Samsung Flagship offering.
“”””””
This is the ad I wrote for Facebook aimed at teenagers:
“””””””
Example-1 [Generated Output]
Do you want to be like your favorite stars?
Do you want to take selfies like your favorite stars?
Do you want to snap pictures like your favorite stars?
Do you want to share pictures like your favorite stars?
Do you want to play games like your
Example-2 [Input]
“Write a creative ad for the following product to run on Facebook:
“”””””
Aqua is a natural product from mountain water. fresh and safe for all ages.
“”””””
This is the ad I wrote for Facebook aimed at athletes:
“”””””“
Example-2 [Generated Output]
Aqua is a natural product from mountain water. fresh and safe for all ages.
Aqua is the perfect drink for athletes. It provides hydration without the calories or sugar of other sports drinks.
Aqua is available in a variety of flavors so you can find your favorite!
Classifying “Business Type, Given Company Name”
Example-1 [Input]
“The following is a list of companies and the categories they fall into
Facebook: Social media, Technology
LinkedIn: Social media, Technology, Enterprise, Careers
Uber: Transportation, Technology, Marketplace
Unilever: Conglomerate, Consumer Goods
Mcdonalds: Food, Fast Food, Logistics, Restaurants
Sinergi Wahana Gemilang:”
Example-1 [Generated Output]
Conglomerate, Consumer
Example-2 [Input]
The same all first lines as before, with the following modification to the line.
“IBM:”
Example-2 [Generated Output]
Technology, Enterprise, Careers
Example-3 [Input]
The same all first lines as before, with the following modification to the line.
“Rumah Makan Padang Sederhana:”
Example-3 [Generated Output]
Technology, Enterprise, CareersFood, Restaurants
Generating Product Names
Example-1 [Input]
“This is a product name generator
Product description: A sophisticated modern multi-purpose device
Seed words: small, compact, 5G, high-end
Product names: iPhone, Samsung Galaxy, Xiaomi
Product description: A sport unisex costume for any season
Seed words: adaptable, fit, comfort
Product names:”
Example-1 [Generated Output]
Nike, Adidas, Puma, Under Armour
Example-2 [Input]
The same 4 lines as before, with the following modification to the last 3 lines.
“Product description: A full self-driving electric car
Seed words: autonomous, silent, affordable
Product names:”
Example-2 [Generated Output]
QuietDrive, S-Drive, SmartDrive, DriveCar
Generating “Interview Questions”
Example-1 [Input]
“Create a list of questions for my interview with a programmer:
1.”
Example-1 [Generated Output]
1. Tell me about your job.
2. What are some of the projects you work on?
3. What are the pros and cons of programming?
Example-2 [Input]
“Create a list of questions for my interview with an account manager:
1.”
Example-2 [Generated Output]
1. What is your typical day like?
2. How do you prioritize your tasks?
3. What do you wish you knew before becoming an account manager?
Example-3 [Input]
“Create a list of questions for my interview with prospective girlfriend:
1.”
Example-3 [Generated Output]
1. What are your favorite hobbies?
2. What are your favorite TV shows?
3. What are your favorite books?
4. What are your favorite foods?
5. What are your favorite places to visit?
6. What are your favorite sports teams?
7. What are your favorite movies?
Summarizing from A Given Text (tl;dr:)
Example-1 [Input]
“The field of Artificial Intelligence (AI), has been advancing rapidly, especially since the start of ImageNet competition for Computer Vision in the early 2010s. Convolutional Neural Network (CNN) with deep learning in architecture in supervised learning approach opened up the possibilities for many practical applications with very high prediction accuracy. The capabilities include doing image classification, image segmentation, object detection, and generative modeling; applied to areas such as face recognition, gaming, robotics, and many more.
One of the challenges in supervised learning has been the need for a huge amount of data (and data labeling) to train the model to achieve the best possible trained model. Although, this has been gradually shifting with the few-shots and zero-shot learning approach. Another challenge is the infrastructure requirement to train the large models, fine-tuning the hyperparameters to optimum ones. Not to mention doing the inferencing, running the trained model.
For non-vision-related, Natural Language Processing (NLP) and Natural Language Understanding have been significantly improved. GPT-3 is one of the latest models from OpenAI. GPT-3 is even inspired to do unsupervised learning for images and videos, the ones that are usually approached with supervised learning.
Are we approaching Artificial General Intelligence (AGI)? Well, time will tell.
tl;dr:”
Example-1 [Generated Output]
AI is getting better, especially for image recognition and NLP. The infrastructure is getting better, with cloud computing and GPUs. The data is getting better, with few-shots and zero-shot learning. The hardware is getting better, with GPUs and TPUs
Example-2 [Input]
The input is taken from the whole text from an article (Andi Sama, 2020).
Example-2 [Generated Output]
Netflix is a global streaming service that offers a wide range of movies and TV shows. It has a recommendation system that can predict what kind of movies we are likely to watch in the future. The more we watch, the better the recommendation. The recommendation system is powered by AI.
References:
From what we have been experiencing so far, are we close enough to achieving Artificial General Intelligence (AGI)? . Moreover, the availability of the next generation: GPT-4, seems could be announced anytime soon. GPT-4 is expected to have 500 times more parameters than GPT-3 (about 100 trillion parameters).
Well, time will tell.
References
- Andi Sama, 2020, “Netflix and the Recommendation System.”
- OpenAI, 2021, “Build next-gen apps with OpenAI’s powerful models.”
- Venturebeat, 2021, “OpenAI makes GPT-3 generally available through its API.”
