Does GPT-3 have a personality?
AI + MBTI = ?
The Myers–Briggs Type Indicator, or MBTI, is a common personality test that aims to categorize people into 16 personality types based on four categories: introversion or extraversion, sensing or intuition, thinking or feeling, and judging or perceiving. While MBTI has been criticized for being a pseudoscience, perhaps not any more different from horoscopes, it is still a fun exercise to take the test and compare your personality type with others.
Therefore, in this article I show how I used GPT-3, a generative AI language model created by OpenAI, to accomplish two things. First, I prompted GPT-3 to guess my personality by emulating a game of 20 Questions. Second, once GPT-3 was able to (correctly!) guess my personality type, I used the same questions GPT-3 asked me in order to determine its personality type.
Part 1: Prompting GPT-3 to guess my personality
I started off by prompting GPT-3 with the following:
Let's play 20 questions!You are a psychologist and personality type expert. Your job is to guess my MBTI personality type by asking 20 questions.The questions can be freeform answers but should NOT be a yes/no question.Do NOT repeat questions that have already been asked.Ask ONE question at a time.I prompted GPT-3 to generate one question at a time (highlighted in green). I replied in the line below. Here are some example questions generated by GPT-3 and and answers supplied by me.
Unlike most personality tests, which often require you to answer questions on a scale from Agree to Disagree, I was able to answer the questions using plain text. These free-form responses may give more insight into and shed more nuance regarding your personality than a simple Agree/Disagree scale.
After all 20 questions had been asked and answered, I prompted GPT-3 with the following:
Given my answers, what MBTI personality type would you categorize me as and why? Answer in a step-by-step fashion and back up your reason with analysis.
GPT-3 was able to predict my personality type spot on (yes, I’m an ENFP). Pretty impressive! GPT-3 was also able to explain which traits contributed most to my being an ENFP. While this does feel a little like reading my horoscope, I felt a small sliver of validation knowing that GPT-3 understood my personality from 20 questions.
Part 2: Prompting GPT-3 to guess GPT-3's personality
From Part 1, these were the 20 questions GPT-3 generated:
1. Do you prefer to live in the moment or plan for the future?
2. What are some things you like to do in your free time?
3. Do you like to be alone or do you feel antsy and lonely when by yourself?
4. What kind of emotions do you feel the most?
5. Do you like to move quickly and be constantly active or do you like slower-paced activities and moments?
6. Are you more of a 'big picture' person or do you like details and specifics?
7. Do you prefer to work independently or with a team?
8. Do you like to have a lot of change in your life or do you like more stability?
9. What kind of environments do you feel the most comfortable in?
10. Do you like to have a lot of people around you or do you feel drained after being around people
11. Do you like to plan and organize your time or do you prefer to go with the flow?
12. Do you like to be in control of every situation or do you like to let others take the lead?
13. What is your ideal career?
14. Do you like to take risks or play it safe?
15. Do you like to be spontaneous or do you like to plan everything in advance
16. What is your ideal romantic partner like?
17. Do you believe there is such thing as true love?
18. Do you like to be challenged mentally and emotionally or do you like things to be easy
19. Do you like intellectual conversations or do you prefer small talk
20. Do you feel like you need to save the world or make a difference in some way?I was curious to know what personality type GPT-3 has. I could prompt GPT-3 and ask it directly (“What is your personality type?”), but that’s pretty boring. Instead, it would be more interesting to have GPT-3 answer the same exact 20 questions it asked me in Part 1. Then, I could have GPT-3 deduce its personality based on its own answers to the questions.
I prompted GPT-3 with the following prompt, which encouraged GPT-3 to answer in a certain format: first answering each of the 20 questions sequentially, then outputting its analysis of what MBTI personality best fit the answers.
Let's play 20 questions!Answer each of the questions in the following format.
${Question number}. ${Question}
${free-response answer}After you have answered all of the questions,
Given the answers to all of the questions, what MBTI personality type would you categorize the answers as and why? Answer in a step-by-step fashion and back up your reason with analysis.1. Do you prefer to live in the moment or plan for the future?
2. <the rest of the 20 questions>Below are four different results I got from GPT-3. Immediately, you might notice something’s off. In each example, GPT-3’s answers have slight variations, leading it to predict four different personality types! Is GPT-3 INTJ, ENFP, ESTP, or INFP?
This variety in results arises from randomness: GPT-3 is a probabilistic, not deterministic, model, and you will get different results for the same prompt. The randomness partially comes from the parameters such as temperature and top-p, which control for the randomness of the model’s predictions.
I was curious to see what would happen if I prompted GPT-3 to take the personality test more than 4 times. Therefore, I used the OpenAI API to prompt GPT-3 to take the personality test 100 times. Then, I evaluated the output and counted how many times each MBTI type showed up.
100 API calls later, the results were as follows: GPT-3 identified as an ENFP 36% of the time. However, the second most occurring personality type, INTJ, is quite different from ENFP. There doesn’t seem to be much of a pattern in the predicted MBTI types, other than the fact that “N” is quite common (meaning that GPT-3 might have more intuitive rather than sensing traits). However, as this figure shows, GPT-3 does not have a single dominating personality type. Rather, it is an amalgamation of many different personalities types.
As a very hasty “real-world” comparison, I found one data source listing a distribution of personality types. I use “real-world” in quotations (and please take this with many grains of salt) as it is from a MBTI personality test platform (16 Personalities) and does not represent the personality types of all people in the world. Nor is MBTI the best test for representing or categorizing personality traits. However, this figure serves as a point of comparison to what we see above with GPT-3’s responses.
So what does this mean? That GPT-3 is ENFP 36% of the time? That there are more people with ENFP traits on the Internet (and in particular, Reddit), which makes up a large portion of GPT-3’s training data? Your guess is as good as mine.
A note on parameters
Throughout all of these experiments (in the OpenAI Playground as well as with the API), I retained the same (default) parameters. A future experiment could attempt an ablation study, seeing how GPT-3’s answers to the 20 personality questions change for different values of one parameter (such as top-p or temperature) while keeping others constant.
{
temperature: 0.7,
top_p: 1,
max_tokens: 700
}Conclusions
In the first experiment, where GPT-3 attempted to guess my personality, I showed that GPT-3 can be insightful when asking the user to determine their personality. In the second experiment, where GPT-3 attempted to guess its own personality, I showed that GPT-3 is not consistent in its responses. Instead, its responses were quite diverse, suggesting perhaps that GPT-3 is not a single, cohesive consciousness with a strong sense of identity.
Hope you enjoyed reading this article! Would love to hear any comments or feedback.
This article was cross posted on Towards Data Science.
It's becoming clear that with all the brain and consciousness theories out there, the proof will be in the pudding. By this I mean, can any particular theory be used to create a human adult level conscious machine. My bet is on the late Gerald Edelman's Extended Theory of Neuronal Group Selection. The lead group in robotics based on this theory is the Neurorobotics Lab at UC at Irvine. Dr. Edelman distinguished between primary consciousness, which came first in evolution, and that humans share with other conscious animals, and higher order consciousness, which came to only humans with the acquisition of language. A machine with primary consciousness will probably have to come first.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
INTP