Two months after OpenAI launched their flagship generative AI product, ChatGPT, we published a pair of blog posts exploring its potential to fuel behavioural science research and addressing some of its limitations. This article reflects on AI’s progress and expert’s concerns, and reconsiders potential use cases of this powerful tool in behavioural science research now and going forward. 

Over the last 18 months, adoption of AI in the workplace has more than doubled, with three in every four knowledge workers now using it. There has been significant improvements in terms of both useability, reliability, and use cases. ChatGPT can now search the internet to return up-to-date answers, and GPT-4o can understand and reply with text, audio and images. It will be available as a desktop application and integrated into Apple’s operating systems. The market of AI providers is growing rapidly and other flagship products such Microsoft’s Copilot will be widely available soon. 

Although it has made considerable progress, experts highlight the serious consequences of AI. They have concerns over abuses of users’ privacy, its ability to generate misinformation, human job losses, and “content satisficing” where we become accustomed to a lower quality of AI-generated output. It is alarming that for every 1,000 researchers trying to make AI more capable, just one is researching how to make it safer. In addition, the expansion of AI is having a tremendous impact on the environment. The US has slowed down its phase out of coal due to the enormous energy demands of AI - using a program like ChatGPT requires ten times more electricity than a browser search.

When it comes to behavioural science research, ChatGPT has also gained significant ground since its public launch. Below, we outline a range of use cases of AI that are either already being used or that we believe can be reliably and effectively be performed by AI in the near future.  

• Help with desk research. When it first launched, ChatGPT was infamous for its fabrication of academic references which made it not just an unreliable tool for academic writing, but arguably a dangerous one. It has somewhat improved in this regard, although one should still use it for desk research with caution.1 It has potential to be used to gather relevant papers for a literature review, and to summarise papers using the TL;DR (too long; didn’t read) prompt. 

• Test surveys before they are launched to real participants. Before launching a survey experiment, it’s important to check for and rectify any coding errors to prevent technical glitches and potential sources of bias during data collection; this is a task that AI could perform. Additionally, AI could be prompted to check that questions make sense and that there are no loopholes in the survey logic. In this way, it could be a useful risk management tool for survey design.

• Act as research participants. Whether AI can accurately replicate human participants in research is an emerging question in behavioural research. If it can, it would reduce significant barriers to conducting research such as time and cost to recruit participants, prevent potential psychological harm to participants, and guarantee representative study populations. 

• Analyse qualitative data. Analysing responses to open-ended survey questions manually can be extremely time consuming, particularly when a survey has thousands of participants. AI may be able to effectively analyse and categorise responses based on keywords and phrases, themes, or tone. It could then produce charts and graphs to visualise this information. Not only would this be more efficient, but it would enable behavioural scientists to include more open-ended questions in future surveys and therefore to gain richer insights than when responses are restrained by a pre-specified list of options. 

• Write, edit, and explain code. ChatGPT is a helpful aid for writing code, although it can get easily confused and produce unusable code. With the desktop version of GPT-4o being able to read your screen, its functionality will improve, making it a helpful tool to support data analysis. 

• Create slide decks. Slide decks are a great way to clearly present information, and are widely used in both consulting and academia. However, they can be time consuming to create, and are rarely visually engaging. An AI tool that can produce high quality slides based on text and images, such as Beautiful.ai or Copilot, would be a valuable aid for behavioural scientists.

The Behaviouralist is excited by the potential for AI to complement our work as we strive to have more time to be creative whilst maintaining academic rigour and high quality outputs. Over the next few months, Kai (Senior Economist) and Ondrej (Senior Behavioural Scientist) will be trialling some of the use cases for AI explored above to see how it can add value to all stages of our project work, from behavioural diagnosis through to data analysis and reporting. 

¹ N.B. I prompted GPT-4 to ‘provide me with five academic papers that have been foundational in the field of behavioural science’ and it produced mixed results. It did produce five real, and significant papers, including Prospect Theory: An Analysis of Decision under Risk (Kahneman and Tversky, 1979) and the Theory of Planned Behaviour (Ajzen, 1991), but also included papers relevant to philosophy and therapy, rather than behavioural science.