All around the world, people are increasingly wise to the advance of AI. More than a few are growing ever more uneasy about it. And yet workers equipped with AI are both more productive and better at their jobs.
These are just a few of many noteworthy observations offered by analysts at Stanford University’s Institute for Human-Centered AI, aka “HAI.” The organization presents its latest findings in Artificial Intelligence Index Report 2024, the seventh in the series.
The report’s authors state the goal of their ongoing exercise is to track, collate, distill and visualize data from broad research related to AI. Here are 10 highlights from the report.
1. The public is pessimistic about AI’s economic impact.
One respected survey found that only 37% of respondents feel AI will improve their job. Just 34% anticipate AI will boost the economy, and 32% believe it will enhance the job market.
2. Demographic differences exist regarding AI optimism.
Some 59% of Gen Z respondents believe AI will improve entertainment options, versus only 40% of Baby Boomers. Additionally, individuals with higher incomes and education levels are more optimistic about AI’s positive impacts on entertainment, health and the economy than their lower-income and less-educated counterparts.
3. ChatGPT is widely known and widely used.
An international survey from the University of Toronto suggests that 63% of respondents are aware of ChatGPT. Of them, around half report using ChatGPT at least once weekly.
4. AI helps medicine take significant strides forward.
In 2023, several significant medical systems were launched, including EVEscape, which enhances pandemic prediction, and AlphaMissence, which assists in AI-driven mutation classification. AI is increasingly being utilized to propel medical advancements.
5. Highly knowledgeable medical AI has arrived.
Over the past few years, AI systems have shown remarkable improvement on the MedQA benchmark, a key test for assessing AI’s clinical knowledge. The standout model of 2023, GPT-4 Medprompt, reached an accuracy rate of 90.2%, marking a 22.6 percentage point increase from the highest score in 2022. Since the benchmark’s introduction in 2019, AI performance on MedQA has nearly tripled.
6. The FDA is approving more and more AI-equipped medical devices.
In 2022, the agency approved 139 AI-related medical devices, a 12.1% increase from 2021. Since 2012, the number of FDA-approved AI-related medical devices has increased more than 45-fold. And AI is increasingly being used for real-world medical purposes.
7. Robust and standardized evaluations for GenAI responsibility are seriously lacking.
New research from the AI Index reveals a significant lack of standardization in responsible AI reporting. Leading developers, including OpenAI, Google, and Anthropic, primarily test their models against different responsible AI benchmarks. This practice complicates efforts to systematically compare the risks and limitations of top AI models.
8. Researchers have discovered more complex vulnerabilities in large language models.
Previously, most efforts to probe these GenAI models for security weaknesses focused on testing adversarial prompts that intuitively made sense to humans. This year, researchers found less obvious strategies to get LLMs to exhibit harmful behavior, like asking the models to infinitely repeat random words.
9. The number of AI incidents continues to rise.
Some 123 incidents were reported in 2023—a 32.3% increase from 2022. Since 2013, AI incidents have grown by over twentyfold. A notable example includes AI-generated, sexually explicit deepfakes of Taylor Swift that were widely shared online.
10. ChatGPT is politically biased.
Researchers find a significant bias in ChatGPT toward Democrats in the United States and the Labour Party in the U.K. This finding raises concerns about the tool’s potential to influence users’ political views, particularly in a year marked by major global elections.
Access the report in full or by chapter here.
