The article discusses Eric Schmidt’s prediction of an AI ‘slowdown’ by 2024 due to scaling laws, which refer to the relationship between the size of a machine learning model and its performance. Schmidt, the former CEO of Google, believes that the rapid progress in AI will hit a bottleneck as the cost of training larger models becomes prohibitively expensive. He cites OpenAI’s GPT-3 model, which required an estimated $4.6 million to train, as an example of the increasing computational costs. Schmidt suggests that this scaling issue could lead to a slowdown in AI advancements, potentially impacting industries relying on AI breakthroughs. However, he also acknowledges the possibility of new hardware or algorithmic breakthroughs that could mitigate this challenge. The article highlights the importance of addressing the scaling laws to sustain the rapid progress in AI and maintain its economic and societal benefits.
Source: https://www.businessinsider.com/eric-schmidt-google-ceo-ai-scaling-laws-openai-slowdown-2024-11