The article discusses the rapidly increasing computational costs associated with training large AI models, as highlighted in a recent report by Epoch AI. Key points include: The cost of training AI models has increased by over 300,000 times since 2012, driven by the growing size and complexity of these models. Training the GPT-3 language model in 2020 cost an estimated $4.6 million, while more recent models like Google’s PaLM and OpenAI’s InstructGPT cost tens of millions of dollars. This exponential growth in compute requirements poses significant financial and environmental challenges, as training these models consumes vast amounts of energy and generates substantial carbon emissions. The report warns that if this trend continues unchecked, the compute required to train a single AI model could eventually surpass the entire world’s computing capacity. To address these issues, the report calls for greater efficiency in AI model training, as well as increased investment in specialized AI hardware and renewable energy sources.
Source: https://time.com/6984292/cost-artificial-intelligence-compute-epoch-report/