The article discusses the launch of Safe Superintelligence Inc., a new company dedicated to developing ethical and safe artificial superintelligence (ASI). ASI refers to AI systems that surpass human intelligence across a wide range of domains. The company aims to address the potential risks and challenges associated with the development of superintelligent AI systems. Key points include: 1) Safe Superintelligence Inc. was founded by Evan Hubinger, a researcher in AI alignment, which focuses on ensuring advanced AI systems behave in accordance with human values and intentions. 2) The company plans to work on technical AI safety problems, such as scalable oversight, corrigibility, and value learning. 3) It will also explore potential governance structures and incentive models to ensure the development of beneficial ASI. 4) The company has raised $5 million in seed funding from investors like Dustin Moskovitz and Sam Altman. 5) Safe Superintelligence Inc. aims to collaborate with other organizations working on AI safety and ethics.
Source: https://time.com/6990076/safe-superintelligence-inc-announced/