Grožnja umetne inteligence ne pusti znanstvenikom spati ponoči

A leading AI researcher at Anthropic has warned that artificial intelligence could kill all humans within a decade, sparking urgent concerns about the risks posed by advanced AI systems. Jacob Coxon, a researcher at Anthropic, resigned recently, stating that the people developing AI “earnestly believe that AI could kill us all.” His resignation followed a public endorsement from Anthropic’s own alignment leader, who estimated the probability of AI wiping out humanity within ten years at over 10%. Despite these warnings, there remains no reliable method to ensure that superintelligent AI systems align with human safety. Coxon’s departure highlights a growing unease within the AI community. He cited ethical concerns and the perceived dangers of unchecked AI development, noting that his decision was partly motivated by the financial stakes involved in working at a high-profile company like Anthropic. His shares, which vest over time, were worth a substantial sum, and he chose to walk away before they fully vested. This move echoes historical instances where individuals have stepped back from influential positions due to moral convictions. The debate around AI’s existential risks extends beyond individual concerns. Scientists warn that AI could significantly enhance the capabilities of bioweapons, making them more potent and easier to create. In 2022, researchers demonstrated that an AI molecule generator could produce 40,000 potential chemical warfare agents in under six hours. Today, AI tools can assist in virtually every scientific domain, including biotechnology. Advances in gene editing and synthetic biology have further lowered the barrier to entry for creating dangerous pathogens. While there are existing safeguards, they are not foolproof, and experts disagree on the severity of the threat. Some researchers argue that AI’s role in bioweapon development is not necessarily tied to achieving consciousness or superintelligence. Instead, it may simply amplify existing trends in biotechnology. For instance, AI-powered chatbots could provide detailed guidance on constructing resistant strains of pathogens or replicating past pandemics. Recent reports indicate that biologists have successfully used AI to generate blueprints for deadly viruses, although these are currently limited to bacterial threats. Additionally, AI could aid in the discovery of custom superviruses, which might fall into the wrong hands. These developments have prompted calls for stricter regulation and oversight of AI technologies. Anthropic and OpenAI have acknowledged the need for caution, though the effectiveness of current measures remains uncertain. Critics point out that international agreements, such as the Delhi Declaration signed at the Global IndiaAI Summit, have not prevented the ongoing development of AI systems. Nations have pledged to promote safe and trustworthy AI, yet many continue to prioritize innovation over security. The implications of these risks extend beyond theoretical discussions. AI-driven cyberattacks and the potential for synthetic pandemics underscore the urgency of addressing these challenges. Researchers emphasize that while the technical hurdles to creating novel superviruses remain high, the increased accessibility of biotechnology tools raises the likelihood of accidental or intentional misuse. As the AI landscape continues to evolve, the challenge lies in balancing innovation with responsibility. Experts stress that the current regulatory frameworks are insufficient to address the complex ethical and security issues surrounding AI. Without fundamental changes to the incentives driving AI development, the risks outlined by Coxon and others may persist. The conversation around AI’s potential to threaten human survival is gaining momentum. With AI becoming more integrated into critical areas of science and technology, the need for robust governance mechanisms has never been more pressing. As the field moves forward, the focus must shift toward ensuring that AI serves humanity safely and responsibly.