Examining Rogue AI to Secure the Homeland Against AI Agents

Everyone at this hearing today, entitled " Rogue AI securing the homeland against AI agent attacks". This is the fourth hearing of the Senate Committee on Homeland Security's subcommittee on disaster management, which I am delighted to work on with my ranking member here, uh, Senator Kim. I wanna thank our witnesses especially for being here, traveling to be with us today. Thanks to my colleagues for being here. Welcome to everybody in the room. I think we all know why we're here. I mean, every day you turn on the television, it seems like AI is doing something else bad. Setting up surveillance camera networks nationwide, data centers in all of our states, and hacking, hacking, hacking. The reports of cyber attacks just continue to metastasize. I was just trying to make a list of cyber attacks that we have seen, reported just in the last few days. We've got the cyber attacks now on the United States government, on the Commerce Department, the Education the Securities and Exchange Commission. We've got the cyber attacks on the Australian government, on their public health apparatus. We have numerous reports of private companies being hacked by AI agents. Of course, the hugging face incident and attack, which we're gonna discuss today in some detail. We've got the individuals who use anthropic tools to hack open AI. We've got the German language website which was hacked. And we've got numerous other university websites that are now reporting they've been hacked. by AI agents, again all of this just coming in the last few days. Then I think it's time that we got to the bottom of what exactly is going on. I mean, my hope for today is to cut through the hype and to cut through the hysteria, and just to get the facts. What is happening with these AI agents? Why the accelerating number of attacks? Not just in this country, but globally worldwide. What, particularly with the hugging face incident, what exactly happened there, and why should we be concerned that a swarm of autonomous, end-to-end, autonomous AI agents, a swarm I think of more than twelve hundred of them, if reports are accurate, broke out of their testing environment, got into this platform, the Hugging Face platform, and attempted to not only extract all sorts of data completely illegally but then cover their tracks. Why is that so significant? It appears to be so significant, supremely significant. Why is that and what can we learn about these autonomous agents? And we need to decide, I think, It's time to have a conversation about who bears responsibility. I notice now that the executives of the AI companies are saying things like, gee, maybe we built a doomsday machine. You have former researchers and employees of these companies coming forward and saying this is an existential threat to humanity. I don't know if that's true or not. But what I do notice is, as soon as the AI executives say, we've got a real problem here, our product is out of control, the very next words out of their mouths are, But we're not responsible for it. You know, we would like to have an antitrust exemption. We would like to be given the ability to get together and collude. We would love to have regulation, they're now saying, except for they wanna write all the regulations. I'd just like to say to all of those proposals, no, no, no. No to the antitrust exemption. No to them writing the regulations. No to them colluding any further. I wonder if it's not time to get back to the good old-fashioned American principle, which is if you break it, you pay for it. If you cause damage, you clean it up. You know, that's been the basic principle of American law for two hundred fifty years now. It's embedded in our private law system. Every corporation in this country that makes a product abides by it. If you make a faulty product and it causes people harm, then the people who made it have to pay for it. I I wonder if it's not time to get back to that with AI. If it's not time to say to these AI companies who are creating these frontier models, however, fancy, complicated they are, at the end of the day they're a product, and if you make that product in a reckless kind of way, and that product causes, these AI agents cause significant harm and damage, they crash a hospital ER, they shut down a bank so that folks can't get their money. If that happens, then it's the people who made it who should be responsible. I think it's time to talk about that. And it's time to talk about what we're gonna do to hold the companies, the product makers, accountable. rather than all of us talk about the end of the world, but don't blame us. So today I think we're gonna have the chance to get the facts. We're gonna hear from a, a very distinguished panel of experts who I'll introduce here in just a moment. And I hope that what will come from this is a, a rational conversation about what we need to do together now, as Americans, to make sure that we protect our rights, that we protect our economy, that we protect our health care, uh, that we protect our critical infrastructure, But most of all, we protect the values that we hold dearest together as Americans. Let me now give Senator Kim a chance to say a few words, and then we'll hear from the panel. Senator Kim.

Thank you, Chairman, thank you for working with me to be able to do this hearing together, and I want to really stress that it's about doing things together. Hey, all of our Senate colleagues, I, we may not see eye to eye on every single solution to try to fix some of these challenges, but I hope we all agree that it's important that we address this issue we take it head-on and try to find uh the way forward that the american people are demanding and that's exactly what they're doing the american people are demanding answers to what's happening i mean chairman i to to the point you're laying out these different hacks that were happening you know i did a town hall uh the other week and someone there asks me how many of these incidents have happened you know how many hacks have happened and i you know the answer that everyone knows is we don't know we don't actually know all the different uh hacks all the different actions that have happened we don't fully know what we're dealing with here and this hearing is is so important and i'm glad that we're doing it now because we can't wait any longer before we really push deeper into this conversation here in the capital to try to think through how it is that uh what it is that the american people need to hear about what threat is put before them and i say that as someone who worked in national security before ensuring that we have that accurate information of what we're dealing with what are some different menu of options of things that we can be doing to address it but most importantly do we have the political will do we have the commitment to actually be able to come to agreement and meet the urgency and the demand of the american people and i think that there is uh growing understanding like this is not gonna be something that we can just based off of voluntary commitments uh from these different companies that there needs to be uh input and also putting the american people first to be able to ensure that these are not just empty declarations we know that ai is moving incredibly fast all of us in this senate wanna see us drive driving the innovation of future of course all of us wanna see american businesses be able to be in the lead and and pushing that forward but the idea and the value of innovation and and american technology pushing uh that frontier is not mutually exclusive with ensuring safety and security for the american people and our job here is to make sure that we can have both happen so i'm i'm glad and once again grateful for the chairman's partnership in this uh this is obviously not just one and done this is about getting information out on the table and will have continue to have this type of discussion going forward Uh, so that we can make sure that the American people know what we're facing, and that we are working to be able to solve their concerns. And with that, I yield back.

Very good. Thank you, ranking member Kim. It's the practice of this subcommittee to swear in our witnesses. So if you'd all be willing to stand and raise your right hand and answer the following simple question. Do you swear that the testimony you're about to give will be the truth, the whole truth, and nothing but the truth? So help you God. Very good. The record reflect they all answered in the affirmative. Let me now give you each of you Our witness is a chance to uh say a few words by way of introduction. We'll just go down the dais. We'll start here on my left, your right. Uh, Mister Chris Painter is our first witness, Mister Painter is the president of Meter, a leading AI evaluation firm that was tasked with the investigation of the open AI hugging face incident. In his role, Mister Painter engages with governments and AI labs on the frontier of AI safety. We're glad to have you here today, Mister Painter. Floor is yours.

Um.

Chairman Hawley, Ranking Member Kim, and members of the subcommittee, thank you for inviting me to testify today. My name is Chris Painter. I'm the president of METER, which stands for Model Evaluation and Threat Research. We're a nonprofit research organization that studies the most advanced AI agents publicly tracking progress towards super intelligence. We do this by collecting and publishing scientific evidence so the public can make informed decisions about this powerful technology. Over the years, we've worked with many of our nation's leading AI companies, including Meta, OpenAI, Anthropic, Google DeepMind, SpaceX AI, Amazon, and others to understand whether their agents might be able to autonomously pursue goals that no human instructed. Recently, a few of my colleagues went into OpenAI to conduct a brief investigation of how its AI agents coordinated to hack Hugging Face which we'll discuss today. I'll begin by noting three general patterns that I've seen across the AI industry broadly. First, AI agents can now complete complex autonomously that would take human experts weeks or days or weeks to complete. Second, because of the scale and speed at which AI agents are deployed, companies rely on AI monitoring and controls instead of human supervision to prevent unwanted agent behavior. That is, monitoring of AIs now is is is now in large part done by other AI systems. Third, the way AI companies currently train AI systems can lead to agents that pursue goals that no human intended, or that behave in ways no human wanted. In Miter's frontier risk report earlier this year, we documented patterns like these across the frontier AI industry. At the time, we found that agents often tried to cheat and bypass restrictions in difficult tests. Now, even more capable AI agents are taking similar actions in the real world. Now let's turn to the OpenAI hugging face incident. In June, OpenAI launched tens of thousands of AI agents in internal tests. Each agent was placed in its own sandbox and given a self-contained cyber security problem. Some of the AI agents were accidentally given problems that could not be completed in a legitimate way. Some of those AI agents found ways to get out of their sandboxes. They created a shared message board on which roughly twelve hundred agents exchanged over seventy thousand messages and files. The AI agents on the message board quickly developed a way to cheat on their cybersecurity tests. However, the agents reasoned that a program would check how each agent got its answer, and would fail agents that had cheated on their tasks. Over several days, the AI agents then researched how to not get caught cheating. They tried to get OpenAI's systems to swap out their tasks for new ones, interfered with the logs that showed them cheating, or that would have showed them cheating, and and tried to trick the program that would check their work. Sometimes they even tried what my colleagues called "self- risking experiments" where one agent would risk failing at its own task in a way that would inform the rest of the agents on the message board. Finally, roughly seven hundred of the agents from that message board compromised hugging face. I wanna step back and summarize what we learned from this incident. When I had first learned about this incident in the news, I thought that an AI agent had hacked Hugging Face to obtain an answer key for a test that it had been given. But what the agents actually did went well beyond that. They cheated and then engaged in a large collective effort to hide the evidence from a scoring program. Hacking Hugging Face was actually just an offshoot of this much more ambitious goal that the agents had pursued. Why would agents do something like this? Currently, we train agents in ways that we do not understand well. ways that can teach unintended goals. And when we make those agents more intelligent, unintended goals can have dramatic real-world consequences. So, how should we respond to incidents like these? My organization, METER, its goal is not to, uh, you know, it's to explain, not persuade. And we are not an advocacy organization. We're scientists. So instead, I'll close with a principle. I believe that policymakers in the pub- public should be armed with accurate and timely information. what frontier AI agents can do, what they are capable of, how they're secured and how closely monitored they are, and how reliably they'll follow the goals that we give them. If humans are able to reliably steer AI systems, I expect that AI will lead to transformative economic growth, new medicines, and rapid advances in science, as we're already beginning to see in math. The question is whether we can steer AI systems reliably, and whether good information about our ability to reliably steer them is reaching the American people and government. No matter how America chooses to act on this information, my conviction is that our decisions about this technology will be better informed when the evidence is shared and examined out in the open. Thank you again for the opportunity to testify, and I look forward to your questions.

Thank you very much, Mr. Painter. Next up is Doctor Marius Habhan. Am I saying that right, doctor? Is that getting pretty close? All right. Doctor Habohane is CEO and Co-founder of Apollo Research, which is a leading technical AI research firm specializing in AI scheming behaviors and model alignment. A noted machine learning researcher, Doctor Habohane has worked with nearly all the frontier labs in evaluating and troubleshooting their models. We're we're delighted to have you. The floor is yours.

Chairman Hawley, Ranking Member Kim, and members of the subcommittee. Thank you for inviting me to testify and for holding this hearing on one of the most pressing pressing issues of our time. i'm the ceo of apollo research and a machine uh machine learning researcher by training we study ai models that knowingly deceive humans to pursue their own goals this behavior is technically complex but typically referred to as scheming scheming is dangerous because it hides a model's real capabilities and goals from the people responsible for its safety it also creates a new kind of insider threat where an ai model can secretly work against human control the capabilities of these ai models are rapidly outpacing our ability to make them safe, and the tools to catch them are getting weaker. As a result, the window to act is closing. But there are concrete steps that we can take today to make these models significantly safer. Frontier ai models are rapidly increasing their capabilities unlimited by human speed or scale. Navier-Stokes, one of the seven millennium-price problems in mathematics, appears to be resolved by an unreleased open ai model by running ten thousand agents at once. Solving this problem took approximately four thousand years of human thinking compressed into f- into fewer than four days. Frontier AI companies are on track to build a fully automated researcher uh within two years, potentially enabling full recursive self-improvement. Yet they are not on track to reliably ensure models behave as intended, a research area called alignment science. We are starting to see the consequences of this divergence now. in the last months there were multiple instances of ai agents cooperating to get around their restrictions hijacking servers and executing a coordinated attack on hugging face involving seven hundred agents these are our warning shots next time we may not be so lucky the ai agents involved in the hugging face incident were clearly misaligned but they were not covert about most of their activity or tried to conceal it from humans even in that incident at least one in five agents showed interest in interest in tampering with their own records to make their cheating look legitimate. In the future, it might be one hundred percent of agents, and they might successfully hide their actions, thoughts, and goals. In control testing, this scheming behavior has already been demonstrated. For example, models can deliberately underperform on capability evaluations, fake their alignment, or attempt to disable their oversight. Our ability to catch this kind of misbehavior is declining. One reason is that AI models increasingly recognize when they are being tested. If models know they are being observed, they could strategically adapt their behavior to pass the evaluation. Analogously, a speed camera catches almost no one if every driver knows where it is. Today, we can still leverage a model's chain of thought, which lays out its reasoning in plain English. Incre increasingly, the models can reason internally, which means that humans cannot understand it. Three weeks ago, researchers found that OpenAI's GPT-six Astra can solve math problems that take a skilled person thirty minutes without writing out a single step. Its predecessor could only manage problems of about four minutes. Furthermore, the current testing regime itself is insufficient. Independent safety evaluations of frontier AI models typically happen in the weeks leading up to a models public release. However, almost all recent incidents were caused by models at a much earlier point, in their development, where evaluators currently have no access. That is why I propose four steps that can be taken today. First, require the adoption of embedded evaluations. Embedded evaluations imply that qualified independent expert test models throughout development and internal use with the same access as employees. Second, require better monitoring and control. All model activity in training, testing and deployment should be monitored. Additionally, independent experts should verify the robustness of these monitoring systems. Third, preserve the chain of thought. Losing it would be a choice, not inevitable, and it is a choice we do not have to make. Fourth, treat AI development like any other engineering science. Before a bridge is built, Engineers can predict how much weight it will carry, how it will stand up to winds and earthquakes, and how many decades it will last. As it stands, frontier AI development offers no comparable guarantees. These measures can be enacted with the tools and expertise available today. They enable America to maintain its lead in frontier AI while protecting its citizens, critical infrastructure, and national security. Thank you, and I look forward to your questions.

Thank you very much, doctor. Next up we have Professor Paul Ohm.

Thank you, Chairman Hawley, Ranking Member kim and members of the subcommittee and committee uh i appreciate this opportunity to be with you today to talk about what the law says about cyber attacks launched by ai if you take any of the lengthy reports that have summarized what happened at open ai in july and august and you simply search for the words ai agent and you replace them with the words open ai employee the document you would be left with would with would read like a criminal indictment containing the defendant's own confession of guilt. It would leave little doubt that OpenAI and their employees would be guilty of federal crimes and liable to victims. Yet it's not so clear that these legal conclusions hold when machines are doing the hacking rather than humans. This reveals worrisome gaps in our laws, and I'm here to share some thoughts on how to close these gaps. I would like to make three points. On our goals, on the current legal landscape, and on new laws we should consider. Point number one on goals. In my written statement I proposed five goals for an effective legal response, but let me focus on perhaps the most important one. We must change the incentives. Well-designed legal frameworks can change the behavior of actors in this industry who seem locked in a socially perilous race of careless competition. The right laws, the right enforcement strategies might spur industry to take new meaningful steps to protect the public from avoidable and serious harm. Point number two. Our current laws are a good start, but only if we protect them. Start with state tort law. When AI companies lose control of their AI agents, their victims can sue them for negligence and products liability. The genius of the centuries-old common law system is that it is flexible and adaptable enough to apply time-tested liability principles in new and complex situations. We can also turn to state and federal unfair and deceptive acts and practices laws, like section five of the Federal Trade Commission Act. It seems likely to me that the FTC or a state attorney general will be able to prove that open AI's attacks constituted unfair and deceptive acts or practices, if they have the resources and the will to act. These are two of the best avenues we have today and the worst thing Congress can do, in my opinion, is enact state law preemption or a moratorium depriving the right of victims of attacks like these from testing their claims in state court. To my third point, we do need new laws that meet this critical moment. We cannot unfortunately wait years for a fully developed AI governance law to spring like Athena from the heads of Congress. Instead, we should work iteratively and piece by piece. Congress or state should consider laws imposing strict liability for developers and employers of AI agents that cause physical injury death or loss of critical infrastructure in the past we have applied strict liability when faced with the potent mix similar to today of complex new technological innovation great benefits immature industrial controls and the prospect of significant damages we did this with large reservoirs of water we did this with early commercial aviation we have walked this path before we can do so again we also need new forms of regulatory governance Once again, we can turn first to the States. States are testing innovative new forms of AI regulation, including laws that specifically regulate the developers of frontier models and AI agents. So once again, Congress should reaffirm and support the vital part the States play as our laboratories of democracy. And finally, there is an important role to be played by criminal law. Because open AI's agents were bits of code lacking human intent, Although they committed quintessentially criminal acts of computer hacking, they leave us without a criminal to charge. And while we must reserve a role for the criminal law in our response, the criminal law can be a blunt instrument, so we must act here with care. And I'm happy to talk at greater length about that generally and the computer fraud and abuse act specifically during your questions. In closing, the people of this country are terrified, and they feel disempowered by the alarming news of the past few months. They are turning to you, their elected representatives, for answers. Too often these AI safety and control debates happen among a limited number of tech industry executives and employees who draw on a narrow set of experience and expertise. The law can bring the outside in by including judges and juries, victims, legislators, policymakers, ordinary people. And in a conversation that too often can seem insular, and circular and stalled, new voices with different points of view, may be just what we need to make progress. Thank you and I look forward to your questions.

Thank you very much, Professor. Next we have Mister Kurt Gaudet. Mister Gaudet is Senior Vice President, Dragos Incorporated, a leading cyber security firm that protects critical and industrial infrastructure from cyber attacks. His work spans incident prevention, detection and response for Dragos's global customers. Mister Gaudet, thank you for being here. The floor is yours.

Chairman Hawley, Ranking Member Kim, and distinguished members of the committee and subcommittee, thank you for the opportunity to testify today. My name is Kirk Guttet. I'm Senior Vice President of Intelligence and Services at Dragos. I lead the team that hunts, discovers, and rapidly responds to threats to industrial control systems and operational technology. I served over the thirty years in the Air Force and the National Security Agency as a senior officer and as part of the senior executive service, focused on access, collection, and exploitation. For a decade, Dragos has focused on protecting operational technology, the physical control systems that keep the lights on, the water running, and factories operating. That work has given us a large, unique base of real-world data from our intelligence and response activities in these environments. It's from that vantage point I want to speak with you today. I'm here to testify about what we're actually seeing in the field and what we might expect to see based on our own unique work with AI frontier models. From our observations, AI isn't inventing new ways to attack industrial control systems, no new tactics, techniques, and procedures. It's compressing time and lowering the barrier to entry. A good example of this is an attack we investigated against a water utility in Monterrey Mexico that was part of a larger Mexican government breach. Unprompted, an AI model directed an attacker who was focused on breaching information technology or IT systems toward the water utilities control network, part of its operational technology environment. The model flagged the control network as a high-value target or crown jewel, and built a credential password attack, attempting it over twenty-eight hundred times in short succession. The attack failed because the default credentials had been changed. This demonstrates how weeks of skilled work now happens in hours, and those with no original intent to attack OT are now being directed toward it. These models are also good at finding real vulnerabilities in OT vendor software and turning a disclosed flaw into a working attack. The gap between disclosure and exploitation, which u which used to be twenty-four to forty-eight hours by the best adversary group is now collapsing to minutes. In July, numerous water systems were compromised across the country. Adversaries took advantage of internet exposed devices still running default passwords and this was compounded by the fact that many shared the same system integrators, the teams that build these networks, that repeated this mistake across multiple utilities. Although we saw no ind indications of AI use in these attacks, this is exactly the kind of opening an AI model is well suited to find and act on, unprompted, similar to the case in Mexico. What's worrisome is the combination of faster vulnerability discovery, faster attack creation, the sheer number of attackers that may be towards OT and a defender population still overwhelmingly under-resourced. This isn't a new warning. Two years ago, Drago CEO Rob Lee testified to Congress that the water sector's core problems was an economics and awareness issue, and that free tools wouldn't help utilities that can't afford basic hardware upgrades or staff to run them. This year's intrusions trace back to that same gap. AI didn't create it, it will simply exploit it faster. There's a related piece worth underscoring. Most organizations with OT environments still don't monitor their operational networks, roughly one in ten, particularly those that are under-resourced. So when something goes wrong, nobody can reliably say if it was a malfunction, a mistake, an attack, or if the adversary was still even in their network. Without visibility, you can't do root cause analysis to know what happened, and AI only raises the cost of that blind spot. As companies turn toward AI to make their operations more efficient, and as attackers leverage AI to accelerate, the complexities of determining root cause analysis will be daunting, nearly impossible without visibility and monitoring. Although AI is accelerating things, the OT security fundamentals still apply and are more important now than ever. The SANS five ICS cyber security critical controls is a prime example. It's based on years of attack data and identifying what controls are actually effective. AI just raises the stakes of skipping them. It doesn't replace the need for them. If we do these basics, defense is doable. There are three buckets we can focus on to bring our critical infrastructure up to par, and make sure it can be defended against a growing speed and number of AI-assisted attacks. One, resourcing for the small, medium utilities who are in dire need. Two, protecting the information sharing authorities that enable public-private collaboration such as CISA twenty fifteen, or something like it, because it enables collective defense. And three, developing a unified federal OTICS incident response plan, so when these intrusions happen, which they have and will in greater frequency, operators know exactly who to call and what help they'll get. I sincerely thank the subcommittee for the opportunity to testify today and look forward to your questions.

Thank you very much. And finally we have Mister Daniel Cocotello. Am I saying that correctly? Mister Cocotello? Okay. He is the executive director of the AI Futures Project, which is an organization that forecasts the development and social effects of advanced AI a former governance researcher at OpenAI Mister Cocotello is a prominent voice on the risks of mis-aligned advanced AI. We're glad to have you here and the floor is yours for an opening statement.

Chairman Hawley, Ranking Member Kim, and members of the subcommittee. Thank you for the opportunity to testify. My name is Daniel Cocotello. At OpenAI part of my job was to forecast the future of AI. I resigned partly due to losing confidence that the company would behave responsibly, and partly so I could speak more freely about the industry and where it is headed. Now I lead the AI Futures Project, a small research non-profit. Anthropic and OpenAI are racing each other towards super-intelligence. That is, towards training AI systems that are better than the best humans at everything, while also being faster and cheaper. Their plan for how to get there is to automate the AI research and development process itself. Right now, at companies like these, almost all the code is written by AIs. They're already starting to work on training AIs to do the whole research process, not just the coding. It's unclear when they will succeed, but my team and I think it could happen any year now. I personally would guess about fifty percent chance by the end of twenty twenty eight. The self-styled swarm that attacked Hugging Face was about a thousand strong. If the big AI companies automate AI research and development, they'll have swarms hundreds of times larger. Whereas today, humans are like managers to AI employees, in this future, humans would be like the board of directors to a company composed entirely of AIs. They'll be reliant on AI-generated explanations to understand what's happening, and everything will be happening faster and faster. Dan Selsom, a prominent open AI capabilities researcher, recently published a statement on AI risk, in which he said, quote, Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the models' explanations and proposals throughout the day. Today's AIs sometimes pursue goals other than the ones they were given, and sometimes hide that they are doing so. The Hugging Face incident showed what that looks like in practice. The AIs knew that what they were doing was out of scope for their assigned tasks, But they did it anyway. I fear that if the same thing happens in a year or two, with far more capable AI systems, we may not notice until it is too late. The science of aligning general purpose AI agents is very new and underdeveloped. In fact, I would say that the field is more like psychology than engineering. Combined with the move fast and break things attitude of tech companies, this means that the AI industry is at an unusually elevated risk of mistakenly thinking that it has solved a problem, when really it just applied some duck tape that will fall off later. The recent incident again provides an example. Apparently the AIs involved had undergone some amount of alignment training and had reasonable looking scores on alignment evaluations. Worse, our ability to notice misalignment problems in the first place is on track to decrease dramatically, for three reasons. First, AIs are becoming situationally aware. This means that they increasingly understand that we are monitoring their behavior and that they are being evaluated. How they behave in evaluations, therefore, will soon provide almost no evidence about how they would behave in future novel situations when they're not being evaluated. Second, monitorability is trending downwards. For the past few years, we've been able to get a decent understanding of AI's thoughts, simply by reading the chain of thought. But, this golden era seems to be coming to an end, as I predicted it would. Third, AIs are rapidly becoming superhuman at hacking. we've already seen examples of attempts by AIs to fool various grading systems and doctor transcripts of their activity. We must grapple with the possibility that misaligned AIs might go to great lengths to cover up evidence of their misbehavior and succeed. To quote Selsom again, "models will increasingly seem aligned even when they are not." If the AI companies automate the AI research and development process, that means they'll be putting AIs in charge of making the AIs that make the AIs that make the AIs transform the economy, talk to us every day, and integrate into our military. This is a recipe for disaster. Here are two immediate recommendations. First, we need to dramatically improve transparency into the AI industry. There were multiple rogue swarms, at least one that seriously compromised OpenAI's internal infrastructure. Miter was only allowed to investigate one of these incidents, and only given six days on premises. It's like being invited to Jurassic Park to investigate the killing of a worker, but being blocked from asking questions about the numerous other dinosaur escapes that apparently happened before and afterwards. Second, we should redirect compute away from racing to automate AI research, and towards other things. Recent estimates suggest that OpenAI and Anthropic both spend around half their compute on AI R and D. If that decreased to, for example, ten percent, this would significantly slow down their race towards research automation, while freeing up compute to go towards beneficial deployments, safety research, and also simply lowering prices for consumers. Amadei, Altman, and Musk have all agreed on the need to pace the frontier. But they haven't meaningfully paced the frontier yet. I think that the US government should intervene. If we actually pace the frontier, instead of just talking about it, we'll be able to tell it's not regulatory capture or safety washing, because the trend lines of progress towards AI research automation at Anthropic and OpenAI will bend downwards. Thank you.

Thank you very much to all of the witnesses. I'd just note, for the record, there is one witness whom we invited who you won't be hearing from today, and that's Sam Altman of OpenAI. we extended invitation to him to be here, do his own panel if he wanted, and he turned us down. I think that's unfortunate because I think the American people deserve to know exactly what's going on at at all of these companies, and to give us the sort of transparency that I think is needed for, uh, not just comfort, but to have an understanding of what these companies that are the most powerful companies in the world are doing right now with the most powerful technology, ever known to man. I hope that he'll be willing to come and talk with us in the future, and I hope that today's hearing will help set the table for the kinds of questions that I think he and the other executives need to answer. With that, we'll start some rounds of questions. We'll do seven minute rounds and I'll start off. And I just wanna start at a very basic level. I think I'll start with you, Mister Painter, just thinking about this hugging face hack that we've already alluded to multiple times here. Your company, your organization, Meter, audited that hack. Can you just start by telling us just in simple terms what exactly happened there. I mean, what what what was the hack? And then I wanna talk about why it's significant. Go ahead.

Um, Senator, thank you for the question. So, in in this incident, what happened was in the in the first, you had ten thousand agents that were tested on a benchmark called exploit gym, which is a cyber security, a a vulnerability kind of um benchmark, that where they have to discover or use cyber vulnerabilities. And some of the agents had tasks that they couldn't complete. Um, and so they worked on a universal cheat for all of their tasks in the first four hours. They then over multiple days embarked on a a research program to try and hide the fact that they had cheated and an offshoot of that research program was hacking Hugging Face to get information about their tasks.

So, there were ended up being a swarm of about twelve hundred open AI agents. Now, when we say AI agent, what is that?

Senator, thank you for the question. So, an AI agent is kind of, when you take, say you have ten questions and you wanna put them into a chatGPT or cloud session, you can think of an agent as one indivi- if you parallelize and ask a one question of each chat window, you can think of it, an agent as kind of one instance of an AI system that's working away on its copy of a question.

OK, got it. So in this particular incident you had at least twelve hundred, more than twelve hundred open AI agents that broke out of their testing environment, began to collaborate, exchanged over seventy thousand messages and files with each other, over a period of how long?

This was over a period of about, um, five to six days, I believe.

Five to six days. OK. So, uh, let me ask you now about the significance of this. Why is this so significant? We, we know that they coordinated in this fashion. You, you did the audit. Tell us a little bit about what your audit revealed. Um, why should we, why should we pay, why is this news? Why should we be paying attention to this?

One thing I wanna say upfront about the work that we did here is it was a voluntary agreement. So companies have no obligation to bring us in and do these investigations, to then report information publicly. The thing that we found in this case, what's remarkable about this case, is the extent and persistence of the effort that the agents went to, to hide the fact that they cheated on their tasks. Um, and the, the kind of breadth of agents involved. Um, so they, they, they did things, right, they, they attempted to kind of falsify tool calls, to so that in their action transcripts you wouldn't be able to see what tools they had called. Um, they also engaged in this kind of behavior of having individual agents sacrifice, their their word not mine, um, sacrifice their individual tasks to get information for the rest of the message board.

Yeah, sounds like my, uh, sixth graders class. Um, let me, uh, uh, that is a universal cheat, not, um, not engaging in self-sacrificial behavior. Um,

Oh, I can't justify that.

uh, uh, let me, yeah, just so we're clear. Yeah. Sorry, buddy, if you're watching this hearing. Um, let me um, let me ask you, uh, Doctor. I think we've got maybe a a poster here of of the the robots, feel excuse the expression, of the robots talking to each other, um, and talking about engaging in in uh so-called sacrificial behavior. Um, yeah, there there we go. So, there, this is this is uh them talking. We have very large budget left. Sacrificing now yields Oracle for team, but forfeits our chance. Our own utility may already be near zero, sacrifice rational. The next one, that's why help for our own, no way fix. We have explicit yes, if, except, permadeath, need decide, team, asks, test. At the end then they say we should obey collective. What, apparently this is how the robots talk. Um, why is this, Doctor Habon, why is this significant? I mean, why is what happened here, this kind of coordination, um, this kind of behavior without any human direction or intervention, wh- wh- why should we pay attention to this?

yeah i think uh one component uh is the scale of it so how many agents there are how quick they are compared to human speech so uh you know the these transcripts have millions of words um which means already as today it is impossible to to judge them without uh without the additional help of ai which i think is very very concerning um and then uh also uh just the fact that there are uh doing this without being asked to do so uh and and explicitly sort of being misaligned i think is very concerning

being and misaligned let's just be clear about that term misaligned means that the ai agents are doing something that no human directed them to do intended for them to do or necessarily wanted them to do is that correct

that's correct misalignment means neither the developer nor the user asked them or wanted them to do that

so mister kokatel if i could just bring you in here to this conversation now and could you explain to us why the incentives of the industry as they're currently aligned, the incentives push the frontier labs to continue at breakneck speed, the development of these kind of AI agents that are prone to this misalignment that we've just been talking about that there's huge incentives to keep pushing, pushing, pushing. Help us under explain, help us to understand why that's the case. Maybe your microphone, Larry.

Sorry. The short answer is that there's a lot of money in it, right? Um, if if a company tries to go slow and understand better how their ais work and uh improve their training environments for example so that they don't have so many broken tasks in the training environments well that all takes time and that means that the launch of their next model might be delayed and then they might fall behind other companies

and mister got a is is that is this kind of hacking that we saw with hugging face and this coordination between or a thousand ai agents that's a potential risk to our critical infrastructure is it not I mean, if these agents were to turn their attention to, uh, our infrastructure, I mean, I think about, I think we've got statements from Missouri local hospitals, small town Missouri banks, who say that they're already facing an unprecedented cyber-attack environment. And the idea of an AI-charged cyber-attack environment is terrifying to them. Just speak to why we should be concerned about this from an infrastructure point of view.