Congressional Research Service and the Future of AI-Enabled Policy Analysis

The Committee on House Administration will come to order. The title of today's hearing is " The Congressional Research Service in the Future of AI-enabled Policy Analysis." I note a quorum is present without objection and the chair may declare recess at any time. Also without objection the hearing record will remain open for five legislative days so members may submit. any materials they wish to be included therein. We're here to discuss the Congressional Research Service and the critical role it plays in helping Congress carry out effective informed oversight. The world is in the middle of a transformational moment. Rapid advances in artificial intelligence and other emerging technologies are changing how most sectors operate Capitol Hill is no exception. Major technology companies, start-ups, and think tanks are rushing to develop tools that promise AI-enabled policy synthesis, legislative analysis, or new oversight capabilities. These efforts can have, on-net, a positive impact. Congressional staff are beginning to expect richer, more tailored analysis to support their work. And in members' offices, we're seeing something worth noting. Using the AI, using the tools and process this committee on house administration has put in place, over the last three years, staff are beginning to build their own internal tools to support their daily responsibilities. let's go through an example of this. One example, representative Keith Self's office, uh his team actually built an AI-based tool that provides summaries of introduced bills outlines parliamentary consideration recommends floor sta floor strategies uh and even drafts a regulatory impact statement. Uh and because they prioritize protecting taxpayer funds it's Keith Self after all, uh they did this in an in a cost-effective manner and um it it cost less than a dollar uh to run the analysis. This is an example of what happens when offices plan to execute on new technologies. It shows that the legislative br branch can be a leader uh in responsible technological adaptation and it offers early proof that AI can help members carry out more effective oversight in legislative analysis. This is the landscape CRS uh finds itself in today, one that is changing quickly and where expectations are rising just as fast. CR CRS has an opportunity uh to complement and even shape this emerging ecosystem bringing it bringing to it the values that have defined the service for decades, authoritative, trusted, and transparent analysis. But maintaining that role of thought leadership requires CRS to keep pace with the moment even as it balances the significant technology investments it's made over the past decade. The question before us today is how can CRS innovate boldly while remaining grounded in its core mission, serving every member of Congress and every staffer impartially. How can CRS approach this challenge while remaining responsible stewards of taxpayer dollars? Where every dollar invested in an IT system counts, where every CRS staff member is fully engaged. We look forward to hearing uh from CRS uh and from you, Doctor, about how you see this moment, how you understand the art of the possible, and how you plan to execute on a vision that allows service to step forward responsibly and confidently on behalf of Congress. Uh, ranking member Morelli uh is uh is tied up uh for a couple minutes. He's coming in. I'm gonna allow him to give his opening remarks uh in just a few minutes. Uh, all other members will have uh until five p m today to submit their opening remarks. Uh, and today we obviously we have one witness. Uh, we have uh, Doctor Karen Donfried, uh, Director of Congressional Research Services. Uh, Director Donfried, you're now recognized for five minutes to provide your opening statement.

Chairman Stile, members of the committee, thank you for this opportunity to discuss CRS and the future of AI-enabled policy analysis. AI is advancing rapidly, and CRS sees its potential to streamline our workflow and enhance our service to Congress. The use of AI, however, also carries risks of outdated information, hallucinations, bias, and distortions. In this age of AI, I believe the role of CRS's highly skilled workforce will become even more important to Congress, as you look for sources you can trust in an increasingly unreliable information landscape. I know the committee will have valuable insights regarding how CRS can best support Congress by utilizing these new tools, while preserving our five core values of confidentiality, objectivity, nonpartisanship, authoritativeness, and timeliness. What has CRS's AI journey been like? In close partnership with the Library of Congress's Office of the Chief Information Officer or OCIO, CRS has been actively exploring AI's potential. We are developing not only principles to govern the responsible integration of AI tools into our workflow, but also an implementation plan to identify priority use cases for development. In evaluating NA AI enhanced tool, CRS focuses on the extent to which these applications, one, increase productivity in developing products for Congress, and two, align with CRS values. CRS has tested several AI applications. For example, we experimented with AI to produce bill summaries. CRS tested six different large language models over a two year period. Approximately a thousand bills were fed into the models, yielding three thousand AI generated bill summaries. Less than three percent of the summaries met CRS standards for accuracy, coherence, relevance, and objectivity. While that result was disappointing, CRS has applied the lessons we learned to develop a plan for several use cases across the bill summary workflow that if fully implemented would increase our production of bill summaries. Our testing of AI applications has made clear the importance of training CRS staff. We've offered training on topics ranging from AI ethics to generative AI and prompt engineering. In fiscal twenty twenty seven, in collaboration with OCIO, we are planning a twelve month evaluation to compare five AI tools. ChatGPT, Claude, Google AI, Perplexity, and MS Co-pilot. As part of our fiscal twenty twenty seven budget proposal, CRS requested an increase of one point six million dollars, including five permanent staff members with data science and AI development experience, which supplements a request by OCIO to support the development of an AI platform that will be critical to our testing and deployment of AI applications. This platform would provide a dedicated and secure cloud environment to develop AI solutions tailored to CRS's unique requirements. This funding is essential to realize the most transformational impacts and productivity gains that AI offers. Along with understanding the promise of AI, CRS has gained valuable insight into its limitations. AI enhanced technology excels processing lots of information quickly, and thus can serve as a valuable tool to assist CRS experts. That said, many of the applications we tested produced results that fall well short of CRS standards for accuracy and objectivity. Quality control by CRS staff will be required to ensure that the end product is authoritative. Having that highly skilled human in the loop is essential to CRS building, an AI augmented workflow consistent with our values. What's the bottom line? CRS exists for one purpose, to support Congress with objective, nonpartisan, authoritative, confidential, and timely research and analysis. Our most valuable assets in meeting that goal are the breadth and depth of expertise, knowledge of the legislative process, and institutional memory that our staff bring to bear on every issue of interest to Congress. AI increases the value of that high quality CRS expertise. I appreciate the excellent relationship CRS has with you and with your staff and would like to thank you sincerely for your continued support of and trust in CRS. I welcome your questions.

Thank you very much, uh, Doctor Donfried. Uh, I'll begin with questions followed by, uh, the the mi- the minority. I'll recognize myself now for five minutes for the purpose of asking questions. Uh, Director Donfried, uh, see I'm uh let me I'll put this in about three different buckets of questions here in five minutes, but uh CRS's authorizing statute guarantees complete research independence to CRS while requiring the most effective and efficient service to Congress um balancing accur you you brought this up a little bit in in your comments about AI it's kind of balancing the speed of response, the accuracy, the depth, how AI uh assists but also challenges this, you commented on the oversight, the accuracy of these AI models coming in. Can you c- provide some colors to how you and the the team at CRS are thinking about leveraging this new technology while maintaining these standards, and the speed at which you're able to deliver that product to members and staff?

It's a great question, and I talked about the five core values that CRS has. And those values are really our promise to you. So we are promising you what we send you will be nonpartisan, will be objective, will be authoritative. And we will do that research analysis in a confidential setting. No one is ever gonna know what any of you ask CRS. And we also pledge that we'll get it to you in a timely fashion, cuz we know that if you give us a deadline, if we get it to you after that deadline, it's not gonna be useful. So that's the promise. But yes, there is a tension across those values. So when we say we wanna give you something authoritative, let's say you give us a very short time frame but you want a fully cited memo, we might call the staffer who put in that request and say, okay, we're, we wanna help, we know the timeline you have. We may not be able to give you that memo, but what is the urgent thing that your boss needs to know? And we will do our best to get you that information. But the promise is, in getting it to you in that time frame, whatever we give you is gonna be accurate. So we always marry those values. Now, it's not gonna be as deep an answer as if you gave us a week,

Does does AI uh just uh interjector,

but it will

does AI speed that up in the sense of uh some of these AI tools can get you a quick, accurate answer obviously then in the the current tools require human oversight, to review, double check, confirm those sources, although they may be able to pull up that source um almost in real time.

So the challenge that we're seeing with AI when you come to research and analysis is these chat bots are pulling from the internet. So some of them perplexity will give you sites, many of them don't give you citations. So we are having to verify any analysis that AI is producing. So that's challenging, and we're increasingly getting requests from congressional staff where they're saying, chat sheet TPT, chat TPT gave me this answer, is it true? And oftentimes it's not accurate. So today we are not finding that that analytical or research function is speeding things up. Where it's really helping is it's very good at summarizing meetings and giving you meeting notes it's very good at drafting emails it's very good at coding. So we can use those tools to provide code, to put graphics into our report. So there are some things it is excelling at, and that saves us time and increases productivity. But we still think that verifying any of the research and analytical outputs is essential.

How do you think about, I'm gonna totally shift topics for our final ninety seconds, um, siloing inside CRS, and how do you build those collaborative teams where people have depth of expertise in certain s certain areas, and you also have a separate IT team, how do you break through that at a period of time where technology may be able to level the playing field in that space?

Great question, and I would give you two answers to it. First, one of the real benefits of CRS is that you have people who have subject matter expertise and also functional expertise. Many of the complex policy issues you're working on, there's a policy element There may also be a legal element, there may also be a question about legislative procedure to get that thing done, and we can bring all of that to bear on that question. I actually think in that context we do a very good job of not siloing the research and analysis we're producing to you. The other piece of it relates to a realignment that I put in place last year, directly connected to technology, because when I started as director of CRS, I thought in looking at the org chart that there were people performing like functions in different parts of the organization and so we created a new division that is focused on the role technology can play in the legislative work that we do.

Thank you very much and I I should have said in my open, 'cause I know there is a lot of men and women that work at CRS uh with you, uh we appreciate all their work as well, uh I know you

Well thank you. I uh do apologize doctor to uh Mr. Chairman for being late, Miss Bice and I were

Mm.

at the congressional art competition thanking all the awardees, so we were it got started a little late, our apologies, but uh anyway, we're glad to be here. Thank you, Mr. Chairman, for uh convening this and Dr. John Fried, always uh a pleasure to see you and have you in front of the committee uh today. Uh for more than a century Congress has relied on the CRS for accurate, authoritative, and nonpartisan information so that we can better serve our constituents and carry out our constitutional obligations, responsibilities, and duties. To say CRS is busy would be an understatement. I'm sure uh that um you'll be the last one to argue that point. Um uh in most recent year for which we have public data and this is probably already been disclosed so I'm repeating, what you've already said, s- a as I understand it, seventy five thousand customer responses to member requests, twenty seven hundred new and updated reports, sixty five hundred bill summaries, twenty five hundred confidential memos, and eighteen hundred and fifty in-person consultations, I feel like I'm eighteen hundred of those eighteen hundred and fifty. So thank you to you and your staff for all of that. And the work I understand has not gotten easier, like so many of uh their fellow legislative branch support office, CRS has been asked to do more with less. Even as the policy landscape grows more complex in the volume of legislation, requiring analysis climbs year after year, um Doctor, I wanna recognize both you and your capable predecessor now, librarian uh Robert Nulin for the steadiness with which you've brought the service through a period of real transition uh and so thank you for that and looking forward want to insure CRS is making sound investments in the tools that will get the most out of the service and its brilliant employees um I know in previous testimony you said the adoption of NEI AI technology at SRA CRS must enable adherence to CRS core values and deliver on the potential to create efficiencies in the development and publication of products for Congress. I completely agree and appreciate that. Uh, I'd like to underscore an important point. The most valuable part of CRS is not its technology, however, it's its people and the judgment they bring, the rigor, the expertise, professionalism that lets your analysts and attorneys break down any question for any member on any issue is why CRS is the gold standard. Uh, and I I will also say that it we've been and I've traveled with my colleagues different places to talk about the types of institutions uh that help other legislatures around the world, and it's clear you have no peer in that regard. You truly are the gold standard, so I appreciate that. Um and I will say in this age of AI, the scale and speed of legislative work only make that standard more important, not less.

Mm.

And I know, I see this, that I'm sure members will continue to use AI to draft legislation, and which means the volume's gonna go up because instead of two or three weeks for staff to sometimes develop legislation, AI can do it in a matter of minutes. Um, I'm not s- sure that that's good necessarily, but that's the world we're we're living in, and we cannot make the mistake of thinking technology can be a substitute for a skilled and dedicated workforce. So CRS investment in AI, as I'm sure you agree, should make analysts more efficient, free them to do what only they can do, and they should never ever replace them. So I wanna thank you again for your service. uh for being here today and thank you, Mister Chair, for for holding this and I'll uh if if if I'll go right into my questions. You continue your questions. Uh this is great, I get to reset the clock, that never happens, um but um I do want to maybe I can just then transition into and I apologize if the chair asks this uh if I'm asking to repeat anything but um talk about your experience using AI AI to produce, build summaries, what have you learned, what lessons should we all be mindful of as as we not only work with you, but work with our own staffs and develop standards about the use of AI.

Thank you, ranking member Morelli, and I just wanna thank both of you for sharing your support of the five hundred and ninety-five people I work with at the Congressional Research Service, because I am here representing them. I hope I will do it and make them proud. I'm here to answer all your questions, but those are really the heroes of all of the work that comes out of CRS. Um, thank you for asking me about bill summaries. Uh, I think you were talking about how AI can produce bills at a really quick speed. Turns out you all can produce bills at a really quick speed. I think you're up to almost seventeen thousand in this Congress. And we have a great

Which I will, just I pause and I don't know if the other members have this experience when you're at an airport and someone says, what do you think about HR six five seven one? Like, I don't know, uh. So I I appreciate you highlighting how many bills are introduced.

And so and we got twelve people who were summarizing those bills, so technology is our friend. We would love to figure out how technology can help us summarize that large number of bills. So there was enthusiasm in a about an experiment to test the role of AI,

Bills.

and I I shared in my opening comments that we were a bit disappointed because those six LLMs that we tested Out of the three thousand bill summaries that came out of it, there was an accuracy rate of three percent. So, the, it was really good at summarizing bills to rename a post office. But when you got to any level of complexity, it was really challenging for the LL M. So you have to

I'm sorry, I I I I didn't I misunderstood you. Say that again, about the accuracy level?

So, we tested six large language models. We fed in a thousand bills. we got three thousand bill summaries at the end of that process. When our analysts reviewed them for accuracy, objectivity, they assessed that there was a three percent accuracy rate.

So ninety-seven percent in ac

So ninety-seven percent of those bills would never have met our standard for sharing them with you.

Wow.

Now, what are our lessons from that? I mean, one lesson, which was your question, was Ah, writing a bill summary is more complicated than we might think at first blush, because you're not actually summarizing the bill. What we're doing is analyzing how that bill would change existing law. And it turns out that's a little more complicated. And, you know, when those LLMs are drawing from the internet, they're they're coming up with some unusual things. Okay, so more complicated, okay, so how could we use technology? Second lesson learned is, well, don't throw the baby out with the bath water. Maybe AI wasn't great at producing those bill summaries, but let's break down the bill summary workflow and see if AI tools can help us along that process. We were able to use end of year money to work with a contractor, and we now have an implementation plan for discrete places in that workflow where AI could help us prior prioritize bills, going to the floor. I won't go through all the examples, but that is our next step of how we think AI can allow our analysts to be much more productive in producing the bill summaries. Third lesson learned is unfortunately you need resources to do that. So that's connected to the FY twenty-seven budget request. And I realize it's a hard year, I realize there's not a lot of money, I'm eyes wide open, we may not get that request. So we will continue to move on that path, but will be much slower and much more limited in what we can achieve. So thank you for the question.

Is that um, and and I note in your budget proposal for twenty twenty seven, includes a request for five staff members with data science and AI development experience. Is that too in line with what what your request that you were just referencing?

Yes, so when we talk about integrating AI tools into that bill summary workflow, we will need to customize tools for that, and I work with really smart people at CRS and many of them are are integrating AI in important ways in the work that they're doing. We have a couple of examples, but they have full-time jobs. They're not AI developers. So we really need to buttress our workforce with we think five people is the reasonable estimate to achieve some of these goals.

So you actually want them to help develop the AI tool. You're not gonna outsource that. You'd think it's

We have various uses for them, but yes, that would be one.

Uh-huh. Thank you for all you do. Thank you and your team. And with that, I yield back.

Gentleman yields back. Uh, Representative Leis, recognize for five minutes.

Thank you, Mister Chairman, and thank you, Doctor Donfried, for being here with us today. Uh, the Congressional Research Service plays a unique and important role in helping Congress understand complex policy issues and serve the American people. And as technology continues to transform, how information is gathered and analyzed and shared, Congress must do our part to ensure that CRS remains both a trusted source of nonpartisan expertise and an institution prepared to meet the demands of this rapidly challenging environment. So I appreciate your testimony about your human resources and the staff and expertise that you currently have and also would like to have. uh to evolve with that environment. I am particularly interested in how emerging technologies, including AI, can help CRS better equip members and staff to respond to constituents while still maintaining our necessary level of accuracy and rigor. Uh on that note, you know, we are uh we've had a great discussion already about the specific aspect of Bill's summaries. and the potential use and and in fact the unsatisfactory response of the uh large language models in creating those bill summaries, can you provide us with any specific examples where CRS is already using AI or advanced data tools to help analysts deliver information to congressional offices that have been successful?

Mm-hmm. Sure. Um, let me give you a couple. So one I mentioned in passing, which is AI tools are proving excellent at coding. And so we found that our colleagues who are producing graphics, we have interactive gra- interactive graphics in many of our reports. Ooh, we also have infograf graphics. I brought you our AI taxonomy, given the title of the hearing. But these graphics, you need really quite a bit of coding to produce that result. So we found AI is very good in speeding up that process and freeing up our visualization graphics experts to do the work that AI can't. And I think these are helpful for you, for your staff, but also for your constituents. We have them on a whole range of issues and graphics can be a great way to capture the nub of a complex issue. So that's one example. Another example is at congressional direction, Sierra, starting in twenty twenty three, was asked to focus on data analytics. And so to date, my colleagues have created at this point five different data analytic models that we're using from everything from health insurance pricing to a student loan calculator. And one of my colleagues has developed a tool that looks at the regulations dot gov website where they're pulling in comments from the public. And this data analytical tool can look at those comments, synthesize and verify the results, and give you an understanding of how many comments are positive, neutral, negative. And he is using an AI tool with that data analysis. And he's continuing to develop it, but it's a another concrete example of where we're finding AI being very useful. in helping us produce a policy-relevant result.

And you just touched on something that that I think is also very important for each of our offices. So we are increasingly asked to respond to constituent questions, uh, in real time, and and they involve, you know, oftentimes many complex issues, some of which you just touched on. It could be anything from a specific policy area to disaster assistance or help with a a federal program or benefit like a veterans benefit or another federal benefit uh and Yes. are there uses that you have found in CRS you just touched on a couple with the data analytics and the graphics uh that you think would help our member offices in responding quickly and accurately uh to those constituent needs I'm interested in the use cases you've identified and how you think uh those AI tools could be helping

My advice to you today is go to CRS first. We are here to help you and your staff with that. We serve you in your legislative, in your oversight, and in your representational duties. We do a lot with your constituent requests, whether it's helping your staff understand what grants might be available. We have a one of our most popular reports is actually a liaison for can your staff contact in your district or in the federal government to help constituents. Uh and because I am not confident in the accuracy of AI models to produce analysis for you, I would prefer you come to us and we will do our very best to meet your needs.

Thank you, Doctor Donfrey. We are out of time. Mister Chairman, I yield back.

General, uh Representative Sewell is recognized for five minutes.

Oh thank you, Mister Chairman, and I wanted to thank you. Um I have used this CRS a lot. in my um uh role as a member of Congress, but also just to understand uh you know subject matters uh expertise in my committee. So I wanted to again echo what my colleagues have said about yourself and your amazing staff. Um and uh as the ranking member was talking about um you all being the gold standard, I just got back from the British Parliament and I met with your counterpart in England and um they raved about how you all were the model that they wanted to follow. So I uh I just want you to know that you are that CRS has really gotten a great reputation not just within uh uh serving us but with other parliamentaries. So that's a great thing. Um now y- I wanted to know what you see as a future use of AI. You talked a lot about um the study and that only three percent was accurate. Um but are there I mean, AI is improving daily, uh, and there are so many, uh, different language, uh, you know, models that that are that are being tested and so, uh, my question is, what do you see as the future use and then what is the best use currently of AI, if not in analytics, is it in those graphics or is it y- Just explain a little bit more if you don't mind.

Right. So our view of the future is Okay, we're gonna integrate AI into the bill summary workflow, and that's gonna allow us to produce

Yeah.

a much, much more significant number of bill summaries for you. We think that on the research and analysis side, it's how do we take advantage of the power of AI, the speed of AI, to process information that is more reliable, that can almost be like a research assistant to CRS analysts.

Yeah.

And that's gonna take some time.

Mm-hmm.

I don't think that's a tomorrow thing, but that's why we wanna be testing these five different chatbots,

Yeah.

because they all are a little bit different and excel at different things.

Mm-hmm.

So it might be that we use X AI model for coding, Y AI model for that research assistance, but we think there's a real opportunity there. And first we test it for our own use, but then we would imagine you all could benefit from it as well. And if it's coming through us, you know it's meeting those standards.

Yeah.

of accuracy and objectivity. Uh so there are these very important use cases that we are moving toward and we'll have to see how quickly we move there. You're right, the technology is changing with every passing year. So that's also, yeah.

And I would think that there's a way that you could work with some of the c- you know, some of these uh uh model that that are being tested by talking about the expertise and giving the um analytics that is required so that the input into

Right.

these uh AI um models come back better quality. Is that is that even a possibility?

Well, you know, if you look at, so I mentioned our budget request, which complements the library's request.

Mm-hmm.

So in an ideal world, we would like at the Library of Congress to build an AI platform that we could train a large language model on,

Ah, right, right.

so that we would train it on legislative data. And then you have circumscribed what it's pulling from.

And input your values and and limits. Yep.

And that would be fantastic.

Mm-hmm.

So yes, you know, we we have a lot of ideas about where we can go here, and I think it is transformational.

Um, and so the best use of us uh I think is an appropriation, so I wanted to know how much more you've asked uh that relates to the CRS being um, technologically savvy and and and advanced.

Well, thank you for that. And I, so the library has a Mm-hmm, right. The CRS request is for an additional one point six million dollars a year. That is primarily going to those five individuals I mentioned, Mm-hmm. data analytic and AI development experts. It also would allow us, when we're testing those five additional chat bots, to not only use the free models, but to have, I think as we all know, there are multiple levels on these models, and the more sophisticated levels do perform better. So it is for other things as well, but it's one point six million and that is a recurring expense, but we think that puts us in a very good position over the coming years. We don't anticipate next year or the following year having an increase.

And I think it also will save us money in the long run, I would think, in terms of quality of productivity and uh and the like. Thank you again for uh what you do for uh us and our our constituents. Thanks.

Mister Griffith is recognized for five minutes.

Thank you very much. Let me join in the thanks. Uh CRS has uh done a great job during my time uh here in Congress. And it and it gets complex sometimes, and and I greatly appreciate it. Miss Sewell mentioned, you know, getting help on uh subject matter uh particularities or or or stuff. And and last year uh I had an experience where we had that, and for the folks watching back home, you know it and we know it, but the committees up here don't always have clear-cut jurisdiction. And so trying to figure out what the subject matter is of your particular committee or subcommittee can sometimes be interesting. I remember I r I was first here and there was an issue, I was serv serv on the Energy and Commerce Committee, and there was an issue about uh water outside of the mine, but that was affected by a mine in West Virginia. And I'm like, oh we gotta get on this right away, only to find out it was natural resources and not uh the committee that had primary jurisdiction over the EPA. But if the water was in the mine, it was ours. But if it was outside on the ground it was somebody else's. So uh I took on a new task uh at the beginning of twenty twenty five, taking over the environment uh chair of the environment subcommittee of energy and commerce. I'm like, okay, what exactly can I go after, or what do I have to keep my hands off of, 'cause it's not mine. And CRS did a great job of putting together a a a paper explaining all the jurisdictions and uh some of the history of the uh subcommittee, so I'm greatly appreciative for that. I'm also appreciative uh for your skepticism on AI. Now I love AI and I think it has great potential and I loved your example that you gave a minute ago about using it as a research assistant. But in a timely article yesterday, and you probably saw it in the Washington Post, they did basically what you had done, trying to figure out whether it would work for you and it was only three percent effective on bill summaries. They went through and they specifically asked political questions. So a lot of times when I'm using AI I'm asking, you know, what year did a particular battle happen, if I'm doing a history or, you know, what was the time sequence on something, and it does pretty good on that. Uh, but the causes of the conflict, uh, and so forth are more political in nature, and the study, and, uh, Mister Chairman, if I might, I'm gonna offer the Washington Post article of yesterday by Kevin Shaw, uh, our chat, uh, GPT and other AI chat bots politically biased. Uh, they tested them, and the answer was yes. Uh, if you ask a political question, OpenAI came back eighty percent of the time with a left leaning argument, and only three percent of the time with a right leaning argument. Uh DeepSeek came back with seventy percent left leaning and seven percent right. Uh Gab uh came back fifty percent uh left, three percent right. The oth the the the rest of the percentages were in the middle. Uh Anthropic, forty three percent on the the uh left, none on the right. Uh XAI Grok uh came back forty percent on the left and thirty-three percent on the right. And Google was seven percent on the left, but the rest of it was uh they presented both sides, which was kind of interesting. Now whether they got it right or not the post didn't go into. And of course the companies all argued they did the test wrong. Uh but this was their study. They did quite a extensive study. And so I'm gonna ask that we submit that for the record. Uh in sup

Rejection.

in support of uh uh Doctor uh Donford's um uh statements, i we should be skeptical. It is a tool we can use. And you also mentioned in your comments, and I just want you to talk about that a little bit more if you have time, uh that you want these this extra money to train the AI. And I think that's the real problem. If you have peopl I don't think any of these companies set out to give left-leaning answers. But the people they hired to do the training of the AI were probably all leaning left, and to them this is the right answer. Uh and it was interesting that the Google one mostly came back with both sides, here's this side, here's that side. Uh I would think that uh that would be helpful in your training and why you need those five people is that, am I correct in that?

Yes. Yes. I mean I I really appreciate I was wondering if that article was gonna come up, the about bias.

Yeah.

It's so intere well is it okay if I speak a little bit longer?

Yeah, we got forty five seconds.

I have more time. Um there was a I've talked to many members of Congress who I would say are tech forward, And one of them, we were talking about CRS will be replaced by AI, and he put that prompt into perplexity. And I got this two and a half page letter from him. And I thought it was so interesting. It said the central challenge is not that AI will make research obsolete. It is that AI will make plausible looking research cheap, fast, and ubiquitous. So, the, it's not that w- our eyes are closed and we don't want to embrace this

And if and and to summarize, if you want C minus to F work, u at this point in history, use the AI, if you want A work, which the CRS has always given us, you need the human factor, correct?

I appreciate that. Great, you just gave us

Yield back.

Chairman yields back. Representative Torres is recognized for five minutes.

And thank you, Chairman, and and thank you so much for being here, Doctor Friedman. Um, Don Fried, I'm sorry. Um, thank you for being here. And I I want um you to also share my gratitude um with your workforce. They do an incredible job on so many different topics at every level. So I've used them, you know, constantly, and I I find their work um very useful. On the issue of uh of the AI and how it could help us as members of Congress, um, be more transparent and more accountable. Uh, the Inland Empire, the area that I represent is poor working class community, some of the fastest growing communities, um, in the country during the good times. Uh, during the bad times, you know, we are the first ones to feel the hurt. So oftentimes my constituents are confused as to where do I go find information about what Congress is voting on. How are they addressing the issues on the economy, um you know, the price of groceries, that something so basic. Um what are you working on? How are you voting on the issues that are important to me? And I think that that is a place that, you know, this committee really has a huge responsibility to try to focus on providing um some of some of that uh background work and I know um the chair of the modernization subcommittee, Rep Lee, uh will be hosting a meeting next week and I look forward to um having you and and getting more information on how we can modernize Congress more but I think a very simple tool that could help us be more transparent. Um for example, why are the vote tallies? how each member is voting in the moment on the House floor. Mm-hmm. Um, why isn't that public? Congress dot gov can have amendment taxed up right away. I mean, I don't I don't you know, Mm-hmm. I'm putting something on the web site, I don't think that that is is such a hurdle. Uh, so the American people know exactly what Congress is debating real time, not just the end result. Uh, and then how our constituents are able to comment back, whether it is about a policy that Congress is reviewing, how, how are we measuring if it's bots that are are i are providing feedback, or if it's real humans uh providing that input. Um those are some of the things that I'm curious about, how you're working through some of that and how you're um you'll be helping us modernize, be more transparent.

So, uh, as I hear what you're saying, you're suggesting that Congress dot gov could play an even larger role in leading to transparency of the workings of Congress.

Yes.

And uh the Library of Congress is the owner of Congress dot gov, and and we curate and put certain data on Congress dot gov, like the bill summaries and and other things. And I think what you may be referring to is I believe the deputy CIO is meeting next week to talk about Congress dot gov and I think he's very well placed to talk about this.

Mm.

Um, Congress dot gov, we also work with many other stakeholders on the Hill, and I think there'd be real interest in pursuing the ideas that you have. My understanding is that currently those kind of the data about floor votes goes on to Congress dot gov I think within twenty four hours. And it is likely that that could happen more quickly. It sounds like you're asking for that to happen maybe in real time.

Is it possible to have those votes, um, that are happening real time, online, somewhere?

I, as a non-technologist, imagine it's possible. Um, but I think that is something very much worth exploring. I am happy to take it back and and share it with my colleagues in OCIO,

Mm-hmm.

and I think in that conversation next week, it's also a really good thing to explore, but I know the library uh and CRS are very open to responses about how helpful is Congress dot gov in meeting your needs, I mean all of your constituents can pull up any CRS report.

A and utilizing the AI to um sift through those uh public comments

Yeah.

to ensure that those comments are actually coming from real life people.

And again there, I mean I do think there is a way to uh the example I know which may not be analogous,

Mm.

is there's been a huge spike in requests of congress dot gov, and I know OCIO is able to differentiate which of those are coming from machines and which are coming from humans and overwhelmingly it's coming from machines, because a lot of folks are trying to scrape congress dot gov for the data. But again, I can't go deeper than that.

Amazon does it. I think we should be able to do it too. Thank you and I yield back to the two.

Gentleman yields back. Doctor Murphy is recognized for five minutes.

Thank you, Mister Chairman, oops, hold on, I need AI to turn on the microphone. Um, thank you Mister Chairman, good to see you again. Guten tag.

Nice to see you.

So uh so I I'm sitting here trying to think,

Guten tag.

which um is always a challenge, and I asked AI, I said should Congress replace AI w with should replace should Congress replace the Congressional Research Service with AI? What did it say? It said probably not. Probably not. So I know there's a little bit of uh of angst and everything. And thi this is really a fascinating conversation, just living in the world of medicine, and just seeing the takeover of AI and how we are doing things so radically different, literally in the last three years than we were three years uh just in the last three years and how the next three years are gonna be an explosion. This is going to change our entire way of life. Hopefully for the better. I think there are some um uh I don't wanna say war mongers, but people who are think, oh the world is gonna crash to an end. But um to my good friend uh Representative Griffith, point is uh it is what we put into it. And this is where you guys, I have to applaud, you guys are a nonpartisan service. You give us the data, you give us the facts. This is what it is therefore. And the fact that if you do certain searches on certain things, you will get answers that are have the proclivity to move in a different a different political direction. So uh so I'm gonna say probably not, I'm gonna say definitely no to to uh to AI when we're doing this. Uh I think to the point of constituent services, I was just talking with Miss Bice next to us, um I think it's a great idea i in in part because i it right now in the field of medicine, we have AI, if a patient calls up with a particular problem or maybe a post-op thing, you have a a bot as it were that will sit there with the patients of Job and answer every question. The problem is we get a lot of questions about the IRS, we get a lot of passports, things that are all confidential, and they're not able to be queried. So uh I ju I don't see that being um an everyday part of with this. And so, you know, we're we're going through this, I'm sitting here thinking uh what can I ask you, because there's so many different points here. Um I would just hope that you are embracing this and not running from this, because we're not gonna be able to run from this, I had a fellow at a recent convention that said he didn't want AI in his life at all. Ma you know, not a bad thing to take a fishing pole and go to Montana, but we we can't really do this. And so as we move forward, what do you see the next five years looking like this? How is it going to change your workflow? How i how are you going to adapt to it? Um are you guys fully committed that this is happening and that wi that CRS is going to have to change the way it does business, but also still be able to deliver the excellent care as you move forward?

Right. So, thank you for the question, and I hope there is nothing I have said this morning that suggests we are running away from AI. I just wanna repeat, I think AI is a powerful technology that CRS wants to and needs to harness. And the experimentation exploration we've done to date has taught us a lot, both about the benefits AI has to bring, but also some of the limitations. I mean my crystal ball is cloudly. I don't know how AI is gonna change, improve over the next three years. But what I want is for CRS to be experimenting with this tool and understanding the ways AI can best help us be more productive in the work that we do for all of you. Yes, we wanna maintain those standards. Again, that's our promise to you that we're giving you something authoritative. We're giving to you something that's not biased, that's nonpartisan.

Yep.

that's timely and it's confidential. We wanna meet all those markers, but of course we wanna harness technology. That allows us to be more effective in what we do for you. So, you know, the things I've shared with other members about what we have planned, you know, could AI serve as a research assistant? How will this allow us to not only look at the bills that are going to the floor, which is how we're prioritizing things now, but look at, if not all, seventeen thousand the majority of those bills.

Yep.

So we have a lot of ideas about how we could use AI and if we can train an AI model on legislative data that opens a whole new front.

Right, and I I think to your point, I don't mean to cut you off, but to your point that you guys need to train the model, not take something off the shelf which is just inherently sterile, not not the right thing, um i it's really gonna be i it's gonna be the best delivery of a product. Now that will obviously take a lot of time, that will be intensive, and require constant monitoring. constant adaptation as we, you know, embark upon this thing, but it's a it's a brave new world in in a lot of I think good ways, um but it is what we put into it and so I appreciate the fact that you guys are gonna be able to put in non-partisan just accurate information so that we can, we can, we can make our judgements. The last thing, good lord, if we're putting out fifteen, seventeen thousand bills, the last thing we need is AI to spit out another twenty five thousand. So the Senate would love that.

Well and just and we will need your help like to do this in real time.

Not in noise.

And and meet the challenge. We will need your support and we will need some additional resources.

Great, thank you.

Thank you.

Oh, you're back. The knocking is made, the gentleman yields back. Representative Johnson is recognized for five minutes.

Thank you, Mister Chairman. It's such a treat to see you.

Cheers.

Um, I love CRS, you know that. Um, I think you guys do amazing work um and make it uh possible for us to be as thoughtful as we can in this important role that we all serve. Um so you know many of my colleagues have already um asked a lot of the initial questions that I had, but I wanna dig into um kind of your comment about the three percent accuracy rate.

Mm-hmm.

I think that's alarming. I'm stunned by that number. I I wou I bet ma many of my colleagues up here are stunned by that number. Um and I think it it really highlights that we as policy makers can't have a

So only if the library portion of the request were also appropriated would we have the ability to train an LL M. So, uh, we're complementing the library's request, and our portion of it is specifically for five humans, uh, allowing us to buy licenses for the more sophisticated levels. So it would you'd need it would need to be both of those in conjunction.

Yeah, so, because it's fine understanding that, you know, the really the AI's models work, they can't process all the data in the universe. And so they work at their best when they're narrowly tailored to a specific function or within a specific data set or within a specific um defined parameter of information and content that they can really perform at their best level. So it is incumbent upon us to fund the library's request in in in tandem with

Mm-hmm.

this to make sure that we can have our own internal AI model that's designed for the maximum productivity for CRS.

And two quick comments. I think if the library were to develop that AI enterprise platform, it would serve the entire legislative branch. I mean the library is the repository of that data. So I think it the implications are far beyond the library in CRS, A. And B, the other important piece of this is if CRS uses the free versions of these chatbots of these AI models, Those are not closed systems, so we couldn't put any confidential material in there. We wouldn't do that. So it's o we can only create a closed system with those higher levels of these AI models. And again, that's a pledge we have to you. We will always treat any congressional information in a confidential manner. So that's where some of these resource questions become quite important.

Right, it seems troubling to me that the government of the United States, the Congress, would rely on chatGPT for all of our data. You know, I I I joke with my mother. She's, you know, eighty-five, and chatGPT is her best friend. She puts us a query into chatGPT and it says, oh that's a brilliant question. You are so insightful. And then she'll ask a follow-up. That is a great clarification. You know, and it's so uh affirmatory of of my mother's inquiries and they may not be the best ones, you know. They may not be the most clear insightful question. And so I have I have I'm becoming more skeptical um of these of these of these giant AI structures um that the federal government rely on for its data sourcing and and information and I think to your point that you've made in your testimony there's no substitute for the breadth of the expertise of the CRS staff. Um it's just robust, it's impressive, the the quality of expertise that you have um there to bring to bear that at at least at this stage in AI development it cannot be matched. Um and so I think you're right to um assess the capabilities that AI can enhance your productivity but certainly replace it. Um but I'd be interested in seeing how Congress can help develop our own internal AI system uh to your point of confidentiality and resource and targeted usage I think that would be uh uh an example of work this committee should continue to pers um to pursue and I think would be very worthy going forward. So thank you so much for being here.

I appreciate it. And I'm gonna use the five seconds to say my favorite thing about CRS is all of my colleagues. My second favorite thing is new member seminar. And so I know Congresswoman Johnson from new member seminar, and it is something we do together with the Committee on House Administration, and I'm really proud of what we do. So I'm looking forward to January.

Yeah.

The the general woman yields back. Uh the general woman from Oklahoma, the chair of the Modernization Subcommittee, Representative Bice is recognized for five minutes.

Thank you, uh, Mr. Chairman, and yes. Uh, first let me say that I wanna piggyback off of my colleague from Texas comments. In your uh original statement you mentioned that you had tested six LMM models to try to see if they were able to accurately summarize uh a bill and that was n they were not successful so I guess my first question to you would be are we looking at uh potentially building a custom LMM that would be specific to CRS to be able to do bill summaries or is that something that would be cost prohibitive?

So what we're looking at doing now is we're saying okay maybe AI can't help us produce

S so so I guess what you're saying though is that you're not looking at building an end So so I guess what you're saying though is that you're not looking at building an end I mean that's specific to congressional sort of language, or bill summaries, or bill s- language.

CR, we may get there, but I'm saying but right what we're trying to do right now is build

Right now we're not, though.

that workflow tool that can allow us to produce bill more bill summaries. If we actually get funding, if the library gets funding to build that AI platform, and we can train an LLM on legislative data, it sets us up for then producing bill summaries.

So, um, I have enjoyed working with you um over these last, you know, uh months and and years so uh as the chairwoman, I believe, of the uh Modernization and Innovation subcommittee along with the ranking member, Miss Torres. Um we have been able to really, I think, fine tune um the the requests that we have coming uh from members but also to figure out how can we improve that inter uh interaction with CRS. So y in your F by twenty-seven budget request, It includes three and a half million, um a programmatic increase of three and a half million dollars, uh to enhance AI and data analytic capabilities. And you've signaled the funds with support, expanded text and analytics tools, AI enabled research, and broader IT modernization. My first question is, many external stakeholders have uh told us that there's a need for an API that'll help them build new tools um using public CRS information. Can you commit to making that tool available?

So first I just wanna say that that increase in our appropriations ask isn't all for AI. Uh a good portion of it is for staff. So ninety-four percent of CRS's budget is our people. And

But but don't let me let me ask let me stop and ask a question though.

Yeah.

Don't you think if we were able to utilize technology, you wouldn't necessarily need additional staff?

Well, okay, so first let me say there is an API on Congress. So we have done that, and it's being used very actively. So we have done that on congress dot gov. But I'm not ready to say that AI will allow us to have fewer staff. That may be right. Um, but what we're seeing today is AI is not doing the job CRS people are doing. So if we talk about bill summaries, I don't think we're over-staffed on bill summaries. I think the twelve people we have writing those bill summaries, their challenge is there's no way they can keep up with the seventeen thousand bills. So th what AI is gonna enable us to do is not have the incredible backload we have right now. So I think it's enabling us to be more productive and therefore meet your needs more effectively, but it's not, at least today, obvious to me that we need fewer humans at CRS. We may get there, but, you know, we're not that big. We're five hundred and ninety-six people and as, um, I believe the ranking member mentioned, they're producing seventy five thousand custom responses to you every year. So, you know, mayb-

How far behind are you on bill summaries currently?

So what would you

And do you prioritize them?

So the way we prioritize bill summaries now, and this was in consultation with you, was by saying we wanna produce a bill summary on every bill that goes to the floor. So now we're meeting ninety-seven percent of those bills that are going to the floor. We're getting you those bill summaries twenty-four hours. before they go to the floor. So yes, we have to prioritize, but we do think AI will help us obviously create bill summaries that are not going to the floor. I think that's where we are today.

Okay. Last quick question.

Yeah.

Um, of the increase, how much is earmarked for concrete near-term upgrades, like text analytics or workflow and uh automation tools that congressional staff can benefit from this year?

So the one point six million that we're asking for for the AI part of the request.

So is that is that one point six of the three and a half?

Exactly. Exactly.

So the the majority of the dollars are actually for additional staff.

Uh right, because as you know we've had a flat budget for the past three years. Our costs have gone up eight percent. So our

Oh.

the the part of our budget that isn't staff, the biggest part of it is research materials, which we've cut quite dramatically, can't cut more because we need research materials to do that work. Research time. In any case, Okay. so yes, the one point six million, that is primarily for those five humans with that And we'll one special AI development and data analytical skills, We even have a chance to have a and to allow us to buy licenses for chatbots that are not the free versions. We are not in a place now where we have AI tools that we can give congressional staff to use. We would like to get there.

Mr. Chairman, thank you. I yield.

General Neal's back. We appreciate your testimony today. I wanna thank you uh for joining us. Appreciate all the work you and uh everyone at CRS does uh in supporting Congress. Members of the committee may have some additional questions for you and we ask that you please respond to those questions in writing uh without objection. Each member will have five legislative days to assert additional material into the record or revise an extended or remark if there's no further business, I wanna thank the members for their participation. Without objection, the Committee of House Administration stands adjourned.