Securing the future of AI
A new, National Science Foundation CAREER grant will allow a Wake Forest professor to create the first safety and security standards for collaborative, real-world AI systems
The emerging AI field of multi-agent reinforcement learning (MARL) has infinite potential for use in health care, disaster response and power grid management, allowing multiple AI systems to work together to manage complex and sometimes life-threatening situations. But the risk of one AI system’s failure leading to the others has kept MARL in the realm of gaming.
With a $598,609 National Science Foundation (NSF) CAREER award, Sarra Alqahtani, associate professor of computer science at Wake Forest University, plans to change that.
Over the next five years, she will work on developing a set of standard algorithms and benchmarks to better ensure that MARL-driven AI systems will continue to work properly, even if one of the systems, or “agents,” fails or is compromised in a deliberate attack.
In MARL, numerous AI agents collaborate to achieve a greater outcome. That could be supporting optimal health for a diabetic patient with heart disease by running a pacemaker and an insulin pump, directing a fleet of self-driving cars to transport passengers to destinations without calamity or coordinating a group of drones on a search-and-rescue mission.
However, Alqahtani said no standard exists for testing the safety and security of such MARL-driven AI systems, only a hodgepodge of de facto guidelines that primarily apply to gaming applications. And that’s not something you want to rely on in “real-world, safety-critical domains,” she said.
“I want to develop algorithms to define safety and security, and make them part of the optimization of the MARL system,” she explained. “We want them to not only collaborate to achieve certain tasks, we want them to collaborate to achieve those tasks safely.”
In previous research, Alqahtani determined that MARL systems are vulnerable. One of the AI agents could be hacked to compromise the entire system.
Her work focuses on reinforcement learning (RL). It is like teaching a child to ride a bike, Alqahtani said. You can explain how to do it, but the child needs to keep trying—and failing—before finally balancing. With MARL, researchers explain the task, the parameters and the goal, and offer rewards for each correct decision until the goal is reached, whether or not one AI agent fails.
Alqahtani grew up in a small Saudi Arabian village and chose to study computer science because it was among the most difficult courses offered in college. She began studying MARL as a post-doctoral associate at the University of Tulsa.
NSF CAREER grants are awarded to junior faculty who exemplify the role of teacher-scholars through research and education.
At Wake Forest, she has introduced classes covering RL at both the graduate and undergraduate level, an uncommon offering for undergrads. With the NSF grant, she will develop a new, hands-on undergraduate curriculum on AI security and safety, as well as a new graduate course on the safety and security of RL.
She is committed to Wake Forest’s teacher-scholar model and has made mentoring students a large part of her work, both in the lab and in the classroom.
“I tell them, ‘You’re going to go beyond just collecting results and generating graphs. You are going to come up with problems, and you are going to solve them and write papers, too,’” she said. “And they do. I believe they can do it. I think that’s what the students need—you to believe in them.”
That belief has paid off: Four of her undergraduates have received the prestigious Goldwater scholarship and the NSF Graduate Research Fellowship Program award in her five years at Wake Forest. Both honors go to students who plan to pursue research careers in science, technology, engineering or math. Alqahtani’s NSF CAREER award also will bring a post-doctoral scholar in computer science to Wake Forest.
Wake Forest students will join Alqahtani in the Amazon forests of Peru to test her MARL safety algorithms. Her interest in the environment has led to a partnership with Wake Forest’s Center for Amazonian Scientific Innovation (CINCIA), where researchers track the devastating effects of illegal artisanal gold mining on the health of the people as well as the trees and animals. Alqahtani is using MARL to teach a fleet of drones to uncover mining sites so the police can be alerted. MARL will help the fleet continue its work even if one drone is disabled.
She will conduct two workshops for CINCIA employees in summer 2027 and 2029, reinforcing a collaborative partnership between Wake Forest students and the Universidad Nacional Amazónica de Madre de Dios, the region where CINCIA is located.
“I want to be able to equip the drones with the ability to recognize unsafe or dangerous areas and avoid them,” she explained. “I’m excited to take my work outside the lab.”
Categories: Awards & Recognition, Research & Discovery
Media Contact
Alicia Roberts
media@wfu.edu
336.758.5237
Recent News
‘Climb to Remember’ at Wake Forest honors lives lost in 9/11 attacks
‘Climb to Remember’ at Wake Forest honors lives lost in 9/11 attacks
