Google has developed a new approach called “Project Naptime,” aiming to empower large language models to conduct research on vulnerabilities and improve automated detection methods.
The giant research company indicated that artificial intelligence technology can contribute to enhancing the ability to identify and analyze threats, as well as detect vulnerabilities that may be missed by current tools.
The Project Zero team announced that they are studying how large language models can mimic researchers’ methods in cybersecurity, such as code auditing and reverse engineering.
Project Naptime involves the interaction of an artificial intelligence agent with a targeted command base, providing the agent with a set of specialized tools designed to simulate the work of a human researcher in the security field.
The goal of this approach is to use advancements in code comprehension and general reasoning to create large language models that can simulate human behavior in discovering and presenting security vulnerabilities.
This approach includes several important components, including a code browsing tool that allows the artificial intelligence agent to navigate through the target code, a tool for running Python scripts in a secure environment, an error monitoring tool to monitor program behavior using different inputs, and a messaging tool to track progress in the task.
Google stated that Project Naptime does not lean towards any specific model and remains neutral in its backend interface. It is also known for its efficiency in monitoring cache overflow bypasses and advanced memory corruption vulnerabilities, according to the CYBERSECEVAL 2 standards released by Meta last April. These standards serve as an assessment framework to identify security risks for large language models.
During Google’s experiments to test for vulnerabilities and security flaws, an improvement in security vulnerability scores of 1.00 and 0.76 was achieved for the GPT-4 Turbo model from OpenAI, compared to previous results of 0.05 and 0.24, respectively.
According to Google, Project Naptime helps large language models conduct research on vulnerabilities that follow the iterative hypothesis-based approach of human security experts. This architectural engineering enhances the agent’s ability to identify and analyze vulnerabilities, ensuring the accuracy and repeatability of results.
