DARPA MUSE: Deep program analysis and big data analytics create a public database

DARPA's MUSE seeks to leverage deep program analyses and big data analytics to create a public database containing mined inferences about salient properties, behaviours and vulnerabilities of software drawn from the hundreds of billions of lines of open source code available today.

The program aims to make significant advances in the way software is built, debugged, verified, maintained and understood, and to enable the automated repair of existing programs and synthesis of new ones.

During the past decade information technologies have driven the productivity gains essential to U.S. economic competitiveness, and computing systems now control significant elements of critical national infrastructure.

As a result, tremendous resources are devoted to ensuring that programs are correct, especially at scale.

Unfortunately, in spite of developers' best efforts, software errors are at the root of most execution errors and security vulnerabilities.

To help improve this state, DARPA has created the Mining and Understanding Software Enclaves (MUSE) program.

MUSE seeks to make significant advances in the way software is built, debugged, verified, maintained and understood.

The collective knowledge gleaned from MUSE's efforts would facilitate new mechanisms for dramatically improving software correctness, and help develop radically different approaches for automatically constructing and repairing complex software.

Suresh Jagannathan

"Our goal is to apply the principles of big data analytics to identify and understand deep commonalities among the constantly evolving corpus of software drawn from the hundreds of billions of lines of open source code available today," said Suresh Jagannathan, DARPA program manager.

"We're aiming to treat programs—more precisely, facts about programs—as data, discovering new relationships (enclaves) among this 'big code' to build better, more robust software."

Central to MUSE's approach is the creation of a community infrastructure that would incorporate a continuously operational specification-mining engine.

This engine would leverage deep program analyses and foundational ideas underlying big data analytics to populate and refine a database containing inferences about salient properties, behaviours and vulnerabilities of the program components in the corpus.

If successful, MUSE could provide numerous capabilities that have so far remained elusive.

"Ideally, we could enable a paradigm shift in the way we think about software construction and maintenance, replacing the existing costly and laborious test/debug/validate cycle with 'always on' program analysis, mining, inspection and discovery," Jagannathan said.

"We could see scalable automated mechanisms to identify and repair program errors, as well as tools to efficiently create new, custom programs from existing components based only a description of desired properties."