Johns Hopkins-led team aims to turn computer systems into digital detectives
Imagine an embassy bombing. Consider the massive amount and varied types of data that investigators would need to review to determine who carried out the attack and how it was accomplished. Such a probe could involve the slow, painstaking examinations of video footage, photos, internet communications, telephone records, and other material.
A Johns Hopkins University-led international team of scientists—supported by an $11-million, five-year U.S. Department of Defense grant—wants to streamline such investigations by developing algorithms for extracting relevant details from multi-modal data. Participating scientists from nine universities in the U.S. and the United Kingdom will convene at JHU’s Homewood campus on Wednesday for their first group meeting on the challenging project.
The team’s ultimate goal is to teach a computer system to “think” like a digital Sherlock Holmes, and to quickly identify the most useful information and ignore details it deems irrelevant.
René Vidal, a Johns Hopkins biomedical engineering professor who is principal investigator on the DOD grant, said a key goal is to develop the technology that will enable a computer to characterize the information content of multi-modal data. Taking images as an example, the goal is to develop methods that can distinguish what is happening in a particular photo and not just store it as another JPEG file.
“In a computer today,” Vidal said, “a picture of a car and a picture of a person are compressed in the same way. We want the computer to recognize what objects are present in a photo or video, what actions are taken, and to see what the contextual relationships among these entities are. That’s what we call the semantic constraints of the scene.”