Joint project uses massive data analysis to predict violent uprising, environmental disasters and global plagues
Could data analysis help us predict the future?
Could it be that an Israeli university, along with one of the world's biggest multinational corporations, has succeeded where the likes of Nostradamus have failed?
A joint project by the Technion, Israel's Institute of Technology and Microsoft, was able to use massive data analysis to predict phenomena such as global plagues, environmental disasters and violent uprisings, with up to 90 percent accuracy.
The Technion's Kira Radinsky and Microsoft's Eric Horvitz constructed a program which collects data spanning a 22-year period from the New York Times and other online resources. It then analyzes this data usings DBPedia, a resource which uses Wikipedia to add layers of information; WordNet, a tool to analyze the meaning of words; and OpenCyc, a general knowledge database.
All this information provides valuable context that's not available in news articles and which is necessary to figure out general rules for what events precede other, bigger events.
For example, the system could infer connections between events in Rwanda and Angola based on the fact that they are both in Africa, have similar GDPs etc. That approach led the software to conclude that, in predicting cholera outbreaks, it should consider a country or city's location, proportion of land covered by water, population density, GDP, and whether there had been a drought the year before.
"I truly view this as a foreshadowing of what's to come," Horvitz told the MIT Technology Review.
In other occurrences, Radinsky and Horvitz's program was able to predict violent uprisings, disease and events with high death tolls with an accuracy rate of 70 to 90 percent. This recent study is the most extensive of its kind, using a total of more than 90 sources.
Helping aid organizations prepare for disaster
Despite the need for further verification, the research team is certain they have a powerful tool in their hands. The program could benefit aid organizations in preparing for events ahead of time and responding faster to crises, says Horvitz. "We've done some reaching out and plan to do some follow-up work with such people."
"Eventually this kind of work will start to have an influence on how things go for people," Horvitz concluded.
Microsoft does not have plans to commercialize Horvitz and Radinsky's research yet, but the project will continue, says Horvitz, who wants to mine more newspaper archives as well as digitized books.