inside big data and the quest of knowledge

For researchers involved in basic science, like myself, Big Data is the Holy Grail: It promises to unearth the mathematical laws that govern society at large. Motivated by this challenge, my lab has spent much of the past decade studying the activity patterns of millions of mobile phone consumers, relying on call patterns provided by mobile phone companies. This data was identical to what NSA muscled away from providers, except that ours was anonymized, processed to help research without harming the participants. In a series of research papers published in the journals Science and Nature, my team confirmed the promise of Big Data by quantifying the predictability of our daily patterns, the threat digital viruses pose to mobile phones and even the reaction people have when a bomb goes off beside them.

We also learned that when it comes to our behavior, we can't use only two scales - one for good and the other for bad. Rather, our activity patterns are remarkably diverse: For any act labeled "unusual" or "anomalous", such as calling people at odd hours or visiting sensitive locations outside our predictable daily routine, we will find millions of individuals who do just that as part of their normal routine. Hence identifying terrorist intent is more difficult than finding a needle in a haystack - it.s more like spotting a particular blade of hay.

Let's face it: Powered by the right type of Big Data, data mining is a weapon. It can be just as harmful, with long-term toxicity, as an atomic bomb. It poisons trust, straining everything from human relations to political alliances and free trade. It may target combatants, but it cannot succeed without sifting through billions of data points scraped from innocent civilians. And when it is a weapon, it should be treated like a weapon.

references

Albert-László Barabási: Scientists must spearhead ethical use of big data