Data Mining: The Last Report of the Office of the US Director of National Intelligence’s (ODNI’s)

Data Mining: The Last Report of the Office of the US Director of National Intelligence’s (ODNI’s)
1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)

In 2011, Earth can be grossly summarized to some stats: 7 billion inhabitants (1), about 2 billion Internet users (2), 5.3 billion Mobile phone users representing 77% of the planet (3),  more than 0.8 billion fixed landline subscribers, 3 billion employed people and about 205 millions unemployed in 2011 (4). All this crowd is frenetically producing everyday a lot of information.

More than 90 Billion emails everyday of which 81% is Spam, 500 millions Facebook users happy to share 30 billion pieces of content they probably wouldn’t have shared in their “real” lives, 5 million of photos are uploaded everyday on FlickR and 24 hours of video uploaded every minutes on Youtube, allowing an audience of more than 2 billion people per day, more than 900 billions text messages sent in a year just by Chinese subscribers and countless landlines, mobile, satellite and Internet phone calls are made every second.

Data is abundant, Information is useful, Knowledge is precious.

Most of us already faced a situation where we unsuccessfully tried to find an email, a document or a piece information that our computers already indexed or filed in a folder. Transpose this schema at the world’s level, try to find the right piece of information in the right context and you will understand the complexity of the task. The new challenge for intelligence agencies is not getting information but rather to getting the right information on time. In this context, intelligence processing is increasingly becoming a form of data mining exercise addressed by the new 2010 report of the US Director of National Intelligence’s.

The Director of National Intelligence (DNI), is the United States government official subject to the authority, direction and control of the President, who is, especially, responsible under the Intelligence Reform and Terrorism Prevention Act of 2004 for overseeing and directing the US National Intelligence Program.

In this context, The National Counterterrorism Center (NCTC) is developping a softtaware called DataSphere that uses network analysis tools on existing data about known and suspected terrorists and their associates. It detects patterns in the data that links individuals with events and actions–including identifying a set of individuals that fit the parameters designated in a threat-intelligence communication.

Information Week provides more details on other initiatives:

“The Intelligence Advanced Research Projects Activity (IARPA) is another agency exploring innovations in data mining to collect and analyze intelligence information, according to the report. Last year IARPA continued to advance development of two programs it created in 2009.

The Knowledge Discovery and Dissemination (KDD) program is aimed at quickly disseminating information from large, complex, and varied data sets so they can be integrated with other data sets already in use. The project also will create analysis tools that can work across the data sets once they’re properly aligned, according to the report. IARPA awarded research contracts for the KDD program last September.

IARPA’s Automated Low-level Analysis and Description of Diverse Intelligence, or ALADDIN, meanwhile, is a video-query program aimed at replacing a manual process already in use, according to the report. The program allows intelligence analysts to search large video data sets to quickly and reliably locate clips showing a particular type of event.”


Other Sources:

Photo Credit: Menwith Hill Surveillance Station By tj.blackwell / FlickR