Mine Safety

Those evil liberals at All Things Considered decided to ask experts about Data-Mining a Mountain of Phone Calls .

Since this is an open source, I’m free to talk about some things I’ve avoided getting into before.

Problem one: the current “watch list” is almost a half million names and some of those names [Edward Kennedy, David Nelson] are not exactly unique. The lack of standardized transliteration between Arabic and English means one name could be on the list multiple times because of spelling variations. The same individual may have more than one name in the Middle East, e.g. Mahmoud Abbas is also known as Abu Masen. This means there is no way of knowing how many people the list represents.

Problem two: to be of any use you have to associate people with the telephone numbers to create the template for the activity you are seeking. If we knew the telephone numbers used by the 9/11 hijackers and looked at those records would that establish the pattern? How would that pattern be different from people planning a wedding or awaiting a birth? Would it be different than finishing a business project? What would be distinctive about the calling pattern of terrorists; what would separate it from other event driven activities?

Problem three: many of the people known to be part of al Qaeda are connected by family and clan affiliations to important business and government officials in the region. The bin Laden family is very important and has contacts around the globe. Many people in the White House, including the Shrubbery and Darth, know any number of people who are acquainted with, or related to Osama bin Laden. Most of the major players in the oil industry know people who know Osama. Establishing anything based on relationships is pretty worthless.

Problem four: like every major computer project ever undertaken by a large organization, this sucker was pushed out the door without any major investment in design and planning. They went out and scooped up the data and started playing with it before the design specifications got beyond the whiteboard. “Find terrorist’s cells” is a goal, not a step in achieving the goal.

The very first thing they should have done would have been to see if there was any discernible pattern in telephone calls related to the 9/11 attack. All of this talk about starting with calls to or from the Middle East ignores the fact that 9/11 activity took place in the US, Germany, and Malaysia, not Afghanistan.

Given the nature of the Pakistani government, I would certainly assume that all telephones in the country are monitored, so I doubt that real terrorists would use them.

They keep hauling in hay, which just makes it more and more difficult to find the needle.

Update: Mark Fiore offers another viewpoint.