Sunday, May 24, 2009

Opening tunnels for data mining

A couple of mines down the road from me in Pulaski County produce iron oxide pigments, not just the sand and gravel that are more common in local "mining" operations.

OK, that's not what the title means by "data mining," but it's something I didn't know before today, and it was easy info to dig up, thanks to a new project called -- an initiative to make more government databases easily accessible to the public. (In my case, browsing links led to a database of active mines at USGS. More interested in coal mines? That took following a few more links.)

In an interesting twist on this project, see Apps for America a contest encouraging data-miners and hackers to build applications that USE such public data.

I'll count the first entry as a "proof of concept," even if it's not particularly useful. It turns FBI most-wanted list photographs into a matching game: FBI Fugitive Concentration

Meanwhile, without entering the contest, SunlightLabs has scraped together Unofficial RSS feed that it found missing from the site itself. It also provides the source code of the Python script that creates the feed, and an analysis of the current contents of

Transparency in government, participation and opportunities for collaboration are all topics of discussion at the Open Government Dialogue site from the National Academy of Public Administration.

Here's a New York Times blog entry on
Throwing Open Uncle Sam’s Data Mine - The Caucus Blog:

"Back on Jan. 21, on President Obama’s first full day in office, he put down a marker on new standards for openness and transparency in government.
His administration has already done a few things, but on Thursday, it took a big step toward its goal and started opening up vast reservoirs of federal data to the online public at a Web site called
So far, there are 47 government data bases available there that you can rummage through, with many more to come over the next months and years. The administration hopes the public will use this information to suggest ways to make the government more efficient, responsive and innovative."

It's a beginning. Those aren't the 47 most fascinating collections of government info, unless you're heavily into geology. But it's a start. On the other hand, at least one database-oriented journalist-blogger sums it up as "lame so far."

I'm still browsing both the Open Government Dialogue and sites. So far the open-the-books attitude reminds me a lot of Carl Malamud's "Reboot .gov" project at, and his creations at -- as I keep browsing, I'll be watching for connections.

No comments:

Post a Comment