INFO 202 Fall 08 Blog

This year the the USDA released the much-anticipated 2007 agricultural census.  This census showed a rise in the number of small farms, and this statistic was celebrated in many farm and food articles and blogs.

Gristmill points out that former USDA Economic Research Service researcher, Michael Roberts, argues that there may not actually be more small farms, there may simply be a difference in what “counts” as a small farm.

The important revelation here is that the USDA uses statistical weighting to arrive at the numbers for these micro-farms since many of these people don’t even self-identify as farmers — and so their precision is entirely a question of their methodology, i.e. how they decide to model the presence/frequency of these small operations. Census weighting is, of course, both controversial and necessary. Counting everything by hand can have a larger margin for error than rigorous statistical modeling. Indeed, this “controversy” is right now at the heart of a monumental battle between Democrats and Republicans over the U.S. Census (just ask Sen. Judd Gregg).

That said, there is nothing inherently wrong with the practice. However, even if your overall approach is solid, if you then change your weighting techniques from year to year, comparing annual changes is all but impossible. And that appears to be exactly what the USDA is doing.

Needless to say, this is a pretty big deal.  Are the number of small farms actually growing?  Or is the current political climate in this realm simply pushing the USDA to fudge their methods a little, causing a shift in their categorization schemes?

As we learn about the traditional categorization systems of IO (Svenonious, Lakoff, etc) and contrast them with the tools of next generation systems like XML and tagging, my thoughts continue to hover about a personal interest of mine which is the “ethics” of journalism and (failed?) attempts at both political and journalistic neutrality. In relation to IO, I am curious if established social definitions of terms, phrases, and colloquialisms (ex. lip product on a farm animal) can be used to deconstruct what is “meant” by an author, news feed, source, interviewee, speaker, etc.

Might a database that tracked the core metaphors implied by words and phrases somehow serve as a dissecting blade or is it up to an active community to closely watch the feeds and tag at will? Can an algorithm parse the subscripts of language? In light of our reading on machine translation, tracking bias seems like an even more distant and difficult goal.

So, two interesting tools showed up in my feedreeder this week and I thought it would be interesting to hear any opinions you all might have.

1) SpinSpotter – a Firefox plug-in that…uses “professionals” to build a set of rules, an algorithm to parse articles against said rules, and a community feedback model to track and tag.

Here is a BusinessWeek article about it.

I’ve installed it, but found that since it is in Beta, there doesn’t seem to be much feedback in the system yet.

2) And… Cognition, a Natural language processing database by Cognition Technologies. They claim that it is the “the largest commercially available Semantic Map of the English language.” RWW Article

They state that their system “Understands the meaning within the context of the text it is processing”, which seems promising. If a series of “loaded” words appears in a phrase, is that a sufficient index to call an author biased?

Anyway, if anyone has an interest in this area of information work, I would certainly like to talk more about it.