Saturday, November 28, 2009

Charming Python: Easy Web data collection with mechanize and Beautiful Soup

Charming Python: Easy Web data collection with mechanize and Beautiful Soup: "At this point, we are done with mechanize; all that is left is to make some sense of that big bunch of HTML files we saved during the fetch() loop. The batch nature of the process lets me separate these cleanly, but obviously in a different program, fetch() and process() might interact more closely. Beautiful Soup makes the post-processing even easier than the initial fetch."