RobotFileParserFixed

patched version of RobotFileParser, integrating fixes from Python 2.3a2 and bug 690214

The RobotFileParserFixed class is accessible via the fmtorrent.master.aggregator.feedfinder module.

Methods

f __init__(self, url='') ...

f can_fetch(self, useragent, url) ...

using the parsed robots.txt decide if useragent can fetch url

See the source for more information.