The BLOGS06 test collection

Modified: 21 March 2006

BLOGS06 is a TREC test collection, created and distributed by the University of Glasgow.

A crawl of Feeds, and associated Permalink and homepage documents (from late 2005 and early 2006).
100,649 feeds were polled once a week for 11 weeks.
At the time of each poll of a feed, the homepage and newly found permalinks were saved.
Each new permalink document was downloaded approximately 2 weeks after first discovery.
Collection is supplied with redirect (URL->URL) tables, in case these are useful e.g. for link-analysis.
Reasonably sized spam component for added realism.

Stats

Total Number of Feeds:	100,649
Total Number of Feeds collected:	753,681
Average feeds collected every day:	10,615
Uncompressed Size:	38.6GB
Compressed Size:	8.0GB

Total number of permalink documents:	3,215,171
Average documents every day:	45,284
Uncompressed Size:	88.8GB
Compressed Size:	12.6GB

Total number of homepage documents:	324,880
Average homepage documents collected every day:	4,576
Uncompressed Size:	20.8GB
Compressed Size:	4.0GB

Total size of collection is 25GB