SIGIR 2013 : News Vertical Search Dataset

This is a dataset containing the news-related queries and ranked documents for those queries from multiple news and user-generated sources during the period of the 10th to the 16th of January 2013. It was designed to be used when investigating real-time news vertical search in the paper 'News Vertical Search: When and What to Display to Users', published at SIGIR 2013 and accessible from:


The dataset is comprised of 3,446 files, each representing a single query topic made at a specific point in time. Each file follows a fixed naming scheme:

<query source>_<query time>_<query number>_<creation time>.topic.csv

where the <query source> is BitlyBurst, representing the Bitly bursting phrases stream, the <query time> is the timestamp when the query was identified, <query number> is the id of the query identified for that time point (queries are collected in batches) and <creation time> is the time the file was written (this is very close to the query time).

Each file is in CSV format. The first line is a comment, starting with a '#' character and provides additional information about the query. Each subsequent line lists a single document ranked for that query. The sources ranked are:

Each line of the file contains the following information


where "uniqueid" is a unique id for the query-document pair, "source" is the <query source> as before, "queryid" is a unique query identifier, "query" is the textual query string, "querydate" is the <query time>, "score" is the document score (if the associated API provides a score), "headline" is the title of the document, "summary" contains the contents for the document and "docdate" lists the time the document was created (if available).

Terms of Use

The dataset is provided free of charge 'as is' for research purposes. By using this dataset, you agree to cite the following publication in any future works and/or publications that use this dataset:

 author = {McCreadie, Richard and Macdonald, Craig and Ounis, Iadh},
 title = {News Vertical Search: When and What to Display to Users},
 booktitle = {Proceedings of SIGIR'13},
 year = {2013},
 location = {Dublin, Ireland},

The dataset can be downloaded here

last edited 2013-07-28 12:18:00 by 0402393m