tomaarsen/natural-questions-hard-negatives

Dataset Card for Natural Questions with mined hard negatives

This dataset is a collection of question-answer-negative triplets and question-answer-negative_1...-negative_5 tuples from the Natural Questions dataset. See Natural Questions for additional information. This dataset can be used directly with Sentence Transformers to train embedding models.

The negative samples have been automatically mined with all-MiniLM-L6-v2 and:

• range_min: 10, i.e. we skip the 10 most similar samples
• range_max: 50, i.e. we only look at the top 50 most similar samples
• margin: 0, i.e. we require negative similarity + margin < positive similarity, so negative samples can't be more similar than the known true answer
• sampling_strategy: "random", i.e. we randomly sample from the candidate negatives rather than taking the "top" negatives
• num_negatives: 5, i.e. we mine 5 negatives per question-answer pair

Dataset Subsets

triplet-all subset

• Columns: "question", "answer", "negative"
• Column types: str, str, str
• Examples:

  {
    'query': 'when did richmond last play in a preliminary final',
    'answer': "Richmond Football Club Richmond began 2017 with 5 straight wins, a feat it had not achieved since 1995. A series of close losses hampered the Tigers throughout the middle of the season, including a 5-point loss to the Western Bulldogs, 2-point loss to Fremantle, and a 3-point loss to the Giants. Richmond ended the season strongly with convincing victories over Fremantle and St Kilda in the final two rounds, elevating the club to 3rd on the ladder. Richmond's first final of the season against the Cats at the MCG attracted a record qualifying final crowd of 95,028; the Tigers won by 51 points. Having advanced to the first preliminary finals for the first time since 2001, Richmond defeated Greater Western Sydney by 36 points in front of a crowd of 94,258 to progress to the Grand Final against Adelaide, their first Grand Final appearance since 1982. The attendance was 100,021, the largest crowd to a grand final since 1986. The Crows led at quarter time and led by as many as 13, but the Tigers took over the game as it progressed and scored seven straight goals at one point. They eventually would win by 48 points – 16.12 (108) to Adelaide's 8.12 (60) – to end their 37-year flag drought.[22] Dustin Martin also became the first player to win a Premiership medal, the Brownlow Medal and the Norm Smith Medal in the same season, while Damien Hardwick was named AFL Coaches Association Coach of the Year. Richmond's jump from 13th to premiers also marked the biggest jump from one AFL season to the next.",
    'negative': 'Collingwood Football Club In 2018 Collingwood made the finals for the first time since 2013, finishing in third place behind West Coast and Richmond. Losing the Second Qualifying Final to West Coast, they went on to beat GWS and the reigning premiers Richmond to reach the 2018 Grand Final, where they were defeated 79-74 by the West Coast Eagles.'
  }
  

• Collection strategy: Reading the natural questions dataset from sentence-transformers/natural-questions and performing hard negative mining with as_triplets=True.
• Deduplified: No

triplet-5 subset

• Columns: "question", "answer", "negative_1", "negative_2", "negative_3", "negative_4", "negative_5"
• Column types: str, str, str, str, str, str, str
• Examples:

  {
    'query': 'when did richmond last play in a preliminary final',
    'answer': "Richmond Football Club Richmond began 2017 with 5 straight wins, a feat it had not achieved since 1995. A series of close losses hampered the Tigers throughout the middle of the season, including a 5-point loss to the Western Bulldogs, 2-point loss to Fremantle, and a 3-point loss to the Giants. Richmond ended the season strongly with convincing victories over Fremantle and St Kilda in the final two rounds, elevating the club to 3rd on the ladder. Richmond's first final of the season against the Cats at the MCG attracted a record qualifying final crowd of 95,028; the Tigers won by 51 points. Having advanced to the first preliminary finals for the first time since 2001, Richmond defeated Greater Western Sydney by 36 points in front of a crowd of 94,258 to progress to the Grand Final against Adelaide, their first Grand Final appearance since 1982. The attendance was 100,021, the largest crowd to a grand final since 1986. The Crows led at quarter time and led by as many as 13, but the Tigers took over the game as it progressed and scored seven straight goals at one point. They eventually would win by 48 points – 16.12 (108) to Adelaide's 8.12 (60) – to end their 37-year flag drought.[22] Dustin Martin also became the first player to win a Premiership medal, the Brownlow Medal and the Norm Smith Medal in the same season, while Damien Hardwick was named AFL Coaches Association Coach of the Year. Richmond's jump from 13th to premiers also marked the biggest jump from one AFL season to the next.",
    'negative_1': 'Richmond, Texas Richmond is located near the center of Fort Bend County. Most of the city is situated on the southwest side of the Brazos River, with a small portion (Richmond Landing) on the northeast side, connected by US Highway 90A. Richmond is bordered to the southwest by the city of Rosenberg. US 90A leads east 8 miles (13\xa0km) to Sugar Land and west through Rosenberg 19 miles (31\xa0km) to East Bernard. Downtown Houston is 30 miles (48\xa0km) to the northeast.',
    'negative_2': 'Richmond Football Club Richmond\'s club mascot is called Tiger "Stripes" Dyer, named after AFL legend Jack "Captain Blood" Dyer. After taking over from Tiggy (Richmond\'s earlier mascot), Stripes displays character and attributes synonymous to the club\'s "never say die" attitude. He remains as a solid foundation for fan engagement and is commonly seen as the team\'s playful and entertaining jokester, prowling the stands and getting among the fans, to be known as the league\'s most loved mascot.[28][neutrality is disputed]',
    'negative_3': "Brisbane Bears However, the club was still struggling off-field. One of the Bears' biggest problems was its lack of support (both on and off the field) in Melbourne, the location of most of its away matches. In mid-1996, the struggling Fitzroy Football Club collapsed due to financial pressures and was seeking to merge its assets with another club. When a merger with North Melbourne in forming the North Fitzroy Kangaroos failed to win the support of the other AFL clubs, a deal for a merger was done between Fitzroy and the Bears. The new team was known as the Brisbane Lions, based at the Gabba, with Northey as the coach of the merged club. As such, the history of the Brisbane Bears as an individual entity ended after the 1996 season, with ten seasons of competition and the third-place finish in 1996 as its best performance. The Bears last match as a separate entity was a preliminary final on Saturday 21 September 1996 at the Melbourne Cricket Ground (where the Bears played their first VFL/AFL game). It was against North Melbourne, their first and last ever opponents, the Bears lost by 38 points to North who would go on to win the premiership that year.When the team came off the MCG, the Bears were finished after a short and troubled existence.",
    'negative_4': 'Virginia Tech–West Virginia football rivalry Virginia Tech held the trophy in six of the nine years in which it was contested, but West Virginia leads the all-time series 28–23–1. The last game was played on September 3, 2017 at FedEx Field in Landover, MD; Virginia Tech won 31–24.',
    'negative_5': 'Virginia in the American Civil War Most of the battles in the Eastern Theater of the American Civil War took place in Virginia because the Confederacy had to defend its national capital at Richmond, and public opinion in the North demanded that the Union move "On to Richmond!" The successes of Robert E. Lee in defending Richmond are a central theme of the military history of the war. The White House of the Confederacy, located a few blocks north of the State Capitol, was home to the family of Confederate leader Jefferson Davis.'
  }
  

• Collection strategy: Reading the natural questions dataset from sentence-transformers/natural-questions and performing hard negative mining with as_triplets=False.
• Deduplified: No