RiddleSense: Answering Riddle Questions as Commonsense Reasoning
release_ekvt3g3oejgqpjwagl7gpytmd4
by
Bill Yuchen Lin, Ziyi Wu, Yichi Yang, Dong-Ho Lee, Xiang Ren
2021
Abstract
A riddle is a mystifying, puzzling question about everyday concepts. For
example, the riddle "I have five fingers but I am not alive. What am I?" asks
about the concept of a glove. Solving riddles is a challenging cognitive
process for humans, in that it requires complex commonsense reasoning abilities
and an understanding of figurative language. However, there are currently no
commonsense reasoning datasets that test these abilities. We propose
RiddleSense, a novel multiple-choice question answering challenge for
benchmarking higher-order commonsense reasoning models, which is the first
large dataset for riddle-style commonsense question answering, where the
distractors are crowdsourced from human annotators. We systematically evaluate
a wide range of reasoning models over it and point out that there is a large
gap between the best-supervised model and human performance -- pointing to
interesting future research for higher-order commonsense reasoning and
computational creativity.
In text/plain
format
Archived Content
There are no accessible files associated with this release. You could check other releases for this work for an accessible version.
Know of a fulltext copy of on the public web? Submit a URL and we will archive it
2101.00376v1
access all versions, variants, and formats of this works (eg, pre-prints)