PMID- 37142627
OWN - NLM
STAT- PubMed-not-MEDLINE
DCOM- 20230507
LR  - 20230508
IS  - 2045-2322 (Electronic)
IS  - 2045-2322 (Linking)
VI  - 13
IP  - 1
DP  - 2023 May 4
TI  - The SciQA Scientific Question Answering Benchmark for Scholarly Knowledge.
PG  - 7240
LID - 10.1038/s41598-023-33607-z [doi]
LID - 7240
AB  - Knowledge graphs have gained increasing popularity in the last decade in science 
      and technology. However, knowledge graphs are currently relatively simple to 
      moderate semantic structures that are mainly a collection of factual statements. 
      Question answering (QA) benchmarks and systems were so far mainly geared towards 
      encyclopedic knowledge graphs such as DBpedia and Wikidata. We present SciQA a 
      scientific QA benchmark for scholarly knowledge. The benchmark leverages the Open 
      Research Knowledge Graph (ORKG) which includes almost 170,000 resources 
      describing research contributions of almost 15,000 scholarly articles from 709 
      research fields. Following a bottom-up methodology, we first manually developed a 
      set of 100 complex questions that can be answered using this knowledge graph. 
      Furthermore, we devised eight question templates with which we automatically 
      generated further 2465 questions, that can also be answered with the ORKG. The 
      questions cover a range of research fields and question types and are translated 
      into corresponding SPARQL queries over the ORKG. Based on two preliminary 
      evaluations, we show that the resulting SciQA benchmark represents a challenging 
      task for next-generation QA systems. This task is part of the open competitions 
      at the 22nd International Semantic Web Conference 2023 as the Scholarly Question 
      Answering over Linked Data (QALD) Challenge.
CI  - (c) 2023. The Author(s).
FAU - Auer, Soren
AU  - Auer S
AD  - TIB-Leibniz Information Centre for Science and Technology, Hannover, Germany.
AD  - L3S Research Center, Leibniz University Hannover, Hannover, Germany.
FAU - Barone, Dante A C
AU  - Barone DAC
AD  - Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, 
      Brazil.
FAU - Bartz, Cassiano
AU  - Bartz C
AD  - Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, 
      Brazil.
FAU - Cortes, Eduardo G
AU  - Cortes EG
AD  - Institute of Informatics, Federal University of Rio Grande do Sul, Porto Alegre, 
      Brazil.
FAU - Jaradeh, Mohamad Yaser
AU  - Jaradeh MY
AD  - TIB-Leibniz Information Centre for Science and Technology, Hannover, Germany.
AD  - L3S Research Center, Leibniz University Hannover, Hannover, Germany.
FAU - Karras, Oliver
AU  - Karras O
AD  - TIB-Leibniz Information Centre for Science and Technology, Hannover, Germany. 
      oliver.karras@tib.eu.
FAU - Koubarakis, Manolis
AU  - Koubarakis M
AD  - Department of Informatics and Telecommunications, National and Kapodistrian 
      University of Athens, Athens, Greece.
FAU - Mouromtsev, Dmitry
AU  - Mouromtsev D
AD  - Laboratory of Information Science and Semantic Technologies, ITMO University, St. 
      Petersburg, Russia.
FAU - Pliukhin, Dmitrii
AU  - Pliukhin D
AD  - Laboratory of Information Science and Semantic Technologies, ITMO University, St. 
      Petersburg, Russia.
FAU - Radyush, Daniil
AU  - Radyush D
AD  - Laboratory of Information Science and Semantic Technologies, ITMO University, St. 
      Petersburg, Russia.
FAU - Shilin, Ivan
AU  - Shilin I
AD  - Laboratory of Information Science and Semantic Technologies, ITMO University, St. 
      Petersburg, Russia.
FAU - Stocker, Markus
AU  - Stocker M
AD  - TIB-Leibniz Information Centre for Science and Technology, Hannover, Germany.
AD  - L3S Research Center, Leibniz University Hannover, Hannover, Germany.
FAU - Tsalapati, Eleni
AU  - Tsalapati E
AD  - Department of Informatics and Telecommunications, National and Kapodistrian 
      University of Athens, Athens, Greece.
LA  - eng
GR  - 819536/ERC_/European Research Council/International
PT  - Journal Article
DEP - 20230504
PL  - England
TA  - Sci Rep
JT  - Scientific reports
JID - 101563288
SB  - IM
PMC - PMC10160120
COIS- The authors declare no competing interests.
EDAT- 2023/05/05 00:42
MHDA- 2023/05/05 00:43
PMCR- 2023/05/04
CRDT- 2023/05/04 23:19
PHST- 2022/09/28 00:00 [received]
PHST- 2023/04/15 00:00 [accepted]
PHST- 2023/05/05 00:43 [medline]
PHST- 2023/05/05 00:42 [pubmed]
PHST- 2023/05/04 23:19 [entrez]
PHST- 2023/05/04 00:00 [pmc-release]
AID - 10.1038/s41598-023-33607-z [pii]
AID - 33607 [pii]
AID - 10.1038/s41598-023-33607-z [doi]
PST - epublish
SO  - Sci Rep. 2023 May 4;13(1):7240. doi: 10.1038/s41598-023-33607-z.