问答系统是自然语言处理和信息检索相关的一个重要学科，在实际工业界有非常多的应用场景，其核心的算法涉及机器学习和深度学习相关知识。

    本资源详细问答系统相关的详细资源，涉及问答系统技术最近的一些发展趋势，典型的问答系统架构，相关的开源数据集，相关比赛，经典论文、书籍、视频教程、课程，重要一些开源项目和代码等等，分享给需要的朋友。

    资源整理自网络，源地址：https://github.com/seriousran/awesome-qa

    带链接版资源下载地址：

    链接: https://pan.baidu.com/s/1V6_sFpiyXRj13VEQkkQkTA 

    提取码: 93yv 

目录

    •最近研究趋势

    •问答系统介绍

    •开源系统

    •问答系统相关竞赛

    •相关论文

    •代码

    •课程

    •PPT

    •数据集整理

    •数据集

    •书籍

    •重要链接

最近研究趋势

    XLNet

    •Original paper

    oXLNet: Generalized Autoregressive Pretraining for Language Understanding, Zhilin Yang, et al., arXiv preprint, 2019.

    BERT

    •Language Model

    oBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, Jacob Devlin, et al., NAACL 2019, 2018.

    oRoBERTa: A Robustly Optimized BERT Pretraining Approach, Yinhan Liu, et al., arXiv preprint, 2019.

    oALBERT: A Lite BERT for Self-supervised Learning of Language Representations, Zhenzhong Lan, et al., arXiv preprint, 2019.

    •QA

    oInvestigating the Successes and Failures of BERT for Passage Re-Ranking, Harshith Padigela, et al., arXiv preprint, May 2019.

    oBERT with History Answer Embedding for Conversational Question Answering, Chen Qu, et al., arXiv preprint, May 2019.

    oUnderstanding the Behaviors of BERT in Ranking, Yifan Qiao, et al., arXiv preprint, Apr 2019.

    oBERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis, Hu Xu, et al., arXiv preprint, Apr 2019.

    oEnd-to-End Open-Domain Question Answering with BERTserini, Wei Yang, et al., arXiv preprint, Feb 2019.

    oA BERT Baseline for the Natural Questions, Chris Alberti, et al., arXiv preprint, Jan 2019.

    oPassage Re-ranking with BERT, Rodrigo Nogueira, et al., arXiv preprint, Jan 2019.

    oSDNet: Contextualized Attention-based Deep Network for Conversational Question Answering, Chenguang Zhu, et al., arXiv, Dec 2018.

    AAAI 2020

    •TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection, Siddhant Garg, et al., AAAI 2020, Nov 2019.

    ACL 2019

    •Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering, Asma Ben Abacha, et al., ACL-W 2019, Aug 2019.

    •Towards Scalable and Reliable Capsule Networks for Challenging NLP Applications, Wei Zhao, et al., ACL 2019, Jun 2019.

    •Cognitive Graph for Multi-Hop Reading Comprehension at Scale, Ming Ding, et al., ACL 2019, Jun 2019.

    •Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Index, Minjoon Seo, et al., ACL 2019, Jun 2019.

    •Unsupervised Question Answering by Cloze Translation, Patrick Lewis, et al., ACL 2019, Jun 2019.

    •SemEval-2019 Task 10: Math Question Answering, Mark Hopkins, et al., ACL-W 2019, Jun 2019.

    •Improving Question Answering over Incomplete KBs with Knowledge-Aware Reader, Wenhan Xiong, et al., ACL 2019, May 2019.

    •Matching Article Pairs with Graphical Decomposition and Convolutions, Bang Liu, et al., ACL 2019, May 2019.

    •Episodic Memory Reader: Learning what to Remember for Question Answering from Streaming Data, Moonsu Han, et al., ACL 2019, Mar 2019.

    •Natural Questions: a Benchmark for Question Answering Research, Tom Kwiatkowski, et al., TACL 2019, Jan 2019.

    •Textbook Question Answering with Multi-modal Context Graph Understanding and Self-supervised Open-set Comprehension, Daesik Kim, et al., ACL 2019, Nov 2018.

    EMNLP-IJCNLP 2019

    •Language Models as Knowledge Bases?, Fabio Petron, et al., EMNLP-IJCNLP 2019, Sep 2019.

    •LXMERT: Learning Cross-Modality Encoder Representations from Transformers, Hao Tan, et al., EMNLP-IJCNLP 2019, Dec 2019.

    •Answering Complex Open-domain Questions Through Iterative Query Generation, Peng Qi, et al., EMNLP-IJCNLP 2019, Oct 2019.

    •KagNet: Knowledge-Aware Graph Networks for Commonsense Reasoning, Bill Yuchen Lin, et al., EMNLP-IJCNLP 2019, Sep 2019.

    •Mixture Content Selection for Diverse Sequence Generation, Jaemin Cho, et al., EMNLP-IJCNLP 2019, Sep 2019.

    •A Discrete Hard EM Approach for Weakly Supervised Question Answering, Sewon Min, et al., EMNLP-IJCNLP, 2019, Sep 2019.

    最新数据集

    •ELI5: Long Form Question Answering, Angela Fan, et al., ACL 2019, Jul 2019

    •CODAH: An Adversarially-Authored Question Answering Dataset for Common Sense, Michael Chen, et al., RepEval 2019, Jun 2019.

QA介绍

    QA类型

    •Single-turn QA: answer without considering any context

    •Conversational QA: use previsous conversation turns

    QA子类型

    •Knowledge-based QA

    •Table/List-based QA

    •Text-based QA

    •Community-based QA

    •Visual QA

    问答系统中的分析和解析

    Lanugage Analysis

    1.Morphological analysis

    2.Named Entity Recognition(NER)

    3.Homonyms / Polysemy Analysis

    4.Syntactic Parsing (Dependency Parsing)

    5.Semantic Recognition

    问答系统三个部分

    1.Fact extraction 

    i.Entity Extraction 

    a.Named-Entity Recognition(NER)

    ii.Relation Extraction 

    2.Understanding the question

    3.Generating an answer

重要事件

    •Wolfram Alpha launced the answer engine in 2009.

    •IBM Watson system defeated top Jeopardy! champions in 2011.

    •Apple's Siri integrated Wolfram Alpha's answer engine in 2011.

    •Google embraced QA by launching its Knowledge Graph, leveraging the free base knowledge base in 2012.

    •Amazon Echo | Alexa (2015), Google Home | Google Assistant (2016), INVOKE | MS Cortana (2017), HomePod (2017)

重要问答系统

    •IBM Watson - Has state-of-the-arts performance.

    •Facebook DrQA - Applied to the SQuAD1.0 dataset. The SQuAD2.0 dataset has released. but DrQA is not tested yet.

    •MIT media lab's Knowledge graph - Is a freely-available semantic network, designed to help computers understand the meanings of words that people use.

QA相关竞赛

经典论文

    •Papers

    o"Learning to Skim Text", Adams Wei Yu, Hongrae Lee, Quoc V. Le, 2017. : Show only what you want in Text

    o"Deep Joint Entity Disambiguation with Local Neural Attention", Octavian-Eugen Ganea and Thomas Hofmann, 2017.

    o"BI-DIRECTIONAL ATTENTION FLOW FOR MACHINE COMPREHENSION", Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, Hananneh Hajishirzi, ICLR, 2017.

    o"Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks", Matthew Francis-Landau, Greg Durrett and Dan Klei, NAACL-HLT 2016.

    https://GitHub.com/matthewfl/nlp-entity-convnet

    o"Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions", Wei Shen, Jianyong Wang, Jiawei Han, IEEE Transactions on Knowledge and Data Engineering(TKDE), 2014.

    o"Introduction to “This is Watson", IBM Journal of Research and Development, D. A. Ferrucci, 2012.

    o"A survey on question answering technology from an information retrieval perspective", Information Sciences, 2011.

    o"Question Answering in Restricted Domains: An Overview", Diego Mollá and José Luis Vicedo, Computational Linguistics, 2007

    o"Natural language question answering: the view from here", L Hirschman, R Gaizauskas, natural language engineering, 2001.

    oEntity Disambiguation / Entity Linking

代码

    •BiDAF - Bi-Directional Attention Flow (BIDAF) network is a multi-stage hierarchical process that represents the context at different levels of granularity and uses bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization.

    oOfficial; Tensorflow v1.2

    oPaper

    •QANet - A Q&A architecture does not require recurrent networks: Its encoder consists exclusively of convolution and self-attention, where convolution models local interactions and self-attention models global interactions.

    oGoogle; Unofficial; Tensorflow v1.5

    oPaper

    •R-Net - An end-to-end neural networks model for reading comprehension style question answering, which aims to answer questions from a given passage.

    oMS; Unofficially by HKUST; Tensorflow v1.5

    oPaper

    •R-Net-in-Keras - R-NET re-implementation in Keras.

    oMS; Unofficial; Keras v2.0.6

    oPaper

    •DrQA - DrQA is a system for reading comprehension applied to open-domain question answering.

    oFacebook; Official; Pytorch v0.4

    oPaper

    •BERT - A new language representation model which stands for Bidirectional Encoder Representations from Transformers. Unlike recent language representation models, BERT is designed to pre-train deep bidirectional representations by jointly conditioning on both left and right context in all layers.

    oGoogle; Official implementation; Tensorflow v1.11.0

    oPaper

课程

    •Question Answering - Natural Language Processing - By Dragomir Radev, Ph.D. | University of Michigan | 2016.

PPT

    •Question Answering with Knowledge Bases, Web and Beyond - By Scott Wen-tau Yih & Hao Ma | Microsoft Research | 2016.

    •Question Answering - By Dr. Mariana Neves | Hasso Plattner Institut | 2017.

数据集整理

    •NLIWOD's Question answering datasets

    •karthinkncode's Datasets for Natural Language Processing

数据集

    •AI2 Science Questions v2.1(2017)

    oIt consists of questions used in student assessments in the United States across elementary and middle school grade levels. Each question is 4-way multiple choice format and may or may not include a diagram element.

    oPaper: http://ai2-website.s3.amazonaws.com/publications/AI2ReasoningChallenge2018.pdf

    •Children's Book Test

    •It is one of the bAbI project of Facebook AI Research which is organized towards the goal of automatic text understanding and reasoning. The CBT is designed to measure directly how well language models can exploit wider linguistic context.

    •CODAH Dataset

    •DeepMind Q&A Dataset; CNN/Daily Mail

    oHermann et al. (2015) created two awesome datasets using news articles for Q&A research. Each dataset contains many documents (90k and 197k each), and each document companies on average 4 questions approximately. Each question is a sentence with one missing word/phrase which can be found from the accompanying document/context.

    oPaper: https://arxiv.org/abs/1506.03340

    •ELI5

    oPaper: https://arxiv.org/abs/1907.09190

    •GraphQuestions

    oOn generating Characteristic-rich Question sets for QA evaluation.

    •LC-QuAD

    oIt is a gold standard KBQA (Question Answering over Knowledge Base) dataset containing 5000 Question and SPARQL queries. LC-QuAD uses DBpedia v04.16 as the target KB.

    •MS MARCO

    oThis is for real-world question answering.

    oPaper: https://arxiv.org/abs/1611.09268

    •MultiRC

    oA dataset of short paragraphs and multi-sentence questions

    oPaper: http://cogcomp.org/page/publication_view/833

    •NarrativeQA

    oIt includes the list of documents with Wikipedia summaries, links to full stories, and questions and answers.

    oPaper: https://arxiv.org/pdf/1712.07040v1.pdf

    •NewsQA

    oA machine comprehension dataset

    oPaper: https://arxiv.org/pdf/1611.09830.pdf

    •Qestion-Answer Dataset by CMU

    oThis is a corpus of Wikipedia articles, manually-generated factoid questions from them, and manually-generated answers to these questions, for use in academic research. These data were collected by Noah Smith, Michael Heilman, Rebecca Hwa, Shay Cohen, Kevin Gimpel, and many students at Carnegie Mellon University and the University of Pittsburgh between 2008 and 2010.

    •SQuAD1.0

    oStanford Question Answering Dataset (SQuAD) is a reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles, where the answer to every question is a segment of text, or span, from the corresponding reading passage, or the question might be unanswerable.

    oPaper: https://arxiv.org/abs/1606.05250

    •SQuAD2.0

    oSQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 new, unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering.

    oPaper: https://arxiv.org/abs/1806.03822

    •Story cloze test

    o'Story Cloze Test' is a new commonsense reasoning framework for evaluating story understanding, story generation, and script learning. This test requires a system to choose the correct ending to a four-sentence story.

    oPaper: https://arxiv.org/abs/1604.01696

    •TriviaQA

    oTriviaQA is a reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions.

    oPaper: https://arxiv.org/abs/1705.03551

    •WikiQA

    oA publicly available set of question and sentence pairs for open-domain question answering.

    IBM Watson研究团队最近几年发表论文

    •2015

    o"Automated Problem List Generation from Electronic Medical Records in IBM Watson", Murthy Devarakonda, Ching-Huei Tsou, IAAI, 2015.

    o"Decision Making in IBM Watson Question Answering", J. William Murdock, Ontology summit, 2015.

    o"Unsupervised Entity-Relation Analysis in IBM Watson", Aditya Kalyanpur, J William Murdock, ACS, 2015.

    o"Commonsense Reasoning: An Event Calculus Based Approach", E T Mueller, Morgan Kaufmann/Elsevier, 2015.

    •2014

    o"Problem-oriented patient record summary: An early report on a Watson application", M. Devarakonda, Dongyang Zhang, Ching-Huei Tsou, M. Bornea, Healthcom, 2014.

    o"WatsonPaths: Scenario-based Question Answering and Inference over Unstructured Information", Adam Lally, Sugato Bachi, Michael A. Barborak, David W. Buchanan, Jennifer Chu-Carroll, David A. Ferrucci*, Michael R. Glass, Aditya Kalyanpur, Erik T. Mueller, J. William Murdock, Siddharth Patwardhan, John M. Prager, Christopher A. Welty, IBM Research Report RC25489, 2014.

    o"Medical Relation Extraction with Manifold Models", Chang Wang and James Fan, ACL, 2014.

    MS Research最近五年发表论文

    •2018

    o"Characterizing and Supporting Question Answering in Human-to-Human Communication", Xiao Yang, Ahmed Hassan Awadallah, Madian Khabsa, Wei Wang, Miaosen Wang, ACM SIGIR, 2018.

    o"FigureQA: An Annotated Figure Dataset for Visual Reasoning", Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, Yoshua Bengio, ICLR, 2018

    •2017

    o"Multi-level Attention Networks for Visual Question Answering", Dongfei Yu, Jianlong Fu, Tao Mei, Yong Rui, CVPR, 2017.

    o"A Joint Model for Question Answering and Question Generation", Tong Wang, Xingdi (Eric) Yuan, Adam Trischler, ICML, 2017.

    o"Two-Stage Synthesis Networks for Transfer Learning in Machine Comprehension", David Golub, Po-Sen Huang, Xiaodong He, Li Deng, EMNLP, 2017.

    o"Question-Answering with Grammatically-Interpretable Representations", Hamid Palangi, Paul Smolensky, Xiaodong He, Li Deng,

    o"Search-based Neural Structured Learning for Sequential Question Answering", Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang, ACL, 2017.

    •2016

    o"Stacked Attention Networks for Image Question Answering", Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Smola, CVPR, 2016.

    o"Question Answering with Knowledge Base, Web and Beyond", Yih, Scott Wen-tau and Ma, Hao, ACM SIGIR, 2016.

    o"NewsQA: A Machine Comprehension Dataset", Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, Kaheer Suleman, RepL4NLP, 2016.

    o"Table Cell Search for Question Answering", Sun, Huan and Ma, Hao and He, Xiaodong and Yih, Wen-tau and Su, Yu and Yan, Xifeng, WWW, 2016.

    •2015

    o"WIKIQA: A Challenge Dataset for Open-Domain Question Answering", Yi Yang, Wen-tau Yih, and Christopher Meek, EMNLP, 2015.

    o"Web-based Question Answering: Revisiting AskMSR", Chen-Tse Tsai, Wen-tau Yih, and Christopher J.C. Burges, MSR-TR, 2015.

    o"Open Domain Question Answering via Semantic Enrichment", Huan Sun, Hao Ma, Wen-tau Yih, Chen-Tse Tsai, Jingjing Liu, and Ming-Wei Chang, WWW, 2015.

    •2014

    o"An Overview of Microsoft Deep QA System on Stanford WebQuestions Benchmark", Zhenghao Wang, Shengquan Yan, Huaming Wang, and Xuedong Huang, MSR-TR, 2014.

    o"Semantic Parsing for Single-Relation Question Answering", Wen-tau Yih, Xiaodong He, Christopher Meek, ACL, 2014.

    Google AI最近五年发表论文

    •2018

    oGoogle QA

    "QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension", Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, Quoc V. Le, ICLR, 2018.

    "Ask the Right Questions: Active Question Reformulation with Reinforcement Learning", Christian Buck and Jannis Bulian and Massimiliano Ciaramita and Wojciech Paweł Gajewski and Andrea Gesmundo and Neil Houlsby and Wei Wang, ICLR, 2018.

    "Building Large Machine Reading-Comprehension Datasets using Paragraph Vectors", Radu Soricut, Nan Ding, 2018.

    oSentence representation

    "An efficient framework for learning sentence representations", Lajanugen Logeswaran, Honglak Lee, ICLR, 2018.

    o"Did the model understand the question?", Pramod K. Mudrakarta and Ankur Taly and Mukund Sundararajan and Kedar Dhamdhere, ACL, 2018.

    •2017

    o"Analyzing Language Learned by an Active Question Answering Agent", Christian Buck and Jannis Bulian and Massimiliano Ciaramita and Wojciech Gajewski and Andrea Gesmundo and Neil Houlsby and Wei Wang, NIPS, 2017.

    o"Learning Recurrent Span Representations for Extractive Question Answering", Kenton Lee and Shimi Salant and Tom Kwiatkowski and Ankur Parikh and Dipanjan Das and Jonathan Berant, ICLR, 2017.

    oIdentify the same question

    "Neural Paraphrase Identification of Questions with Noisy Pretraining", Gaurav Singh Tomar and Thyago Duque and Oscar Täckström and Jakob Uszkoreit and Dipanjan Das, SCLeM, 2017.

    •2014

    o"Great Question! Question Quality in Community Q&A", Sujith Ravi and Bo Pang and Vibhor Rastogi and Ravi Kumar, ICWSM, 2014.

    Facebook AI最近5年发表论文

    •2018

    oEmbodied Question Answering, Abhishek Das, Samyak Datta, Georgia Gkioxari, Stefan Lee, Devi Parikh, and Dhruv Batra, CVPR, 2018

    oDo explanations make VQA models more predictable to a human?, Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav, Prithvijit Chattopadhyay, and Devi Parikh, EMNLP, 2018

    oNeural Compositional Denotational Semantics for Question Answering, Nitish Gupta, Mike Lewis, EMNLP, 2018

    •2017

    oDrQA

    Reading Wikipedia to Answer Open-Domain Questions, Danqi Chen, Adam Fisch, Jason Weston & Antoine Bordes, ACL, 2017.

重要书籍

    •Natural Language Question Answering system Paperback - Boris Galitsky (2003)

    •New Directions in Question Answering - Mark T. Maybury (2004)

    •Part 3. 5. Question Answering in The Oxford Handbook of Computational Linguistics - Sanda Harabagiu and Dan Moldovan (2005)

    •Chap.28 Question Answering in Speech and Language Processing - Daniel Jurafsky & James H. Martin (2017)

重要链接

    •Building a Question-Answering System from Scratch— Part 1

    •Qeustion Answering with Tensorflow By Steven Hewitt, O'REILLY, 2017

    •Why question answering is hard