Skip to content

Commit

Permalink
minor updates
Browse files Browse the repository at this point in the history
  • Loading branch information
Sewon Min committed Jun 15, 2021
1 parent 79b461a commit f268dac
Show file tree
Hide file tree
Showing 4 changed files with 55 additions and 37 deletions.
2 changes: 1 addition & 1 deletion explorer.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<div class="collapse navbar-collapse" id="menu-list">
<button class="title">
<a href="https://faviq.github.io/">
FaVIQ: Fact Verification from Information seeking Questions
FaVIQ: Fact Verification from Information-seeking Questions
</a>
</button>
<button><a class="active" href="https://faviq.github.io/explorer.html">Explore</a></button>
Expand Down
2 changes: 1 addition & 1 deletion index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<div class="" id="menu-list">
<button class="title">
<a href="https://faviq.github.io/">
FaVIQ: FAct Verification from Information seeking Questions
FaVIQ: FAct Verification from Information-seeking Questions
</a>
</button>
<button><a href="https://faviq.github.io/explorer.html">Explore</a></button>
Expand Down
6 changes: 5 additions & 1 deletion static/explorer_index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,11 @@
<p><span class="label label-text">Text</span> ` + data['positive_evidence']['text'] + `</p>
`;
$('#content').append(getPanel("Positive Passage", htmlText_pos));
$('#content').append('<h4>&#8251 Note that the positive passage is not golden, but is the one retrieved from TF-IDF (the top passage that contains the answer to the original question).</h4>');
$('#content').append(`<h5>
&#8251 Note that the positive passage is not groundtruth,
but is obtained through TF-IDF and post-processing (details in Appendix C.2 of the paper).
'N/A' indicates TF-IDF fails to retrieve any reasonable passage.
</h5>`);

// htmlText_neg = `
// <p><span class="label label-id">ID</span> ` + data['negative_evidence']['id'] + `</p>
Expand Down
82 changes: 48 additions & 34 deletions static/index.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,14 @@
"nq": "https://www.mitpressjournals.org/doi/pdf/10.1162/tacl_a_00276",
"ambigQA": "https://arxiv.org/pdf/2004.10645.pdf",
"ours": "",
"ours-bibtex": "",
"ours-bibtex": "https://raw.githubusercontent.com/faviq/faviq/main/park2021faviq.txt",
"dpr": "https://arxiv.org/pdf/2004.04906.pdf",
"drqa_github": "https://github.com/facebookresearch/DrQA/tree/master/scripts/retriever",
"explore":"https://faviq.github.io/explorer.html",
"kilt":"https://arxiv.org/pdf/2009.02252.pdf"
"kilt":"https://arxiv.org/pdf/2009.02252.pdf",
"readme":"https://github.com/faviq#faviq",
"baselines":"https://github.com/faviq/faviq/tree/main/codes",
"request-test-data": "https://github.com/faviq#request-test-data"
};

$( window ).init(function(){
Expand All @@ -23,13 +26,20 @@
<h3>About</h3>
<br>
<p>
<b>Fa</b>ct <b>V</b>erification from <b>I</b>nformation seeking <b>Q</b>uestions (<b>FaVIQ</b>) is a challenging and realistic fact verification dataset,
consisting of claims transformed from information-seeking questions by a pre-trained language model.
Specifically, we use the ambiguity in information-seeking questions and their disambiguation and automatically convert them to true and false claims.
These claims are natural and verifying these claims requires a complete understanding of the evidence.
We introduce <b>FaVIQ</b>
(<b>Fa</b>ct <b>V</b>erification from <b>I</b>nformation-seeking <b>Q</b>uestions),
a challenging and realistic fact verification dataset that reflects confusions raised by real users.
We use the ambiguity in information-seeking questions and their disambiguation, and
automatically convert them to true and false claims.
These claims are natural, and require a complete understanding of the evidence for verification.
FaVIQ serves as a challenging benchmark for natural language understanding, and
improves performance in professional fact checking.
</p>
<p>
Details can be found in our paper:
</p>
<blockquote>
Jungsoo Park, Sewon Min, Jaewoo Kang, Luke Zettlemoyer, Hannaneh Hajishirzi.
Jungsoo Park*, Sewon Min*, Jaewoo Kang, Luke Zettlemoyer, Hannaneh Hajishirzi.
<a href="` + references["ours"] + `" target="_blank">"FaVIQ: FAct Verification from Information seeking Questions"</a>.
[<a href="` + references["ours-bibtex"] + `" target="_blank">BibTeX</a>]
</blockquote>
Expand All @@ -44,53 +54,57 @@
<h3>Data</h3>
<br>
<p>
The data consists of <b>A set</b> and <b>R set</b>, where the former has 26k claims converted from ambiguous questions in <a href="` + references["ambigQA"] + `" target="_blank">AmbigQA</a> and
the latter has 188k claims converted from <a href="` + references["nq"] + `" target="_blank">NQ</a>.
A single instance (in json format) consists of claim, label, and evidence texts (both in positive and negative) used to train
<a href="` + references["dpr"] + `" target="_blank">DPR</a> retriever module.
Note that the evidence texts can be not available for some samples.
We are hiding the test set from A set, since the test set of <a href="` + references["ambigQA"] + `" target="_blank">AmbigQA</a> is hidden.
Please contact us if you need the test set from A set.
Visit <a href="` + references["explorer"] + `" target="_blank">Explore</a> to see some samples!
The data consists of <b>A set</b> and <b>R set</b>.
<b>A set</b> is our main dataset, consisting of 26k claims converted from ambiguous questions and their disambiguations.
<b>R set</b> is an additional dataset, consisting of 188k claims converted from regular question-answer pairs.
Please refer to <a href="` + references["readme"] + `" target="_blank">README</a> for the detailed data format.
Visit <a href="` + references["explorer"] + `" target="_blank">Explorer</a> to see some samples!
</p>
<p>
<em>&#8251 Test data of the A set is hidden. In order to get the test data,
please follow the instructions in <a href="` + references["request-test-data"] + `" target="_blank">README</a>.</em>
</p>
<br>
`);
$('#intro-content').append("<div class='readme row' style='margin-top: 10px;'>" + card1 + card2 + "</div>");
$('#intro-content').append("<div class='readme row' style='margin-top: 10px;'>" + card1 + card2 + "</div>");
$('#intro-content').append('<hr>');
$('#intro-content').append(`
<h3>Resources</h3>
<br>
<p>
For solving the fact verification task, an external knowledge source is required.
We use the English Wikipedia from 08/01/2019 following <a href="` + references["kilt"] + `" target="_blank">KILT</a> where we modified the original version
for creating a collection of passages each having approximately 100 tokens.
Note that for experiments in our paper, we concatenated the title with the passage.
We release baseline implementations and checkpoints <a href="` + references["baselines"] + `" blank="_blank">here</a>.
We use the English Wikipedia from 08/01/2019 following <a href="` + references["kilt"] + `" target="_blank">KILT</a>
for primary knowledge sources with minor modification.
Download it here!
Please refer to the paper for the details of the baselines and how we use Wikipedia as knowledge sources.
</p>
<br>
`);
$('#intro-content').append("<div class='readme row' style='width: 50%; margin: 0 auto;'>" + card3 + "</div>");
$('#intro-content').append("<div class='readme row' style='width: 50%; margin: 0 auto;'>" + card3 + "</div>");
$('#intro-content').append('<hr>');
$('.panel').width($('#intro-content').width()/3-30);
$('.panel').css("margin-right", "10px");
$('#intro-content').append(`
<h3>References</h3>
<br>
<ul><li style="font-size:18px">Natural Questions: a Benchmark for Question Answering` + loadCitation("nq", "Kwiatkowski et al. (TACL 2019)") + ` </li>
<li style="font-size:18px">AmbigQA: Answering Ambiguous Open-domain Questions` + loadCitation("ambigQA", "Min et al. (EMNLP 2020)") + ` </li>
<li style="font-size:18px">KILT: a Benchmark for Knowledge Intensive Language Tasks` + loadCitation("kilt", "Petroni et al. (NAACL 2021)") + ` </li>
</ul>
<p>
<ul>
<li>For information-seeking questions and regular question-answer pairs, we use ` +
loadCitation("nq", "Natural Questions: a Benchmark for Question Answering (Kwiatkowski et al. TACL 2019)") + ` </li>
<li>For ambiguity/disambiguations of information-seeking questions, we use ` +
loadCitation("ambigQA", "AmbigQA: Answering Ambiguous Open-domain Questions (Min et al. EMNLP 2020)") + ` </li>
<li>For the knowledge source (Wikipedia), we use ` +
loadCitation("kilt", "KILT: a Benchmark for Knowledge Intensive Language Tasks (Petroni et al. NAACL 2021)") + ` </li>
</ul>
</p>
`);
$('#intro-content').append('<hr>');

// load references
$('#intro-content').append(`
<h3>Contact</h3>
<br>
<p style="font-size:18px">
For any questions about the code or data, please contact Jungsoo Park(<a class="icons-sm email-ic" href="mailto:[email protected]" target="_blank"><i class="fa fa-envelope-o"></i> Email</a>)
or Sewon Min
(<a class="icons-sm email-ic" href="mailto:[email protected]" target="_blank"><i class="fa fa-envelope-o"></i> Email</a>)
<p>
For any questions about the code or data, please contact
Jungsoo Park (<a class="icons-sm email-ic" href="mailto:[email protected]" target="_blank"><i class="fa fa-envelope-o"></i> Email</a>)
or Sewon Min (<a class="icons-sm email-ic" href="mailto:[email protected]" target="_blank"><i class="fa fa-envelope-o"></i> Email</a>)
or leave <a href="` + github + `/issues"><i class="fa fa-github"></i> issues</a>.
</p>
`);
Expand Down Expand Up @@ -127,7 +141,7 @@
}

function loadCitation(citation, keyword) {
var text = `, <a target="_blank" href="` + references[citation] + `">` + keyword + `</a>`;
var text = `<a target="_blank" href="` + references[citation] + `">` + keyword + `</a>`;
if (citation==="ours") {
text += ` [<a target="_blank" href="` + references[citation+"-bibtex"] + `">BibTeX</a>]`
}
Expand Down

0 comments on commit f268dac

Please sign in to comment.