Add new results of ACE and CL-KL (sebastianruder#564)

honghanhh · Aug 25, 2021 · d26ddb3 · d26ddb3
1 parent d133727
commit d26ddb3
Show file tree

Hide file tree

Showing 5 changed files with 8 additions and 5 deletions.
diff --git a/english/dependency_parsing.md b/english/dependency_parsing.md
@@ -29,8 +29,8 @@ are excluded from the evaluation. Evaluation metrics are unlabeled attachment sc
 | Model                                                                        |  POS  |  UAS  |  LAS  | Paper / Source                                                                                                                    | Code                                                                           |
 | ---------------------------------------------------------------------------- | :---: | :---: | :---: | --------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
 | Label Attention Layer + HPSG + XLNet (Mrini et al., 2019)                    | 97.3  | 97.42 | 96.26 | [Rethinking Self-Attention: Towards Interpretability for Neural Parsing](https://khalilmrini.github.io/Label_Attention_Layer.pdf) | [Official](https://github.com/KhalilMrini/LAL-Parser)                          |
+| ACE + fine-tune (Wang et al., 2020) | - | 97.20 | 95.80 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | HPSG Parser (Joint) + XLNet (Zhou et al, 2020)                            | 97.3  | 97.20 | 95.72 | [Head-Driven Phrase Structure Grammar Parsing on Penn Treebank](https://www.aclweb.org/anthology/2020.findings-emnlp.398.pdf)                        | [Official](https://github.com/DoodleJZ/HPSG-Neural-Parser)                     |
-| ACE + fine-tune (Wang et al., 2020) | - | 97.20 | 95.70 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | Second-Order MFVI + BERT (Wang et al., 2020) | - | 96.91 | 95.34 | [Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training](https://arxiv.org/pdf/2010.05003.pdf) | [Official](https://github.com/wangxinyu0922/Second_Order_Parsing)|
 | CVT + Multi-Task (Clark et al., 2018)                                        | 97.74 | 96.61 | 95.02 | [Semi-Supervised Sequence Modeling with Cross-View Training](https://arxiv.org/abs/1809.08370)                                    | [Official](https://github.com/tensorflow/models/tree/master/research/cvt_text) |
 | CRF Parser (Zhang et al., 2020)                                              |   -   | 96.14 | 94.49 | [Efficient Second-Order TreeCRF for Neural Dependency Parsing](https://www.aclweb.org/anthology/2020.acl-main.302)                | [Official](https://github.com/yzhangcs/crfpar)                                 |

diff --git a/english/named_entity_recognition.md b/english/named_entity_recognition.md
@@ -17,9 +17,11 @@ corpus tagged with four different entity types (PER, LOC, ORG, MISC). Models are
 
 | Model           | F1  |  Paper / Source | Code |
 | ------------- | :-----:| --- | --- |
+| ACE + document-context (Wang et al., 2021) | 94.6 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | LUKE (Yamada et al., 2020) | 94.3 | [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://www.aclweb.org/anthology/2020.emnlp-main.523/) | [Official](https://github.com/studio-ousia/luke) |
+| CL-KL (Wang et al., 2021) | 93.85 | [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://arxiv.org/abs/2105.03654) | [Official](https://github.com/Alibaba-NLP/CLNER)|
 | InferNER (Moemmur et al., 2021) | 93.76| [InferNER: an attentive model leveraging the sentence-level information for Named Entity Recognition in Microblogs](https://journals.flvc.org/FLAIRS/article/view/128538) | |
-| ACE + fine-tune (Wang et al., 2020) | 93.5 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
+| ACE (Wang et al., 2021) | 93.6 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | CNN Large + fine-tune (Baevski et al., 2019) | 93.5 | [Cloze-driven Pretraining of Self-attention Networks](https://arxiv.org/pdf/1903.07785.pdf) | |
 | RNN-CRF+Flair | 93.47 | [Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition](https://www.aclweb.org/anthology/D19-1367/) | |
 | CrossWeigh + Flair (Wang et al., 2019)♦ | 93.43 | [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://www.aclweb.org/anthology/D19-1519/) | [Official](https://github.com/ZihanWangKi/CrossWeigh) |
@@ -50,6 +52,7 @@ Links: [CoNLL++](https://github.com/ZihanWangKi/CrossWeigh) (including direct do
 
 | Model                                   |  F1   | Paper / Source                           | Code                                     |
 | --------------------------------------- | :---: | ---------------------------------------- | ---------------------------------------- |
+| CL-KL (Wang et al., 2021) | 94.81 | [Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning](https://arxiv.org/abs/2105.03654) | [Official](https://github.com/Alibaba-NLP/CLNER)|
 | CrossWeigh + Flair (Wang et al., 2019)♦ | 94.28 | [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://www.aclweb.org/anthology/D19-1519/) | [Official](https://github.com/ZihanWangKi/CrossWeigh) |
 | Flair embeddings (Akbik et al., 2018)♦ | 93.89 | [Contextual String Embeddings for Sequence Labeling](https://drive.google.com/file/d/17yVpFA7MmXaQFTe-HDpZuqw9fJlmzg56/view) | [Flair framework](https://github.com/zalandoresearch/flair)
 | BiLSTM-CRF+ELMo (Peters et al., 2018) | 93.42 | [Deep contextualized word representations](https://arxiv.org/abs/1802.05365) | [AllenNLP Project](https://allennlp.org/elmo) [AllenNLP GitHub](https://github.com/allenai/allennlp) |

diff --git a/english/semantic_parsing.md b/english/semantic_parsing.md
@@ -217,7 +217,7 @@ In each dataset, there is a in-domain (ID) and out-of-domain (OOD) test set. The
 
 | Model           | DM ID | DM OOD | PAS ID | PAS OOD | PSD ID | PSD OOD | Paper / Source | Code |
 | --------------- | :-----: |  :-----:|:-----: |  :-----:|:-----: |  :-----:| --------------- | ---- |
-| ACE + fine-tune (Wang et al., 2020) | 95.3 | 92.6 | 95.3 | 93.9 | 83.6 | 83.2| [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
+| ACE + fine-tune (Wang et al., 2020) | 95.6 | 92.6 | 95.8 | 94.6 | 83.8 | 83.4| [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 |Transition-based Pointer Network+Char+Lemma+BERT (Fernández-González & Gómez-Rodríguez, 2020)|94.4|91.0|95.1|93.4|82.6|82.0|[Transition-based Semantic Dependency Parsing with Pointer Networks](https://www.aclweb.org/anthology/2020.acl-main.629/)|[Official](https://github.com/danifg/SemanticPointer)|
 |Second-Order+Biaffine+Char+Lemma (Wang et al., 2019)|94.0|89.7|94.1|91.3|81.4|79.6|[Second-Order Semantic Dependency Parsing with End-to-End Neural Networks](https://www.aclweb.org/anthology/P19-1454/)|[Official](https://github.com/wangxinyu0922/Second_Order_SDP)|
 |Biaffine+Char+Lemma (Dozat and Manning, 2018)|93.7|88.9|93.9|90.6|81.0|79.4|[Simpler but More Accurate Semantic Dependency Parsing](https://www.aclweb.org/anthology/P18-2077/)|[Official](https://github.com/tdozat/Parser-v3)|

diff --git a/english/sentiment_analysis.md b/english/sentiment_analysis.md
@@ -152,7 +152,7 @@ Subtask 1 results (SemEval-2014 Task 4 for Laptop and SemEval-2016 Task 5 for Re
 
 | Model           | Laptop (F1) | Restaurant (F1) |  Paper / Source |  Code |
 | ------------- | :-----:| :-----:| --- | --- |
-| ACE + fine-tune (Wang et al., 2020) | 85.0 | 81.2 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
+| ACE + fine-tune (Wang et al., 2020) | 87.4 | 81.3 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | BERT-PT (Hu, Xu, et al., 2019) | 84.26 | 77.97 | [BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis](https://arxiv.org/pdf/1904.02232.pdf) | [official](https://github.com/howardhsu/BERT-for-RRC-ABSA)
 | DE-CNN (Hu, Xu, et al., 2018) | 81.59 | 74.37 | [Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction](https://www.aclweb.org/anthology/P18-2094) | [official](https://github.com/howardhsu/DE-CNN)
 | MIN (Li, Xin, et al., 2017) | 77.58 | 73.44 | [Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction] | 

diff --git a/english/shallow_syntax.md b/english/shallow_syntax.md
@@ -21,7 +21,7 @@ for testing. Models are evaluated based on F1.
 
 | Model           | F1 score  |  Paper | Source |
 | ------------- | :-----:| --- | --- |
-| ACE + fine-tune (Wang et al., 2020) | 96.90 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
+| ACE + fine-tune (Wang et al., 2020) | 97.30 | [Automated Concatenation of Embeddings for Structured Prediction](https://arxiv.org/pdf/2010.05006.pdf) | [Official](https://github.com/Alibaba-NLP/ACE)|
 | Flair embeddings (Akbik et al., 2018) | 96.72 | [Contextual String Embeddings for Sequence Labeling](http://aclweb.org/anthology/C18-1139) | [Flair](https://github.com/flairNLP/flair) |
 | JMT (Hashimoto et al., 2017) | 95.77 | [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://www.aclweb.org/anthology/D17-1206) |
 | Low supervision (Søgaard and Goldberg, 2016) | 95.57 | [Deep multi-task learning with low level tasks supervised at lower layers](http://anthology.aclweb.org/P16-2038) |