wordembeddings

Overview of the content of this repo:

• I used the GloVe pre-trained word vectors. Those are explored and converted to a pickle file for local usage in GloVe_pretrained_vectors.ipynb.
• A scikit-learn compatible Transformer to convert text features to numerical vectors based on the wordembeddings is included wordembeddings.py.
• Applied on data
  • Semantic Textual Similarity (STS) Tasks: sts_tasks.ipynb
    • In this notebook we try to replicate the experimental results on textual similarity tasks from A Simple but Tough-to-Beat Baseline for Sentence Embeddings (Arora et al. 2017, https://openreview.net/pdf?id=SyK00v5xx)
    • This just calculates the similarity between two sentences based on the distance between the sentence embeddings (without applying supervised model)
    • Some different methods are used to convert the set of word embeddings (from the sentence) to a single sentence embedding (average, weighted averaged, with SVD reduction)
  • SICK similarity task: sts_sick_supervized.ipynb
    • Similarity between two sentences, but now learn similarity with supervised model based on the sentence embeddings (instead of just the distance between the embeddings)
    • Implemented a Keras model similar to the one described in Wieting et al., 2016 (https://arxiv.org/pdf/1511.08198.pdf)
    • Compared with scikit-learn using MLPRegressor, but don't get as good results
  • SICK entailment task: sts_sick_entailment.ipynb
    • Classification problem: based on two sentences determine whether those are NEUTRAL, ENTAILMENT, or CONTRADICTION
    • Applied Keras model similar to the one described in Wieting et al., 2016, sklearn MLPClassifier using word embeddings and using TF-IDF.