Linguistically motivated deep learning models for measuring semantic textual similarity

Abstract

Over the past few years, Natural Language Processing (NLP) has swiftly shifted from newlinestatistical feature-based methods to deep neural network-based models. These models newlinerely solely on input words to learn abstract representations of sentence semantics, newlinerendering linguistic features like Parts-of-Speech (POS) tags and parse trees no newlinelonger a necessity. This research shows how deep learning models can still benefit newlinefrom linguistic features by composing better sentence representations, particularly in newlinesemantic similarity-related tasks. newlineSemantic textual similarity refers to the degree of equivalence in the meaning of newlinetwo text snippets irrespective of their words and syntax. Its applications include but newlineare not limited to semantic relatedness scoring, paraphrase identification, recognizing newlinetextual entailment, question answering, machine translation evaluation, and automatic newlinetext summarization. Recurrent Neural Network (RNN) and its recursive variant, newlinenamely Tree-RNN, are the state-of-the-art models used in language processing. They newlinerepeatedly apply the same neural network on each word to compose sentence vectors newlineirrespective of the semantic role or syntactic functions of the words. We address newlinethis limitation of RNNs and Tree-RNNs by proposing three Deep Learning (DL) newlinemodels that use grammar-based non-uniform neural nets for semantic composition. newlineExperiments were conducted using two benchmark datasets Sentence Involving newlineCompositional Knowledge (SICK) and Stanford Sentiment Treebank (SST). newlineThe first contribution addresses the inability of Tree-RNN models in semanti- newlinecally differentiating sentences with identical parse trees. We show that grammatical newlinerelations, also known as typed dependencies, are essential to identify such differ- newlineences. We propose a dependency tree-based RNN model that can efficiently learn