Detecting Paraphrases in the Marathi Language

Abstract

Paraphrasing refers to writing that either differs in its textual content or is dissimilar in rearrangement of words but conveys the same meaning. Identifying a paraphrase is exceptionally important in various real life applications such as Information Retrieval, Plagiarism Detection, Text Summarization and Question Answering. A large amount of work in Paraphrase Detection has been done in English and many Indian Languages. However, there is no existing system to identify paraphrases in Marathi. This is the first such endeavor in the Marathi Language.

A paraphrase has differently structured sentences, and since Marathi is a semantically strong language, this system is designed for checking both statistical and semantic similarities of Marathi sentences. Statistical similarity measure does not need any prior knowledge as it is only based on the factual data of sentences. The factual data is calculated on the basis of the degree of closeness between the word-set, word-order, word-vector and word-distance. Universal Networking Language (UNL) speaks about the semantic significance in sentence without any syntactic points of interest. Hence, the semantic similarity calculated on the basis of generated UNL graphs for two Marathi sentences renders semantic equality of two Marathi sentences. The total paraphrase score was calculated after joining statistical and semantic similarity scores, which gives a judgement on whether there is paraphrase or non-paraphrase about the Marathi sentences in question.