[see also the research page for related information]
Various software modules and data sets that are/were used in my research. They are made available under the terms of GNU General Public License. Both data and software are distributed without any warranty.
For any questions regarding the content of this page, please contact Rada Mihalcea, rada at cs.unt.edu
Efficient Indexer for the Google Web 1T Ngram corpus
This is an efficient indexer for the Google Web 1T Ngram corpus, along with a client-server model for fast querying. The software also accepts queries with wildcards. download (July 15, 2012).
Wikipedia Interlingual Links Evaluation Dataset
This resource contains manual annotations for 195 pairs of articles in Wikipedia, covering four language pairs. download. The metafile, containing all the candidate interlingual links for ten language pairs can also be downloaded (July 15, 2012).
Sentiment Lexicons in Spanish
This resource contains two polarity lexicons in Spanish. The lexicons have been automatically or semi-automatically generated. [download] (April 3, 2012).
Measuring the Semantic Relatedness between Words and ImagesThis dataset contains the list of synsets from ImageNet and related data used in an experiment to compute semantic relatedness between words and images. download (January 12, 2011).
Text Mining for Automatic Image Tagging
This dataset contains images, texts and gold-standard annotations of 300 image-text pairs randomly collected over the web. download (August 23, 2010).
Learning to Identify Educational Material (LIEM)
The data set is a collection of 862 documents annotated for its educative-ness value along with other user selected features. download (August 11, 2009).
Cross-Lingual Semantic Relatedness (CLSR)
A validated translation of the original Miller-Charles (Miller and Charles, 1998) and WordSimilarity-353 (Finkelstein et al., 2001) in Spanish, Romanian, and Arabic. download (August 10, 2009)
Data for Automatic Short Answer Grading
A collection of short student answers and grades for a course in Computer Science. The data set consists of 21 questions with 30 student answers each. [download] (February 10, 2009)
- Veronica Perez Rosas, Carmen Banea, Rada Mihalcea, Learning Sentiment Lexicons in Spanish, in Proceedings of the International Conference on Language Resources and Evaluations (LREC 2012), Istanbul, Turkey, May 2012.
Multilingual Subjectivity Analysis: Gold Standard and Training Data
Michael Mohler and Rada Mihalcea, Text-to-text Semantic Similarity for Automatic Short Answer Grading, in Proceedings of the European Chapter of the Association for Computational Linguistics (EACL 2009), Athens, Greece, March 2009. pdf]
- Gold standard data for multilingual subjectivity analysis. The data set consists of 500 sentences in English, Romanian and Spanish, manually annotated for subjectivity. [download] (October 25, 2008)
Rada Mihalcea, Carmen Banea and Jan Wiebe, Learning Multilingual Subjective Language via Cross-Lingual Projections, In Proceedings of the Association for Computational Linguistics (ACL 2007), Prague, June 2007. [pdf]
- Multilingual training data, automatically annotated for subjectivity, in English, Romanian, and Spanish. [download] (October 25, 2008)
- Carmen Banea, Rada Mihalcea, Janyce Wiebe and Samer Hassan, Multilingual subjectivity analysis using machine translation, In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2008), Honolulu, Hawaii, October 2008. [pdf]
- Multilingual training data, automatically annotated for subjectivity, in English, Arabic, French, German, Romanian, and Spanish. [download] (August 20, 2010)
Carmen Banea, Rada Mihalcea and Janyce Wiebe, Multilingual subjectivity: are more languages better, In Proceedings of the International Conference on Computational Linguistics (COLING 2010), Beijing, China, August 2010. [pdf]
GWSD: Unsupervised Graph-based Word Sense Disambiguatio
- GWSD is a system for unsupervised all-words graph-based word sense disambiguation download GWSD 1.0 (September 13, 2007).
- Ravi Sinha and Rada Mihalcea, Unsupervised Graph-based Word Sense Disambiguation Using Measures of Word Semantic Similarity, In Proceedings of the IEEE International Conference on Semantic Computing (ICSC 2007), Irvine, CA, September 2007. [pdf]
- Rada Mihalcea, Unsupervised Large-Vocabulary Word Sense Disambiguation with Graph-based Algorithms for Sequence Data Labeling, In Proceedings of the Joint Conference on Human Language Technology / Empirical Methods in Natural Language Processing (HLT/EMNLP), Vancouver, October, 2005. [pdf]
Affective Text: Data Annotated for Emotions and Polarity
- Affective Text is a data set consisting of 1000 test headlines and 200 development headlines, each of them annotated with the six Eckman emotions and the polarity orientation. [download] (July 13, 2007).
- Carlo Strapparava and Rada Mihalcea, SemEval-2007 Task 14: Affective Text, in Proceedings of the 4th International Workshop on the Semantic Evaluations (SemEval 2007), Prague, Czech Republic, June 2007. [pdf]
- Read more about the task here.
SenseLearner: All-Words Word Sense Disambiguation Tool
- SenseLearner 2.0 [download] (June 13, 2005).
- Changes in version 2.0: a client-server model that allows for significantly faster tagging; simpler input file format (the SemCor-like format is not anymore required)
- SenseLearner 1.0 (beta) [download] (Nov 18, 2004)
Benchmark for the evaluation of back-of-the-book indexing systems
- A benchmark for the evaluation of systems for back-of-the-book indexing [download].
The benchmark is described in:
Andras Csomai and Rada Mihalcea, Creating a Testbed for the Evaluation of Automatically Generated Back-of-the-
book Indexes, in Proceedings of the Conference on Computational Linguistics and Intelligent Text Processing (CICLing), LNCS, Mex
ico City, February 2006. [pdf]
FrameNet - WordNet verb sense mapping
- FnWnVerbMap 1.0 [download]
A mapping between verb lexical units in FrameNet II and verb senses in WordNet. The mapping process is described in:
Lei Shi and Rada Mihalcea, Putting Pieces Together: Combining FrameNet, VerbNet and WordNet for Robust Semantic Parsing, Cicling 2005, Mexico [pdf]
Resources and Tools for Romanian NLP
- Romanian corpus of newspaper articles (and two novels), 50 mil. words. [research purpose only - send a request to rada at cs unt edu]
- Romanian sense tagged data, 39 ambiguous words [download]
- Romanian-English parallel texts, sentence-aligned, 1 mil. words (each side) [download; research purpose only - send a request to rada at cs unt edu]
- Romanian-English word aligned data (2003) [download]
- Romanian-English word aligned data (2005) [download]
- Romanian-English dictionary (38,000 entries) [download]
- For other resources and tools for Romanian, see the ConsiLR webpage.
Open Mind Word Expert Sense Tagged Data
- OMWE 1.0: Sense tagged data for 288 nouns, created within the Open Mind Word Expert framework during one year of activity (2002) [download]
- OMWE 2.0: Sense tagged data for nouns, verbs, adjectives, created within the Open Mind Word Expert framework. These data sets were used during the Senseval-3 evaluations.
- Romanian OMWE: Data for 39 ambiguous words in Romanian [download]
- English OMWE: Data for 57 ambiguous words, annotated with WordNet/Wordsmyth senses [download]
- English-Hindi OMWE: Data for 41 English words annotated with their corresponding Hindi translation [download]
TWA Sense Tagged Data
- Sense tagged data for six words with two-way ambiguities (bass, crane,
motion, palm, plant, tank). [download]
Resources for Word Alignment
All these available from the webpage of the HLT/NAACL 2003 workshop on Building and Using Parallel Texts: Data Driven Machine Translation and Beyond.
- Word aligned data for Romanian-English, English-French.
- Parallel texts for training.
- Code for word alignment evaluation.
Texts semantically annotated with WordNet 1.6 senses (created at Princeton
University), and automatically mapped to WordNet 1.7, WordNet 1.7.1, WordNet 2.0, WordNet 2.1, WordNet 3.0
- SemCor 1.6 [download]
- SemCor 1.7 [download]
- SemCor 1.7.1 [download]
- SemCor 2.0 [download]
- SemCor 2.1 [download]
- SemCor 3.0 [download]
A mapping between synsets offsets in various WordNet versions.
- WordNet 1.6 - 1.7 [download]
- WordNet 1.6 - 1.7.1 [download]
- WordNet 1.7 - 1.7.1 [download]
- WordNet 1.6 - 2.0 [download]
- WordNet 1.7.1 - 2.0 [download]
Senseval-2 and Senseval-3 English all-words data converted into SemCor format
- Evaluation software for text filtering systems, implements the normalized utility, F-measure, precision, and recall, as defined in the TREC 2002 Filtering task. Straightforward usage, follows closely the TREC 2002 Filtering guidelines. [download].
- More soon...
QA Data Set: Annotated questions
Annotations for about 5,500 questions used in an analysis of information requests. Questions are drawn from the Excite log, respectively the TREC QA benchmark. This is the data set used in the experiments reported in:
- Rada Mihalcea, The Semantic Wildcard, in Proceedings of the LREC 2002 Workshop on "Using Semantics for Information Retrieval and Filtering: State of the Art and Future Research", Las Palmas, Spain, May 2002.