add cedr dataset

datasets/cedr/README.md

@@ -0,0 +1,181 @@
+---
+annotations_creators:
+- crowdsourced
+language_creators:
+- found
+languages:
+- ru-RU
+licenses:
+- apache-2.0
+multilinguality:
+- monolingual
+pretty_name: The Corpus for Emotions Detecting in Russian-language text sentences
+size_categories:
+- unknown
+source_datasets:
+- original
+task_categories:
+- text-classification
+task_ids:
+- sentiment-classification
+- multi-label-classification
+- text-classification-other-emotion-classification
+---
+
+# Dataset Card for [cedr]
+
+## Table of Contents
+- [Table of Contents](#table-of-contents)
+- [Dataset Description](#dataset-description)
+  - [Dataset Summary](#dataset-summary)
+  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
+  - [Languages](#languages)
+- [Dataset Structure](#dataset-structure)
+  - [Data Instances](#data-instances)
+  - [Data Fields](#data-fields)
+  - [Data Splits](#data-splits)
+- [Dataset Creation](#dataset-creation)
+  - [Curation Rationale](#curation-rationale)
+  - [Source Data](#source-data)
+  - [Annotations](#annotations)
+  - [Personal and Sensitive Information](#personal-and-sensitive-information)
+- [Considerations for Using the Data](#considerations-for-using-the-data)
+  - [Social Impact of Dataset](#social-impact-of-dataset)
+  - [Discussion of Biases](#discussion-of-biases)
+  - [Other Known Limitations](#other-known-limitations)
+- [Additional Information](#additional-information)
+  - [Dataset Curators](#dataset-curators)
+  - [Licensing Information](#licensing-information)
+  - [Citation Information](#citation-information)
+  - [Contributions](#contributions)
+
+## Dataset Description
+
+- **Homepage:** https://github.com/sag111/CEDR
+- **Repository:** https://github.com/sag111/CEDR
+- **Paper:** https://www.sciencedirect.com/science/article/pii/S1877050921013247
+- **Leaderboard:**
+- **Point of Contact:** sag111@mail.ru
+
+### Dataset Summary
+
+The Corpus for Emotions Detecting in Russian-language text sentences of different social sources (CEDR) contains 9410  comments labeled for 5 emotion categories (joy, sadness, surprise, fear, and anger). 
+
+Here are 2 dataset configurations:
+- "main" - contains "text", "labels", and "source" features;
+- "enriched" - includes all "main" features and "sentences".
+
+Dataset with predefined train/test splits.
+
+### Supported Tasks and Leaderboards
+
+This dataset is intended for multi-label emotion classification.
+
+### Languages
+
+The data is in Russian.
+
+## Dataset Structure
+
+### Data Instances
+
+Each instance is a text sentence in Russian from several sources with one or more emotion annotations.
+
+### Data Fields
+
+The main configuration includes:
+- text: the text of the sentence;
+- labels: the emotion annotations;
+- source: the tag name of the corresponding source
+
+In addition to the above, the raw data includes:
+- sentences: text tokenized and lemmatized with [udpipe](https://ufal.mff.cuni.cz/udpipe)
+  - 'forma': the original word form;
+  - 'lemma': the lemma of this word
+
+### Data Splits
+
+The dataset includes a set of train/test splits. 
+with 7528, and 1882 examples respectively.
+
+## Dataset Creation
+
+### Curation Rationale
+
+The formed dataset of examples consists of sentences in Russian from several sources (blogs, microblogs, news), which allows creating methods to analyse various types of texts. The created methodology for building the dataset based on applying a crowdsourcing service can be used to expand the number of examples to improve the accuracy of supervised classifiers.
+
+### Source Data
+
+#### Initial Data Collection and Normalization
+
+Data was collected from several sources: posts of the Live Journal social network, texts of the online news agency Lenta.ru, and Twitter microblog posts.
+
+Only those sentences were selected that contained marker words from the dictionary of [the emotive vocabulary of the Russian language](http://lexrus.ru/default.aspx?p=2876). We manually formed a list of marker words for each emotion by choosing words from different categories of the dictionary.
+
+In total, 3098 sentences were selected from LiveJournal posts, 2847 sentences from Lenta.Ru, and 3465 sentencesfrom Twitter. After selection, sentences were offered to annotators for labeling.
+
+#### Who are the source language producers?
+
+Russian-speaking LiveJournal and Tweeter users, and authors of news articles on the site lenta.ru.
+
+### Annotations
+
+#### Annotation process
+
+Annotating sentences with labels of their emotions was performed with the help of [a crowdsourcing platform](https://yandex.ru/support/toloka/index.html?lang=en).
+
+The annotators’ task was: “What emotions did the author express in the sentence?”. The annotators were allowed to put an arbitrary number of the following emotion labels: "joy", "sadness", "anger", "fear", and "surprise".
+
+If the accuracy of an annotator on the control sentences (including the trial run) became less than 70%, or if the accuracy was less than 66% over the last six control samples, the annotator was dismissed. 
+
+Sentences were split into tasks and assigned to annotators so that each sentence was annotated at least three times. A label of a specific emotion was assigned to a sentence if put by more than half of the annotators.
+
+#### Who are the annotators?
+
+Only those of the 30% of the best-performing active users (by the platform’s internal rating) who spoke Russian and were over 18 years old were allowed into the annotation process. Moreover, before a platform user could be employed as an annotator, they underwent a training task, after which they were to mark 25 trial samples with more than 80% agreement compared to the annotation that we had performed ourselves.
+
+### Personal and Sensitive Information
+
+The text of the sentences may contain profanity.
+
+## Considerations for Using the Data
+
+### Social Impact of Dataset
+
+[More Information Needed]
+
+### Discussion of Biases
+
+[More Information Needed]
+
+### Other Known Limitations
+
+[More Information Needed]
+
+## Additional Information
+
+### Dataset Curators
+
+Researchers at AI technology lab at NRC "Kurchatov Institute". See the author [list](https://www.sciencedirect.com/science/article/pii/S1877050921013247).
+
+### Licensing Information
+
+The GitHub repository which houses this dataset has an Apache License 2.0.
+
+### Citation Information
+If you have found our results helpful in your work, feel free to cite our publication. This is an updated version of the dataset, the collection and preparation of which is described here:
+```
+@article{sboev2021data,
+  title={Data-Driven Model for Emotion Detection in Russian Texts},
+  author={Sboev, Alexander and Naumov, Aleksandr and Rybka, Roman},
+  journal={Procedia Computer Science},
+  volume={190},
+  pages={637--642},
+  year={2021},
+  publisher={Elsevier}
+}
+```
+
+### Contributions
+
+Thanks to [@naumov-al](https://github.com/naumov-al) for adding this dataset.

datasets/cedr/cedr.py

@@ -0,0 +1,174 @@
+# coding=utf-8
+# Copyright 2020 The HuggingFace Datasets Authors and the current dataset script contributor.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Lint as: python3
+"""CEDR dataset"""
+
+import json
+import os
+
+import datasets
+
+
+# TODO: Add BibTeX citation
+# Find for instance the citation on arxiv or on the dataset repo/website
+_CITATION = """\
+@article{sboev2021data,
+  title={Data-Driven Model for Emotion Detection in Russian Texts},
+  author={Sboev, Alexander and Naumov, Aleksandr and Rybka, Roman},
+  journal={Procedia Computer Science},
+  volume={190},
+  pages={637--642},
+  year={2021},
+  publisher={Elsevier}
+}
+"""
+
+_LICENSE = """http://www.apache.org/licenses/LICENSE-2.0"""
+
+# TODO: Add description of the dataset here
+# You can copy an official description
+_DESCRIPTION = """\
+This new dataset is designed to solve emotion recognition task for text data in Russian. The Corpus for Emotions Detecting in
+Russian-language text sentences of different social sources (CEDR) contains 9410 sentences in Russian labeled for 5 emotion
+categories. The data collected from different sources: posts of the LiveJournal social network, texts of the online news
+agency Lenta.ru, and Twitter microblog posts. There are two variants of the corpus: main and enriched. The enriched variant
+is include tokenization and lemmatization. Dataset with predefined train/test splits.
+"""
+
+# TODO: Add a link to an official homepage for the dataset here
+_HOMEPAGE = "https://github.com/sag111/CEDR"
+
+# TODO: Add link to the official dataset URLs here
+# The HuggingFace dataset library don't host the datasets but only point to the original files
+# This can be an arbitrary nested dict/list of URLs (see below in `_split_generators` method)
+_URLs = {
+    "main": "https://drive.google.com/u/0/uc?id=1flukWj075o7InAlBZm7htXEqZM6D-LfA&export=download",
+    "enriched": "https://drive.google.com/u/0/uc?id=1w9fJf9SwxtEABy5uhougn_2ZlxKBbZHV&export=download",
+}
+
+
+# TODO: Name of the dataset usually match the script name with CamelCase instead of snake_case
+class cedr(datasets.GeneratorBasedBuilder):
+    """This dataset is designed to solve emotion recognition task for text data in Russian."""
+
+    VERSION = datasets.Version("0.1.1")
+
+    # This is an example of a dataset with multiple configurations.
+    # If you don't want/need to define several sub-sets in your dataset,
+    # just remove the BUILDER_CONFIG_CLASS and the BUILDER_CONFIGS attributes.
+
+    # If you need to make complex sub-parts in the datasets with configurable options
+    # You can create your own builder configuration class to store attribute, inheriting from datasets.BuilderConfig
+    # BUILDER_CONFIG_CLASS = MyBuilderConfig
+
+    # You will be able to load one or the other configurations in the following list with
+    # data = datasets.load_dataset('my_dataset', 'first_domain')
+    # data = datasets.load_dataset('my_dataset', 'second_domain')
+    BUILDER_CONFIGS = [
+        datasets.BuilderConfig(
+            name="main", version=VERSION, description="This part of CEDR dataset covers a main version"
+        ),
+        datasets.BuilderConfig(
+            name="enriched", version=VERSION, description="This part of CEDR dataset covers a enriched version"
+        ),
+    ]
+
+    DEFAULT_CONFIG_NAME = "main"  # It's not mandatory to have a default configuration. Just use one if it make sense.
+
+    def _info(self):
+        # TODO: This method specifies the datasets.DatasetInfo object which contains informations and typings for the dataset
+        if self.config.name == "main":  # This is the name of the configuration selected in BUILDER_CONFIGS above
+            features = datasets.Features(
+                {
+                    "text": datasets.Value("string"),
+                    "labels": datasets.features.Sequence(datasets.Value("string")),
+                    "source": datasets.Value("string"),
+                    # These are the features of your dataset like images, labels ...
+                }
+            )
+        else:  # This is an example to show how to have different features for "first_domain" and "second_domain"
+            features = datasets.Features(
+                {
+                    "text": datasets.Value("string"),
+                    "labels": datasets.features.Sequence(datasets.Value("string")),
+                    "source": datasets.Value("string"),
+                    "sentences": [[{"forma": datasets.Value("string"), "lemma": datasets.Value("string"),}]]
+                    # These are the features of your dataset like images, labels ...
+                }
+            )
+        return datasets.DatasetInfo(
+            # This is the description that will appear on the datasets page.
+            description=_DESCRIPTION,
+            # This defines the different columns of the dataset and their types
+            features=features,  # Here we define them above because they are different between the two configurations
+            # If there's a common (input, target) tuple from the features,
+            # specify them here. They'll be used if as_supervised=True in
+            # builder.as_dataset.
+            supervised_keys=None,
+            # Homepage of the dataset for documentation
+            homepage=_HOMEPAGE,
+            # License for the dataset if available
+            license=_LICENSE,
+            # Citation for the dataset
+            citation=_CITATION,
+        )
+
+    def _split_generators(self, dl_manager):
+        """Returns SplitGenerators."""
+        # TODO: This method is tasked with downloading/extracting the data and defining the splits depending on the configuration
+        # If several configurations are possible (listed in BUILDER_CONFIGS), the configuration selected by the user is in self.config.name
+
+        # dl_manager is a datasets.download.DownloadManager that can be used to download and extract URLs
+        # It can accept any type or nested list/dict and will give back the same structure with the url replaced with path to local files.
+        # By default the archives will be extracted and a path to a cached folder where they are extracted is returned instead of the archive
+        my_urls = _URLs[self.config.name]
+        data_dir = dl_manager.download_and_extract(my_urls)
+        return [
+            datasets.SplitGenerator(
+                name=datasets.Split.TRAIN,
+                # These kwargs will be passed to _generate_examples
+                gen_kwargs={"filepath": os.path.join(data_dir, self.config.name, "train.jsonl"), "split": "train",},
+            ),
+            datasets.SplitGenerator(
+                name=datasets.Split.TEST,
+                # These kwargs will be passed to _generate_examples
+                gen_kwargs={"filepath": os.path.join(data_dir, self.config.name, "test.jsonl"), "split": "test"},
+            ),
+        ]
+
+    def _generate_examples(
+        self, filepath, split  # method parameters are unpacked from `gen_kwargs` as given in `_split_generators`
+    ):
+        """Yields examples as (key, example) tuples."""
+        # This method handles input defined in _split_generators to yield (key, example) tuples from the dataset.
+        # The `key` is here for legacy reason (tfds) and is not important in itself.
+
+        with open(filepath, encoding="utf-8") as f:
+            for id_, row in enumerate(f):
+                data = json.loads(row)
+                if self.config.name == "main":
+                    yield id_, {
+                        "text": data["text"],
+                        "source": data["source"],
+                        "labels": data["labels"],
+                    }
+                else:
+                    yield id_, {
+                        "text": data["text"],
+                        "source": data["source"],
+                        "sentences": data["sentences"],
+                        "labels": data["labels"],
+                    }

datasets/cedr/dataset_infos.json

@@ -0,0 +1 @@
+{"main": {"description": "This new dataset is designed to solve emotion recognition task for text data in Russian. The Corpus for Emotions Detecting in\nRussian-language text sentences of different social sources (CEDR) contains 9410 sentences in Russian labeled for 5 emotion\ncategories. The data collected from different sources: posts of the LiveJournal social network, texts of the online news\nagency Lenta.ru, and Twitter microblog posts. There are two variants of the corpus: main and enriched. The enriched variant\nis include tokenization and lemmatization. Dataset with predefined train/test splits.\n", "citation": "@article{sboev2021data,\n  title={Data-Driven Model for Emotion Detection in Russian Texts},\n  author={Sboev, Alexander and Naumov, Aleksandr and Rybka, Roman},\n  journal={Procedia Computer Science},\n  volume={190},\n  pages={637--642},\n  year={2021},\n  publisher={Elsevier}\n}\n", "homepage": "https://github.com/sag111/CEDR", "license": "http://www.apache.org/licenses/LICENSE-2.0", "features": {"text": {"dtype": "string", "id": null, "_type": "Value"}, "labels": {"feature": {"dtype": "string", "id": null, "_type": "Value"}, "length": -1, "id": null, "_type": "Sequence"}, "source": {"dtype": "string", "id": null, "_type": "Value"}}, "post_processed": null, "supervised_keys": null, "task_templates": null, "builder_name": "cedr", "config_name": "main", "version": {"version_str": "0.1.1", "description": null, "major": 0, "minor": 1, "patch": 1}, "splits": {"train": {"name": "train", "num_bytes": 1423880, "num_examples": 7528, "dataset_name": "cedr"}, "test": {"name": "test", "num_bytes": 351868, "num_examples": 1882, "dataset_name": "cedr"}}, "download_checksums": {"https://drive.google.com/u/0/uc?id=1flukWj075o7InAlBZm7htXEqZM6D-LfA&export=download": {"num_bytes": 693026, "checksum": "d81e6d19679a903773b8776c4c0f68755d55596e6b34866fbaa9d39d2e385bd3"}}, "download_size": 693026, "post_processing_size": null, "dataset_size": 1775748, "size_in_bytes": 2468774}, "enriched": {"description": "This new dataset is designed to solve emotion recognition task for text data in Russian. The Corpus for Emotions Detecting in\nRussian-language text sentences of different social sources (CEDR) contains 9410 sentences in Russian labeled for 5 emotion\ncategories. The data collected from different sources: posts of the LiveJournal social network, texts of the online news\nagency Lenta.ru, and Twitter microblog posts. There are two variants of the corpus: main and enriched. The enriched variant\nis include tokenization and lemmatization. Dataset with predefined train/test splits.\n", "citation": "@article{sboev2021data,\n  title={Data-Driven Model for Emotion Detection in Russian Texts},\n  author={Sboev, Alexander and Naumov, Aleksandr and Rybka, Roman},\n  journal={Procedia Computer Science},\n  volume={190},\n  pages={637--642},\n  year={2021},\n  publisher={Elsevier}\n}\n", "homepage": "https://github.com/sag111/CEDR", "license": "http://www.apache.org/licenses/LICENSE-2.0", "features": {"text": {"dtype": "string", "id": null, "_type": "Value"}, "labels": {"feature": {"dtype": "string", "id": null, "_type": "Value"}, "length": -1, "id": null, "_type": "Sequence"}, "source": {"dtype": "string", "id": null, "_type": "Value"}, "sentences": [[{"forma": {"dtype": "string", "id": null, "_type": "Value"}, "lemma": {"dtype": "string", "id": null, "_type": "Value"}}]]}, "post_processed": null, "supervised_keys": null, "task_templates": null, "builder_name": "cedr", "config_name": "enriched", "version": {"version_str": "0.1.1", "description": null, "major": 0, "minor": 1, "patch": 1}, "splits": {"train": {"name": "train", "num_bytes": 4797891, "num_examples": 7528, "dataset_name": "cedr"}, "test": {"name": "test", "num_bytes": 1183936, "num_examples": 1882, "dataset_name": "cedr"}}, "download_checksums": {"https://drive.google.com/u/0/uc?id=1w9fJf9SwxtEABy5uhougn_2ZlxKBbZHV&export=download": {"num_bytes": 1822522, "checksum": "3b0ee43108ca6a52ce21037d35c99538a4a80e9dba5bd3d02b3ff17d4d89b2b7"}}, "download_size": 1822522, "post_processing_size": null, "dataset_size": 5981827, "size_in_bytes": 7804349}}