This page provides details on the annotations used in the paper Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection.
The paper analyses the annotations from the SemEval 2024 Task 4. The files below include the annotations of each annotator as well as the gold labels for two group of annotators employed in the campaign:
If you use the data you can cite the paper (bibtex available here) as:
“Davide Bassi, Dimitar Iliyanov Dimitrov, Bernardo D’Auria, Firoj Alam, Maram Hasanain, Christian Moro, Luisa Orrù, Gian Piero Turchi, Preslav Nakov, and Giovanni Da San Martino. 2025. Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection. In Findings of the Association for Computational Linguistics: ACL 2025, pages 17918–17929, Vienna, Austria. Association for Computational Linguistics.
If you are interested in the task and get access to a larger set of examples and their gold labels you can:
Register to the Task 4 at SemEval 2024 at: https://propaganda.math.unipd.it/semeval2024task4/
Once registered you will get access to train and dev data of the shared task which corresponds to the dataset described in the paper.
You can upload your submissions for the test set and check the state of the art on the corresponding leaderboard