TEJAS Journal of Technologies and Humanitarian Science

A Comparative Study of debate speech analysis using Argumentation Mining in LLMs

Shweta Agarwal
Shri Ramswaroop Memorial University, Barabanki, India, 225003

Author

Megha Agarwal
Shri Ramswaroop Memorial University, Barabanki, India, 225003

Author

Shobhit Sinha
Shri Ramswaroop Memorial University, Barabanki, India, 225003

Author

📜 License:

This work is licensed under a Creative Commons Attribution 4.0 International License

• Share — Copy and Redistribute the material
• Adapt — Remix, Transform, and build upon the material
• The licensor cannot revoke these freedoms as long as you follow the license terms.

Abstract:

Argument mining (AM) is an automated course of mining arguments, their mechanisms, and relationships from textual data. As online debate platforms become increasingly prevalent, there is a growing demand for effective AM techniques, particularly to support subsequent analytical tasks. One specific area within AM is Relation-Based Argument Mining (RbAM), which aims to identify agreement (support) and disagreement (attack) relationships among arguments. However, RbAM presents significant challenges, and existing methods often fall short of achieving satisfactory performance in this domain. Our experiments focus on two open-source LLMs, namely Llama-2 and Mistral, which consist of ten files each. By harnessing the capabilities of these LLMs, we aim to address the limitations of current RbAM methods and pave the way for more effective and efficient argument mining techniques. Our findings underscore the potential of LLMs in advancing the state-of-the-art in RbAM and contribute to the broader goal of enhancing argument understanding and analysis in the era of online debates and discussions.

References

[1] A. A. Zubaer, M. Granitzer, and J. Mitrović, “Performance analysis of large language models in the domain of legal argument mining,” Front. Artif. Intell., vol. 6, 2023.

[2] A. Peldszus and M. Stede, “Joint prediction in MST-style discourse parsing for argumentation mining,” in Proc. EMNLP, Lisbon, Portugal, Sep. 2015, pp. 938–948.

[3] B. Konat et al., “A corpus of argument networks: Using graph properties to analyze divisive issues,” in Proc. LREC, Portorož, Slovenia, May 2016, pp. 3899–3906.

[4] T. Brown et al., “Language models are few-shot learners,” in Adv. Neural Inf. Process. Syst., vol. 33, pp. 1877–1901, 2020.

[5] C. Stab and I. Gurevych, “Parsing argumentation structures in persuasive essays,” Comput. Linguist., vol. 43, no. 3, pp. 619–659, Sep. 2017.

[6] C. Stab et al., “Cross-topic argument mining from heterogeneous sources,” in Proc. EMNLP, Brussels, Belgium, Oct.–Nov. 2018, pp. 3664–3674.

[7] D. A. Furman et al., “High-quality argumentative information in low-resource approaches improve counter-narrative generation,” in Findings ACL: EMNLP, Singapore, 2023.

[8] D. Trautmann et al., “Relational and fine-grained argument mining: The LMU Munich project ReMLAV,” Datenbank-Spektrum, vol. 20, no. 2, pp. 99–105, Jul. 2020.

[9] L. Ein-Dor et al., “Corpus-wide argument mining—A working solution,” in Proc. AAAI, vol. 34, no. 5, 2020.

[10] E. Cabrio and S. Villata, “Combining textual entailment and argumentation theory for supporting online debates interactions,” in Proc. ACL, Jeju Island, Korea, Jul. 2012, pp. 208–212.

[11] E. Cabrio and S. Villata, “NODE: A benchmark of natural language arguments,” in Proc. COMMA, 2014, pp. 449–450.

[12] E. Cabrio and S. Villata, “Five years of argument mining: A data-driven analysis,” in Proc. IJCAI, Stockholm, Sweden, Jul. 2018, pp. 5427–5433.

[13] E. Frantar et al., “GPTQ: Accurate post-training quantization for generative pre-trained transformers,” 2022.

[14] F. Boltužić and J. Šnajder, “Back up your stance: Recognizing arguments in online discussions,” in Proc. Workshop Argumentation Mining, Baltimore, MD, USA, Jun. 2014, pp. 49–58.

[15] G. Chen et al., “Exploring the potential of large language models in computational argumentation,” 2023.

[16] A. Q. Jiang et al., “Mixtral of experts,” arXiv:2401.04088, 2024.

[17] J. Lawrence and C. Reed, “Argument mining: A survey,” Comput. Linguist., vol. 45, no. 4, pp. 765–818, 2019.

[18] J. Park and C. Cardie, “A corpus of eRulemaking user comments for measuring evaluability of arguments,” in Proc. LREC, Miyazaki, Japan, May 2018.

[19] L. Carstens and F. Toni, “Towards relation-based argumentation mining,” in Proc. Workshop Argumentation Mining, Denver, CO, USA, Jun. 2015, pp. 29–34.