Data Augmentation with Large Language Models for Aspect-Based Sentiment Analysis in Low-Resource Scenarios

Sentiment analysis (SA) is a research area in natural language processing (NLP) which involves the computational classification of individuals' sentiments, opinions and emotions. This mostly involves categorizing sentiments into three polarities: positive, neutral and negative.

Social media services like X (now rebranded as Twitter) and Facebook have gained popularity as sources of data for sentiment analysis due to the low barrier for posting messages, enabling a multitude of users around the world to express their opinions on various topics daily. Accordingly, content from social media is used for sentiment analysis in many fields, for example in the political context, financial sector or healthcare.

Sentiment analysis can be applied at both document- and sentence-level. However, if a document or sentence comprises a mixture of different sentiments, it might not be possible to assign a positive, negative or neutral label. As an illustrative example, consider a sentence of a restaurant review wherein positive sentiment is expressed towards the food while, concurrently, negative sentiment is expressed when addressing the food's price. To address this issue, Aspect-Based Sentiment Analysis (ABSA) has been extensively studied as it goes beyond assessing general sentiment and instead delves into a more granular examination of sentiment by linking particular aspects or attributes with corresponding sentiment polarities.

This work aims to apply LLMs for generating annotated examples for ABSA for the mitigation of data scarcity. A human-annotated dataset of 3,000 sentences from restaurant reviews posted on TripAdvisor in the German language (low-resource language in ABSA), serve as the foundation for this study. Using increasing amounts of examples synthesized with LLMs in addition to a given amount of 500 real examples for training smaller German language models based on BERT architecture, it is evaluated how the models' performance differs when using only real examples. For generating a synthetic example, 25 random samples are drawn from the 500 real examples, serving as few-shot examples. Additionally, a condition is examined, for which we assume a significantly smaller set of 25 real examples. In this case, we consistently employ these 25 examples as the few-shot examples for generating all synthetic examples.

• Data acquisition: German language reviews posted on TripAdvisor
• Data annotation: ABSA Annotation of 5,000 sentences from the reviews (aspect term, aspect category, sentiment polarity)
• LLM augmentation: GPT-3.5 (175B parameters) and Llama2-70B (70B parameters)
• LLM language analysis: Analysis of the language diversity of annotated examples
• Evaluation:

  The performance of the following tasks is analysed using synthetic data:

  • Aspect Category Detection (ACD)
  • Aspect Category Sentiment Analysis (ACSA)
  • End-to-End Aspect-Based Sentiment Analysis (E2E ABSA)
  • Target Aspect Sentiment Detection (TASD)

• Python: pandas, numpy, spaCy, NLTK, scikit-learn and transformers
• Fundamentals of Deep Learning (DL), Natural Language Processing (NLP) and Data Augmentation
• Large Language Models (LLMs): llama.cpp and experience with the OpenAI API
• Small Language Models (SLMs): Sequence Classification, Token Classification, and Seq-2-Seq Text Generation
• Familiarity with NVIDIA GPUs and their utilization for accelerated computing

[1] Li, G., Wang, H., Ding, Y., Zhou, K., & Yan, X. (2023). Data augmentation for aspect-based sentiment analysis. International Journal of Machine Learning and Cybernetics, 14(1), 125-133.

[2] Zhang, W., Deng, Y., Liu, B., Pan, S. J., & Bing, L. (2023). Sentiment Analysis in the Era of Large Language Models: A Reality Check. arXiv preprint arXiv:2305.15005.

[3] Møller, A. G., Dalsgaard, J. A., Pera, A., & Aiello, L. M. (2023). Is a prompt and a few samples all you need? Using GPT-4 for data augmentation in low-resource classification tasks. arXiv preprint arXiv:2304.13861.

[4] Wang, S., Liu, Y., Xu, Y., Zhu, C., & Zeng, M. (2021, November). Want To Reduce Labeling Cost? GPT-3 Can Help. In Findings of the Association for Computational Linguistics: EMNLP 2021 (pp. 4195-4205).

[5] Meyer, S., Elsweiler, D., Ludwig, B., Fernandez-Pichel, M., & Losada, D. E. (2022, July). Do we still need human assessors? prompt-based gpt-3 user simulation in conversational ai. In Proceedings of the 4th Conference on Conversational User Interfaces (pp. 1-6).

[6] Li, K., Chen, C., Quan, X., Ling, Q., & Song, Y. (2020). Conditional augmentation for aspect term extraction via masked sequence-to-sequence generation. arXiv preprint arXiv:2004.14769.

[7] Zhang, W., Li, X., Deng, Y., Bing, L., & Lam, W. (2022). A survey on aspect-based sentiment analysis: Tasks, methods, and challenges. IEEE Transactions on Knowledge and Data Engineering.

[8] Li, G., Wang, H., Ding, Y., Zhou, K., & Yan, X. (2023). Data augmentation for aspect-based sentiment analysis. International Journal of Machine Learning and Cybernetics, 14(1), 125-133.