A combination of autoregressors and autoencoders using XLNet for sentiment analysis
Abstract
In this paper sentiment analysis has been performed in order to evaluate the performance
of XLNet on this particular task. XLNet is rather a ground-breaking network on language
understanding which uses the perks of both autoregressive models and autoencoders. While
BERT uses autoencoders and Transformers use autoregression, XLNet combines the aforementioned networks’ attributes in order to achieve higher performance in many NLP tasks,
such as sentiment analysis, question answering, reading comprehension, natural language
understanding etc. In this work we evaluate the XLNet model in several sentiment classification tasks in terms of accuracy and efficiency. The XLNet reaches state of the art results
and outperforms BERT which is the previous state of the art model on natural language
processing.
This was an assignment for the course of Deep learning in PhD program of National Technical Unicersity of Athens
- Team composed of 3 persons
- Runs has been made on HPC-ARIS through batch scripts
- Course grade 10/10 (excellent)
- Full report formatted as a paper in here
- Code for 2 sentiment analysis tasks out of 3 (implemented by the author of this repo) in here
- Data available here
