In order to facilitate explainable decision-making, this project will concentrate on developing methods to explain a model’s predictions by providing extractive rationales. We propose a new framework for unsupervised NLP encoder. We will develop a statistical model that uses sentence compression to extract the most vital information for text classification. It will use unsupervised summarization by forcing the model to distill the most important portions of a given document and predict a label using only this information, resulting in meaningful and effective extractive rationales. We propose a novel method for jointly learning to extract these portions and leverage them for classification using the Gumbel reparameterization trick. We will then evaluate how faithful our model’s these rationales are to human-annotated rationales using the ERASER benchmark.