Welcome to the SLUE Benchmark

                        █▌ ██─   
               ▓    ─ ▌   █    ╗▄
             ─ █  █  █  █  ██▄  ▌ █   ▄    
          ╙    ▌ ▌   ██  ▌  █ █   ██▀    
                    ██▀    ▌  ██    

We are pleased to present the Spoken Language Understanding Evaluation (SLUE) benchmark, aimed at accomplishing the following objectives:

  • Track research progress on multiple SLU tasks

  • Facilitate the development of pre-trained representations by providing fine-tuning and eval sets for a variety of SLU tasks

  • Foster the open exchange of research by focusing on freely available datasets that all academic and industrial groups can easily use

For this benchmark, we provide:

  • New annotation of publicly available, natural speech data for training and evaluation

  • A benchmark suite including code to download and pre-process the SLUE datasets, train the baseline models, and evaluate performance on SLUE tasks

Refer to Toolkit and Paper for more details.

ASAPP     https://www.ttic.edu/img/logo.png     _images/ntu-logo.png     _images/cmu-logo.png    

SLUE committees

Suwon Shon - ASAPP
Felix Wu - ASAPP
Ankita Pasad - TTIC
Chyi-Jiunn Lin - NTU
Siddhant Arora - CMU
Roshan Sharma - CMU
Wei-Lun Wu - NTU
Hung-Yi Lee - NTU
Karen Livescu - TTIC
Shinji Watanabe - CMU

Questions and issues

For open discussion, we will use GitHub issue page as our official Q&A boards.

Logo of the SLUE

It is important to note that the SLUE logo does not depict a snake; rather, it is an audio waveform in time-domain drawn with ASCII characters. We believe this text-drawn waveform appropriately represents what SLUE stands for, and so we used a generator to create the image and tweaked it slightly.