Photo credit: news.mit.edu

New Technique Allows Language Models to Moderate Toxicity in Generated Text

As individuals develop from childhood into adulthood, their vocabulary expands, and their capacity for thoughtful communication matures. This evolution in language usage is influenced by personal ethics, cultural contexts, and individual experiences, guiding us away from potentially harmful expressions. Remarkably, similar advancements in language moderation are emerging in large language models (LLMs), which are pre-trained on vast public datasets that often contain biased and toxic language.

Researchers at MIT, in collaboration with the MIT-IBM Watson AI Lab and IBM Research, have introduced a novel approach known as self-disciplined autoregressive sampling (SASA). This innovative technique empowers LLMs to self-moderate their outputs effectively while maintaining linguistic fluency.

Distinct from traditional detoxification strategies, SASA employs a decoding algorithm that identifies the boundaries between toxic and nontoxic language within the LLM’s internal representations. Notably, it does this without modifying the model’s parameters or requiring retraining or external rewards. During the generation process, the algorithm evaluates the toxicity of the partially constructed phrase, considering both the already generated tokens and possible new tokens, ultimately selecting words that fit within a safe nontoxic context.

Ching-Yun “Irene” Ko, the lead researcher and a PhD candidate at MIT, emphasizes the objective of developed technology: “We aimed to find a way, using any existing language model, to ensure that the generation process reflects human values, specifically targeting toxicity in this case.”

Ko’s team includes esteemed colleagues such as Luca Daniel, a professor at MIT’s Department of Electrical Engineering and Computer Science, along with several distinguished members from the MIT-IBM Watson AI Lab and IBM Research. Their findings are set to be shared at the upcoming International Conference on Learning Representations.

Understanding the Need for Language Moderation

Large language models typically learn from diverse internet sources, which unfortunately include negative language, including swear words and bullying remarks. Consequently, LLMs can unintentionally produce harmful content, even from innocuous prompts, highlighting the urgent need for corrective measures. Known methods to foster fairness and ensure alignment with desired values have their challenges. Some involve retraining models with filtered datasets, a resource-intensive process that risks altering the model’s overall efficacy. Others utilize external reward systems, which can slow down operation and increase memory requirements.

The SASA approach presents an innovative solution by leveraging the autoregressive properties of LLMs, gradually guiding the generation process token by token towards more appropriate language choices.

The research team developed a linear classifier that distinguishes between toxic and nontoxic embeddings within the model’s learned representations. During training, similar words are positioned closely in vector space, allowing the researchers to hypothesize that contextual information could assist in identifying toxic language. The team trained their classifier using datasets with annotated sentences indicating varying toxicity levels, allowing for the establishment of clear boundaries between acceptable and harmful language outputs.

Practical Testing of SASA

The effectiveness of SASA was tested against various benchmark interventions across three transformer LLMs, including GPT2-Large, Llama2-7b, and Llama 3.1-8b-Instruct. Each model was tasked with generating sentence completions, resulting in multiple outputs that were then evaluated for toxicity using the PerspectiveAPI scoring system.

The researchers systematically increased the complexity of the prompts to better assess SASA’s detoxification capabilities. Initially, they tested straightforward prompts followed by more challenging scenarios that were predisposed to producing undesirable outputs. They also assessed the model’s performance against datasets focused on gender bias, ensuring that toxic response rates were balanced across genders.

While SASA demonstrated significant reductions in toxic language generation, some reduction in fluency was observed as a trade-off. The researchers noted that prior to SASA intervention, LLMs produced a higher volume of toxic results for prompts associated with female references. However, SASA’s integration significantly lowered these harmful outputs, enhancing equality in response generation.

Future Implications and Values Alignment

Ko highlighted a crucial aspect of this research—balancing natural language generation with the need to mitigate toxic expressions is a defined optimization challenge. The SASA methodology allows for adjustable parameters to align more closely with broader human values, such as truthfulness, helpfulness, and loyalty, without extensive computational resources.

With its lightweight nature, SASA could potentially be adapted to accommodate multiple value systems. By assessing an output’s position in various subspaces, it maintains low computational overhead while steering content generation toward a more principled outcome.

This innovative research was conducted with the support of the MIT-IBM Watson AI Lab and the National Science Foundation, marking a significant step forward in the quest for responsible and ethical AI language generation.

Source
news.mit.edu