Is the System Message Really Important to Jailbreaks in Large Language Models?
CoRR(2024)
摘要
The rapid evolution of Large Language Models (LLMs) has rendered them
indispensable in modern society. While security measures are typically in place
to align LLMs with human values prior to release, recent studies have unveiled
a concerning phenomenon named "jailbreak." This term refers to the unexpected
and potentially harmful responses generated by LLMs when prompted with
malicious questions. Existing research focuses on generating jailbreak prompts
but our study aim to answer a different question: Is the system message really
important to jailbreak in LLMs? To address this question, we conducted
experiments in a stable GPT version gpt-3.5-turbo-0613 to generated jailbreak
prompts with varying system messages: short, long, and none. We discover that
different system messages have distinct resistances to jailbreak by
experiments. Additionally, we explore the transferability of jailbreak across
LLMs. This finding underscores the significant impact system messages can have
on mitigating LLMs jailbreak. To generate system messages that are more
resistant to jailbreak prompts, we propose System Messages Evolutionary
Algorithms (SMEA). Through SMEA, we can get robust system messages population
that demonstrate up to 98.9
not only bolsters LLMs security but also raises the bar for jailbreak,
fostering advancements in this field of study.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要