Advancing Beyond Identification: Multi-bit Watermark for Large Language Models

We show the viability of tackling misuses of large language models beyond theidentification of machine-generated text. While existing zero-bit watermarkmethods focus on detection only, some malicious misuses demand tracing theadversary user for counteracting them. To address this, we propose Multi-bitWatermark via Position Allocation, embedding traceable multi-bit informationduring language model generation. Through allocating tokens onto differentparts of the messages, we embed longer messages in high corruption settingswithout added latency. By independently embedding sub-units of messages, theproposed method outperforms the existing works in terms of robustness andlatency. Leveraging the benefits of zero-bit watermarking, our method enablesrobust extraction of the watermark without any model access, embedding andextraction of long messages (≥ 32-bit) without finetuning, and maintainingtext quality, while allowing zero-bit detection all at the same time. Code isreleased here: https://github.com/bangawayoo/mb-lm-watermarking