Click here to flash read.
Instruction tuning has emerged as a promising approach to enhancing large
language models in following human instructions. It is shown that increasing
the diversity and number of instructions in the training data can consistently
enhance generalization performance, which facilitates a recent endeavor to
collect various instructions and integrate existing instruction tuning datasets
into larger collections. However, different users have their unique ways of
expressing instructions, and there often exist variations across different
datasets in the instruction styles and formats, i.e., format inconsistency. In
this work, we study how format inconsistency may impact the performance of
instruction tuning. We propose a framework called "Unified Instruction Tuning"
(UIT), which calls OpenAI APIs for automatic format transfer among different
instruction tuning datasets. We show that UIT successfully improves the
generalization performance on unseen instructions, which highlights the
importance of format consistency for instruction tuning. To make the UIT
framework more practical, we further propose a novel perplexity-based denoising
method to reduce the noise of automatic format transfer. We also train a
smaller offline model that achieves comparable format transfer capability than
OpenAI APIs to reduce costs in practice.
