Certainty-Flow Tuning for Calibrated Instruction Refusal Behavior in Large Language Models
DOI:
https://doi.org/10.66238/ijrss.v6i2.134Keywords:
Certainty-Flow Tuning, Instruction Refusal, Model Calibration, Alignment, SafetyAbstract
The rapid advancement of large language models has brought unprecedented capabilities alongside significant challenges in safety alignment. Current alignment techniques frequently result in miscalibrated refusal behaviors, where models either comply with malicious instructions or overly refuse benign queries due to superficial structural similarities to unsafe prompts. This paper introduces certainty-flow tuning, a novel framework designed to achieve calibrated instruction refusal behavior by aligning model responses with their internal confidence regarding prompt safety. The certainty-flow mechanism evaluates the layer-wise propagation of uncertainty during the forward pass, adjusting the gradient updates to penalize unjustified confidence in refusal for ambiguous or benign instructions. Through extensive theoretical analysis and empirical evaluation, this research demonstrates that certainty-flow tuning significantly reduces false refusal rates while maintaining robust defenses against adversarial attacks. The proposed methodology captures the epistemic uncertainty inherent in safety evaluations, translating it into a continuous flow metric that modulates the refusal token generation probability. Our findings suggest that integrating certainty-flow metrics into the fine-tuning pipeline offers a highly effective paradigm for balancing helpfulness and harmlessness, ensuring that language models refuse instructions only when internal safety boundary confidence exceeds dynamically calibrated thresholds.
References
Chen, S., Zhu, T., Wang, Z., Zhang, J., Wang, K., Zhou, R., ... & Li, M. (2025). Why Do LLM Agents Fail in Exploring New Environments? A World-Modeling Perspective. arXiv preprint arXiv:2510.15047.
Wang, Y., Fan, S., Wang, M., Gao, S., & Wang, C. (2026, April). DAMR: Efficient and adaptive context-aware knowledge graph question answering with LLM-guided MCTS. In International Conference on Learning Representations (Vol. 2026, pp. 25548-25570).
Tian, Y., & Yang, Z. (2026, May). SAEC: scene-aware enhanced edge-cloud collaborative industrial vision inspection with multimodal LLM. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 19747-19751). IEEE.
Jiang, J., Yang, P., Zhang, R., & Liu, F. (2026, July). Towards efficient large language model serving: A survey on system-aware kv cache optimization. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 38450-38476).
Yu, H., Zhang, J., Chen, C., Xiang, T., Fang, Y., Niebles, J. C., & Adeli, E. (2026, March). Socialgen: Modeling multi-human social interaction with language models. In 2026 International Conference on 3D Vision (3DV) (pp. 1-17). IEEE.
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT.
Li, G., Zhang, J., Tan, M. H. Y., Wang, B., & Huang, W. (2026). SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay. arXiv preprint arXiv:2608.03063.
Lyu, S., Mao, S., & Ma, Y. (2025). Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising. arXiv preprint arXiv:2511.03272.
Yao, Z., Li, R., Chen, X., Cheng, J., Le, C., Yue, M., ... & Dong, J. (2026). PreDiff-LM: Pretrained Discrete Masked Diffusion Language Modeling with Hybrid Attention. arXiv preprint arXiv:2607.25157.
Wang, Z., Zheng, S., & Zhang, H. (2026, August). Can Large Language Models Reason about Event-Time Stream-Processing Semantics?. In 2026 7th International Conference on Computer Vision and Data Mining (ICCVDM) (pp. 404-409). IEEE.
Liu, H., & Henao, R. (2025, April). Learning to substitute words with model-based score ranking. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 11551-11565).
Fang, S., Shen, I., Wang, Y., Tsai, Y. H., Yang, Y., Zhou, S., ... & Yang, M. H. (2025). Meshllm: Empowering large language models to progressively understand and generate 3d mesh. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 14061-14072).
Li, P., Yu, X., Peng, H., Xian, Y., Wang, L., Sun, L., ... & Yu, P. S. (2024). Relational prompt-based pre-trained language models for social event detection. ACM Transactions on Information Systems, 43(1), 1-43.
Fensham, R., Sumner, T. D., Cutter, N., Buchanan, G., Liu, R., Munoz, J., ... & Van Der Plaat, D. (2024). Towards a National Data Architecture for Cultural Collections: Designing the Australian Cultural Data Engine. Digital Humanities Quarterly, 18(2).
Wu, J., Chen, S., Heo, I., Gutfraind, S., Liu, S., Li, C., ... & Sharps, M. (2025). Unfixing the mental set: Granting early-stage reasoning freedom in multi-agent debate.
Liu, R., McKay, D., & Buchanan, G. (2024). Sociotechnical factors affecting interoperability of digital humanities. Proceedings of the Association for Information Science and Technology, 61(1), 218-231.
Mo, Z. (2025, May). Development of an Intelligent Retrieval and Recommendation System for Chinese Language Educational Resources Based on DNN Technology. In 2025 2nd International Conference on Intelligent Computing and Robotics (ICICR) (pp. 170-175). IEEE.
Liu, X., Yu, Z., Yao, Y., Yang, K., Zeng, H., & Chen, C. P. (2026). A Comprehensive Survey of Cross-Modal Retrieval: Breaking Through Modal Barriers. ACM Computing Surveys.
Li, B., Gu, B., & Ding, Z. (2025). LLM-based Personalized Portfolio Recommender: Integrating Large Language Models and Reinforcement Learning for Intelligent Investment Strategy Optimization. arXiv preprint arXiv:2512.12922.
Xiong, F., Xu, H., Wang, Y., Cheng, R., Wang, Y., & Chu, X. (2025, November). Hs-star: Hierarchical sampling for self-taught reasoners via difficulty estimation and budget reallocation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 5539-5555).
Hu, Y., Yang, Z., Zhao, C., & Ji, W. (2026, May). Adaptive guidance semantically enhanced via multimodal llm for edge-cloud object detection. In ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 12962-12966). IEEE.
Yang, H., Xie, Z., Xiong, S., & Huang, B. (2026). DepthBenchCAD: When Does Deeper Auditing Yield More Reliable Conclusions?. arXiv preprint arXiv:2609.15122.
Yao, Z., Li, R., Chen, X., Cheng, J., Le, C., Yue, M., ... & Dong, J. (2026). Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization. arXiv preprint arXiv:2607.25136.
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692.
Lyu, S., Deng, Y., Liu, G., Qi, Z., & Wang, R. (2025, June). Transferable Modeling Strategies for Low-Resource LLM Tasks: A Prompt and Alignment-Based Approach. In 2025 7th International Conference on Artificial Intelligence Technologies and Applications (ICAITA) (pp. 225-229). IEEE.
Cohen, J., Rosenfeld, E., & Kolter, J. Z. (2019). Certified adversarial robustness via randomized smoothing. In Proceedings of ICML.
M. Emani, S. Foreman, V. Sastry, Z. Xie, S. Raskar, W. Arnold, R. Thakur,
V. Vishwanath, M.E. Papka, A comprehensive performance study of
large language models on novel ai accelerators. (2023). arXiv preprint
arXiv:2310.04607
Zhu, R., Ma, Z., Wu, J., Gao, J., Wang, J., Lin, D., & He, C. (2025, April). Utilize the flow before stepping into the same river twice: Certainty represented knowledge flow for refusal-aware instruction tuning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 24, pp. 26157-26165).
Meng, X., Zhang, Y., & Lin, J. (2026). PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data. arXiv preprint arXiv:2609.10612.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Daphne Hung, Joyce Wan
This work is licensed under a Creative Commons Attribution 4.0 International License.
All articles in the International Journal of Research & Social Sciences are published under the Creative Commons Attribution 4.0 (CC BY 4.0) license. Users may share and adapt the content, including for commercial purposes, provided proper credit is given to the authors and the journal.
