I create artificial agents that have the skills and incentives to assist humans. My work specifically focuses on the following topics:
Learning from human feedback: I worked on reinforcement learning from human feedback (RLHF) since the pre-LLM days [EMNLP’17]. Recently, I have been developing frameworks for learning via abstract, pragmatic language-based communication [ICML’21, ArXiv’23].
Learning to ask questions: agents that can ask questions enjoy improved interpretability, safety, and utility! I have written a series of papers to disseminate this idea [EMNLP’15’, CVPR’19, EMNLP’19, ICML’22].
Modeling human cognition: current models implement a very primitive “model of thought” [ToM@ICML’23]. To communicate effectively with humans, their cognition must be robust and human-compatible. I recently improved the cognitive capbility of instruction-generation models [ACL’23].
More facts:
My real name is Nguyễn Xuân Khánh . My first name (Khanh) means “joy” or “happiness”. Please do not confuse it with Khan or Kahn :(
I was born in Việt Nam , a peaceful country (click here for inspiration to visit us).
I am also proud to be a PTNK (Phổ Thông Năng Khiếu) alumnus.
New paper on task-oriented cognitive capabilities. TLDR; we found and improved the deficiency in the pragmatic capability of instruction generation models. Received outstanding paper award at the ToM workshop at ICML 2023.
Aug 17, 2022
I will be organizing InterNLP workshop at NeurIPS 2022. Please submit your papers if interested!
Agents that can decide when and what question to ask, and incoporate answer to make progress
@inproceedings{nguyen2022hari,author={Nguyen, Khanh and Bisk, Yonatan and Daum{\'e} III, Hal},title={A framework for learning to request rich and contextually useful information from humans},booktitle={ICML},month=jul,year={2022},}
Framework for learning from language feedback with theoretical guarantees
@inproceedings{nguyen2021iliad,title={Interactive learning from activity description},author={Nguyen, Khanh and Misra, Dipendra and Schapire, Robert and Dud{\'\i}k, Miro and Shafto, Patrick},booktitle={ICML},year={2021},}
Improve machine translation with reinforcement learning from noisy ratings
Machine translation is a natural candidate problem for reinforcement learning from human feedback: users provide quick, dirty ratings on candidate translations to guide a system to improve. Yet, current neural machine translation training focuses on expensive human-generated reference translations. We describe a reinforcement learning algorithm that improves neural machine translation systems from simulated human feedback. Our algorithm combines the advantage actor-critic algorithm (Mnih et al., 2016) with the attention-based neural encoder-decoder architecture (Luong et al., 2015). This algorithm (a) is well-designed for problems with a large action space and delayed rewards, (b) effectively optimizes traditional corpus-level machine translation metrics, and (c) is robust to skewed, high-variance, granular feedback modeled after actual human behaviors.
@inproceedings{nguyen2017banditnmt,title={Reinforcement learning for bandit neural machine translation with simulated human feedback},author={Nguyen, Khanh and Daum{\'e} III, Hal and Boyd-Graber, Jordan},booktitle={EMNLP},month=sep,year={2017},address={Copenhagen, Denmark},publisher={Association for Computational Linguistics},url={https://www.aclweb.org/anthology/D17-1153},doi={10.18653/v1/D17-1153},pages={1464--1474},}
Calibrated probabilities for detecting errors of NLP models
@inproceedings{nguyen15calibration,title={Posterior calibration and exploratory analysis for natural language processing models},author={Nguyen, Khanh and O{'}Connor, Brendan},booktitle={EMNLP},year={2015},address={Lisbon, Portugal},publisher={Association for Computational Linguistics},url={https://www.aclweb.org/anthology/D15-1182},doi={10.18653/v1/D15-1182},pages={1587--1598},}
. My first name (Khanh) means “joy” or “happiness”. Please do not confuse it with 
, a peaceful country (click here for inspiration to visit us).
