Transformers Can Achieve Length Generalization But Not Robustly (AI summary)
Yongchao Zhou, Uri Alon, Xinyun Chen, Xuezhi Wang, Rishabh Agarwal, Denny Zhou
Read more
DoRA: Weight-Decomposed Low-Rank Adaptation (AI summary)
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Min-Hung Chen
Read more
Mixtures of Experts Unlock Parameter Scaling for Deep RL (AI summary)
Johan Obando-Ceron, Ghada Sokar, Timon Willi, Clare Lyle, Jesse Farebrother, Jakob Foerster, Gintare Karolina Dziugaite, Doina Precup, Pablo Samuel Castro
Read more
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data (AI summary)
Mateusz Łajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszyńska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokolova, Thomas Drugman
Read more
Suppressing Pink Elephants with Direct Principle Feedback (AI summary)
Louis Castricato, Nathan Lile, Suraj Anand, Hailey Schoelkopf, Siddharth Verma, Stella Biderman
Read more
Policy Improvement using Language Policy Improvement using Language Feedback Models (AI summary)
Victor Zhong, Dipendra Misra, Xingdi Yuan, Marc-Alexandre Côté
Read more
Scaling Laws for Fine-Grained Mixture of Experts (AI summary)
Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro, Michał Krutul, Szymon Antoniak, Kamil Ciebiera, Krystian Król, Tomasz Odrzygóźdź, Piotr Sankowski, Marek Cygan, Sebastian Jaszczur
Read more
Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrapping (AI summary)
Haoyu Wang, Guozheng Ma, Ziqiao Meng, Zeyu Qin, Li Shen, Zhong Zhang, Bingzhe Wu, Liu Liu, Yatao Bian, Tingyang Xu, Xueqian Wang, Peilin Zhao
Read more
ODIN: Disentangled Reward Mitigates Hacking in RLHF (AI summary)
Lichang Chen, Chen Zhu, Davit Soselia, Jiuhai Chen, Tianyi Zhou, Tom Goldstein, Heng Huang, Mohammad Shoeybi, Bryan Catanzaro
Read more
Direct Language Model Alignment from Direct Language Model Alignment from Online AI Feedback (AI summary)
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, Mathieu Blondel
Read more