Adam-mini: Use Fewer Learning Rates To Gain More (AI summary)
Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Yinyu Ye, Zhi-Quan Luo, Ruoyu Sun
Read more
On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Surve (AI summary)
Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, Haobo Wang
Read more
Following Length Constraints in Instructions (AI summary)
Weizhe Yuan, Ilia Kulikov, Ping Yu, Kyunghyun Cho, Sainbayar Sukhbaatar, Jason Weston, Jing Xu
Read more
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees (AI summary)
Yuhui Li, Fangyun Wei, Chao Zhang, Hongyang Zhang
Read more
GraphReader: Building Graph-based Agent to Enhance Long-Context Abilities of Large Language Models (AI summary)
Shilong Li, Yancheng He, Hangyu Guo, Xingyuan Bu, Ge Bai, Jie Liu, Jiaheng Liu, Xingwei Qu, Yangguang Li, Wanli Ouyang, Wenbo Su, Bo Zheng
Read more
From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries (AI summary)
Hitesh Wadhwa, Rahul Seetharaman, Somyaa Aggarwal, Reshmi Ghosh, Samyadeep Basu, Soundararajan Srinivasan, Wenlong Zhao, Shreyas Chaudhari, Ehsan Aghazadeh
Read more
From Artificial Needles to Real Haystacks: Improving Retrieval Capabilities in LLMs by Finetuning on Synthetic Data (AI summary)
Zheyang Xiong, Vasilis Papageorgiou, Kangwook Lee, Dimitris Papailiopoulos
Read more
Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8B (AI summary)
Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, Wanli Ouyang
Read more
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs (AI summary)
Ziyan Jiang, Xueguang Ma, Wenhu Chen
Read more
Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs (AI summary)
Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, Tom Goldstein
Read more