More papers on AI are published than ever before but each paper tends to only present its part of the picture—and it becomes difficult to recognize the larger story to which a paper is connected. To encourage the community to explore broader research narratives, we co-organized the Big Picture Workshop at EMNLP 2023. We received a number of high-quality submissions that distill important research topics, from narrative understanding to modern generation techniques.
My favorite part of the workshop, however, were the invited talks. We had asked researchers from different labs working on the same topic to reflect on and consolidate their often disagreeing contributions. The result were talks that were more nuanced and engaging than the—typically one-sided—scientific presentations. The talks covered topics from in-context learning to attention as explanation and morality.
In-context learning (ICL) is one of the most important emerging phenomena of LLMs but it is still not clearly understood what factors contribute to its success. At EMNLP 2022, two papers with seemingly contradictory hypotheses were published: In “Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?”, Sewon Min and others argued that random labels perform similarly to using ground-truth labels in the demonstrations. On the other hand, Kang Min Yoo and Junyeob Kim found that ground-truth labels are important in “Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations”. So, do we need ground-truth labels for ICL or not?
Source: Sebastian Ruder — Read the full article on the original site ↗