## Introduction
[image:6]
That being said, welcome to the MLC NLP interest group! We're a very welcoming group who meets every **Sunday 9 AM PST** to discuss NLP papers, discuss ongoing projects, and provide feedback to each other. We're a very diverse group ranging from little to several years of experience in the field so don't hesitate to join regardless of past experience - there'll always be something to learn. We have published a few papers and blog posts in the past - if you're interested in learning more about what we do, consider joining our discord or checking out some of the past presentations and work we've done! Link to join discord - discord.gg/nNJ4GBPZm9
| | |
| -------- | --------- |
| **Reading Group Day & Time** | Sunday, 9am PST |
|**Channel**: | [`#natural-language-processing`](https://discord.com/channels/785992161132806174/831904937969451019) |
### Previous Meetings
| Date | **Paper Title** | Presenter | Presentation |
| -------| ----------- | ----------- | -----------|
| Dec 3'23 | [OpenAI Assistant API and function calling](https://platform.openai.com/docs/assistants/overview) | Akash | [Recording](https://youtu.be/D1qRy9GId2c)
| Nov 5'23 | [Query Rewriting for Retrieval-Augmented Large Language Models](https://arxiv.org/abs/2305.14283) | Aishwarya | [Recording](https://youtu.be/Zl9c_2aaVlo)
| Oct 29 '23 | [Self-Refine: Iterative Refinement with Self-Feedback](https://selfrefine.info/) | Jay | [Recording](https://youtu.be/rPGSeGGMyy4)
| Oct 22 '23 | [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://arxiv.org/abs//2310.11511) | Aiswarya | [Slides](https://docs.google.com/presentation/d/1DP-TZGKrKxPT0ECZIbBJmUNNxxbejRWUsPYDp44OkUA/edit#slide=id.g2482856e5e1_0_24) [Recording](https://www.youtube.com/watch?v=iGhoH1Q1eE8)
| Oct 15 '23 | [Decomposing Language Models Into Understandable Components](https://www.anthropic.com/index/decomposing-language-models-into-understandable-components?fbclid=IwAR20fQYqxRNEGyB3Dc_AhrfbmugWfqTqQKFx4Gvv7B7Y8iHOWaK8wOUpZio) | Aiswarya | [Slides](https://docs.google.com/presentation/d/1qR8uUcTbFGN8PYZj1GJbzaBUnL0dgvFUwxzRi1oKVg8/edit#slide=id.g282f72e6e2e_0_43) [Recording](https://drive.google.com/file/d/1HWaX1gCgTRDFMAJVsweyO29oxth3UxIp/view?usp=sharing)
| Oct 1 '23 | [A Survey on Large Language Model based Autonomous Agents Part 2](https://arxiv.org/abs/2308.11432v1) | Akash | [Recording](https://youtu.be/3QYX7UormWA)
| Sep 24 '23 | [A Survey on Large Language Model based Autonomous Agents Part 1](https://arxiv.org/abs/2308.11432v1) | Aiswarya | [Slides](https://docs.google.com/presentation/d/1fpfVe9o1IThI2AIRZIr5iFYZgou2zA12cBYjfR2KoKk/edit#slide=id.g282f72e6e2e_0_43) [Recording](https://youtu.be/rDljQuwM2o8)
| Sep 10 '23 | [Direct Preference Optimization](https://arxiv.org/abs/2305.18290) | Atsushi | [Slides](https://docs.google.com/presentation/d/1Kqsu27WhYohJNdc1uXWsTyMHQc4l4UdvEZZodXem6a8/edit#slide=id.p) [Recording](https://youtu.be/wjy4JlRwbS4)
| Sep 3 '23 | [Giraffe: Adventures in Expanding Context Lengths in LLMs](https://arxiv.org/abs/2308.10882) | Aiswarya | [Slides](https://docs.google.com/presentation/d/10A_nBRP3K8cJGVRV075ITm1YValGBrvvYyYeTfGWUsY/edit#slide=id.p) |
| Aug 27 '23 | [Gorilla: Large Language Model Connected with Massive APIs](https://arxiv.org/abs/2305.15334) | Aiswarya | [Slides](https://docs.google.com/presentation/d/14fkbEsinFnPuGM7bhTIQL13ZwEKhav9S_5wbqMrZTZU/edit#slide=id.p) |
| July - Aug '23 | Finetuning LLM Project
| Jun 18 '23 | QLora code discussion | Akash | [Recording](https://drive.google.com/file/d/1HiHODXCJFJ3T_V7I6aAq6Gjv3TwCiMqI/view?usp=sharing) |
| Jun 4 '23 | [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314) | Jay | [Recording](https://youtu.be/dPKY7BPyv7w) |
| May 28 '23 | [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://arxiv.org/abs/2212.10560) | Aiswarya | [Slides]( https://docs.google.com/presentation/d/1GSd9cPDHt4du9X8B8seIn9nFSMfVfdl3rSjsTWOwvZg/edit?usp=sharing) [Recording](https://youtu.be/8moobJ6bb04) |
| May 21 '23 | [PeFT: Parameter-Efficient Fine-Tuning](https://arxiv.org/abs/2205.05638) | Jay | [Recording](https://youtu.be/VEGO3ivazSY) |
| Apr 30 '23 | [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) | Pavan | [Slides](https://docs.google.com/presentation/d/1m5UbyI0IAuFAk6p8AM5R73V2yoEIIdH1Fs0Nsd3v0yU/edit?usp=sharing) [Recording](https://youtu.be/pSI1rO5xnZ4) |
| Apr 23 '23 | [Sparks of Artificial General Intelligence: Early experiments with GPT-4 PART 2](https://arxiv.org/abs/2303.12712) | Jay | [Recording](https://drive.google.com/file/d/1WjuJhQWj4qh6OpeSgxfHwvlZzOJ8tuB5/view?usp=sharing) |
| Apr 9 '23 | [Sparks of Artificial General Intelligence: Early experiments with GPT-4 PART 1](https://arxiv.org/abs/2303.12712) | Akash | [Recording](https://www.youtube.com/watch?v=Sya1j3zVlHY) |
| Apr 2 '23 | [Successive Prompting for Decomposing Complex Questions](https://arxiv.org/pdf/2301.07597v1.pdf) | Aiswarya | [Recording](https://drive.google.com/file/d/1z3rituvJxtnpyfwUlO-a-HNY2AIYUbFw/view?usp=sharing) |
| Mar 26 '23 | [Mixture of Soft Prompts for Controllable Data Generation](https://arxiv.org/abs/2303.01580) | Jay | [Recording](https://drive.google.com/file/d/1RuA2JU8Wczn4uisuL9jnx87ks01GASjC/view?usp=sharing) |
| Mar 19 '23 | Kubeflow and MLOps| pappachuck | [Recording](https://www.youtube.com/watch?v=pjaFZOlptE4) |
| Mar 12 '23 | [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://arxiv.org/pdf/2301.07597v1.pdf) | Akash | [Slides](https://docs.google.com/presentation/d/1dI7RKlDXrUQIS9XvKv5ywc1IEFMVwy7cYCmDiuPrDjA/edit#slide=id.gd4f5f6d927_0_24) [Recording](https://www.youtube.com/watch?v=6CvcYYUuUng) |
| Feb 26 '23 | Meeting for Unit test generator and Meme Generation project
| Feb 19 '23 | [MemPrompt: Memory-assisted Prompt Editing with User Feedback](https://arxiv.org/abs/2201.06009) | Aiswarya | [Slides](https://docs.google.com/presentation/d/1YxI4bqJwZsAeLy0EYxUgSQbxlvGUSuyQYhw626VqZFQ/edit?usp=sharing) [Recording](https://www.youtube.com/watch?v=JdXnhWpRpu4) |
| Feb 5 '23 | [Automating Code Review Activities by Large-Scale Pre-training](https://arxiv.org/pdf/2203.09095.pdf) | Aiswarya | [Slides](https://docs.google.com/presentation/d/1p7crYrDai7gbItq7QHKrRetF5xc0lvjdvFUKbNGjOLE/edit?usp=sharing) [Recording](https://www.youtube.com/watch?v=FMhsRkZmXrg) |
| Jan 29 '23 | [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://arxiv.org/pdf/2108.12409.pdf) | Jay & Akash | [Slides](https://drive.google.com/file/d/10loerHlDHA0qhrNnbfaojSyRbWyIK8ym/view?usp=sharing) [Recording](https://www.youtube.com/watch?v=KvWpw5tZ1gc) |
| Jan 22 '23 | [Cramming: Training a Language Model on a Single GPU in One Day](https://arxiv.org/pdf/2212.14034.pdf) | Jay | [Slides](https://drive.google.com/file/d/1ibwvHs9wUjW5i1UHi5cp3u9z4bOd_f-5/view?usp=sharing) [Recording](https://www.youtube.com/watch?v=POUGSPZaMsk) |
| Jan 15 '23 | [RLHF and InstructGPT](https://arxiv.org/pdf/2203.02155.pdf) | Akash & Aiswarya | [Slides](https://docs.google.com/presentation/d/1fCuL8fI6UCwMb4TAzktynz2evSY8-Lx0v9z6_VgJrBk/edit#slide=id.gd4f5f6d927_0_24) [Recording](https://www.youtube.com/watch?v=MKb4orC58-M) |
| Dec 18 | [Fine Tuning Language Models to find agreement among humans with diverse preferences](https://arxiv.org/pdf/2211.15006.pdf) | Aiswarya | [Link](https://docs.google.com/presentation/d/1wFSBHl2TQW84BnZwd9D9X-A8rTzChxXHwP1vRVnnrsI/edit#slide=id.gd4f5f6d927_0_24) |
| Dec 11 | [Cicero: Human-level play in the game of Diplomacy by combining language models with strategic reasoning](https://www.science.org/doi/10.1126/science.ade9097) | Akash | [Link](https://docs.google.com/presentation/d/1Kk3a3skmBzaPbM0KhEg-BOWxDg--n5UxbyzqDCDttOs/edit?usp=sharing) |
| Dec 4 | [Scaling Instruction Fine Tuned Language Models](https://arxiv.org/abs/2210.11416) | Jay | [Link](https://drive.google.com/file/d/1XTm_PQhb4jKoru_6X5V-T3OZ8uMJlAkH/view?usp=sharing) |
| Nov 27 | [Large Language Models can Self Improve](https://arxiv.org/abs/2210.11610) | Aiswarya | [Link](https://docs.google.com/presentation/d/1Hq7JpCVv2poHD7Q44uTzpeD2C4AYlO4wqEVvDyO3A3Y/edit?usp=sharing) |
| Nov 20 | [Mind's Eye: Grounded Language Model Reasoning through Simulation](https://arxiv.org/abs/2210.05359) | Akash | [Link](https://docs.google.com/presentation/d/1OqN3iHDedsvdIAHuxol0BIAw3JbldKG9B2ep0_EyK0k/edit#slide=id.g150c4297ed7_0_23) |
| Nov 13 | [Large Language Models are Human Level Prompt Engineers](https://arxiv.org/abs/2211.01910) | Aiswarya | [Link](https://docs.google.com/presentation/d/15VPsvYgDJIKM-23h9ndBrf6VuH5k9NA21nurygIdeFo/edit#slide=id.p)|
| Nov 6 | [Language-Mediated Robot Task Planning](https://say-can.github.io/) | David Fagan | [Link](https://docs.google.com/presentation/d/1Rxjd6I1NambXzmT-Mzg4nOjItA71WyXFJMYlDK-w0mk/edit#slide=id.p) |
| Oct 16 | [Measuring and Improving Consistency in Pretrained Language Models](https://arxiv.org/abs/2102.01017) | Aiswarya | [Link](https://docs.google.com/presentation/d/1n9ziyWY5_blHVmFbegaZwwzffXD52BRbb7ptGtE1i8M/edit#slide=id.p) |
| Oct 9 | [Locating and Editing Factual Associations in GPT](https://arxiv.org/abs/2202.05262) | Aiswarya | [Link](https://docs.google.com/presentation/d/1xuZ96HmML1cU3XH9qyim4b_flqGjn29bRPBE0gbPJWY/edit#slide=id.p) |
| Sept 18 | [Knowledge Neurons in Pretrained Transformers](https://arxiv.org/abs/2104.08696) | Jay | [Link](https://docs.google.com/presentation/d/1dFFwAlroEtOhyYY2sEJPTxVny8uN4KYWCHOkar2r7yg/edit#slide=id.p) |
| Sept 4 | [A Mathematical Framework for Transformer Circuits](https://transformer-circuits.pub/2021/framework/index.html) | Aiswarya | [Link](https://docs.google.com/presentation/d/1yHJ6429a4lNE3o92X-poSa3Fw_EHHjSg2lfqlKwwqks/edit?usp=sharing) |
| Aug 28 | [Data Distributional Properties Drive Emergent In-Context Learning in Transformers](https://arxiv.org/abs/2205.05055) | Akash Kumar | [Link](https://docs.google.com/presentation/d/1D8tQckzfO3_bD1wT0O9hhBhah9-Ubq-sltWaXB-5b0c/edit#slide=id.p) |
| Aug 21 | [GenIE: Generative Information Extraction](https://arxiv.org/abs/2112.08340) | Aiswarya | [Link](https://docs.google.com/presentation/d/1xS8pdTVABGxMZu6V-2wE4Y64jFfsLaxCEm0uy_1lnu4/edit?usp=sharing) |
| Aug 14 | [HydraSum](https://arxiv.org/pdf/2110.04400.pdf) | Aiswarya | [Link](https://docs.google.com/presentation/d/1q17uxdsjHTonNstDjwYZtXW54uSRvgxuHmErNGEzumM/edit) |
| Aug 7 | [Autoregressive Entity Retrieval](https://arxiv.org/abs/2005.11401) | Adithya | [Link](https://docs.google.com/presentation/d/1GG8DrQnfkflSffzlaVVQ6lXbTf4K3mO2Q29ilnk7GI0/edit#slide=id.p) |
# Completed Projects
| **Title** | **Description** | **Completion Date** |
| ----------| --------------- | ------------------- |
ICLR Blog Post | Select a previous ICLR paper and write an informal and accessible blog post. Highlight ambiguities, provide better visualization, etc. | 04/10/2022 |
# ICLR Blog Post Submissions
- [An Understanding of Learning from Demonstrations for Neural Text Generation](https://iclr-blog-track.github.io/2022/03/25/text-gen-via-lfd/)
Authors: Aiswarya Sankar, Pavan Kantharaju
- [Discovering Non-Monotonic Autoregressive Ordering for Text Generation Models using Sinkhorn Distributions](https://iclr-blog-track.github.io/2022/03/25/non-monotonic-autoregressive-ordering/)
Author: Ashutosh Kumar
...
More to come
Additional presentations -
- [Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding](https://docs.google.com/presentation/d/1rINybn4Fo99I3evSjvTvEBmR8_2K8MhY-0RiuKpCp-k/edit?usp=sharing)
- [Contrastive Learning with Adversarial Perturbations for Conditional Text Generation
](https://docs.google.com/presentation/d/1bf_U-0z55wHJBJ1PpYT2QT_j_YY8bjYo5Xk1-FrgWuA/edit?usp=sharing)
- [Text Generation by Learning from Demonstrations](https://docs.google.com/presentation/d/1_ZbiMFgCta2pSUSCqRmK9FYYQG1f5wuFYu6OCYCoNJE/edit?usp=sharing)
- [Insertion Transformer](https://docs.google.com/presentation/d/1Hv-e9UUGEKM7WEAT-FxF545cnBEvWkRoYQ_DxqfGokU/edit?usp=sharing)
- [Discovering Non-monotonic Autoregressive Orderings with Variational Inference](https://docs.google.com/presentation/d/1uxKOF_C6LRO15r0wjsZR16HJvd6hV1AaYLTvDiKQGjI/edit?usp=sharing)
and many others! Please check out our notion page if you're interested in watching the recordings for past meetings.
If you think you have a paper suggestion which you'd like to present or discuss, come and join us on discord :)
