This week, our final projecr team was created! We decided to work on the open source projects by Huggingface, and we’ve been reading the relevant materials and making our first contributions. Click to see more details!

The Motivation to Choose Huggingface

There are several reasons that we chose the open source Huggingface for our final project:

• Huggingface is one of the most commonly used Machine Learning library in the community, with efficient, scalable implementation of SOTA AI models. I believe reading the source code as well as trying to modify it will deeply enhance our understandings of AI. Given our group members plan to pursue career in the AI industry in the future, the experience of contributing to Huggingface will be a great boost.
• The primary programming language of many Huggingface projects are Python, in which we are quite fluent.
• Huggingface provides many open source projects with different difficulty levels for contribution.

Progress and Obstacles

Our group started working on Huggingface by getting familiar with the relevant material - which is quite a long list, including forums and issues of Huggingface, how to access NYU HPC, and relevant techniques for running and debugging LLMKs (vLLM and pdb). I’ve learned a lot from these materials, and will probably apply what I’ve learned to other courses and projects. Some of the problems we are facing are:

• Most LLMs require a lot of computing resource (e.g. a competent GPU) to run, yet NYU Greene HPC is very crowded, making it often hard to get a GPU. A possible workaround is to focus on models with smaller sizes which require less VRAM and time to run.
• The most popular open-source projects of Huggingface are Transformers, diffusers, and datasets, where most issues are non-trivial, requiring good knowledge in AI and debugging to solve. In fact, we checked the issues with “good first issues” tag and noticed many of them are not very beginner-friendly. Luckily, many of the new issues are relatively easy, even though they are not tagged “good first issues”. We’ve been sharing good and solvable issues in our Slack channel as well.

Personally, I have made my first contribution to Huggingface! It was an issue related to generation with KV-Cache. Initially I thought there was a bug in code and used pdb to debug the source code line by line, yet in the end I realized this was a problem with model precision (16-bits), and the problem can be solved by changing the precision to 32-bits. Moreover, later I found a previous issue where detailed explanation for this phenomenon is provided. I included in my answer as a reference.