After the Spring Break, my team and I continued working on contributions to the Huggingface Open Source Projects. Also, our TA Shivam Balikondwar made a presentation on his journey as an open-source contributor. Click to see more!

My Contributions to Huggingface so far

I’ve been working on Huggingface Transformers throughout March and have made several contributions that I’m proud of:

• issue #36536 proposed an explanation for inconsistent generation results w/ and w/o KV-cache. In the beginning I thought this would be bug that needs a PR to fix, yet it turned out the inconsistency was caused by model precision, so I simply proposed an explanation with reference to previous discussions. Issue closed.
• issue #36598 proposed an explanation for lm_head missing in model.state_dict(). It turns out lm_head is tied to the model’s embedding and thus sharing weights with the embedding, which is why it’s not present in the state dict. Issue closed.
• PR #36779 a new release of Transformers which added Gemma 3 support has serious bugs in loading previous models like BERT. After arduous debugging, I was able to find the logic errors in the model loading code and fixed them. However, the maintainers told me they do not accept PRs to dev branches so the PR was finally closed.
• PR #37153 this PR fixed a type inconsistency problem among different functions, which can potentially cause bugs. Notably, I had an interesting discussion on the specific implementation with the person that raised the issue. I believed my choice was simpler and more consistent with the style of the whole codebase, and it was finally accepeted by the user and the maintainers. PR merged.

My hopes for our group project

Throughout this semester, I hope I could make ~10 contributions to Huggingface. My team and I are discussing working on 2 feature request issues together:

• Improve auxiliary_in_channels default behavior in UperNet
• Include “time” as option to save_strategy

I believe these feature requests are of significance (especially the second one) with reasonable workload. The PRs will be submitted in the near future.

Reflections on Shivam’s presentation

This week our TA Shivam Balikondwar gave a presentation on his journey as an open-source contributor in class. I was quite amazed that Shivam was not a CS major student during his bachelor’s study. Nevertheless, he was able to find open-source projects that interest him and are truly important (Chromium, Hyperledger) to work on pesistently, which eventually paved his way to nice jobs.
Besides, Shivam was so nice to give me suggestions on my group project. I was afraid to work on GPU/efficiency/parallelism-related issues in Huggingface because I don’t have sufficient knowledge in these areas. Shivam suggested that I should leverage my strengths when making the contributions. He also offered to help me look at the project if needed.