Where Are Large Language Models for Code Generation on GitHub?

The study titled "Where Are Large Language Models for Code Generation on GitHub?" investigates the use of Large Language Models (LLMs) like ChatGPT and Copilot in software development, particularly focusing on their code generation capabilities as reflected in GitHub projects. The research highlights that ChatGPT and Copilot are the most commonly used LLMs for code generation, while other models have minimal presence. It notes that projects utilizing these models tend to be smaller and less recognized, often led by individuals or small teams, yet they show signs of continuous development. The primary programming languages for generated code include Python, Java, and TypeScript, with a focus on data processing and transformation tasks. In contrast, C/C++ and JavaScript are used for algorithm implementation and user interface code. The generated code snippets are generally short and exhibit low complexity. The study also finds that LLM-generated code is present in a limited number of projects and undergoes fewer modifications compared to human-written code, with bug-related changes being particularly rare. Additionally, comments accompanying the generated code often lack detail, typically only indicating the code's source without providing context on prompts or modifications. The findings suggest implications for both researchers and practitioners in understanding the integration and effectiveness of LLMs in real-world coding scenarios.