AI Community of Practice Fine-Tuning of LLM's

Columbia University’s AI Community of Practice, led by the Emerging Technologies team, convened its October session with a focus on fine-tuning large language models (LLMs). The session, presented by Mark Chen, Machine Learning Engineer at CUIT, highlighted the increasing importance of fine-tuning for a variety of specialized use cases, ranging from customer service to technical document processing.

Key Features and Advantages of Fine-Tuning for Specialized Use Cases

• Data Security: Fine-tuning smaller models locally ensures that sensitive data remains on-device, avoiding the risks associated with transferring data to external servers. This is especially important for organizations handling proprietary or confidential information.

• Efficiency: Fine-tuned, smaller models can outperform larger, more generalized models like GPT-4 for specific tasks. They require less computational power, making them faster and more cost-effective for targeted operations.

• Customization: Fine-tuning allows organizations to tailor models to their unique needs. Mark explained how models can be trained to adapt to domain-specific tasks, such as responding to customer queries or summarizing technical reports.

Technical Approach

Mark outlined the technical process behind fine-tuning, focusing on low-rank adaptation. By updating only a small percentage of a model’s parameters, fine-tuning makes it possible to achieve specialized performance without the resource demands of full-weight updates. This approach allows models to be fine-tuned on consumer-grade hardware, making it an accessible option for many users.

Mark demonstrated how fine-tuning can be applied to small, open-source models, transforming them into efficient tools for niche tasks. One example showcased how a fine-tuned model was used to summarize complex documents, maintaining context and accuracy while improving processing speed.

Challenges and Solutions

• Overfitting: One of the key challenges in fine-tuning is preventing overfitting, where the model memorizes the training data instead of learning to generalize. Mark explained the importance of monitoring the evaluation loss throughout the training process to ensure the model doesn’t overfit.

• Data Requirements: Fine-tuning requires high-quality data. Mark recommended having at least 1,000 examples in question-and-answer format to ensure successful fine-tuning. In some cases, larger models like GPT-4 can be used to generate training data, followed by human review to guarantee quality.

Future Developments and Use Cases

Mark highlighted several exciting future directions for fine-tuning:

• Text-to-SQL Models: Fine-tuning can be used to teach AI models how to convert natural language into SQL queries, making database interactions more efficient.

• Multilingual Chatbots: Fine-tuned models can seamlessly switch between languages, allowing organizations to serve diverse audiences in multiple languages, such as English and Spanish.

• Agentic Systems: Fine-tuned models could act as orchestrators in more complex AI systems, determining which tasks or larger models to activate based on the context of user queries.

Conclusion and Next Steps

The session concluded with an invitation for attendees to explore fine-tuning within their own projects. Mark encouraged participants to collaborate and share their experiences with fine-tuning. The next session of Columbia’s AI Community of Practice is scheduled for November, continuing the exploration of AI and machine learning applications.

Participants are encouraged to stay engaged by attending the monthly sessions, held on the fourth Friday of every month, and sharing their own work in AI and ML. For more information on the AI Community of Practice, visit Columbia University's Emerging Technologies page.