AI: Community of Practice

The AI Community of Practice at Columbia University, led by the Emerging Technologies team, held its first session of 2025, focusing on OpenAI tools and their evolving role in education. This session featured an engaging presentation by Joe Casson, OpenAI’s Education Lead for Solutions Engineering, who provided insights into higher education use cases and demonstrated OpenAI's latest advancements.

Highlights of the Meeting

1. Welcome and Context:

   • Parixit Dave, Senior Director for Emerging Technologies, introduced the session.
   • The AI Community of Practice, started in January 2024, continues to foster collaboration and exploration of AI and machine learning across Columbia.
      

2. OpenAI's Contributions to Education:

   • Joe Casson emphasized OpenAI’s mission to support universities with AI tools like ChatGPT and the newly introduced O1 models.
   • He discussed their application in personalized learning, operational efficiency, curriculum development, and research enhancement.
      

3. Live Demonstrations:

   • Joe showcased OpenAI’s tools in action, including:
     • Custom GPTs for tailored use cases, such as generating curriculum content and analyzing university mission statements.
     • The new Canvas feature for collaborative content iteration.
     • File analysis capabilities for document-based workflows.
   • The session also touched on Operator, a new tool enabling task automation, such as conducting research and summarizing data.
      

4. Key Topics Discussed:

   • Strategies for promoting ethical AI use among students.
   • Addressing academic integrity concerns.
   • Enhancing transparency through citations and data attribution.
      

5. Future Outlook:

   • OpenAI teased upcoming features for educators, emphasizing collaboration with university leaders and researchers to align AI development with academic goals.

The session concluded with an invitation to participate in February’s meeting and explore OpenAI’s resources further. Columbia remains committed to integrating AI responsibly into teaching, research, and operations.

You can access a recording of the AICoP in the News section here.

The AI Community of Practice at Columbia University, led by the Emerging Technologies team, held its final session of the year with a spotlight on two transformative technologies: LibreChat and AI Avatars. This November session provided attendees with demonstrations and discussions on how these innovations are reshaping productivity, training, and ethical considerations in artificial intelligence.

LibreChat: A Cost-Effective and Versatile AI Solution

The session began with a presentation by Parixit Dave, Senior Director for Emerging Technologies at CUIT, who introduced LibreChat, the upcoming replacement for CU GPT. CU GPT, a Columbia University innovation, offered a cost-efficient alternative to individual ChatGPT licenses by using OpenAI APIs. LibreChat, however, takes this further, integrating a wider range of models and functionalities to support diverse academic and professional needs.

Key Features of LibreChat

1. Model Diversity:
   LibreChat supports multiple models, including OpenAI’s GPT series, Anthropic’s Claude, and Google’s Gemini, as well as custom GPTs, enabling users to compare and select the best tool for their tasks.

2. Real-Time Processing:
   With rapid real-time streaming of responses, LibreChat minimizes waiting times for outputs.

3. Document Analysis:
   Users can upload documents such as PDFs, Word files, and Excel spreadsheets for analysis and summary generation.

4. Internet Search:
   The platform integrates real-time internet search capabilities, ensuring users can access the latest data beyond pre-trained model cutoffs.

5. Image Creation and Processing:
   LibreChat enables users to generate and analyze images, opening possibilities for use cases in medical imaging, education, and creative projects.

Productivity Enhancements

Parikshit showcased examples of LibreChat's capabilities, from drafting formal emails to generating complex project statements of work (SOWs) and conducting financial analysis. These demonstrations highlighted the tool’s potential to dramatically enhance productivity while maintaining cost efficiency.

The session concluded with an invitation to explore LibreChat further upon its planned launch in early 2025, positioning it as a central tool for advancing Columbia’s AI capabilities.

AI Avatars: Transforming Training and Communication

The second half of the session was led by John P. Martin, Emerging Technologist at CUIT, who presented on the use of AI avatars for creating realistic, automated video content. John shared insights from a recent project with the Maryland School of Public Health, where over 400 videos were produced in under three weeks using AI-driven tools like Wondershare Virbo for avatars and ElevenLabs for voice generation.

Applications and Advantages

1. Efficient Content Production:
   AI avatars were employed to deliver training videos with realistic lip-syncing and audio, bypassing the need for on-camera human actors and extensive video editing.

2. Customization and Cloning:
   ElevenLabs allowed for the cloning of voices to maintain a consistent tone across videos while adapting to different scripts.

3. Ethical Considerations:
   John emphasized transparency in AI-generated content, suggesting that creators attribute videos to AI to ensure audiences are informed.

Technical Overview

John explained the underlying technology, Stable Diffusion, which powers both AI-generated images and avatars. Stable Diffusion processes noise into refined outputs through iterative denoising, leveraging large datasets of visual and audio data. This mechanism ensures high-quality, realistic results in both static and dynamic applications.

Expanding Use Cases

From training modules to research communication, the applications for AI avatars and LibreChat are vast:

• Training and Education: Quickly create instructional videos for courses or corporate training.
• Research: Enhance study designs by anonymizing and preserving audio data for longitudinal analysis.
• Creative Content: Leverage avatars for storytelling, marketing, or virtual events.

Looking Ahead

The session ended with an invitation to join upcoming discussions in 2025 as Columbia University continues to lead in exploring the intersection of AI, ethics, and practical applications. With tools like LibreChat and AI Avatars at the forefront, the university is fostering an environment where innovation and responsibility go hand in hand.

The AI Community of Practice remains a vital platform for sharing insights, challenges, and advancements, ensuring Columbia stays at the cutting edge of AI technologies

You can access a recording of the AICoP in the News section here.

Columbia University’s AI Community of Practice, led by the Emerging Technologies team, convened its October session with a focus on fine-tuning large language models (LLMs). This session, presented by Mark Chen, Machine Learning Engineer at CUIT, explored the growing importance of fine-tuning for a variety of specialized use cases, from localized data processing to enhanced AI behavior customization.

Key Features and Advantages of Fine-Tuning for Specialized Use Cases

• Data Security: Fine-tuning smaller models locally ensures that sensitive data remains on-device, reducing latency and bypassing the risks associated with sending data to external servers. This is especially relevant for organizations dealing with sensitive or proprietary information.

• Efficiency: Fine-tuned, smaller models can outperform large, generalized models like GPT-4 for specific tasks. These models require less computational power, making them faster and cheaper for narrow, task-specific operations.

• Customization: Fine-tuning allows users to adapt models to their unique needs. Mark emphasized how models could be trained for various applications, such as summarizing complex documents or adopting specialized behaviors like customer service interactions or technical troubleshooting.

Technical Approach

Mark provided insights into the technical methods used to fine-tune models, such as low-rank adaptation, which involves updating only a fraction of a model’s parameters to optimize its performance for a specific task. This method reduces the computational demand, making it feasible to fine-tune models on consumer-grade hardware, and contrasts with the more resource-intensive full-weight updates.

The session demonstrated how fine-tuning can make open-source models more efficient for tasks like summarizing technical or research documents, highlighting the practical applications of this technique in a range of fields.

Challenges and Solutions

• Overfitting: Mark discussed the issue of overfitting, where models trained for too long on limited data begin to memorize the training set rather than generalizing effectively. Monitoring evaluation metrics and adjusting training steps were suggested as key strategies to prevent overfitting.

• Data Requirements: Fine-tuning requires high-quality datasets, with at least 1,000 examples formatted in question-and-answer style. Mark recommended generating training data using larger models like GPT-4 and verifying outputs to ensure accuracy.

Future Developments and Use Cases

The session explored potential use cases for fine-tuned models, including:

• Chatbots: Fine-tuning can help create more specialized chatbots tailored to different domains, whether it's technical support, customer service, or instructional assistants.

• Text-to-SQL Models: Fine-tuning could enable LLMs to translate natural language queries into structured SQL code, streamlining database interactions.

• Multilingual Systems: Fine-tuned models can be trained to provide seamless responses in multiple languages, broadening their applicability for diverse user bases.

• Agentic Systems: Small fine-tuned models could serve as decision-makers in more complex AI ecosystems, determining which tasks or larger models to delegate to based on user input.

Conclusion and Next Steps

The session concluded with an invitation for attendees to explore fine-tuning in their own projects. Mark encouraged participants to collaborate and share their experiences. Columbia’s AI Community of Practice will continue to host these monthly sessions, offering a platform for exploring innovative AI applications across a wide range of fields.

You can access a recording of the AICoP in the News section here.

Columbia University’s AI Community of Practice, led by the Emerging Technologies team, kicked off its fall semester with a focus on the collaboration between academia and industry. This community provides a forum for faculty, staff, and industry partners to discuss AI and machine learning (ML) applications across teaching, research, and administrative areas. In this session, Anthropic was the featured guest, with a deep dive into their AI platform, Claude.

Key Features and Advantages of Claude for Enterprise

• Data Security: Anthropic emphasized the importance of AI safety, urging participants to avoid inputting confidential or protected health information (PHI) in the free version of Claude, as data may be used for training purposes. Claude’s enterprise version offers stronger safeguards, ensuring that data used within organizations remains secure.
• Custom Knowledge Base: Claude allows users to upload documents and other content to create project-specific knowledge bases. This enables tailored AI interactions that reflect the organization's needs, helping users access and process vast amounts of information efficiently.
• Expanded Context Windows: Claude’s enterprise version offers a large token limit, allowing users to work with complex documents, such as research papers or grant applications, without losing valuable context. This enhances its usefulness in academic settings where large volumes of data need to be processed.
• Integration with GitHub: While currently limited to GitHub, Anthropic is working on expanding integration capabilities with other platforms like Google Drive and Microsoft OneDrive. This allows developers and faculty to seamlessly access their code bases and other relevant documents without leaving the Claude environment.

Technical Approach

Claude leverages Constitutional AI, a safety-driven framework designed to prevent harmful or biased outputs, making it particularly well-suited for responsible AI usage. Claude's team continuously refines these safeguards to ensure that the model cannot be tricked into producing undesirable responses, a process known as "jail breaking."

Participants also discussed how Claude could assist with various practical use cases in academia, such as helping faculty and researchers apply for grants by analyzing research documents and providing suggestions based on the uploaded data.

Future Plans and Improvements

Anthropic is focused on enhancing Claude’s capabilities, including plans for internet search functionality and additional API integrations. These future developments aim to provide greater flexibility and usability for both research and administrative purposes. The company is also working on achieving HIPAA compliance to better support users from medical institutions.

With its commitment to safety and performance, Anthropic’s Claude is positioned to become a powerful tool in academia, helping institutions like Columbia University harness the potential of AI for a variety of use cases, from research to administrative tasks.

The session concluded with a commitment to ongoing collaboration, and a follow-up session was proposed to explore the API capabilities of Claude in more detail.

You can access a recording of the AICoP in the News section here.

Columbia University's AI Community of Practice, spearheaded by Columbia University Information Technology (CUIT), has embarked on a pioneering project to develop CU-GPT, an AI-driven interface designed to serve as a cost-effective and secure alternative to ChatGPT. This initiative aims to enhance the academic and administrative experience by integrating advanced AI capabilities into the university's infrastructure.

Key Advantages of CU-GPT

• Cost Efficiency: CU-GPT operates on a pay-as-you-go model, significantly reducing costs compared to ChatGPT's enterprise license. Departments can set daily usage limits, and users can monitor their real-time spending, ensuring transparent and manageable costs.
• Data Security: Running on Columbia's infrastructure, CU-GPT leverages an enterprise contract with OpenAI, ensuring robust data security measures. This setup protects the intellectual property and research data of users, addressing privacy concerns.
• Customizability and Flexibility: CU-GPT recognizes user roles and responsibilities, providing tailored access to specific GPT instances relevant to different departments and research needs. This ensures that users have access to pertinent AI tools and resources.
• Scalability and Performance: Utilizing AWS Lambda, CU-GPT offers scalable computing power, efficiently handling multiple user requests. This serverless architecture ensures cost-effectiveness by charging only for the compute time used, without maintaining continuous infrastructure.
• Enhanced User Experience: The platform maintains user chat history, allows for file uploads, and features an intuitive interface similar to ChatGPT, making it accessible and user-friendly.

Technical Implementation 

• Architecture: CU-GPT's architecture includes document vectorization, converting plain text into numerical embeddings that capture the semantic meaning of documents. This allows for more accurate retrieval based on the context rather than just keywords. The system uses cosine similarity algorithms to rank documents according to their relevance to user queries. The results are combined with a large language model to produce detailed summaries and recommendations.
• Scalability Considerations: To ensure high performance and accuracy, the CU-GPT team focuses on robust computing strategies. These include pre-building vector databases and exploring efficient computational methods, such as running large language models (LLMs) in-house to manage and reduce long-term costs. The AWS Lambda architecture supports scalable computing, handling high query volumes efficiently.

Future Developments

The CU-GPT team is continuously working on enhancing the platform’s capabilities, including potential support for image processing and expanded customization options. They are also pursuing HIPAA compliance to securely handle health-related data. As the project progresses, it aims to set new standards for AI integration in academic research and administrative processes, demonstrating Columbia University's commitment to leveraging advanced technology for the benefit of its community.

You can access a recording of the AICoP in the News section here.

Columbia Libraries/Information Services, the heart of the university's intellectual life, embarked on a journey to improve discovery processes using AI and large language models (LLMs), particularly focusing on the Columbia CLIO library search system. The system uses retrieval augmented generation (RAG) to enhance search capabilities, connecting an AI system to a database of documents and using vector databases to capture semantic meanings.

Key Advantages of AI-enhanced University Library Search System

• Specificity: ability to handle complex research topics and provide highly relevant documents by understanding the semantic meaning of the queries.
• Natural Language: allows for natural language queries, making it easier for users, especially students, to find relevant documents without needing specialized search syntax.
• Translation: ability to find and translate documents from different languages, expanding access to non-English documents.
• Feedback Loop: ability to refine search results based on user feedback, continuously improving the relevance of the documents provided.

Technical Implementation 

• Architecture: RAG begins with creating vector embeddings for all documents in a database so that they can be searched according to their semantic meaning. The user’s query is first used by the AI system/LLM to retrieve the most relevant documents from the databases by using their vector embeddings. The documents are included in the final prompt to the LLM, allowing the LLM to answer queries using its natural language capabilities and the source material from the database. The vector database can provide more contextually relevant search results compared to traditional keyword searches.
• Scalability considerations: When it comes to scalability, considerations include scaling the AI system to handle large query volumes while managing costs. One strategy is to pre-build vector databases and explore more efficient computational methods, like running LLMs in-house to spread out costs over time.

You can access a recording of the AICoP in the News section here.

This session provided a comprehensive understanding of the transformative effects of Attention technology in large language models (LLMs), exploring both its groundbreaking applications and the challenges it presents in terms of computational demands and potential ethical concerns. Attention is central to the functionality of LLMs, such as those used in GPT (Generative Pre-trained Transformer) architectures.

Overview of Attention Mechanism

• LLMs are based on a transformer architecture, heavily relying on a mechanism known as Attention to process language data.
• Attention helps the model determine which parts of the input data are relevant, which improves its ability to generate coherent and contextually appropriate responses. It is pivotal for the performance improvements seen in models like GPT.

Practical Applications and Implications

• Attention in models applies in practical applications from simple text generation to complex tasks like multimodal inputs (integrating text, image, and video).
• Computational demand of these models and their reliance on large, diverse datasets may not always be of high quality or free from bias.
• Conversation also touched on potential future advancements in AI and how attention-based models could revolutionize various fields by processing complex, multimodal data.

You can access a recording of the AICoP in the News section here.

The highly informative session demonstrated different ways to create custom bots using large language models (LLMs) and how to practically apply these concepts. These methods include no-code options, open-source tools, and more technical solutions such as retrieval-augmented generation (RAG) fine-tuning using custom datasets.

Key highlights included:

• Creating custom bots using ChatGPT Enterprise offers an accessible, no-code solution to tailor LLMs for specific needs.
• Running open-source LLMs on proprietary hardware or utilizing cloud computing platforms provides various options based on technical expertise and resource availability.
• RAG fine-tuning technique offers paths to significantly enhance the performance and accuracy of LLMs using custom datasets.
• Technical discussions centered around the importance of clean, well-prepared datasets for training and the potential for custom LLMs to be integrated into various applications and services.

The meeting discussed the practical aspects of using LLMs at different technical levels. The focus was on customization, data privacy, and finding a balance between model accuracy and resource investment. It highlighted the fast-paced development of LLM technology and tools and suggested that more user-friendly and effective solutions for fine-tuning and customization are likely to emerge soon.

You can access a recording of the AICoP in the News section here.

AICoP convened a session dedicated to the overview of ChatGPT Enterprise for Columbia University. The agenda encompasses an in-depth presentation of ChatGPT Enterprise, highlighting its features, enhanced security and privacy, and benefits of ChatGPT's custom GPTs to the Columbia University community. 

Columbia University ChatGPT Enterprise

• CUIT has finalized an enterprise-wide license agreement for ChatGPT, marking a significant step forward in incorporating AI tools into the university's toolkit.
• The process involved extensive reviews, including security, architecture, and legal considerations, to ensure data protection and compliance.

Security and Data Protection

• A 'walled garden' approach ensures high-level data protection, privacy, and encryption, with compliance with GDPR and HIPAA.
• User data remains private and is not shared externally or used for model fine-tuning without permission.

Features and Benefits

• ChatGPT offers advanced AI capabilities, including the most mature AI model with image creation, data analytics, and code interpretation features.
• Enterprise customers can create internal-only GPTs for specific business needs, departments, or proprietary datasets, without coding.
• The enterprise license ensures dedicated, reliable, and scalable access, distinguishing it from free or commercial versions.

Integration and Use Cases

• ChatGPT demonstrates the potential for integration into Columbia's systems, such as CourseWorks, to enhance educational tools and create personalized learning experiences.
• CUIT demoed various use cases and initiated discussions on API usage for broader application and customization possibilities, including building and sharing bots for specific departmental or research needs.

You can access a recording of the AICoP in the News section here.

Columbia Technology Ventures (CTV), the technology transfer arm of Columbia University, provided insights into the potential use of ChatGPT to enhance various office functions and automate tasks.

• Experiments and Projects: CTV showcased various AI-integrated projects, including the automation of mass email campaigns using a Python package, the on-the-fly drafting of legal language for licensing agreements, improving negotiation efficiency, and analyzing data tables to streamline internal business functions using ChatGPT (DAT GPT). One of the most significant findings was ChatGPT's proficiency in high-skill tasks such as drafting legal language and performing data analysis.

• Automation Potential: The exploration revealed a considerable potential for automating repetitive tasks, which could benefit those unfamiliar with coding. However, exporting non-textual files remains a challenge, albeit one expected to improve as technology advances.

• Learning and Future Applications: CTV is optimistic about further integrating ChatGPT into their workflows, emphasizing the need for continued experimentation and adaptation to leverage AI tools effectively. The session emphasized the importance of setting up system prompts and scope prompts to guide ChatGPT's interactions, enhancing its efficiency and relevance to specific tasks.

• Community Feedback and Interest: The exploration generated significant interest among the participants, with discussions on how to set up effective prompts for ChatGPT and the potential for its application in various projects.

You can access a recording of the AICoP in the News section here.