AI: Community of Practice

Columbia Libraries/Information Services, the heart of the university's intellectual life, embarked on a journey to improve discovery processes using AI and large language models (LLMs), particularly focusing on the Columbia CLIO library search system. The system uses retrieval augmented generation (RAG) to enhance search capabilities, connecting an AI system to a database of documents and using vector databases to capture semantic meanings.

Key Advantages of AI-enhanced University Library Search System

• Specificity: ability to handle complex research topics and provide highly relevant documents by understanding the semantic meaning of the queries.
• Natural Language: allows for natural language queries, making it easier for users, especially students, to find relevant documents without needing specialized search syntax.
• Translation: ability to find and translate documents from different languages, expanding access to non-English documents.
• Feedback Loop: ability to refine search results based on user feedback, continuously improving the relevance of the documents provided.

Technical Implementation 

• Architecture: RAG begins with creating vector embeddings for all documents in a database so that they can be searched according to their semantic meaning. The user’s query is first used by the AI system/LLM to retrieve the most relevant documents from the databases by using their vector embeddings. The documents are included in the final prompt to the LLM, allowing the LLM to answer queries using its natural language capabilities and the source material from the database. The vector database can provide more contextually relevant search results compared to traditional keyword searches.
• Scalability considerations: When it comes to scalability, considerations include scaling the AI system to handle large query volumes while managing costs. One strategy is to pre-build vector databases and explore more efficient computational methods, like running LLMs in-house to spread out costs over time.

This session provided a comprehensive understanding of the transformative effects of Attention technology in large language models (LLMs), exploring both its groundbreaking applications and the challenges it presents in terms of computational demands and potential ethical concerns. Attention is central to the functionality of LLMs, such as those used in GPT (Generative Pre-trained Transformer) architectures.

Overview of Attention Mechanism

• LLMs are based on a transformer architecture, heavily relying on a mechanism known as Attention to process language data.
• Attention helps the model determine which parts of the input data are relevant, which improves its ability to generate coherent and contextually appropriate responses. It is pivotal for the performance improvements seen in models like GPT.

Practical Applications and Implications

• Attention in models applies in practical applications from simple text generation to complex tasks like multimodal inputs (integrating text, image, and video).
• Computational demand of these models and their reliance on large, diverse datasets may not always be of high quality or free from bias.
• Conversation also touched on potential future advancements in AI and how attention-based models could revolutionize various fields by processing complex, multimodal data.

The highly informative session demonstrated different ways to create custom bots using large language models (LLMs) and how to practically apply these concepts. These methods include no-code options, open-source tools, and more technical solutions such as retrieval-augmented generation (RAG) fine-tuning using custom datasets.

Key highlights included:

• Creating custom bots using ChatGPT Enterprise offers an accessible, no-code solution to tailor LLMs for specific needs.
• Running open-source LLMs on proprietary hardware or utilizing cloud computing platforms provides various options based on technical expertise and resource availability.
• RAG fine-tuning technique offers paths to significantly enhance the performance and accuracy of LLMs using custom datasets.
• Technical discussions centered around the importance of clean, well-prepared datasets for training and the potential for custom LLMs to be integrated into various applications and services.

The meeting discussed the practical aspects of using LLMs at different technical levels. The focus was on customization, data privacy, and finding a balance between model accuracy and resource investment. It highlighted the fast-paced development of LLM technology and tools and suggested that more user-friendly and effective solutions for fine-tuning and customization are likely to emerge soon.

AICoP convened a session dedicated to the overview of ChatGPT Enterprise for Columbia University. The agenda encompasses an in-depth presentation of ChatGPT Enterprise, highlighting its features, enhanced security and privacy, and benefits of ChatGPT's custom GPTs to the Columbia University community. 

Columbia University ChatGPT Enterprise

• CUIT has finalized an enterprise-wide license agreement for ChatGPT, marking a significant step forward in incorporating AI tools into the university's toolkit.
• The process involved extensive reviews, including security, architecture, and legal considerations, to ensure data protection and compliance.

Security and Data Protection

• A 'walled garden' approach ensures high-level data protection, privacy, and encryption, with compliance with GDPR and HIPAA.
• User data remains private and is not shared externally or used for model fine-tuning without permission.

Features and Benefits

• ChatGPT offers advanced AI capabilities, including the most mature AI model with image creation, data analytics, and code interpretation features.
• Enterprise customers can create internal-only GPTs for specific business needs, departments, or proprietary datasets, without coding.
• The enterprise license ensures dedicated, reliable, and scalable access, distinguishing it from free or commercial versions.

Integration and Use Cases

• ChatGPT demonstrates the potential for integration into Columbia's systems, such as CourseWorks, to enhance educational tools and create personalized learning experiences.
• CUIT demoed various use cases and initiated discussions on API usage for broader application and customization possibilities, including building and sharing bots for specific departmental or research needs.

Columbia Technology Ventures (CTV), the technology transfer arm of Columbia University, provided insights into the potential use of ChatGPT to enhance various office functions and automate tasks.

• Experiments and Projects: CTV showcased various AI-integrated projects, including the automation of mass email campaigns using a Python package, the on-the-fly drafting of legal language for licensing agreements, improving negotiation efficiency, and analyzing data tables to streamline internal business functions using ChatGPT (DAT GPT). One of the most significant findings was ChatGPT's proficiency in high-skill tasks such as drafting legal language and performing data analysis.

• Automation Potential: The exploration revealed a considerable potential for automating repetitive tasks, which could benefit those unfamiliar with coding. However, exporting non-textual files remains a challenge, albeit one expected to improve as technology advances.

• Learning and Future Applications: CTV is optimistic about further integrating ChatGPT into their workflows, emphasizing the need for continued experimentation and adaptation to leverage AI tools effectively. The session emphasized the importance of setting up system prompts and scope prompts to guide ChatGPT's interactions, enhancing its efficiency and relevance to specific tasks.

• Community Feedback and Interest: The exploration generated significant interest among the participants, with discussions on how to set up effective prompts for ChatGPT and the potential for its application in various projects.