AICoP: AI for Justice - Building Chatbots for Legal Aid

Columbia AI Community of Practice Showcases Legal Aid Chatbot for Housing Justice

In its final session before the summer break, Columbia University’s AI Community of Practice—led by the Emerging Technologies team—highlighted a powerful example of AI applied to real-world challenges: a custom legal aid chatbot built by Columbia Law School’s “Lawyering in the Digital Age” clinic.

The featured presenters, Professor Conrad Johnson and Basem Aly, walked attendees through the development and deployment of an AI-powered chatbot used by the Legal Aid Society to help paralegals support tenants facing eviction in New York City.

Why This Matters

NYC sees over 127,000 eviction cases a year. Legal Aid’s Housing Justice Helpline is staffed by just 15 paralegals and a few supervisors. Demand far outweighs supply.

The chatbot acts as a real-time legal assistant, helping paralegals quickly answer complex housing questions without having to wait on supervisors or comb through long internal wikis. This frees up time, reduces frustration, and increases access to timely legal help.

Inside the Chatbot: Design and Use

The chatbot is built on GPT-4 using Retrieval-Augmented Generation (RAG). It doesn’t search the web. It only pulls from a vetted knowledge base created by Legal Aid and refined with input from clinic students.

Key features include:

• Closed-Loop RAG: Responses come only from internal docs, reducing hallucinations.
• Built-in Escalation: Red-flag terms (e.g. domestic violence) trigger supervisor review.
• Translation + Email: One-click translation into Spanish or formatting into a client email.
• Chain-of-Thought: The bot supports follow-up questions and multi-part legal issues.

Performance and Evaluation

• Students shadowed paralegals, tested the bot, and stress-tested it with edge cases.
• A 10-question rubric evaluated the tool’s reliability—ranging from simple to highly complex legal scenarios.
• Supervising attorneys review interactions weekly, monthly, and quarterly for quality control.

Legal Aid reported faster calls, clearer responses, and fewer bottlenecks. The chatbot has now been adopted as a core tool—and won $1.1 million from the Robin Hood Foundation’s AI Poverty Challenge.

Organizational Impact

This project went beyond just building a tool:

• It changed Legal Aid’s official AI policy—from blanket bans to cautious adoption.
• It helped staff at all levels (from paralegals to supervisors) develop AI literacy.
• It exposed students to ethical, responsible AI design in a high-stakes legal context.

Looking Ahead

Legal Aid plans to replicate the model for other service lines and explore integration with Microsoft Copilot. They're also considering hiring internal AI engineers to maintain and expand their systems.

This project shows how AI can meet real human needs—when designed with transparency, constraints, and users in mind.

Faculty and staff who want to explore similar AI applications in service-based environments can contact the Emerging Technologies team at [email protected].