AICoP: Practical AI - Clear Prompts, Useful Context

Columbia AI Community of Practice Explores Prompt and Context Engineering

Columbia University’s AI Community of Practice hosted a webinar on how to move beyond basic prompt tricks into a deeper practice called context engineering. The event was led by John P. Martin and featured an in-depth presentation and demos by Spencer Ames of CUIT’s Emerging Technologies team.

What Are Prompt Engineering and Context Engineering?

Prompt engineering focuses on crafting clear instructions inside a single message to a large language model. It controls what the model sees as the “prompt.”

Context engineering widens that scope. It means designing everything the model sees in its context window:

• Instructions
• Examples
• Sources and citations
• Data and documents
• Conversation history
• Tools and outputs from earlier steps

Prompt engineering is no longer “dead.” It is one piece inside context engineering rather than the whole job.

Why It Matters

The speakers showed how good context can change AI from “hit or miss” to a reliable partner in real work:

• Reduces hallucinations by grounding the model in curated sources
• Saves time on teaching the model the same things again and again
• Supports repeatable workflows for teaching, research, and administration
• Gives users a way to apply policy, privacy, and style rules directly in the prompt

Key Concepts

Attention as a scarce resource

• The context window is limited.
• Too little context and the model guesses.
• Too much context and the model gets distracted.
• The goal: smallest set of inputs that raises the odds of a good answer.

Memory and “starting fresh”

Spencer explained two types of memory:

• Short-term (inside one chat): the model remembers earlier messages in the same conversation.
• Long-term (across chats): system-level memory where it can remember things like your role or long projects over time.

To avoid bloated chats, he shared a prompt the team uses often:

“Give me a summary of this chat and a master prompt to start a new chat.”

This keeps the useful parts and resets the rest.

ROSES prompting template

The team shared the ROSES framework as a practical pattern:

• Role – Who is the AI supposed to be?
• Objective – What outcome do you want?
• Scenario – What background or context matters?
• Expected Solution – What form should the answer take? Length, tone, format.
• Steps – What process should the model follow? Scan sources, draft, then refine, etc.

This structure makes prompts easier for both humans and models to read.

Guardrails and delimitation

Spencer stressed clear behavioral rules, such as:

• “Always be professional.”
• “Always admit limitations.”
• “Never break departmental policy.”
• “Always cite your sources.”

He also showed how delimiters improve results:

• XML-style tags
• Brackets
• Markdown sections

These boundaries help the model separate instructions from data.

Demonstrations

During the session, Spencer walked through several live demos:

• A weak prompt for a climate data science course description that led to hallucinated tools and fake course details
• A revised version using ROSES and detailed context about tools (Pandas, Matplotlib) and a final project on New York City’s urban heat island effect
• A multi-source setup where he uploaded several articles on prompt and context engineering and asked the model to design an agenda and talking points for this very presentation

He then showed a separate chat where the task changed from “prepare a presentation” to “draft an article,” and he trimmed the sources down to only what that article needed. That change illustrated how context engineering means choosing what not to include as much as what to include.

Practical Use Cases

Attendees saw how these methods apply across roles:

• Faculty
  • Design better course descriptions and assignments
  • Build repeatable prompts for quizzes, rubrics, and feedback
• Researchers
  • Structure literature review prompts
  • Keep sources explicit and cited in outputs
• Administrators
  • Draft sensitive emails with clear guardrails
  • Summarize reports and meetings with less risk of hallucination

Spencer also encouraged using multiple LLMs in parallel—pasting the same prompt into different tools to compare strengths for writing, coding, or research. John noted that CUIT plans for CU Chat to support more models in one interface over time.

Security and Governance

The Q&A focused heavily on privacy, copyright, and platform choices:

• Consumer AI tools often train on user data.
• Columbia’s contracts with providers like OpenAI and Google prevent that for approved services.
• Sensitive or copyrighted content should be handled only inside secure, university-managed tools.
• Columbia Libraries described work on MCP (Model Context Protocol) servers to connect licensed content, such as Academic Commons and archival finding aids, to LLMs in a controlled way.

Participants were encouraged to treat fair use conservatively and consult library and legal guidance when working with licensed or copyrighted material.

Support and Training

The Emerging Technologies team offers several support options:

• Weekly AI office hours on Fridays (10–11 a.m.) covering ChatGPT, CU Chat, Gemini, and NotebookLM
• Recorded AI Community of Practice sessions posted on the CUIT Emerging Technologies site
• A public prompt library and articles on:
  • Intro prompt engineering
  • Advanced prompt engineering
  • Context engineering
  • Evaluation of AI outputs

Faculty and staff can register for sessions and access resources through the Emerging Technologies website.

Takeaway

The webinar framed context engineering as the next step for anyone who wants to get real work done with AI. By treating attention as scarce, structuring prompts with ROSES, setting strong guardrails, and picking the right sources, Columbia users can make their models more reliable partners across teaching, research, and administration.

Try It Yourself

During the session, the speakers shared prompts that anyone at Columbia can test right away:

• Start a fresh project from a long chat:
  • “Give me a summary of this chat and a master prompt to start a new chat.”
• Ask the model to help you write a better prompt:
  • “Here is my draft prompt and my goal. Rewrite this prompt using role, objective, scenario, expected solution, and steps. Then list three questions you need me to answer to improve it.”
• Add guardrails for an important email:
  • “You are helping me draft an email to my department. Always be professional. Always admit limitations. Never break Columbia policy. Ask one clarification question before drafting.”
     

Faculty and staff interested in applying these practices in CU Chat or in custom AI agents can contact the Emerging Technologies team at [email protected].

Useful Links:

Slide deck

How to talk to AIs - Prompt engineering 101

How to talk to AIs - Context Engineering 101

How to talk to AIs - Advanced Context Engineering

Prompt library article