Prompting ‘high-quality’ conversations with an AI

Company MindGym

Team Product Designers, User Researcher, Product manager, Engineering, Behavioural Science and Data Science teams

Role Content & UX designer

Zoomed in grid of face emojis on a digital application

Challenge

Differentiate our AI assistant from ChatGPT

We designed an MVP for an AI Slack application (using an LLM) to help people with challenges they were facing at work. My role was to map the user journey and develop character and conversation design principles.

We led several stakeholder workshops to understand how the opportunities and risks of the product aligned with the needs of our customers. Based on this, we formed a set of principles for leveraging our IP within a conversational AI application.

Process

Explore and define ‘high-quality’ conversations

I carried out extensive prompt engineering experiments using ChatGPT to test the quality of conversations it was capable of. I developed core prompts to instruct the AI to respond in our tone of voice and style and in ways that differentiated its value from ChatGPT. This process helped define standards for ‘high-quality’ conversations. I documented explorations and reflections to share across design, engineering, data science, and behavioural science teams.

Crafting prompts for consistency

Working alongside data scientists, we developed a set of prompts/instructions for the AI that generated relatively stable and consistent responses. For example, specific instruction around hyperbole, metaphors, short sentences, and providing validating responses.

Table with columns 'interaction style', 'prompt' and 'learnings and reflections' for evaluating consistency in ChatGPT responses

Using natural conversation starters

From mapping user flows, I identified ‘dead ends’ in the conversation paths and opportunities for re-engagement. We developed a concept of ‘conversation starters’ as a more natural and customer-centric exchange. The aim was to offer value while helping customers understand the functionality of the product.

Example AI conversation starter in a Slack application about body language in meetings

Content patterns with Slack Blockkit templates

I created editable content patterns for different types of responses using Slack Blockkit templates in partnership with a UI designer. This simple code was documented for the data science team to allow them to format any new content changes efficiently while maintaining consistency in the designs.

Example Slack block kit user interface template with customisable sections

Biggest challenge

Keeping ethics in the conversation

Raising ethical issues can be challenging in AI product development, as the complexity of solutions can be a barrier to rapid prototyping. Nevertheless, I understood our responsibility to do the best we could. I leaned on the wisdom of existing principles for responsible AI development to facilitate open discussions within the product team. For example, I recommended the AI not use ‘emojis’ to avoid biases associated with anthropomorphism and advocated for transparency in the limitations of the AI.