A Discussion on How to Design a Generative AI Product
The brief exploration of a (kinda) new design material: Part 3
As with any product material, designing with generative artificial intelligence (genAI) comes with its own set of specific needs and considerations. Understanding these is crucial for leveraging artificial intelligence (AI) to its fullest potential while avoiding common pitfalls.
One of the first decisions we face is with what kind of genAI we work, if it’s custom trained or if we build on top of an existing one, and if which one. The most likely is the latter option, and while we of course can (and should) define requirements, we should most likely not define the actual used model. As with every decision about the tech stack, that’s a decision that should be done at least in collaboration with the people whose domain this is. Thus for AI, we should discuss that with the data scientists or AI engineers.
With the emergence of intent-based outcome specifications, we need to think about a completely new idea of how users interact. Unlike traditional software, where inputs are explicit and predefined, AI systems can operate on more abstract and intent-driven inputs. This means we must carefully consider how users will interact with the product, ensuring that the system can understand and appropriately respond to their intentions.
The data we have at our disposal is the most critical question to answer, that basically is the resource we can work with. The quality and nature of the data directly influence the possibilities, performance, and fairness of AI systems. We can only work with the data we have, additionally, biased or unrepresentative data can lead to skewed results and perpetuate existing inequalities. Therefore, it is essential to curate and preprocess data carefully, ensuring it is diverse, accurate, and relevant.
And, of course, the AI’s capabilities must align with the product’s intended purpose. This alignment ensures that the AI adds value without complicating the user experience or diluting the product’s core offerings. Understanding user expectations from the interactions with the AI helps in designing interfaces and interactions that feel intuitive and helpful rather than intrusive or overwhelming. And yes, this applies to every medium and every product, but with AI it is particularly important to take a close look here because our product and its interface themself suddenly become a dedicated agent that we have to take into account.
Disclaimer: I’m not an AI expert by trade, just a fellow designer trying to keep up with the world we live in.
Challenges
Let’s use a support communication tool for small and medium-sized enterprises (SMEs) as an example case study; how to improve their communication, especially their direct dialog with their customers, and thus the customer experience (CX) they offer? They often face significant communication challenges that can hinder their growth and operational efficiency. These challenges primarily arise from limited resources and the lack of robust systems that larger corporations typically possess. The main issues include:
- Resource Constraints: SMEs often operate with fewer staff members and tighter budgets, making it difficult to manage extensive communication loads both internally and with customers. This can lead to delays, missed messages, and decreased customer satisfaction.
- Scalability: As businesses grow, the communication systems employed by SMEs often fail to scale effectively, leading to bottlenecks and inefficiencies that can stifle growth.
- Integration Issues: Many SMEs use a variety of communication tools that are not fully integrated. This can result in disjointed communication flows, data silos, and increased chances of errors.
- Customer Engagement: Engaging with customers effectively and consistently is crucial for SMEs. However, without the right tools, personalizing communication and tracking customer interactions can be challenging.
- Compliance and Security: Ensuring communication compliance with regulations and maintaining security standards is vital, yet challenging, for many SMEs due to the complexities and costs involved.
A bit of Dreaming
Let’s speculate, let’s discuss what a dream AI support could bring to the table to improve the CX when we talk about the communication of SMEs, in this case, particularly in the customer support — meet Alex:
After a long day, Alex notices his smart home system’s motion sensor isn’t working. Frustrated, he contacts customer support through the company’s messenger app.
“Hi, Alex! I am Mona, your AI support agent. How can I assist you today?” the AI greets him. Alex types, “The motion sensor in my living room isn’t working. It doesn’t seem to detect any movement.”
The AI quickly analyzes his message and his customer profile. “I’m sorry to hear about the trouble. Here is an article that might help: [Troubleshooting Motion Sensors].” Alex follows the guide but the sensor remains unresponsive. He informs the AI assistant, “I’ve tried all the steps, but it’s still not working.”
“I see,” the AI responds. “Let me escalate this to our technical support team. Please hold on for a moment while I connect you with an expert. That might roughly take 5 minutes”
About 4 minutes later Sarah joins the chat, “Hi Alex, this is Sarah from technical support. I see you’ve been having trouble with your motion sensor. Let’s see how we can fix this.”
She asks Alex a few questions and then guides him through updating the sensor’s firmware. The update works. “The sensor is detecting movement again. Thank you!” Alex types. “I’m glad we could resolve the issue,” Sarah replies.
The AI assistant offers some maintenance tips and asks for feedback for the whole case. Alex appreciates the proactive advice and provides positive feedback. He feels satisfied. The AI and human support resolved his issue quickly and effectively, reinforcing his trust in the company.
Okay, that sounds nice. But to get the full picture, we need to also see Sarah’s side:
Sarah receives a notification from the AI system about a new high-priority case. The AI assistant, Mona, has already provided a detailed summary: Alex, a customer, is having issues with his smart home system’s motion sensor. Mona has attempted basic troubleshooting steps, but the issue persists.
She takes a moment to review Alex’s profile and the steps he has already taken. This preparation allows her to quickly understand the problem and think about possible solutions. She is not a native English speaker, when she joins the chat Mona supports her and translates her input: “Hi Alex, this is Sarah from technical support. I see you’ve been having trouble with your motion sensor. Let’s see how we can fix this.”
With a few targeted questions, she understands the issue and suspects a firmware update might resolve the problem. Sarah guides Alex through the firmware update process. Once completed, the sensor starts working again. “The sensor is detecting movement again. Thank you!” Alex types.
“I’m glad we could resolve the issue,” Sarah replies. Mona steps in to offer maintenance tips and collect feedback. Alex provides positive feedback, feeling satisfied with the support.
Implications
For Alex the AI provided immediate support and escalated efficiently to a human counterpart while giving him an understanding of how long this might take. Thus his experience was smooth and coherent; pains were relieved: long waiting or unknown response times, potentially a lot of forth and back.
On Sarah's side, the AI did the first-level support and gave her a structured summary to enable her to hit the ground running with this support case. She was also supported during the handling by extended support tools like the translation handling and the AI took over again when the case was solved. With that she was able to handle the case as quickly as possible while providing quality support; pains relieved: the obstacle of language, effort to get familiar with the case, and automated handling of the aftercare.
Reality Check
Much of that is technically more or less possible today.
The automation of first-level support could be achieved using a Retrieval-Augmented Generation (RAG) model, which accesses a knowledge base for facts rather than generating the whole answer. This model would enable contextual search, allowing customers to ask questions in natural language and receive the most relevant articles, tutorials, or FAQs. For more complex issues, the AI could potentially provide interactive, step-by-step guidance, helping customers resolve problems independently. However, trust can become an issue as it’s seemingly unclear what the AI is doing and what is not.
By analyzing incoming messages, AI could determine the urgency and sentiment, assigning priority levels accordingly. While this is doable, it’s crucial to remember that AI identifies patterns but doesn’t understand meaning. The effectiveness of such systems would largely depend on the amount of data available to fine-tune the model. Similarly, skill-based routing could direct cases to the most suitable support agents based on their expertise and workload. This would require detailed employee profiles, making it another data-dependent feature. Providing feedback to users on expected response times could also be achievable, similar to practices in telephone support, again relying on sufficient data.
Speaking of analysis; AI could create and maintain detailed profiles for each customer, continuously updated with preferences, behaviors, and interaction history. When a customer initiates a chat, the AI assistant could use these profiles to provide personalized greetings, responses, and support, ensuring a tailored experience. This would also hinge on the data available and would benefit greatly from access to sources not directly linked to the conversation.
Translation services might be integrated using tools like DeepL, enabling agents to communicate effectively in multiple languages. Additionally, AI could convert bullet points into coherent text with an appropriate tone of voice, and perform grammar checks using tools like Grammarly. It should be noted that both are possible with bigger genAI models, like GPT-4, without the need for a specialized genAI, but the quality of these services drastically improves using optimized environments. Even then, these capabilities should be fine-tuned for better performance, ensuring that the support provided is accurate and professional.
Finally, Systems can be automated to collect feedback from customers on their support experience, and while AI is not strictly necessary for this task, it can significantly streamline the process and enhance the overall interaction.
Thus, the main challenges are not technical but come down to data availability, competence in how to use the data and trust in AI systems. SMEs often struggle to provide the necessary amount of data to create effective AI solutions. Moreover, they also typically lack the expertise to fine-tune AI models, and building trust in the system is another critical issue.
Solutions
So, how can we mitigate these issues and improve both the CX and the experience for support agents, while building calibrated trust and not overwhelm them with the data we need?
Inspiration and Research
Any proper design process starts with thorough research and inspiration.
Birdeye offers a lot of what we discuss above, it uses AI to improve customer experience by supporting the human agent, automating review answers, and feedback collection. Their platform employs AI to analyze customer sentiment, generate insights, and provide businesses with actionable data and they even offer some chatbot functionality. However, a lot of these features remain general and thus on a surface level without in-depth fine-tuning and personalization to a specific company.
Intercom offers features like chatbots, automated responses, and machine learning-powered message routing, and helps businesses provide timely and relevant support to their customers. Intercom’s AI capabilities enable quick resolution of common queries, allowing human agents to focus on more complex issues. The setup and complexity of these systems are more tailored toward big enterprise customers and thus do not fit the needs of most SMEs.
Finally, also worth a look: Notion has integrated AI into its platform to streamline workflows and enhance productivity. By incorporating AI-driven features such as task automation, smart suggestions, and personalized templates, Notion improves user experience and helps users manage their projects more efficiently. Studying Notion’s approach can provide valuable insights into how AI can be seamlessly integrated into productivity tools to offer significant user benefits.
But this is of course just (very) an exemplary elaboration, other services to investigate include (but are by far not limited to) uberall, redeye, reply.io, planify x, userlike, or even slack.
Proposal
To reiterate: Our primary objective is to enhance the CX through the integration of AI in customer support. To achieve this, we need to emphasize how AI can improve efficiency, personalization, and overall satisfaction for both customers and support agents. Additionally, we must provide an accessible way to fine-tune the system to our needs and establish a path to calibrated trust.
As a foundational step, we could implement easily achievable goals that provide immediate value. In this scenario, we will make an initial fine-tuning with a self-exploration of the company and prioritize basic functionalities that enhance the support process. This includes a summary feature that allows AI to generate concise summaries of customer issues for quick reference by support agents. Additionally, we will implement an AI button that offers several key features like AI-powered translation, tone and voice adjustment, a grammar check, converting bullet points into well-structured text, and a fact-check against a vector database initialized by an FAQ. The AI will provide warnings if the information is questionable or unknown and can be configured to auto-learn new valid data and thus be fine-tuned on the go. These features will help us build trust in AI’s capability and start to build the foundation of data we can build upon without the need for deeper knowledge or training.
Building on the data accumulated, we will then introduce direct response suggestions in the second iteration. This will enable AI to provide agents with quick reply options, helping them respond more efficiently and effectively. With the enriched database, the AI will have a deeper understanding of common issues and customer preferences, allowing it to generate more relevant and precise responses. This enhanced capability will streamline the support process and improve overall customer satisfaction. Importantly, it’s still in the hands of a human support agent. While AI offers suggestions, the final decision and control remain with the human agents, ensuring no autonomous AI handling. Additionally, this gives us the possibility to further fine-tune our database.
With this foundation, we have built trust in the system and the data to maintain valuable support. We can now move to the third iteration, developing and deploying fully automated first-level support, where AI independently handles initial customer interactions and escalates to human agents only when necessary. This approach ensures efficient handling of routine inquiries while reserving human expertise for more complex issues, further enhancing the overall support experience.
Outside of our Bubble
While it’s crucial to consider the benefits and applications of AI in customer support, it’s equally important to acknowledge the broader technical framework and implementation challenges that lie outside our immediate expertise.
Building a robust AI product requires a combination of high-quality data, powerful computing infrastructure or third-party APIs, and a focus on robustness, security, and ethical considerations.
Implementing AI systems comes with its own set of challenges. These range from data integration issues to the technical intricacies of deploying AI services at scale. Addressing these challenges requires collaboration with professionals who possess the necessary technical expertise. As with many technological advancements, we don’t need to, and probably even can’t, know how to do everything ourselves. For instance, we don’t need to understand the exact process of fine-tuning an AI model. However, we do need to grasp the implications of these processes and how to effectively communicate our goals and requirements to the professionals responsible for implementation. This includes developers who are responsible for integrating AI solutions into existing systems and ensuring they function smoothly. It also includes data scientists and AI engineers who handle the complexities of AI, from data preparation to model training and fine-tuning. These professionals need clear descriptions of the desired outcomes and the business logic behind AI applications to tailor their efforts effectively.
Next Steps
As we move forward, thorough testing and a tailored product strategy are essential. By addressing both head-on, we can ensure our AI solutions meet and exceed user expectations, delivering a seamless customer experience.
Testing
UX Testing a genAI product presents unique challenges due to its dynamic nature. Unlike traditional software, where testing can be straightforward with predefined inputs and expected outputs, AI systems can learn and evolve and are arguably an agent on their own, making their behavior less predictable. It requires a thoughtful and comprehensive approach that combines traditional UX testing methods with specialized techniques tailored to AI systems. By defining clear objectives, recruiting diverse participants, designing realistic scenarios, leveraging AI-powered tools, and continuously iterating based on user feedback, we can ensure that your AI product delivers a seamless and satisfying user experience. For more detailed information on specific testing methods and tools, we can refer to sources such as HeadSpin’s guide on leveraging generative AI in continuous testing, Applause’s insights on generative AI testing, and UX Collective’s Ben’s best practices for designing AI products.
Further Iterations
You might have noticed that we have not yet addressed the whole AI contact data thing; the implementation of profiles for each customer, which are used to customize the experience. This would be a logical next step, particularly now that we have a better understanding of which data is important relative to our business and the customers we interact with and thus build upon the foundation we just discussed.
Epilogue
Now that we have our proposed design, let’s consider how it might impact the companies that implement it. Fast customer support is a critical component of a positive CX. It enhances customer satisfaction, builds trust, and provides a competitive edge, ultimately leading to increased sales and customer loyalty. Our proposal also adds tangible value to the business, reducing operative efforts and improving the experience on the side of the support agent.
However, it is important to acknowledge the potential negative aspects of using AI in customer support. AI systems can sometimes lack the empathy and personal touch that human agents provide, potentially leading to customer frustration in complex or sensitive situations. There are also concerns about data privacy and the security of customer information, as well as the need for continuous monitoring and updating to ensure AI systems remain accurate and effective. Addressing these challenges is essential to maximize the benefits of AI while minimizing any negative impacts.
This article is part of a three-part series in which I look at what AI is, what it can be used for, and how it can be used as a design material:
