Building Trustworthy AI: Early Experiments in Conversational Search at Macmillan

For large, content-rich organisations, the rise of generative AI presents immense opportunity and significant risk. For an organisation like Macmillan, where our cancer information for the public must be accurate and trustworthy, simply plugging our data into an LLM is a high-risk strategy. How do you innovate responsibly when the stakes are this high? Our answer is to experiment, learn, and keep a human in the loop. We need to understand how our users will actually interact with this new technology. To do this, we're starting small and safe. We've begun with several distinct experiments: using an LLM to help people tailor searches for Coffee Morning recipes, and creating a semantic, conversational search across our cancer content.

In this session, we'll share the honest story of our first steps, including user feedback and live demos of our current progress. We'll detail our tech-agnostic experimental approach, designed to search across any source of structured data across our sprawling estate, and explain how you can do the same. More importantly, we'll share what we're learning about user behaviour and the practical challenges of building a trustworthy, AI-powered conversational layer.