Business leaders don't need another talk-to-docs demo. They're looking for automated workflows that scale without relying on IT.

This week, I had the pleasure of diving into a rock-and-roll week of workshops, courtesy of our IBM Transformation office. We combined AI & GenAI education, use case ideation, and business case charter building — all for one of our major telco clients.

Key insights from discussions with directors, VPs, and C-suite participants:

1. Gen AI Playgrounds Are Insufficient

Business leaders have already explored tools like ChatGPT and Gemini — they don't need another talk-to-documents demo. They're looking for automated workflows that can scale across the organization without relying on IT. No-Code AI has been here for half a decade, but it will accelerate with Gen AI.

2. Scaling Challenges Persist

This year, I've seen multiple prompt-engineering-based call classification use cases fail to achieve strong F1 scores. More awareness around advanced techniques like LLM-as-judge and chain-of-thought (CoT) reasoning is needed. We should also reframe fine-tuning — costs have significantly decreased (e.g., OpenAI's GPT-4o mini, SLMs), and tools like Snorkel AI simplify the labeling process.

3. Where Are the Growth Use Cases?

Developing growth-focused use cases remains more challenging than cost-saving ones. It's critical to push the boundaries of creativity within teams to close this gap. The technology is there.

4. Scrutiny Must Be Universal

A recognizable brand doesn't guarantee a superior LLM implementation. Companies that were hesitant about AI just two years ago are now integrating Gen AI into their product offerings. Rigorous testing for accuracy and consistency is vital, regardless of the brand or market position.