How to reduce hallucinations when using LLMs for data analysis tasks?

I'm building a system where users ask questions about their data in natural language, and an LLM generates SQL queries and interprets results.

The problem: The LLM often "hallucinates" insights that aren't supported by the actual data.

Example:

• User asks: "What's our top-selling product?"
• LLM correctly generates SQL
• But then adds: "This is likely due to the recent marketing campaign" (we had no such campaign)

What I've tried:

1. Strict system prompts saying "only state facts from the data"
2. Few-shot examples of good vs bad responses
3. Temperature = 0 for deterministic output

Still getting hallucinations. How do production data analysis tools handle this? Should I use a separate verification step?