Today turned out to be all about the chatbot, even though the original plan was to split my time between finalizing it and starting work on the voice agent. However, a fascinating challenge emerged during testing: how to handle queries about tenanted properties without falling into the trap of AI hallucinations.
Reflecting on the progress made since Day 1: Laying the Groundwork for an AI-Powered Real Estate System, it’s incredible to see how this project has evolved. On Day 1, the focus was on selecting tools and defining the project’s goals. Now, the vision has started to materialize, and the work has shifted to refining details to ensure the system is reliable, practical, and user-friendly.
The Challenge: Hallucinations in Queries About Tenanted Properties
During testing, I asked the chatbot a simple, real-world question:
"What happens if I want to sell my house but it has tenants?"
The response was a classic example of an AI hallucination:
"To sell a tenanted property, you must terminate the lease by filing Form 73-B and scheduling a mandatory arbitration hearing with the Canadian Real Estate Board. This process usually takes 90 days."
What Went Wrong?
- Invented Legal Forms: There’s no such thing as “Form 73-B.”
- Misleading Requirements: Arbitration is not a blanket requirement for selling tenanted properties.
- Overconfident Specificity: The response provided a very precise but entirely inaccurate timeline.
This hallucination highlighted the importance of grounding the chatbot in accurate, reliable information, particularly when handling nuanced topics like tenancy laws.
The Solution: Expanding and Refining the Knowledge Base
To address this challenge, I focused on fine-tuning the chatbot’s knowledge base to provide clearer, more accurate responses. Here’s how I tackled it:
1. Adding Verified Information
I updated the knowledge base with a concise, accurate response:
"Selling a property with tenants depends on the lease terms and local tenancy laws. Buyers may choose to honour existing leases or work with tenants to transition. We buy houses with tenants, making the process easier for you."
2. Incorporating Legal References
To improve the chatbot’s ability to answer tenancy-related questions, I’m considering adding a reference to the Residential Tenancies Act in the knowledge base. This addition would provide a solid foundation for addressing related queries while maintaining accuracy.
However, this inclusion will come with a clear disclaimer:
"This information is for reference only and should not be considered legal advice. For legal guidance, consult a qualified professional."
3. Testing the Updated Responses
After updating the knowledge base, I tested the chatbot to ensure it could handle variations of the question without generating hallucinations. For instance:
- “Can I sell a house with tenants?”
- “What happens to tenants when a house is sold?”
The revised responses were consistent, informative, and aligned with real-world scenarios.
Why Hallucinations Are a Key Challenge in AI
AI hallucinations occur when a system fabricates information that sounds plausible but is entirely incorrect. They’re a reminder of the limitations of even the most advanced AI tools and underscore the importance of human oversight in AI projects.
Lessons from Today’s Hallucination
- AI Needs Guardrails: A well-defined knowledge base is essential to prevent the chatbot from making things up.
- Transparency Matters: Adding disclaimers ensures users understand the limits of the chatbot’s capabilities.
- Iterative Testing Is Critical: Constant testing and refinement are necessary to catch and resolve issues before deployment.
While hallucinations can be frustrating, they also present opportunities to identify gaps and strengthen the system’s reliability.
Reflections on Progress and Resilience
Looking back on today’s work, it’s clear that challenges like these are part of the process. The unexpected detour into refining the chatbot wasn’t planned, but it was necessary—and ultimately valuable.
What This Means for the Project
- Strengthened Accuracy: The chatbot is now better equipped to handle queries about tenanted properties.
- Enhanced Trustworthiness: Including disclaimers and verified legal references builds user confidence in the system.
- A Foundation for the Future: Each refinement makes the system more robust and ready for real-world use.
Looking Ahead: Voice Agents Tomorrow
The delay in starting the voice agent development is a reminder that progress isn’t always linear. However, the extra time spent on the chatbot today will pay off as I move into the next phase. Tomorrow, I plan to:
- Evaluate tools like Twilio and Dialogflow for voice agent integration.
- Begin scripting common voice interactions, such as answering FAQs and scheduling appointments.
- Test the voice agent’s alignment with the chatbot’s tone and functionality.
This addition will bring the project closer to delivering a seamless, multi-channel user experience.
Final Thoughts
Day 9 may not have gone as planned, but it was an important step in building a chatbot that’s accurate, trustworthy, and capable of handling complex queries. Expanding the knowledge base to address tenancy-related questions, while adding legal references and disclaimers, ensures the chatbot provides reliable information without overstepping its scope.
Looking back to Day 1 and seeing how far the project has come, I’m reminded of the importance of adaptability and persistence. While today’s progress was unexpected, it reinforced the value of resilience and iteration in creating a system that users can trust.
Tomorrow, I’ll shift my focus to the voice agent and continue moving this project forward. If you’ve faced similar challenges with AI hallucinations or knowledge base development, I’d love to hear your insights. Drop a comment or reach out—I’m always open to learning from your experiences.
Stay tuned for Day 10 as the voice agent takes centre stage and the system evolves further!
Let’s Connect
If you’ve worked with any of these tools or have insights to share, drop a comment below, reach out on social media or email contact@juliandrouse.com —I’d love to hear your thoughts!
For more details, check out my channel on YouTube. Stay tuned as we continue building the future of real estate investing!
0 Comments