Designing AI Tools: From Concept to Implementation
I had a particular problem: how do you create an application that allows for AI to make changes, but for a human to be in the loop to either approve or change what the AI did or suggested? I had thought of a commit plan flow where the AI would propose a change, the user would approve it, then the AI would act. But how do you actually do that? Do you just make heaps of duplicates? Change the duplicate then delete it once it's been approved? What if the user abandons mid-session and then we just have heaps of duplicates?
I eventually landed on just closing the loop. AI makes the change, no human in the loop at all. The human just slows down the change anyway. Wouldn't it be better if you just said "make this change" and it did it? I was wrong. My users didn't want to use the tool at all.
Building AI tools that actually work in practice requires more than just connecting an LLM to a MCP server. After shipping several AI-powered tools that eventually delivered real ROI, I've learned some key lessons about developing AI tools for in-house projects. Here's what I wish I'd known when I started.
Start with the Problem, Not the Technology
The biggest mistake I see teams make is falling in love with AI capabilities rather than focusing on user problems. Your first question shouldn't be "What can AI do?" but rather "What friction exists in my users' workflows?"
This is one of the mistakes I made, I immediately went straight to what can I automate with AI in a chat interface. Instead of trying to solve real problems that my users were facing. I went straight to making an MCP server for my backend and hooking an LLM to it. When years of software engineering principles have said, start with user requirements, I made the mistake of getting mixed up in the hype.
AI tools succeed when they remove cognitive load, not when they showcase impressive technology. The best AI features often feel invisible—users accomplish their goals faster without thinking about the AI powering it.
Don't try to Remove the Human from the Loop
This is a key takeaway. I started developing a tool that I originally thought would solve all our data entry needs. You would ask it to do an update and it would do it. Remove inactive products from this set. Update these records with these values, etc. This important job was done by some key support colleagues and I thought this would let them focus more on data quality and less on data entry. They never used it.
I started pondering why. And it occurred to me when someone told me to try Claude CLI and I thought I like being in the loop. I don't like background agents, I like being behind the wheel. And click—there it was. They felt the same. I threw out the old tools that actually updated the data. I started making a new UI that focused on putting tools back into human hands and keeping them in control, but still being powered by AI. AI no longer updated the data, it updated the UI.
"Remove inactive products from this set" became "highlight all inactive products in this set." Delete button is clicked by human. Data is updated in the backend via a REST API. All of a sudden, it started getting used, and people thought it was amazing. It became a tool that made doing their job easier, instead of trying to do their job.
One of its strongest features that have really made an impact is how useful AI validation can be. We have custom validation rules that users can tell the AI to remember. Because our tools are used for a wide variety of different verticals, from Internet, to Banking. And the schema is completely customizable, validation has always been a weak point in our system because we just don't know what fields our users might add. This customizable AI validation was such a game changer for data quality. Users could now set validation rules, and when the user is done editing the AI checks the state, before the user saves it in the backend.
Design Principles for AI Tools
1. Natural Interfaces Beat Chat Interfaces
Everyone defaults to building chat interfaces, but they're often the wrong choice. Chat works best when:
- The conversation itself has value
- Users can just fire and forget—ask and it's done.
But consider these alternatives:
- Contextual suggestions - AI recommendations that appear where users already work
- Smart defaults - AI that pre-fills forms or suggests configurations
- Progressive enhancement - Traditional interfaces with AI-powered shortcuts
2. Embrace Imperfection with Graceful Degradation
One of the tools in my UI lets the AI generate an RSQL filter. It could do this well most of the time but often got some field names wrong. It would get the user 80% of the way there, and then the UI had IntelliSense input for RSQL that allowed the user to easily fix any mistakes made by the LLM.
AI will make mistakes. Design for it:
- Show confidence levels - Let users know when the AI is uncertain
- Make corrections easy - One-click fixes are better than starting over
- Provide escape hatches - Always offer a way to bypass AI and do things manually
- Learn from corrections - Use user feedback to improve future responses
3. Design for Trust and Transparency
Users need to understand what your AI is doing:
- Explain reasoning - Show the "why" behind AI decisions
- Indicate data sources - Make it clear what information the AI is using
- Set clear boundaries - Be explicit about what your AI can and cannot do
- Provide audit trails - Let users trace AI actions, especially in critical workflows
Implementation Best Practices
Start with Manual Processes
Before building AI automation, implement the underlying processes manually. This helps you:
- Understand the real requirements
- Identify edge cases
- Train your AI with examples from real usage
- Build user trust gradually
Measure What Matters
Traditional metrics often don't apply to AI tools. Focus on:
- Task completion rate - Can users accomplish their goals?
- Time to completion - How much faster is the AI-powered flow?
- Error recovery - How quickly can users fix AI mistakes?
- Adoption patterns - Which features do users actually use?
- Quality over quantity - Better to solve one problem perfectly than many poorly
Common Pitfalls to Avoid
The "Magic Wand" Trap
Managers don't want to micromanage telling AI agents what to do, employees still want to do their job. Don't try to make an AI tool that just does the job, make an AI tool that makes someone better at their job.
Over-Engineering Early
Resist the urge to build complex AI orchestration systems from the start. Simple, reliable tools beat complex, unreliable ones every time.
Ignoring Non-Technical Users
If your AI tool requires prompt engineering skills, you've probably failed. Design for your least technical users, not your most technical ones.
Forgetting About Maintenance
AI tools require ongoing maintenance in ways traditional software doesn't:
- Model performance can drift over time
- User expectations evolve as AI capabilities improve
- You still have existing software maintenance too
Moving Forward
The best AI tools feel like superpowers, not sci-fi. They solve real problems for real people without making them think about AI at all. Focus on your users' workflows, start simple, and iterate based on actual usage rather than theoretical capabilities.
Remember: the goal isn't to build an impressive AI tool—it's to make your users' lives better. Everything else is just implementation details.
What's Next?
In my next post, I'll dive deeper into specific implementation patterns I've found effective, including code examples and architecture diagrams from real AI tools that are running in production today.
Have you built AI tools that deliver real value? I'd love to hear about your experiences. Reach out on GitHub or connect with me on LinkedIn.