Desktop AI tutor
Clicky for Windows
Hold a shortcut and ask about anything on screen. Clicky talks you through it and draws directly over the app to show exactly where to look or click.
Capture, transcription and provider setup now overlap, while the overlay stays responsive through slow devices and network calls.
- Demo: Hold the shortcut, ask a question and get an explanation over the app already on screen.
- Architecture: Independent capture, audio and provider work overlaps before the final answer is painted.
- Knowledge map: The Obsidian graph makes app-specific context visible instead of burying it in an opaque database.
- Provider settings: Provider choices are explicit and API keys stay in Windows Credential Manager.
- PyCon Top 3: Top 3 at the OpenAI-backed PyCon Singapore 2026 Hackathon, with S$2K+ in prizes.
What it does
Clicky starts when a user holds a shortcut and asks a question about the current screen. It captures the screen, transcribes the question, talks the user through the next step and draws directly over the current app to show where to look or click.
The result
Clicky finished Top 3 at the OpenAI-backed PyCon Singapore 2026 Hackathon, with S$2K+ in prizes. The Windows installer reached 90 verified downloads. The app now has 431 passing tests across the parts desktop software usually breaks: global shortcuts, audio devices, installers, provider credentials and overlay state.
The latency work
The first pipeline ran capture, transcription and model setup in sequence. I timed each stage, moved independent work earlier and kept the overlay responsive while network calls were in flight.
That cut a measured multi-sentence path from 3.7 seconds to 1.7 seconds. Provider latency still varies, but the app no longer adds avoidable serial work before the request leaves the machine.
What I learned
Desktop software fails at the seams. The test suite grew because each release found a new one, not because test count was the goal.