Rota AI: How to Run AI Voice Dictation Completely Offline on Windows
5. How to Run AI Voice Dictation Completely Offline on Windows
TL;DR: Install Ollama, download a Whisper model, configure Rota AI to use it. Your voice never touches the internet. Works on my Dell G15 with 16GB RAM. Small model runs on 8GB too.
I have a problem. I do not trust cloud APIs with my voice data.
Your voice is biometric data. It is literally as unique as your fingerprint. Once it leaves your machine, you do not control what happens to it anymore. It could be stored. It could be used for training. It could be breached.
I know, I know. "Nothing to hide." But that is not the point. The point is it should be your choice.
That is why I built offline mode into Rota AI and honestly, it is one of my favorite features.
Why Offline Matters
about cloud-based dictation. Every time you speak, your audio gets packaged up, sent to a server somewhere, transcribed, and sent back. That happens in milliseconds so it feels instant. But your voice data still left your machine.
With offline mode:
- Your audio stays on your laptop. Period.
- No internet connection needed to dictate
- No API costs, no rate limits, no waiting
- Works on flights, in cafes with bad wifi, anywhere
- No account needed anywhere
The tradeoff: you need enough RAM to run the model. and local transcription is slower than cloud. But for most people, it is good enough.
What You Need
Minimum specs:
- 8GB RAM (for the small model)
- 16GB RAM recommended (for medium model)
- About 2GB free disk space
- Windows 10 or 11
- Decent microphone (your laptop mic works, external is better)
I run it on my Dell G15. i5 12th gen, 16GB RAM, RTX 3050. The medium model runs without issues. Small runs on anything.
Step by Step
1. Install Ollama
Go to ollama.com and download the Windows installer. Run it. That is it.
Ollama runs as a background service. You do not need to open anything.
2. Download a Whisper Model
Open a terminal and run:
ollama pull whisper-small
Or for better accuracy:
ollana pull whisper-medium
The small model is about 250MB. Medium is about 500MB. They download once and stay on your machine forever.
3. Configure Rota AI
Open Rota AI settings. Go to Transcription > Backend. Select "Ollama Local." Choose your model size. Hit save.
Done. Now when you press F9 and speak, the audio stays on your machine, gets transcribed by your local Whisper model, gets cleaned up, and gets injected into your app.
No internet. No cloud. Nobody else hears you.
Model Size Comparison
| Model | Size | RAM Needed | Accuracy | Speed |
|---|---|---|---|---|
| Small | 250MB | 8GB | Decent | Fast |
| Medium | 500MB | 16GB | Good | Medium |
| Large | 1.5GB | 16GB+ | Best | Slow |
I use medium. It is the sweet spot between speed and accuracy. If you have 8GB RAM, use small. It is honestly not that bad.
Real Numbers
On my Dell G15 with Groq cloud: transcription takes about 1-2 seconds for a 10-second clip.
On the same machine with Ollama medium: about 3-5 seconds for the same clip.
With Ollama small: about 2-3 seconds.
So yeah, local is slower. But it is not painfully slow. It is like, you finish speaking, count to 3, and the text appears. Completely usable.
When You Actually Need Offline
Here are the real scenarios where offline mode saves you:
- Traveling. Airplane, train, cafe with terrible wifi. Works everywhere.
- Privacy-sensitive work. If you are dictating something personal, financial, or medical, keep it local.
- API limits hit. Groq free tier gives you plenty but if you somehow hit the limit, local has no limits.
- Internet outage. Rare but when it happens, you still want to dictate.
- Paranoia. Some people just do not want their voice on someone else's server. Respect.
Does It Actually Work Though?
Yeah tbh. I was skeptical at first. I thought local transcription would be garbage compared to cloud.
It is not. It is close. Like, maybe 90% as accurate as Groq's cloud for English. For other languages it varies. But for daily dictation, emails, notes, coding? Totally fine.
The main difference is speed. Cloud is 1-2 seconds. Local is 3-5 seconds. That is literally the only difference I notice day to day.
FAQ
Is offline dictation as accurate as cloud? Close but not identical. Maybe 90% as accurate for English. Good enough for daily use.
How much RAM do I need? 8GB for small model. 16GB for medium. 16GB+ for large.
Does it work without internet at all? Yes. After you download the model once, you never need internet again.
Can I use Rota AI offline on Mac? Rota AI is Windows only. For Mac offline dictation, try SuperWhisper or Wispr Flow.
How long does the model take to download? Small model is 250MB. On a decent connection, 5-10 minutes.
Should I use cloud or local? Cloud for speed and accuracy first. Local for privacy and offline use. You can switch between them in settings whenever.
Written by Karthik Krishnan. I built offline mode because I got paranoid about my voice data. Ended up being one of the features I am most proud of.