All projects12 of 13 · Dukaan Dost
12VOICE AI

Dukaan Dost

A Voice Agent That Takes Phone Orders for a Kirana Store

  • AI / ML
  • Web & Backend
  • Languages

One phone call, from a voice to a total

An example call. The words, items and prices are invented for this page, and the waveform is not a recording. In the project itself, all voices, shops and orders are synthetic.

Gnani Timbre speaks the reply, and the agent reads back the total.

Caller, in Hinglish

Bhaiya, do kilo chawal, ek kilo cheeni aur aadha kilo toor dal bhej dena.

In English: Brother, please send two kilos of rice, one kilo of sugar and half a kilo of toor dal.

Order slip · example
The model saysCode works out
ItemHow manyPriceAmount
Rice2 kg₹62 a kg₹124
Sugar1 kg₹46 a kg₹46
Toor dal0.5 kg₹155 a kg₹78
Total₹248

AgentThat is 3 items. Your total is ₹248.

The problem and the idea

A voice agent for a neighbourhood kirana store. A customer calls and orders in Telugu, Hindi, English or Hinglish; the agent matches each item to the shop's stock list, confirms quantities and prices, reads back the total, keeps the shop's credit book and writes an order summary the shopkeeper sees on a dashboard. Built for Gnani.ai's Great Indian AI Internship Challenge on Gnani's speech models. All voices, shops and orders are synthetic.

What I built

  • The model never does the money: it only says which items and how many; code checks stock, rounds, totals and writes every sentence that states an amount
  • Callers can interrupt, and the agent remembers only what was actually heard
  • Scripted credit and loan calls: 8 of 8 exactly right, first audio 2.2 s at the median
  • Many shops, one agent, each with a PIN-protected dashboard
  • Took two real phone calls on a trial number; it is a demonstration line, not a production one

The result

  • 8 of 8Scripted credit and loan calls exactly right
  • 2.2 sMedian time to first audio
  • 4Ways to speak: Telugu, Hindi, English, Hinglish

The model never does the money

An example order. The shelf, the prices, the rounding and the wording are invented for this page, and this is not the agent's own code. In the project itself, all voices, shops and orders are synthetic.

Model

Which items and how many. Nothing else. Change what it heard:

  • Rice2 kg
  • Sugar1 kg
  • Toor dal0.5 kg

Code

Checks stock, rounds, totals, and writes every sentence that states an amount.

  • Rice₹124In stock · 2 kg × ₹62
  • Sugar₹46In stock · 1 kg × ₹46
  • Toor dal₹77.50 → ₹78In stock · 0.5 kg × ₹155
  • Total₹248

What code writes for the agent to say

That is 3 items. Your total is ₹248.

Stack

  • Python
  • Gnani Prisma (speech to text)
  • Gnani Timbre (text to speech)
  • WebSockets
  • WebRTC VAD
  • Docker