Discover Jordan Assistant
A travel assistant that looks facts up in a database through function calling, then answers in text, voice and a generated image.
Overview
gpt-4.1-mini is given a ticket-price tool backed by a SQLite table of Jordanian destinations. It loops through tool calls until it can answer, and each reply is voiced with gpt-4o-mini-tts and illustrated with DALL·E 3 in a Gradio interface.
Problem
Chat models guess prices. For answers that must come from data, the model has to call a tool and speak from what the tool returns.
Architecture
- Gradio BlocksChatbot, image panel and auto-playing audio.
- Chat modelgpt-4.1-mini with one registered tool.
- Tool + SQLiteParameterised lookup of price and landmark for ten destinations.
- Voice + imagegpt-4o-mini-tts speaks the reply; DALL·E 3 draws the destination.
AI pipeline
- 01
Input
A user message such as "I want to visit Petra".
- 02
LLM
The model decides to call the price tool.
- 03
Tools
The loop runs while the model keeps requesting tool calls, executing a SQLite query each time.
- 04
Output
Final text reply, spoken audio and a generated image.
- Function calling with a tool loop
- Text-to-speech
- Image generation
Engineering
- A real tool loop
- The model can call the tool several times in one turn; the loop continues until it returns a final message.
- Parameterised SQL
- Lookups are normalised to lowercase and use bound parameters rather than string-built queries.
Challenges
- No challenges are documented in the repository.
Results & limits
Measured
- No evaluation. Prices are sample data, not real fares.
Known limits
- Adapts a well-known course pattern to a Jordan theme.
- Ten rows of sample data and no error handling.
- Every reply triggers a speech call, and tool lookups trigger an image call, which costs API credit.
Stack
- Python
- OpenAI API
- SQLite
- Gradio
Demo
No public demo for this one yet. The source repository has setup instructions.