Mohammad Ahmed
All work

Multimodal tool-calling agentDec 2025Demo notebook

Discover Jordan Assistant

A travel assistant that looks facts up in a database through function calling, then answers in text, voice and a generated image.

Discover Jordan assistant: chat about Petra with a generated pop-art image and an audio waveform

Overview

gpt-4.1-mini is given a ticket-price tool backed by a SQLite table of Jordanian destinations. It loops through tool calls until it can answer, and each reply is voiced with gpt-4o-mini-tts and illustrated with DALL·E 3 in a Gradio interface.

Problem

Chat models guess prices. For answers that must come from data, the model has to call a tool and speak from what the tool returns.

Architecture

  1. Gradio BlocksChatbot, image panel and auto-playing audio.
  2. Chat modelgpt-4.1-mini with one registered tool.
  3. Tool + SQLiteParameterised lookup of price and landmark for ten destinations.
  4. Voice + imagegpt-4o-mini-tts speaks the reply; DALL·E 3 draws the destination.

AI pipeline

  1. 1

    Input

    A user message such as "I want to visit Petra".

  2. 2

    LLM

    The model decides to call the price tool.

  3. 3

    Tools

    The loop runs while the model keeps requesting tool calls, executing a SQLite query each time.

  4. 4

    Output

    Final text reply, spoken audio and a generated image.

  • Function calling with a tool loop
  • Text-to-speech
  • Image generation

Engineering

A real tool loop
The model can call the tool several times in one turn; the loop continues until it returns a final message.
Parameterised SQL
Lookups are normalised to lowercase and use bound parameters rather than string-built queries.

Challenges

  • No challenges are documented in the repository.

Results & next steps

Measured

  • No evaluation. Prices are sample data, not real fares.

Known limits and next steps

  • Adapts a well-known course pattern to a Jordan theme.
  • Ten rows of sample data and no error handling.
  • Every reply triggers a speech call, and tool lookups trigger an image call, which costs API credit.

Stack

  • Python
  • OpenAI API
  • SQLite
  • Gradio

Demo

No public demo for this one yet. The source repository has setup instructions.