Building a Speech-to-Speech Chatbot with Multi-turn Memory and Logging

Source:https://chatgpt.com/share/68871063-ec5c-8006-bbac-194dbee7a20f

Got it! You want the chatbot to keep conversation context (multi-turn memory), so it feels like a continuous conversation—just like ChatGPT’s chat history.

Here’s What We'll Do:

We'll maintain a chat history list of messages (user + assistant) and send that entire history to the OpenAI API for every turn. This way, GPT will respond with full context.

Modified Parts:

  • Maintain a chat_history list (system, user, assistant messages).
  • On every user input, append to history.
  • Send the entire chat history to OpenAI.
  • GPT will now reply aware of the entire conversation.

Updated Code Snippet (Multi-turn Memory Added)

python

# === Global Chat History for Context === chat_history = [ {"role": "system", "content": "You are a friendly AI assistant."} ] # === Chatbot with Contextual Memory === def chatbot_response_with_memory(user_input): # Add user's message to history chat_history.append({"role": "user", "content": user_input}) # Call OpenAI with full history response = openai.ChatCompletion.create( model="gpt-3.5-turbo", messages=chat_history ) reply = response['choices'][0]['message']['content'].strip() # Add assistant's reply to history chat_history.append({"role": "assistant", "content": reply}) return reply # === Update listen_and_respond() to use memory-aware function === def listen_and_respond(): with sr.Microphone() as source: append_chat("System", "Listening...") audio = recognizer.listen(source) try: user_input = recognizer.recognize_google(audio) append_chat("You", user_input) except sr.UnknownValueError: append_chat("System", "I couldn't understand you.") return except sr.RequestError as e: append_chat("System", f"Speech Recognition error: {e}") return if user_input.lower() in ['exit', 'quit']: root.quit() return # Get contextual chatbot reply reply = chatbot_response_with_memory(user_input) append_chat("Bot", reply) # Speak the reply in real-time stream speak_text_stream(reply)

✅ Now it will:

  • Remember past conversation turns.
  • Handle multi-turn dialogue smoothly.
  • You can even programmatically clear/reset memory if needed by resetting chat_history.

Want to take it further?

  • Add Hotword Detection ("Hey Assistant") to start listening automatically?
  • Save chat history to a file (chat logs)?
  • GUI with Chat Bubbles, Avatar, Animated Mouth Movement? Let me know how “fancy” you want to go next!