Building a Local AI Chatbot on MacBook Using Ollama, Gemma 4 and Python Gradio
Introduction
I've been exploring local AI models recently and wanted to try running an LLM directly on my MacBook rather than depending entirely on cloud-based AI services.
There are several open-weight models available today, and tools like Ollama make it quite easy to download and run them locally.
For this experiment, I decided to use Google's Gemma 4 model and build a simple web-based chat interface using Python and Gradio.
The objective was simple: get a local AI model running on my MacBook and make it accessible through a browser, similar to a basic ChatGPT interface.
In this post, I'll walk through the steps I followed, from installing Ollama to building and running the Python application.
My environment
| Machine | MacBook Pro M5 |
| Memory | 24GB Unified Memory |
| Operating System | macOS |
| Local LLM runtime | Ollama |
| AI Model | Gemma 4 E4B |
| Programming Language | Python |
| Web Interface | Gradio |
| API Integration | OpenAI Python SDK |
1. Installing Ollama
The first step was installing Ollama on macOS.
Ollama provides a simple way to download and run language models locally.
I used the following command in Terminal:
curl -fsSL https://ollama.com/install.sh | shOnce the installation was completed, I checked the installed version:
ollama --versionMy installation returned:
ollama version is 0.40.13This confirmed that Ollama was installed successfully.
2. Downloading Gemma 4
Next, I downloaded Google's Gemma 4 E4B model.
ollama pull gemma4:e4bThe model download was approximately 9.5GB.
After downloading, I verified the model using:
ollama listOutput:
NAME ID SIZE
gemma4:e4b eb2177adf764 9.5 GBThe model was now available locally through Ollama.
3. Running Gemma 4 locally
Before developing the web application, I wanted to make sure the model was working correctly.
I started it through Terminal:
ollama run gemma4:e4bI tested it with a simple greeting:
>>> hello
Hello! How can I help you today?The model responded successfully.
I also tried a few general knowledge and technical questions to understand how it handled different types of queries.
This was my first confirmation that I could run a language model directly on my MacBook.
4. Creating a customised model using a Modelfile
Ollama provides a feature called Modelfile that allows us to customise a model's behaviour.
Instead of using the base model directly, we can create our own model configuration with system instructions and generation parameters.
For example:
# Custom AI Assistant
FROM gemma4:e4b
SYSTEM """
You are a helpful personal assistant.
Provide clear, accurate and practical answers.
Explain technical concepts with examples.
Keep responses concise and professional.
"""
PARAMETER temperature 0.5
PARAMETER num_ctx 8192A few important settings:
FROM: Specifies the base model.
SYSTEM: Defines the assistant's behaviour.
temperature: Controls the randomness of generated responses.
num_ctx: Sets the context window.
I used a temperature of 0.5 and an initial context window of 8,192 tokens.
To create the customised model:
ollama create my-assistant -f ModelfileTo verify:
ollama listAnd to run it:
ollama run my-assistantThis creates a customised model based on the original Gemma model.
5. Building a web-based chat interface using Python and Gradio
Running the model through Terminal is useful for testing, but I wanted something more convenient.
I decided to use Gradio because it provides a simple way to create a browser-based interface without developing a frontend application from scratch.
Gradio's ChatInterface provides message bubbles, an input field and a send button.
Installing the dependencies
I installed Gradio and the OpenAI Python SDK:
pip install gradio openaiOne interesting thing about Ollama is that it provides an OpenAI-compatible API.
This means we can use the OpenAI Python SDK to communicate with Ollama without connecting to OpenAI's cloud service.
The local API endpoint is:
http://localhost:11434/v1Creating the Python application
I created a Python file called app.py.
The application uses Gradio for the user interface and the OpenAI SDK to communicate with the local Ollama model.
The complete implementation is below.
import os
import gradio as gr
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama"
)
MODEL = os.environ.get(
"GEMMA_MODEL",
"my-assistant:latest"
)
def normalize_history(history):
"""Accept Gradio messages or legacy tuple history."""
messages = []
for turn in history or []:
if isinstance(turn, dict):
messages.append({
"role": turn["role"],
"content": turn["content"]
})
elif isinstance(turn, (list, tuple)) and len(turn) == 2:
user_msg, bot_msg = turn
if user_msg:
messages.append({
"role": "user",
"content": user_msg
})
if bot_msg:
messages.append({
"role": "assistant",
"content": bot_msg
})
return messages
def chat(message, history):
# Rebuild the conversation for the model.
messages = [
{
"role": "system",
"content": "You are a helpful assistant."
}
]
messages += normalize_history(history)
messages.append({
"role": "user",
"content": message
})
stream = client.chat.completions.create(
model=MODEL,
messages=messages,
stream=True
)
reply = ""
for chunk in stream:
content = chunk.choices[0].delta.content
if content:
reply += content
yield reply
def build_demo():
css = """
.gradio-container {
height: 100vh !important;
}
"""
with gr.Blocks(
fill_height=True,
css=css
) as demo:
kwargs = dict(
fn=chat,
title="Local Gemma 4 Chat",
description="Powered entirely by Gemma 4 running on this machine.",
fill_height=True
)
try:
gr.ChatInterface(
type="messages",
**kwargs
)
except TypeError:
gr.ChatInterface(**kwargs)
return demo
if __name__ == "__main__":
demo = build_demo()
# Default: localhost only.
demo.launch()Understanding the implementation
There are a few important things happening in this application.
1. Connecting to Ollama
client = OpenAI(
base_url="http://localhost:11434/v1",
api_key="ollama"
)Although I'm importing the OpenAI SDK, the application is communicating with Ollama running locally.
The API key is a placeholder used for compatibility. It is not an actual OpenAI API key.
2. Maintaining conversation history
The normalize_history() function converts the conversation history into the message format expected by the API.
This allows the model to receive previous messages as context instead of treating every question as a completely new conversation.
The function supports both the messages format and the older tuple-based format used by Gradio.
3. Streaming responses
One feature I wanted was to see the answer appearing progressively rather than waiting for the complete response.
This is handled through:
stream = client.chat.completions.create(
model=MODEL,
messages=messages,
stream=True
)The application receives response chunks from Ollama.
Using Python's yield statement, Gradio updates the chat interface as the response grows.
This makes the interaction feel much closer to a regular AI chat application.
6. Running the application
Once the Python application was ready, I started it using:
python app.pyGradio started the local web server.
I opened the browser and accessed:
http://127.0.0.1:7860The local Gemma 4 chat interface was now available through the browser.
7. Testing the application
I tested the application with different types of questions.
Some of my tests included:
General conversation and simple questions.
Oracle RMAN backup commands.
Historical questions about the Earth's shape.
Questions about the origin and naming of colours.
General knowledge questions.
The model was able to generate responses and stream them into the browser interface.
This was a useful exercise because it allowed me to test both the language model and the Python integration.
8. Final result
The result is a simple browser-based chatbot running locally on my MacBook.
I can interact with Gemma 4 through the browser without sending prompts to an external AI API.
The application is still a basic chatbot at this stage. It does not yet have persistent memory, document search or the ability to automate macOS tasks.
However, it provides a good foundation for developing a more capable personal AI assistant.
9. What's next?
This is just the beginning of my local AI experimentation.
I would like to extend this application with a few additional capabilities:
Persistent memory: Allow the assistant to retain useful information between conversations.
Document search: Enable it to search local PDFs, technical documents and personal notes.
Voice interaction: Add speech-to-text and text-to-speech capabilities.
Mac automation: Integrate Python and AppleScript to automate selected tasks.
Native macOS application: Eventually build a dedicated desktop interface using SwiftUI.
My longer-term objective is to build a personal AI assistant that can help with technical work, learning, research and everyday productivity.
Conclusion
I found this to be a straightforward way to get started with local LLM development on macOS.
Ollama handles the model execution, Gemma 4 provides the language capabilities, and Gradio makes it easy to expose everything through a browser-based interface.
What started as a simple experiment with a local language model has now become the foundation for building my own personal AI assistant.
There is still plenty to explore, particularly around memory, RAG, voice interaction and automation.
I'll cover those areas as I continue developing this project.
Technologies used: macOS | Apple Silicon | Ollama | Gemma 4 E4B | Python | Gradio | OpenAI Python SDK
Project type: Personal local AI experimentation
