Building a Local AI Chatbot on MacBook Using Ollama, Gemma 4 and Python Gradio

Introduction

I've been exploring local AI models recently and wanted to try running an LLM directly on my MacBook rather than depending entirely on cloud-based AI services.

There are several open-weight models available today, and tools like Ollama make it quite easy to download and run them locally.

For this experiment, I decided to use Google's Gemma 4 model and build a simple web-based chat interface using Python and Gradio.

The objective was simple: get a local AI model running on my MacBook and make it accessible through a browser, similar to a basic ChatGPT interface.

In this post, I'll walk through the steps I followed, from installing Ollama to building and running the Python application.

My environment

Component                      Configuration 

Machine	MacBook Pro M5
Memory	24GB Unified Memory
Operating System	macOS
Local LLM runtime	Ollama
AI Model	Gemma 4 E4B
Programming Language	Python
Web Interface	Gradio
API Integration	OpenAI Python SDK

1. Installing Ollama

The first step was installing Ollama on macOS.

Ollama provides a simple way to download and run language models locally.

I used the following command in Terminal:

curl -fsSL https://ollama.com/install.sh | sh

Once the installation was completed, I checked the installed version:

ollama --version

My installation returned:

ollama version is 0.40.13

This confirmed that Ollama was installed successfully.

2. Downloading Gemma 4

Next, I downloaded Google's Gemma 4 E4B model.

ollama pull gemma4:e4b

The model download was approximately 9.5GB.

After downloading, I verified the model using:

ollama list

Output:

NAME          ID              SIZE
gemma4:e4b    eb2177adf764    9.5 GB

The model was now available locally through Ollama.

3. Running Gemma 4 locally

Before developing the web application, I wanted to make sure the model was working correctly.

I started it through Terminal:

ollama run gemma4:e4b

I tested it with a simple greeting:

>>> hello

Hello! How can I help you today?

The model responded successfully.

I also tried a few general knowledge and technical questions to understand how it handled different types of queries.

This was my first confirmation that I could run a language model directly on my MacBook.

4. Creating a customised model using a Modelfile

Ollama provides a feature called Modelfile that allows us to customise a model's behaviour.

Instead of using the base model directly, we can create our own model configuration with system instructions and generation parameters.

For example:

# Custom AI Assistant

FROM gemma4:e4b

SYSTEM """
You are a helpful personal assistant.

Provide clear, accurate and practical answers.
Explain technical concepts with examples.
Keep responses concise and professional.
"""

PARAMETER temperature 0.5
PARAMETER num_ctx 8192

A few important settings:

• FROM: Specifies the base model.

• SYSTEM: Defines the assistant's behaviour.

• temperature: Controls the randomness of generated responses.

• num_ctx: Sets the context window.

I used a temperature of 0.5 and an initial context window of 8,192 tokens.

To create the customised model:

ollama create my-assistant -f Modelfile

To verify:

ollama list

And to run it:

ollama run my-assistant

This creates a customised model based on the original Gemma model.

5. Building a web-based chat interface using Python and Gradio

Running the model through Terminal is useful for testing, but I wanted something more convenient.

I decided to use Gradio because it provides a simple way to create a browser-based interface without developing a frontend application from scratch.

Gradio's ChatInterface provides message bubbles, an input field and a send button.

Installing the dependencies

I installed Gradio and the OpenAI Python SDK:

pip install gradio openai

One interesting thing about Ollama is that it provides an OpenAI-compatible API.

This means we can use the OpenAI Python SDK to communicate with Ollama without connecting to OpenAI's cloud service.

The local API endpoint is:

http://localhost:11434/v1

Creating the Python application

I created a Python file called app.py.

The application uses Gradio for the user interface and the OpenAI SDK to communicate with the local Ollama model.

The complete implementation is below.

import os

import gradio as gr
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

MODEL = os.environ.get(
    "GEMMA_MODEL",
    "my-assistant:latest"
)


def normalize_history(history):
    """Accept Gradio messages or legacy tuple history."""

    messages = []

    for turn in history or []:

        if isinstance(turn, dict):
            messages.append({
                "role": turn["role"],
                "content": turn["content"]
            })

        elif isinstance(turn, (list, tuple)) and len(turn) == 2:

            user_msg, bot_msg = turn

            if user_msg:
                messages.append({
                    "role": "user",
                    "content": user_msg
                })

            if bot_msg:
                messages.append({
                    "role": "assistant",
                    "content": bot_msg
                })

    return messages


def chat(message, history):

    # Rebuild the conversation for the model.
    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        }
    ]

    messages += normalize_history(history)

    messages.append({
        "role": "user",
        "content": message
    })

    stream = client.chat.completions.create(
        model=MODEL,
        messages=messages,
        stream=True
    )

    reply = ""

    for chunk in stream:

        content = chunk.choices[0].delta.content

        if content:
            reply += content
            yield reply


def build_demo():

    css = """
    .gradio-container {
        height: 100vh !important;
    }
    """

    with gr.Blocks(
        fill_height=True,
        css=css
    ) as demo:

        kwargs = dict(
            fn=chat,
            title="Local Gemma 4 Chat",
            description="Powered entirely by Gemma 4 running on this machine.",
            fill_height=True
        )

        try:
            gr.ChatInterface(
                type="messages",
                **kwargs
            )

        except TypeError:
            gr.ChatInterface(**kwargs)

    return demo


if __name__ == "__main__":

    demo = build_demo()

    # Default: localhost only.
    demo.launch()

Understanding the implementation

There are a few important things happening in this application.

1. Connecting to Ollama

client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"
)

Although I'm importing the OpenAI SDK, the application is communicating with Ollama running locally.

The API key is a placeholder used for compatibility. It is not an actual OpenAI API key.

2. Maintaining conversation history

The normalize_history() function converts the conversation history into the message format expected by the API.

This allows the model to receive previous messages as context instead of treating every question as a completely new conversation.

The function supports both the messages format and the older tuple-based format used by Gradio.

3. Streaming responses

One feature I wanted was to see the answer appearing progressively rather than waiting for the complete response.

This is handled through:

stream = client.chat.completions.create(
    model=MODEL,
    messages=messages,
    stream=True
)

The application receives response chunks from Ollama.

Using Python's yield statement, Gradio updates the chat interface as the response grows.

This makes the interaction feel much closer to a regular AI chat application.

6. Running the application

Once the Python application was ready, I started it using:

python app.py

Gradio started the local web server.

I opened the browser and accessed:

http://127.0.0.1:7860

The local Gemma 4 chat interface was now available through the browser.

7. Testing the application

I tested the application with different types of questions.

Some of my tests included:

• General conversation and simple questions.

• Oracle RMAN backup commands.

• Historical questions about the Earth's shape.

• Questions about the origin and naming of colours.

• General knowledge questions.

The model was able to generate responses and stream them into the browser interface.

This was a useful exercise because it allowed me to test both the language model and the Python integration.

8. Final result

The result is a simple browser-based chatbot running locally on my MacBook.

I can interact with Gemma 4 through the browser without sending prompts to an external AI API.

The application is still a basic chatbot at this stage. It does not yet have persistent memory, document search or the ability to automate macOS tasks.

However, it provides a good foundation for developing a more capable personal AI assistant.

9. What's next?

This is just the beginning of my local AI experimentation.

I would like to extend this application with a few additional capabilities:

1. Persistent memory: Allow the assistant to retain useful information between conversations.

2. Document search: Enable it to search local PDFs, technical documents and personal notes.

3. Voice interaction: Add speech-to-text and text-to-speech capabilities.

4. Mac automation: Integrate Python and AppleScript to automate selected tasks.

5. Native macOS application: Eventually build a dedicated desktop interface using SwiftUI.

My longer-term objective is to build a personal AI assistant that can help with technical work, learning, research and everyday productivity.

Conclusion

I found this to be a straightforward way to get started with local LLM development on macOS.

Ollama handles the model execution, Gemma 4 provides the language capabilities, and Gradio makes it easy to expose everything through a browser-based interface.

What started as a simple experiment with a local language model has now become the foundation for building my own personal AI assistant.

There is still plenty to explore, particularly around memory, RAG, voice interaction and automation.

I'll cover those areas as I continue developing this project.

---

Technologies used: macOS | Apple Silicon | Ollama | Gemma 4 E4B | Python | Gradio | OpenAI Python SDK

Project type: Personal local AI experimentation