Overview

Meta’s Segment Anything Model (SAM) changed the game for image segmentation. SAM 2.1 can be run locally on your own machine, while the latest SAM 3 is available through Meta’s online playground. In this post, I run SAM 2.1 on an Apple Silicon Mac with MPS GPU acceleration and compare it with the SAM 3 online demo.

SAM 2.1 Local vs SAM 3 Online — Architecture Comparison

flowchart LR
    subgraph Local["SAM 2.1 Local"]
        A["User Input <br/> Points/Boxes"] --> B["SAM 2.1 Tiny <br/> 74.5 MB Model"]
        B --> C["PyTorch MPS <br/> Apple Silicon GPU"]
        C --> D["Gradio Web UI <br/> localhost:7860"]
    end

    subgraph Cloud["SAM 3 Online"]
        E["User Input <br/> Text/Click"] --> F["SAM 3 <br/> Meta Server"]
        F --> G["Cloud GPU <br/> Inference"]
        G --> H["Web Browser <br/> aidemos.meta.com"]
    end

    style Local fill:#e8f5e9,stroke:#2e7d32
    style Cloud fill:#e3f2fd,stroke:#1565c0

SAM 2.1 on Apple Silicon Mac

The ice-ice-bear/sam2-mac-test repository provides a ready-to-run SAM 2.1 setup for Apple Silicon Macs.

Key Features

• MPS GPU Acceleration: Uses PyTorch’s Metal Performance Shaders backend to run inference on M1/M2/M3/M4 GPUs
• Multi-point Segmentation: Place include/exclude points for fine-grained segmentation with undo/clear support
• Segment Everything Mode: Segment all objects in an image at once
• Gradio Web UI: Browser-based interface accessible right away
• SAM 2.1 Tiny Model: Lightweight 74.5 MB model, auto-downloaded on first run

Quick Start

git clone https://github.com/ice-ice-bear/sam2-mac-test.git
cd sam2-mac-test
uv sync
uv run python app.py

Open http://127.0.0.1:7860 in your browser to access the Gradio UI.

Performance

Benchmarks on an M1 MacBook:

The Tiny model keeps memory usage low, and MPS acceleration provides significant speedup over CPU-only inference.

Tech Stack

• SAM 2.1: Via the Ultralytics library
• PyTorch MPS: Apple Silicon GPU backend
• Gradio: Web UI framework
• uv: Package manager

Meta SAM 3 Online Playground

Meta offers the latest SAM 3 as an online demo at aidemos.meta.com/segment-anything.

What Sets SAM 3 Apart

• Text-prompt Segmentation: Find objects using natural language — “find animal”, “find person”
• One-click Effects: Apply blur, clone, desaturate, and more with a single click
• Motion Trails: Add motion effects to segmented objects
• Contour Lines / Bounding Boxes: Various visualization options
• Video Segmentation: Track Anything feature for object tracking in video
• Community Templates: Use effects created by other users

SAM 2.1 Local vs SAM 3 Online Comparison

Which One Should You Choose?

SAM 2.1 local is the right choice when:

• You don’t want sensitive images leaving your machine
• You need to integrate segmentation into an automated pipeline
• You want to modify or extend the model
• You need to work offline

SAM 3 online demo is the right choice when:

• You want to find objects using text prompts
• You need quick access to effects like blur and cloning
• You need video segmentation
• You want to try it out without any installation

Wrapping Up

Running SAM 2.1 locally is a practical option for Apple Silicon Mac users. The 74.5 MB Tiny model delivers usable segmentation results, and MPS acceleration makes good use of the GPU. The SAM 3 online demo takes it further with text prompts and a rich set of effects. Depending on your use case, combining local and cloud approaches gives you the best of both worlds.

Links

• ice-ice-bear/sam2-mac-test (GitHub)
• Meta AI Demos — Segment Anything
• Ultralytics SAM 2 Documentation
• PyTorch MPS Backend