README

This is an automatic full segmentation tool based on Segment-Anything-2 and Segment-Anything-1. Our tool performs automatic full segmentation of the video, enabling the tracking of each object and the detection of possible new objects.

Demo

chicken-demo.mp4

sea-demo.mp4

dog-demo.mp4

Environment Setup

First, clone this repository and submodules

#SSH
git clone git@github.com:zrporz/AutoSeg-SAM2.git --recursive

or

#HTTPS
git clone https://github.com/zrporz/AutoSeg-SAM2.git --recursive

The code requires python>=3.10, as well as torch>=2.3.1 and torchvision>=0.18.1 We use SAM1 to provide static segmentation results and use the SAM2 to track the static segmentation results. You can install them by the following commands

### install sam1 and sam2 modules
pip install -e submodule/segment-anything-1
pip install -e submoudle/segment-anything-2
### download checkpoints
cd checkpoints/sam1
bash download.sh
cd ../sam2
bash download.sh

Prepare your data

Please organize your video data as follows

|-<video dir>
    |-000001.jpg
    |-000002.jpg
    |-000003.jpg
    |-000004.jpg
    ...

or you can use our demo datasets chickenchicken and put it under videos/chickenchicken, then run

bash scripts/chickenchicken.sh

Citation

@software{AutoSeg_SAM2,
  author = {Zrporz},
  title = {AutoSeg-SAM2},
  year = {2024},
  publisher = {GitHub},
  url = {https://github.com/zrporz/AutoSeg-SAM2},
  version = {Latest}, 
  license = {MIT}, 
  note = {Automated image segmentation tool based on Segment Anything Model (SAM)}
}