[Under Review] Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentation

This repository contains the source code of "Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentation".

✅Updates

• Dec. 6th, 2024: Exact paper is released at arXiv.
• ...

Get Started

Environment

• Ubuntu 20.04, with Python 3.8.0, PyTorch 1.12.0, CUDA 11.6, multi gpus(8) - Nvidia RTX 3090.
• You can install all dependencies with the provided requirements file.

pip install -r requirements.txt

Data Preparations

PASTIS dataset

The original PASTIS dataset is accessible here. We follow the TSViT to divide each sample into 24x24 patches by running the script:

python data/PASTIS24/data2windows.py --rootdir <...> --savedir <...> --HWout 24

The reorganized directory should be:

PASTIS
├── pickle24x24
│   ├── 40562_9.pickle
│   └── ...
├── fold-paths
│   ├── fold_1_paths.csv
│   └── ...

In addition, we generate multi-class labels for each patch by running the following script:

python data/PASTIS24/seg2cls_label.py --pickle_path <...>/PASTIS/pickle24x24 

Germany dataset

The original Germany dataset is accessible here, we can download the dataset (40GB) via:

wget https://zenodo.org/record/5712933/files/data_IJGI18.zip

The size of each sample in Germany dataset is 24x24, so we only need to generate the multi-class labels with the above script without splitting.

Usage

Step 1: Train Exact_cls classification network.

(to be released)

Step 2: Generate CB-CAMs and pseudo labels.

(to be released)

Step3: Train segmentation network with the pseudo labels.

python tools/train_seg.py --config configs/PASTIS24/TSViT_fold1.yaml

Main Results

PASTIS Benchmark

Results of pseudo labels.

Method	OA	mIoU
baseline	81.2	69.5
ours-Exact	84.1	75.6

Results of segmentation network (TSViT segmentation model trained with different pseudo labels).

Method	OA	mIoU
baseline	77.2	57.8
ours-Exact	80.2	62.0

Citation

Please cite our work if you find it helpful to your research.

@misc{zhu2025exact,
      title={Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentation}, 
      author={Hao Zhu and Yan Zhu and Jiayu Xiao and Tianxiang Xiao and Yike Ma and Yucheng Zhang and Feng Dai},
      year={2024},
      eprint={2412.03968},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Acknowledgement

This repo is built upon TSViT and PASTIS, thanks for their excellent works!