Ship Landmark: An Informative Ship Image Annotation and Its Applications

Introduction

By summarizing the locations of such areas in ships, we define 20 ship landmarks and build the Ship Landmark Dataset (SLAD), the first ship dataset with landmark annotations. We also provide a benchmark for ship landmark detection by evaluating state-of-the-art landmark detection methods on the newly built SLAD. This repository provides the source code for our research on structure-based ship perception, including the vanilla ship recognition, keypoint-aware ship recognition, structure-to-texture ship generation, etc.

If you are having trouble obtaining the image, you can access it using the following link.

Code Overview

The structure of the repository is as follows：

• ./data: Dataset annotation and associated scripts and image storage locations.
• ./configs: Parameter configuration module.
• ./model: Our main model with all its building blocks.
• ./utils: Utility functions for training, inference, and postprocessing.
• ./scripts: Scripts related to model training, inference, image and code download, visualization.
• ./visualization: CAM image visualization and experimental performance saving.
• ./demo: Demonstration of ship detection task using mmpose.

Ship Landmark Detection

Preparation

According to the mmpose official documentation (https://github.com/open-mmlab/mmpose/tree/dev-1.x), configure mmpose. The current operation assumes in the current directory.

mv ./demo/slad_dataset.py mmpose-main/mmpose/datasets/datasets/body/
mv -f ./demo/__init__.py mmpose-main/mmpose/datasets/datasets/body/
mv ./demo/slad.py mmpose-main/configs/base/datasets/
mkdir ./mmpose-main/configs/body_2d_keypoint/topdown_heatmap/slad/
mv ./demo/td-hm_hrnet-w32_8xb64-210e_slad-256x192.py mmpose-main/configs/body_2d_keypoint/topdown_heatmap/slad/

Example demonstration Example demonstration using ./demo/td-hm_hrnet-w32_8xb64-210e_slad-256x192.py, following the naming conventions similar to the mmpose official website: 'td' denotes top-down, 'hm' signifies heatmap, 'hrnet-w32' represents the algorithm name, '8xb64' indicates the number of GPUs and batch size, '210e' represents the number of epochs, 'slad' indicates the usage of the SLAD dataset, and '256x192' denotes the width and height of the input image.

Since the SLAD dataset is provided in the COCO standard format, configuration settings can be referenced from the COCO format. The main modifications involve setting dataset_type to 'SLADDataset' and data_root to 'data/slad/'. Here, the data format follows the COCO standard layout for organizing image data and annotations. To simplify modifications, bbox_file can be directly replaced with the COCO file.

Implementation

Environment installation

conda create -n ships_class python==3.6
conda activate ships_class
pip install -r requirements.txt

Data preparation

To download and retrieve images from the annotation files in /data/anno/, please use the following command. Before executing the script, kindly sign the AgreementForSLAD.docx file located in /data/ and send it to [email protected] via email. We will proceed with the image download once we receive and confirm your signed agreement. If you encounter difficulties or come across missing data, please inform us of the file names so we can assist you promptly.

sh ./scripts/data_preparation_class.sh
sh ./scripts/data_preparation.sh

Training recognition Networks

• Train networks using the vanilla ship images.

sh ./scripts/train_resnet_vallinia.sh

• Train networks using keypoint-aware ship images.

sh ./scripts/train_resnet_add.sh

• CAM image visualization If further exploration of the CAM image is required, the raw data is stored in *.npy.

sh ./scripts/cam_download.sh
sh ./scripts/cam_vallina_resnet.sh
sh ./scripts/cam_add_resnet.sh

Structure-to-Texture Generation Network For generating network selection, we used CycleGAN and pix2pix in PyTorch, without any additional modifications, and completed the experiments using only paired ship structure-texture images as data.

• Data preparation and training

sh ./scripts/gan_download.sh

• Test with test dataset

python test.py --dataroot ./datasets/ships --name ships_pix2pix --model pix2pix --direction BtoA --num_test 1000 --epoch 200

• Test with multi-view structural images generated from 3D data

python test.py --dataroot ./datasets/multi_view_ships --name ships_pix2pix --model pix2pix --direction BtoA --num_test 114 --epoch 200

Contact

If you have any questions about the code and data, please contact. email ([email protected])

For SLAD, please refer to:

@article{zhang2024ship,
  title={Ship Landmark: An Informative Ship Image Annotation and Its Applications},
  author={Zhang, Mingxin and Zhang, Qian and Song, Ran and Rosin, Paul L and Zhang, Wei},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2024},
  publisher={IEEE}
}