Fine-Grained Self-Supervised Learning with Jigsaw puzzles for medical image classification

Wongi Park1, Jongbin Ryu1,
1Ajou University
Paper arXiv Code

Abstract

Classifying fine-grained lesions is challenging due to minor and subtle differences in medical images. This is because learning features of fine-grained lesions with highly minor differences is very difficult in training deep neural networks. Therefore, in this paper, we introduce Fine-Grained Self-Supervised Learning(FG-SSL) method for classifying subtle lesions in medical images. The proposed method progressively learns the model through hierarchical block such that the cross-correlation between the fine-grained Jigsaw puzzle and regularized original images is close to the identity matrix. We also apply hierarchical block for progressive fine-grained learning, which extracts different information in each step, to supervised learning for discovering subtle differences. Our method does not require an asymmetric model, nor does a negative sampling strategy, and is not sensitive to batch size. We evaluate the proposed fine-grained self-supervised learning method on comprehensive experiments using various medical image recognition datasets. In our experiments, the proposed method performs favorably compared to existing state-of-the-art approaches on the widely-used ISIC2018, APTOS2019, and ISIC2017 datasets.

FG-SSL Framework


Overall illustration of our framework.

Overall workflow of the proposed FG-SSL method. We generate Jigsaw puzzles and distorted images from an original training sample to learn fine-grained self-supervision. By comparing the features obtained from fine-grained Jigsaw puzzles and distorted images, we minimize the empirical cross-correlation with the identity matrix for progressively training our networks. For further details, please refer to our paper.



Factor Analysis.

To show the effectiveness of the progressive Jigsaw puzzles, we compare ours with other self-supervised learning methods in Table 2a.

Visualization

Grad-CAM [44] visualization. A*, B*, and C* represent the result of sequentially adding the Rotation [41], SimSiam [42], and BarlowTwins [23] methods with the proposed fine-grained self-supervised learning. At the same time, A, B, and C are taken from the baseline networks without our self-supervised learning.



BibTeX

@article{park2024fine,
      title={Fine-Grained Self-Supervised Learning with Jigsaw puzzles for medical image classification},
      author={Park, Wongi and Ryu, Jongbin},
      journal={Computers in Biology and Medicine},
      volume={174},
      pages={108460},
      year={2024},
      publisher={Elsevier}
    }