Pikaliov/DCN_custom_op
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial commit: DCNv4 custom op mirror setup

Browse Source 
- Add enhanced README with project structure and quick start guide
- Initialize repository with DCNv4 CUDA extension (PyTorch module)
- Include classification, detection, and segmentation subdirectories
- Reference upstream OpenGVLab DCNv4 implementation

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Pikaliov
2026-06-11 10:30:44 +03:00
commit 1b3206b6a7
290 changed files with 41632 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
detection/configs/coco/README.md Normal file
View File

@@ -0,0 +1,51 @@
# COCO
## Introduction
Introduced by Lin et al. in [Microsoft COCO: Common Objects in Context](https://arxiv.org/pdf/1405.0312v3.pdf)
The MS COCO (Microsoft Common Objects in Context) dataset is a large-scale object detection, segmentation, key-point detection, and captioning dataset. The dataset consists of 328K images.
Splits: The first version of MS COCO dataset was released in 2014. It contains 164K images split into training (83K), validation (41K) and test (41K) sets. In 2015 additional test set of 81K images was released, including all the previous test images and 40K new images.
Based on community feedback, in 2017 the training/validation split was changed from 83K/41K to 118K/5K. The new split uses the same images and annotations. The 2017 test set is a subset of 41K images of the 2015 test set. Additionally, the 2017 release contains a new unannotated dataset of 123K images.
## Model Zoo
### Mask R-CNN + FlashInternImage
| backbone | schd | box mAP | mask mAP |Config | Download |
| :-----------------: | :---------: | :-----: |:------: | :-----: | :---: |
| FlashInternImage-T | 1x | 48.0 | 43.1 | [config](./mask_rcnn_flash_intern_image_t_fpn_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_t_fpn_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_t_fpn_1x_coco.log) |
| FlashInternImage-T | 3x | 49.5 | 44.0 | [config](././mask_rcnn_flash_intern_image_t_fpn_3x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_t_fpn_3x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_t_fpn_3x_coco.log) |
| FlashInternImage-S | 1x | 49.2 | 44.0 | [config](./mask_rcnn_flash_intern_image_s_fpn_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_s_fpn_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_s_fpn_1x_coco.log) |
| FlashInternImage-S | 3x | 50.5 | 44.9 | [config](./mask_rcnn_flash_intern_image_s_fpn_3x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_s_fpn_3x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_s_fpn_3x_coco.log) |
| FlashInternImage-B | 1x | 50.1 | 44.5 | [config](./mask_rcnn_flash_intern_image_b_fpn_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_b_fpn_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_b_fpn_1x_coco.log) |
| FlashInternImage-B | 3x | 50.6 | 45.4 | [config](./mask_rcnn_flash_intern_image_b_fpn_3x_coco.py)| [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_b_fpn_3x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/mask_rcnn_flash_internimage_b_fpn_3x_coco.log) |
- Training speed is measured with A100 GPUs using current code and may be faster than the speed in logs.
- Some logs are our recent newly trained ones. There might be slight differences between the results in logs and our paper.
- Please set `with_cp=True` to save memory if you meet `out-of-memory` issues.
### Cascade Mask R-CNN + FlashInternImage
| backbone | schd | box mAP | mask mAP | Config | Download |
| :------------: | :---------: | :-----: | :------: | :---: | :---: |
| FlashInternImage-L | 1x | 55.6 | 48.2 | [config](./cascade_flash_intern_image_l_fpn_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/cascade_flash_internimage_l_fpn_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/cascade_flash_internimage_l_fpn_1x_coco.log)
| FlashInternImage-L | 3x | 56.7 | 48.9 | [config](./cascade_flash_intern_image_l_fpn_3x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/cascade_flash_internimage_l_fpn_3x_coco.pth) |
- Training speed is measured with A100 GPUs using current code and may be faster than the speed in logs.
- Some logs are our recent newly trained ones. There might be slight differences between the results in logs and our paper.
- Please set `with_cp=True` to save memory if you meet `out-of-memory` issues.
### DINO + FlashInternImage
| backbone | lr type | pretrain | schd | box mAP | Config | Download |
| :------------: | :---------: |:---------: | :---------: | :-----: | :---: | :-----:
| FlashInternImage-T | layer-wise lr | ImageNet-1K | 1x | 54.7 | [config](./dino_4scale_flash_internimage_t_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_t_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_t_1x_coco.json) |
| FlashInternImage-S | layer-wise lr | ImageNet-1K | 1x | 55.3 | [config](./dino_4scale_flash_internimage_s_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_s_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_s_1x_coco.log) |
| FlashInternImage-B | layer-wise lr | ImageNet-1K | 1x | 56.0 | [config](./dino_4scale_flash_internimage_b_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_b_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_b_1x_coco.log) |
| FlashInternImage-L | 0.1x backbone lr | ImageNet-22K | 1x | 58.8 | [config](./dino_4scale_flash_internimage_l_1x_coco.py) | [ckpt](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_l_1x_coco.pth) \| [log](https://huggingface.co/OpenGVLab/DCNv4/resolve/main/dino_4scale_flash_internimage_l_1x_coco.log) |

148
detection/configs/coco/cascade_flash_intern_image_l_fpn_1x_coco.py Normal file
View File

@@ -0,0 +1,148 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/cascade_mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_1x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_l_22k_384.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=160,
depths=[5, 5, 22, 5],
groups=[10, 20, 40, 80],
mlp_ratio=4.,
drop_path_rate=0.4,
norm_layer='LN',
layer_scale=1.0,
offset_scale=2.0,
post_norm=True,
with_cp=False,
dcn_output_bias=True,
mlp_fc2_bias=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[160, 320, 640, 1280],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5),
roi_head=dict(
bbox_head=[
dict(
type='DCNv4FCBBoxHead',
with_dcn=False,
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.1, 0.1, 0.2, 0.2]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0)),
dict(
type='DCNv4FCBBoxHead',
with_dcn=False,
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.05, 0.05, 0.1, 0.1]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0)),
dict(
type='DCNv4FCBBoxHead',
with_dcn=False,
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.033, 0.033, 0.067, 0.067]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0))
]))
# By default, models are trained on 8 GPUs with 2 images per GPU
data = dict(samples_per_gpu=2)
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=37, layer_decay_rate=0.94,
depths=[5, 5, 22, 5]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# Bbox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.556
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.744
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.604
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.388
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.598
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.720
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.670
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.670
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.670
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.505
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.714
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.823
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.482
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.720
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.526
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.289
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.514
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.676
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.588
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.588
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.588
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.424
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.629
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.749

196
detection/configs/coco/cascade_flash_intern_image_l_fpn_3x_coco.py Normal file
View File

@@ -0,0 +1,196 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/cascade_mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_3x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_l_22k_384.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=160,
depths=[5, 5, 22, 5],
groups=[10, 20, 40, 80],
mlp_ratio=4.,
drop_path_rate=0.4,
norm_layer='LN',
layer_scale=1.0,
offset_scale=2.0,
post_norm=True,
with_cp=True,
dcn_output_bias=True,
mlp_fc2_bias=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[160, 320, 640, 1280],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5),
roi_head=dict(
bbox_head=[
dict(
type='ConvFCBBoxHead',
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.1, 0.1, 0.2, 0.2]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0)),
dict(
type='ConvFCBBoxHead',
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.05, 0.05, 0.1, 0.1]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0)),
dict(
type='ConvFCBBoxHead',
num_shared_convs=4,
num_shared_fcs=1,
in_channels=256,
conv_out_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=80,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0., 0., 0., 0.],
target_stds=[0.033, 0.033, 0.067, 0.067]),
reg_class_agnostic=False,
reg_decoded_bbox=True,
norm_cfg=dict(type='SyncBN', requires_grad=True),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='GIoULoss', loss_weight=10.0))
]))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
# augmentation strategy originates from DETR / Sparse RCNN
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='AutoAugment',
policies=[
[
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333), (576, 1333),
(608, 1333), (640, 1333), (672, 1333), (704, 1333),
(736, 1333), (768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(type='Resize',
img_scale=[(400, 1333), (500, 1333), (600, 1333)],
multiscale_mode='value',
keep_ratio=True),
dict(type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=True),
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']),
]
# we use 4 nodes to train this model, with a total batch size of 64
data = dict(
samples_per_gpu=4,
train=dict(pipeline=train_pipeline))
# optimizer
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001 * 2, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=37, layer_decay_rate=0.90,
depths=[5, 5, 22, 5], offset_lr_scale=0.01))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
resume_from = None
custom_hooks = [
dict(
type='ExpMomentumEMAHook',
resume_from=resume_from,
momentum=0.0001,
priority=49)
]
# Bbox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.567
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.754
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.615
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.410
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.612
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.729
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.685
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.685
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.685
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.532
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.733
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.825
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.490
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.732
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.537
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.301
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.527
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.677
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.600
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.600
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.600
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.445
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.644
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.751

180
detection/configs/coco/dino_4scale_flash_internimage_b_1x_coco.py Normal file
View File

@@ -0,0 +1,180 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2024 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/datasets/coco_detection.py',
'../_base_/default_runtime.py',
'../_base_/schedules/schedule_1x.py',
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_b_1k_224.pth'
model = dict(
type='DINO',
backbone=dict(
type='FlashInternImage',
core_op='DCNv4',
channels=112,
depths=[4, 4, 21, 4],
groups=[7, 14, 28, 56],
mlp_ratio=4.,
drop_path_rate=0.3,
norm_layer='LN',
layer_scale=1.0,
offset_scale=0.5,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
neck=dict(
type='ChannelMapper',
in_channels=[224, 448, 896],
kernel_size=1,
out_channels=256,
act_cfg=None,
norm_cfg=dict(type='GN', num_groups=32),
num_outs=4),
bbox_head=dict(
type='DINOHead',
num_query=900,
num_classes=80,
in_channels=2048,
sync_cls_avg_factor=True,
as_two_stage=True,
with_box_refine=True,
dn_cfg=dict(
type='CdnQueryGenerator',
noise_scale=dict(label=0.5, box=1.0),
group_cfg=dict(dynamic=True, num_groups=None, num_dn_queries=100)),
transformer=dict(
type='DinoTransformer',
two_stage_num_proposals=900,
encoder=dict(
type='DetrTransformerEncoder',
num_layers=6,
transformerlayers=dict(
type='BaseTransformerLayer',
attn_cfgs=dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
feedforward_channels=2048,
ffn_dropout=0.0, # 0.1 for DeformDETR
operation_order=('self_attn', 'norm', 'ffn', 'norm'))),
decoder=dict(
type='DinoTransformerDecoder',
num_layers=6,
return_intermediate=True,
transformerlayers=dict(
type='DetrTransformerDecoderLayer',
attn_cfgs=[
dict(
type='MultiheadAttention',
embed_dims=256,
num_heads=8,
dropout=0.0),
dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
],
feedforward_channels=2048,
ffn_dropout=0.0,
operation_order=('self_attn', 'norm', 'cross_attn', 'norm',
'ffn', 'norm')))),
positional_encoding=dict(
type='SinePositionalEncoding',
num_feats=128,
temperature=20,
normalize=True),
loss_cls=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=5.0),
loss_iou=dict(type='GIoULoss', loss_weight=2.0)),
# training and testing settings
train_cfg=dict(
assigner=dict(
type='HungarianAssigner',
cls_cost=dict(type='FocalLossCost', weight=2.0),
reg_cost=dict(type='BBoxL1Cost', weight=5.0, box_format='xywh'),
iou_cost=dict(type='IoUCost', iou_mode='giou', weight=2.0))),
test_cfg=dict(max_per_img=300))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
# train_pipeline, NOTE the img_scale and the Pad's size_divisor is different
# from the default setting in mmdet.
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='AutoAugment',
policies=[
[
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(
type='Resize',
img_scale=[(400, 4200), (500, 4200), (600, 4200)],
multiscale_mode='value',
keep_ratio=True),
dict(
type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=False),
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
# By default, models are trained on 8 GPUs with 4 images per GPU
data = dict(
samples_per_gpu=4,
train=dict(pipeline=train_pipeline))
# optimizer
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0002, weight_decay=0.0001,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=0.9,
depths=[4, 4, 21, 4]))
optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=0.1, norm_type=2))
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[11])
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# fp16 = dict(loss_scale=512.)

184
detection/configs/coco/dino_4scale_flash_internimage_l_1x_coco.py Normal file
View File

@@ -0,0 +1,184 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2024 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/datasets/coco_detection.py',
'../_base_/default_runtime.py',
'../_base_/schedules/schedule_1x.py',
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_l_22k_384.pth'
model = dict(
type='DINO',
backbone=dict(
type='FlashInternImage',
core_op='DCNv4',
channels=160,
depths=[5, 5, 22, 5],
groups=[10, 20, 40, 80],
mlp_ratio=4.,
drop_path_rate=0.4,
norm_layer='LN',
layer_scale=1.0,
offset_scale=2.0,
post_norm=True,
with_cp=True,
dcn_output_bias=True,
mlp_fc2_bias=True,
dw_kernel_size=3,
out_indices=(1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
neck=dict(
type='ChannelMapper',
in_channels=[320, 640, 1280],
kernel_size=1,
out_channels=256,
act_cfg=None,
# norm_cfg=norm_cfg,
norm_cfg=dict(type='GN', num_groups=32),
num_outs=4),
bbox_head=dict(
type='DINOHead',
num_query=900,
num_classes=80,
in_channels=2048,
sync_cls_avg_factor=True,
as_two_stage=True,
with_box_refine=True,
dn_cfg=dict(
type='CdnQueryGenerator',
noise_scale=dict(label=0.5, box=1.0),
group_cfg=dict(dynamic=True, num_groups=None, num_dn_queries=100)),
transformer=dict(
type='DinoTransformer',
two_stage_num_proposals=900,
encoder=dict(
type='DetrTransformerEncoder',
num_layers=6,
transformerlayers=dict(
type='BaseTransformerLayer',
attn_cfgs=dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
feedforward_channels=2048,
ffn_dropout=0.0, # 0.1 for DeformDETR
operation_order=('self_attn', 'norm', 'ffn', 'norm'))),
decoder=dict(
type='DinoTransformerDecoder',
num_layers=6,
return_intermediate=True,
transformerlayers=dict(
type='DetrTransformerDecoderLayer',
attn_cfgs=[
dict(
type='MultiheadAttention',
embed_dims=256,
num_heads=8,
dropout=0.0),
dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
],
feedforward_channels=2048,
ffn_dropout=0.0,
operation_order=('self_attn', 'norm', 'cross_attn', 'norm',
'ffn', 'norm')))),
positional_encoding=dict(
type='SinePositionalEncoding',
num_feats=128,
temperature=20,
normalize=True),
loss_cls=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=5.0),
loss_iou=dict(type='GIoULoss', loss_weight=2.0)),
# training and testing settings
train_cfg=dict(
assigner=dict(
type='HungarianAssigner',
cls_cost=dict(type='FocalLossCost', weight=2.0),
reg_cost=dict(type='BBoxL1Cost', weight=5.0, box_format='xywh'),
iou_cost=dict(type='IoUCost', iou_mode='giou', weight=2.0))),
test_cfg=dict(max_per_img=300))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
# train_pipeline, NOTE the img_scale and the Pad's size_divisor is different
# from the default setting in mmdet.
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='AutoAugment',
policies=[
[
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(
type='Resize',
img_scale=[(400, 4200), (500, 4200), (600, 4200)],
multiscale_mode='value',
keep_ratio=True),
dict(
type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=False),
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
# By default, models are trained on 8 GPUs with 2 images per GPU
data = dict(
samples_per_gpu=2,
train=dict(pipeline=train_pipeline))
# optimizer
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
paramwise_cfg=dict(
custom_keys={
'backbone': dict(lr_mult=0.1),
}))
optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=0.1, norm_type=2))
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[11])
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# fp16 = dict(loss_scale=512.)

180
detection/configs/coco/dino_4scale_flash_internimage_s_1x_coco.py Normal file
View File

@@ -0,0 +1,180 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/datasets/coco_detection.py',
'../_base_/default_runtime.py',
'../_base_/schedules/schedule_1x.py',
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_s_1k_224.pth'
model = dict(
type='DINO',
backbone=dict(
type='FlashInternImage',
core_op='DCNv4',
channels=80,
depths=[4, 4, 21, 4],
groups=[5, 10, 20, 40],
mlp_ratio=4.,
drop_path_rate=0.3,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
neck=dict(
type='ChannelMapper',
in_channels=[160, 320, 640],
kernel_size=1,
out_channels=256,
act_cfg=None,
norm_cfg=dict(type='GN', num_groups=32),
num_outs=4),
bbox_head=dict(
type='DINOHead',
num_query=900,
num_classes=80,
in_channels=2048,
sync_cls_avg_factor=True,
as_two_stage=True,
with_box_refine=True,
dn_cfg=dict(
type='CdnQueryGenerator',
noise_scale=dict(label=0.5, box=1.0),
group_cfg=dict(dynamic=True, num_groups=None, num_dn_queries=100)),
transformer=dict(
type='DinoTransformer',
two_stage_num_proposals=900,
encoder=dict(
type='DetrTransformerEncoder',
num_layers=6,
transformerlayers=dict(
type='BaseTransformerLayer',
attn_cfgs=dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
feedforward_channels=2048,
ffn_dropout=0.0, # 0.1 for DeformDETR
operation_order=('self_attn', 'norm', 'ffn', 'norm'))),
decoder=dict(
type='DinoTransformerDecoder',
num_layers=6,
return_intermediate=True,
transformerlayers=dict(
type='DetrTransformerDecoderLayer',
attn_cfgs=[
dict(
type='MultiheadAttention',
embed_dims=256,
num_heads=8,
dropout=0.0),
dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
],
feedforward_channels=2048,
ffn_dropout=0.0,
operation_order=('self_attn', 'norm', 'cross_attn', 'norm',
'ffn', 'norm')))),
positional_encoding=dict(
type='SinePositionalEncoding',
num_feats=128,
temperature=20,
normalize=True),
loss_cls=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=5.0),
loss_iou=dict(type='GIoULoss', loss_weight=2.0)),
# training and testing settings
train_cfg=dict(
assigner=dict(
type='HungarianAssigner',
cls_cost=dict(type='FocalLossCost', weight=2.0),
reg_cost=dict(type='BBoxL1Cost', weight=5.0, box_format='xywh'),
iou_cost=dict(type='IoUCost', iou_mode='giou', weight=2.0))),
test_cfg=dict(max_per_img=300))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
# train_pipeline, NOTE the img_scale and the Pad's size_divisor is different
# from the default setting in mmdet.
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='AutoAugment',
policies=[
[
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(
type='Resize',
img_scale=[(400, 4200), (500, 4200), (600, 4200)],
multiscale_mode='value',
keep_ratio=True),
dict(
type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=False),
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
# By default, models are trained on 8 GPUs with 4 images per GPU
data = dict(
samples_per_gpu=4,
train=dict(pipeline=train_pipeline))
# optimizer
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0002, weight_decay=0.0001,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=0.9,
depths=[4, 4, 21, 4]))
optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=0.1, norm_type=2))
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[11])
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# fp16 = dict(loss_scale=512.)

179
detection/configs/coco/dino_4scale_flash_internimage_t_1x_coco.py Normal file
View File

@@ -0,0 +1,179 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/datasets/coco_detection.py',
'../_base_/default_runtime.py',
'../_base_/schedules/schedule_1x.py',
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_t_1k_224.pth'
model = dict(
type='DINO',
backbone=dict(
type='FlashInternImage',
core_op='DCNv4',
channels=64,
depths=[4, 4, 18, 4],
groups=[4, 8, 16, 32],
mlp_ratio=4.,
drop_path_rate=0.2,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=False,
with_cp=True,
out_indices=(1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
neck=dict(
type='ChannelMapper',
in_channels=[128, 256, 512],
kernel_size=1,
out_channels=256,
act_cfg=None,
norm_cfg=dict(type='GN', num_groups=32),
num_outs=4),
bbox_head=dict(
type='DINOHead',
num_query=900,
num_classes=80,
in_channels=2048,
sync_cls_avg_factor=True,
as_two_stage=True,
with_box_refine=True,
dn_cfg=dict(
type='CdnQueryGenerator',
noise_scale=dict(label=0.5, box=1.0),
group_cfg=dict(dynamic=True, num_groups=None, num_dn_queries=100)),
transformer=dict(
type='DinoTransformer',
two_stage_num_proposals=900,
encoder=dict(
type='DetrTransformerEncoder',
num_layers=6,
transformerlayers=dict(
type='BaseTransformerLayer',
attn_cfgs=dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
feedforward_channels=2048,
ffn_dropout=0.0, # 0.1 for DeformDETR
operation_order=('self_attn', 'norm', 'ffn', 'norm'))),
decoder=dict(
type='DinoTransformerDecoder',
num_layers=6,
return_intermediate=True,
transformerlayers=dict(
type='DetrTransformerDecoderLayer',
attn_cfgs=[
dict(
type='MultiheadAttention',
embed_dims=256,
num_heads=8,
dropout=0.0),
dict(
type='MultiScaleDeformableAttention',
embed_dims=256,
dropout=0.0),
],
feedforward_channels=2048,
ffn_dropout=0.0,
operation_order=('self_attn', 'norm', 'cross_attn', 'norm',
'ffn', 'norm')))),
positional_encoding=dict(
type='SinePositionalEncoding',
num_feats=128,
temperature=20,
normalize=True),
loss_cls=dict(
type='FocalLoss',
use_sigmoid=True,
gamma=2.0,
alpha=0.25,
loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=5.0),
loss_iou=dict(type='GIoULoss', loss_weight=2.0)),
# training and testing settings
train_cfg=dict(
assigner=dict(
type='HungarianAssigner',
cls_cost=dict(type='FocalLossCost', weight=2.0),
reg_cost=dict(type='BBoxL1Cost', weight=5.0, box_format='xywh'),
iou_cost=dict(type='IoUCost', iou_mode='giou', weight=2.0))),
test_cfg=dict(max_per_img=300))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
# train_pipeline, NOTE the img_scale and the Pad's size_divisor is different
# from the default setting in mmdet.
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(
type='AutoAugment',
policies=[
[
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(
type='Resize',
img_scale=[(400, 4200), (500, 4200), (600, 4200)],
multiscale_mode='value',
keep_ratio=True),
dict(
type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=False),
dict(
type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
# By default, models are trained on 8 GPUs with 4 images per GPU
data = dict(
samples_per_gpu=4,
train=dict(pipeline=train_pipeline))
# optimizer
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0002, weight_decay=0.0001,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=30, layer_decay_rate=0.9,
depths=[4, 4, 18, 4]))
optimizer_config = dict(_delete_=True, grad_clip=dict(max_norm=0.1, norm_type=2))
# learning policy
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[11])
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# fp16 = dict(loss_scale=512.)

83
detection/configs/coco/mask_rcnn_flash_intern_image_b_fpn_1x_coco.py Normal file
View File

@@ -0,0 +1,83 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_1x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_b_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=112,
depths=[4, 4, 21, 4],
groups=[7, 14, 28, 56],
mlp_ratio=4.,
drop_path_rate=0.3,
norm_layer='LN',
layer_scale=1.0,
offset_scale=0.5,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[112, 224, 448, 896],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5),
)
# By default, models are trained on 8 GPUs with 2 images per GPU
data = dict(samples_per_gpu=2)
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=1.0,
depths=[4, 4, 21, 4]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# Bbox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.5005
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.717
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.543
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.322
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.540
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.652
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.617
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.617
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.617
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.433
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.658
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.774
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.445
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.687
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.478
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.244
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.477
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.637
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.556
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.556
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.556
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.375
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.595
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.720

125
detection/configs/coco/mask_rcnn_flash_intern_image_b_fpn_3x_coco.py Normal file
View File

@@ -0,0 +1,125 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_3x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_b_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=112,
depths=[4, 4, 21, 4],
groups=[7, 14, 28, 56],
mlp_ratio=4.,
drop_path_rate=0.4,
norm_layer='LN',
layer_scale=1.0,
offset_scale=0.5,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[112, 224, 448, 896],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5))
# By default, models are trained on 8 GPUs with 2 images per GPU
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='AutoAugment',
policies=[
[
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333), (576, 1333),
(608, 1333), (640, 1333), (672, 1333), (704, 1333),
(736, 1333), (768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(type='Resize',
img_scale=[(400, 1333), (500, 1333), (600, 1333)],
multiscale_mode='value',
keep_ratio=True),
dict(type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=True),
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']),
]
# we use 4 nodes to train this model, with a total batch size of 64
data = dict(
samples_per_gpu=4,
train=dict(pipeline=train_pipeline))
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001 * 2, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=0.9,
depths=[4, 4, 21, 4], offset_lr_scale=0.01))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# Bbox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.506
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.726
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.554
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.361
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.549
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.651
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.622
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.622
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.622
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.476
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.661
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.764
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.454
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.700
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.492
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.270
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.492
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.638
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.565
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.565
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.565
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.408
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.607
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.717

84
detection/configs/coco/mask_rcnn_flash_intern_image_s_fpn_1x_coco.py Normal file
View File

@@ -0,0 +1,84 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_1x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_s_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=80,
depths=[4, 4, 21, 4],
groups=[5, 10, 20, 40],
mlp_ratio=4.,
drop_path_rate=0.3,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[80, 160, 320, 640],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5),
)
# By default, models are trained on 8 GPUs with 2 images per GPU
data = dict(samples_per_gpu=2)
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=1.0,
depths=[4, 4, 21, 4]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# BBox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.492
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.707
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.539
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.328
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.531
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.647
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.609
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.609
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.609
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.431
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.650
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.768
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.440
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.678
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.476
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.245
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.470
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.633
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.551
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.551
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.551
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.372
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.591
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.714

126
detection/configs/coco/mask_rcnn_flash_intern_image_s_fpn_3x_coco.py Normal file
View File

@@ -0,0 +1,126 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_3x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_s_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=80,
depths=[4, 4, 21, 4],
groups=[5, 10, 20, 40],
mlp_ratio=4.,
drop_path_rate=0.4,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=True,
with_cp=True,
dw_kernel_size=3,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[80, 160, 320, 640],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5))
# By default, models are trained on 8 GPUs with 2 images per GPU
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='AutoAugment',
policies=[
[
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333), (576, 1333),
(608, 1333), (640, 1333), (672, 1333), (704, 1333),
(736, 1333), (768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(type='Resize',
img_scale=[(400, 1333), (500, 1333), (600, 1333)],
multiscale_mode='value',
keep_ratio=True),
dict(type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=True),
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']),
]
# we use 4 nodes to train this model, with a total batch size of 64
data = dict(
samples_per_gpu=8,
train=dict(pipeline=train_pipeline))
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001 * 2, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=33, layer_decay_rate=1.0,
depths=[4, 4, 21, 4]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# BBox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.505
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.720
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.552
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.341
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.545
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.647
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.623
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.623
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.623
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.461
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.657
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.769
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.449
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.690
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.484
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.252
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.480
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.635
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.560
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.560
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.560
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.396
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.596
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.715

81
detection/configs/coco/mask_rcnn_flash_intern_image_t_fpn_1x_coco.py Normal file
View File

@@ -0,0 +1,81 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_1x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_t_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=64,
depths=[4, 4, 18, 4],
groups=[4, 8, 16, 32],
mlp_ratio=4.,
drop_path_rate=0.2,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=False,
with_cp=True,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[64, 128, 256, 512],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5)
)
# By default, models are trained on 8 GPUs with 2 images per GPU
data = dict(samples_per_gpu=2)
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=30, layer_decay_rate=1.0,
depths=[4, 4, 18, 4]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# BBox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.480
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.695
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.528
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.303
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.515
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.629
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.599
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.599
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.599
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.408
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.637
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.750
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.431
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.667 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.463 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.225
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.461
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.622
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.543
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.543
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.543
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.352
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.581
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.705

124
detection/configs/coco/mask_rcnn_flash_intern_image_t_fpn_3x_coco.py Normal file
View File

@@ -0,0 +1,124 @@
# --------------------------------------------------------
# DCNv4
# Copyright (c) 2023 OpenGVLab
# Licensed under The MIT License [see LICENSE for details]
# --------------------------------------------------------
_base_ = [
'../_base_/models/mask_rcnn_r50_fpn.py',
'../_base_/datasets/coco_instance.py',
'../_base_/schedules/schedule_3x.py',
'../_base_/default_runtime.py'
]
pretrained = 'https://huggingface.co/OpenGVLab/DCNv4/resolve/main/flash_intern_image_t_1k_224.pth'
model = dict(
backbone=dict(
_delete_=True,
type='FlashInternImage',
core_op='DCNv4',
channels=64,
depths=[4, 4, 18, 4],
groups=[4, 8, 16, 32],
mlp_ratio=4.,
drop_path_rate=0.2,
norm_layer='LN',
layer_scale=1.0,
offset_scale=1.0,
post_norm=False,
with_cp=True,
out_indices=(0, 1, 2, 3),
init_cfg=dict(type='Pretrained', checkpoint=pretrained)),
# We leverage the FPN implemented in ViTDet for stable training,
# and we don't benefit from this FPN in terms of performance.
neck=dict(
type='FPN_vitdet',
in_channels=[64, 128, 256, 512],
out_channels=256,
norm_cfg=dict(type='LN', requires_grad=True),
use_residual=True,
num_outs=5),)
# By default, models are trained on 8 GPUs with 2 images per GPU
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True, with_mask=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='AutoAugment',
policies=[
[
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333), (576, 1333),
(608, 1333), (640, 1333), (672, 1333), (704, 1333),
(736, 1333), (768, 1333), (800, 1333)],
multiscale_mode='value',
keep_ratio=True)
],
[
dict(type='Resize',
img_scale=[(400, 1333), (500, 1333), (600, 1333)],
multiscale_mode='value',
keep_ratio=True),
dict(type='RandomCrop',
crop_type='absolute_range',
crop_size=(384, 600),
allow_negative_crop=True),
dict(type='Resize',
img_scale=[(480, 1333), (512, 1333), (544, 1333),
(576, 1333), (608, 1333), (640, 1333),
(672, 1333), (704, 1333), (736, 1333),
(768, 1333), (800, 1333)],
multiscale_mode='value',
override=True,
keep_ratio=True)
]
]),
dict(type='Normalize', **img_norm_cfg),
dict(type='Pad', size_divisor=32),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels', 'gt_masks']),
]
# we use 4 nodes to train this model, with a total batch size of 64
data = dict(
samples_per_gpu=2,
train=dict(pipeline=train_pipeline))
optimizer = dict(
_delete_=True, type='AdamW', lr=0.0001, weight_decay=0.05,
constructor='CustomLayerDecayOptimizerConstructor',
paramwise_cfg=dict(num_layers=30, layer_decay_rate=1.0,
depths=[4, 4, 18, 4]))
optimizer_config = dict(grad_clip=None)
# fp16 = dict(loss_scale=dict(init_scale=512))
evaluation = dict(save_best='auto')
checkpoint_config = dict(
interval=1,
max_keep_ckpts=1,
save_last=True,
)
# BBox
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.495
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.707
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.543
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.339
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.532
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.641
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.607
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.607
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.607
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.443
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.643
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.752
# Segm
# Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.440
# Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = 0.677
# Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = 0.474
# Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.255
# Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.473
# Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.624
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.545
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = 0.545
# Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = 0.545
# Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.380
# Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.582
# Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.704