Request to add DINO object detector

[tcourat]

Model description

DINO (do not confuse it with the DINO image encoder from META) is a SOTA DETR-like object detector, improving the denoising training, query initialization, and box prediction. It is based on a combination of the enhancement brought by DN-DETR , DAB-DETR , and Deformable DETR.
As it is used as backbone for many other DETR architecture (e.g Co-DETR which is SOTA on COCO test-dev : https://paperswithcode.com/sota/object-detection-on-coco), it would be nice to have it in transformers.

Additionnaly, a slighly improved version of DINO, called Stable-DINO, also exist, and should be easily added on top of DINO (only a few lines of code).

Open source status

• The model implementation is available
• The model weights are available

Provide useful links for the implementation

Paper : https://arxiv.org/abs/2203.03605
Code : https://github.com/IDEA-Research/DINO

Code for Stable-DINO : https://github.com/IDEA-Research/Stable-DINO

[qubvel]

Hi @court thanks for opening the issue! That would be a great addition to our object-detection series, would you like to work on it, or should we seek anyone who will take this up?

[tcourat]

Hi @tcourat thanks for opening the issue! That would be a great addition to our object-detection series, would you like to work on it, or should we seek anyone who will take this up?

I do not plan to work on it at the moment as I am not familiar with HF integration practices, but since I believe this would be a great addition to HF I created this request.

[Ruhaan838]

Hi, @tcourat @qubvel. If you don't mind, I need to contribute to transformers. Can I do that?

[qubvel]

Hi @Ruhaan838, sure, we would appreciate your contribution 🤗 Feel free to ping me when PR is open

[Ruhaan838]

Hi @qubvel I got a problem with the official repo the problem is like this

• so first I fork the DINO repo and tried to understand it at some point, I got the codebase and I started to make the inference to do the simple forward pass it required some dependence for the model building named MultiScaleDeformableAttention
• I am trying to install it from pypi and it's not installing gives me some weird error

ERROR: Could not find a version that satisfies the requirement MultiScaleDeformableAttention (from versions: none)
ERROR: No matching distribution found for MultiScaleDeformableAttention

• so if you have any other Idea in your mind can you tell me I am trying to figure out this thing for 2 days.

help me if you have time :)

[qubvel]

Hey, you should probably use another python version to install it, check pypi for requirements

[qubvel]

Alternatively you can try to clone and install this folder
https://github.com/fundamentalvision/Deformable-DETR/tree/main/models/ops

[konstantinos-p]

Hello! I've noticed that this issue has been inactive for a while. I have to use the DINO DETR for some work tasks so I implemented a version for the transformers library. @qubvel if you'd like you can check my draft PR and we can take it from there?