Skip to content

Commit d839fb2

Browse files
author
xukuan
committed
first commit
0 parents  commit d839fb2

77 files changed

Lines changed: 10108 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
6+
# C extensions
7+
*.so
8+
9+
# Distribution / packaging
10+
.Python
11+
build/
12+
develop-eggs/
13+
dist/
14+
downloads/
15+
eggs/
16+
.eggs/
17+
lib/
18+
lib64/
19+
parts/
20+
sdist/
21+
var/
22+
wheels/
23+
*.egg-info/
24+
.installed.cfg
25+
*.egg
26+
MANIFEST
27+
28+
# PyInstaller
29+
# Usually these files are written by a python script from a template
30+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
31+
*.manifest
32+
*.spec
33+
34+
# Installer logs
35+
pip-log.txt
36+
pip-delete-this-directory.txt
37+
38+
# Unit test / coverage reports
39+
htmlcov/
40+
.tox/
41+
.coverage
42+
.coverage.*
43+
.cache
44+
nosetests.xml
45+
coverage.xml
46+
*.cover
47+
.hypothesis/
48+
.pytest_cache/
49+
50+
# Translations
51+
*.mo
52+
*.pot
53+
54+
# Django stuff:
55+
*.log
56+
local_settings.py
57+
db.sqlite3
58+
59+
# Flask stuff:
60+
instance/
61+
.webassets-cache
62+
63+
# Scrapy stuff:
64+
.scrapy
65+
66+
# Sphinx documentation
67+
docs/_build/
68+
69+
# PyBuilder
70+
target/
71+
72+
# Jupyter Notebook
73+
.ipynb_checkpoints
74+
75+
# pyenv
76+
.python-version
77+
78+
# celery beat schedule file
79+
celerybeat-schedule
80+
81+
# SageMath parsed files
82+
*.sage.py
83+
84+
# Environments
85+
.env
86+
.venv
87+
env/
88+
venv/
89+
ENV/
90+
env.bak/
91+
venv.bak/
92+
93+
# Spyder project settings
94+
.spyderproject
95+
.spyproject
96+
97+
# Rope project settings
98+
.ropeproject
99+
100+
# mkdocs documentation
101+
/site
102+
103+
# mypy
104+
.mypy_cache/
105+
106+
# Dataset
107+
CityScapes/
108+
Lane/
109+
110+
# Generated
111+
scores/
112+
113+
# tmp files
114+
*.pyc
115+
runs/
116+
*.pickle
117+
*.pth
118+
saving/*
119+
*.swp

‎README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Object Descriptor
2+
3+
This repo contains source codes for the arXiv preprint ["A Robust Object Encoding Method"](https://arxiv.org/abs/2105.00327)
4+
5+
6+
## Demo
7+
Object matching comparison when the objects are non-rigid and the view is changed, left is the result of our method while right is the result of NetVLAD
8+
9+
![](experiments/demo/object-matching1.gif) ![](experiments/demo/object-matching2.gif)
10+
11+
Relocalization in KITTI
12+
13+
![](experiments/demo/kitti-relocalization.gif)
14+
15+
16+
## Dependencies
17+
* Python
18+
* PyTorch
19+
* OpenCV
20+
* Matplotlib
21+
* NumPy
22+
* Yaml
23+
24+
25+
## Data
26+
Four datasets are used in our experiments.
27+
28+
### KITTI Odometry
29+
For relocalization experiment. Three sequences are selected, and they are "00", "05" and "06".
30+
31+
### KITTI Tracking
32+
For multi-object matching experiment. Four sequences are selected, and they are "0002", "0003", "0006", "0010".
33+
34+
### VOT Datasets
35+
For single-object matching experiment. We select three sequences from VOT2019 datasets and they are "bluecar", "bus6" and "humans_corridor_occ_2_A", because the tracked objects in these sequences are included in coco datasets, which are the data we used to train mask-rcnn.
36+
37+
### OTB Datasets
38+
For single-object matching experiment. We select five sequences and they are "BlurBody", "BlurCar2", "Human2", "Human7" and "Liquor".
39+
40+
41+
## Examples
42+
43+
### Relocalization in KITTI Datasets
44+
45+
1. Extract object descrptors
46+
```
47+
python experiments/place_recogination/online_relocalization.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_MIDDLE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
48+
```
49+
50+
2. Compute precision-recall curves
51+
```
52+
python experiments/place_recogination/offline_process.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
53+
```
54+
55+
3. Compute top-K relocalization results
56+
```
57+
python experiments/place_recogination/offline_topK.py -c config/experiment_tracking.yaml -g 1 -d PATH_TO_DATASET -n PATH_TO_MIDDLE_RESULTS -s PATH_TO_SAVE_RESULTS
58+
```
59+
60+
### Object Matching in OTB, VOT or KITTI Tracking Datasets
61+
62+
* Run multi-object matching experiment in KITTI Tracking Datasets
63+
Modify the [config file](config/experiment_tracking.yaml) and run
64+
```
65+
python experiments/object_tracking/object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
66+
```
67+
68+
* Run single-object matching experiment in OTB or VOT Datasets
69+
Modify the [config file](config/experiment_tracking.yaml) and run
70+
```
71+
python experiments/object_tracking/single_object_tracking.py -c config/experiment_tracking.yaml -g 1 -s PATH_TO_SAVE_RESULTS -d PATH_TO_DATASET -m PATH_TO_MODELS
72+
```

‎config/compare_tracking.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
intervals: [1, 3, 5] # select lines, the number represent the interval
2+
title: kitti # figure title
3+
colors: ['green', 'red', 'blue', 'yellow', 'darkviolet', 'sandybrown'] # lines colors
4+
linewidth: 3 # line width
5+
xlabel: recall # x-axis name
6+
ylabel: precision # y-axis name
7+
fontsize: 20 # font size
8+
figsize: (10, 10) # fingure size, inch
9+
dpi: 100 # dots per inch

‎config/experiment_place_recognition.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
data:
2+
name: 'kitti'
3+
nclass: 81
4+
normal_size: [384, 1280] # min_size, max_size
5+
cache_in_memory: false
6+
validation_size: 96
7+
model:
8+
superpoint:
9+
cell : 8
10+
detection_threshold: 0.2
11+
maskrcnn:
12+
add_maskrcnn: true
13+
trainable_layers: 5 # backbone trainable layers
14+
fix_backbone: true
15+
backbone_type: 'resnet50'
16+
image_mean: [0.45, 0.45, 0.45]
17+
image_std: [0.225, 0.225, 0.225]
18+
gcn:
19+
descriptor_dim: 256
20+
points_encoder_dims: [2, 4, 8, 16]
21+
hidden_dim: 512
22+
dropout: 0
23+
alpha: 0.2
24+
nheads: 4
25+
nout: 2048

‎config/experiment_tracking.yaml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
data:
2+
### kitti datasets
3+
name: 'kitti'
4+
normal_size: [384, 1280] # min_size, max_size
5+
### OTB or VOT datasets
6+
# name: 'otb'
7+
# normal_size: [384, 1280] # min_size, max_size
8+
###
9+
nclass: 81
10+
cache_in_memory: false
11+
validation_size: 96
12+
model:
13+
superpoint:
14+
cell : 8
15+
detection_threshold: 0.2
16+
maskrcnn:
17+
add_maskrcnn: true
18+
trainable_layers: 5 # backbone trainable layers
19+
fix_backbone: true
20+
backbone_type: 'resnet50'
21+
image_mean: [0.45, 0.45, 0.45]
22+
image_std: [0.225, 0.225, 0.225]
23+
gcn:
24+
descriptor_dim: 256
25+
points_encoder_dims: [2, 4, 8, 16]
26+
hidden_dim: 512
27+
dropout: 0
28+
alpha: 0.2
29+
nheads: 4
30+
nout: 2048

‎config/tracking_compare_plot.yaml‎

Whitespace-only changes.

‎config/train_gcn_coco.yaml‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
data:
2+
name: 'coco'
3+
nclass: 81
4+
# TRAIN: 'coco_2014_train'
5+
TRAIN: 'coco_2014_train'
6+
VAL: 'coco_2014_minival'
7+
TEST: 'coco_2014_valminusminival'
8+
normal_size: [320, 320] # min_size, max_size
9+
cache_in_memory: false
10+
validation_size: 96
11+
augmentation:
12+
photometric:
13+
enable: true
14+
primitives: [
15+
'random_brightness', 'random_contrast', 'additive_speckle_noise',
16+
'additive_gaussian_noise', 'add_shade', 'motion_blur']
17+
params:
18+
random_brightness: {max_change: 50}
19+
random_contrast: {max_change: [0.5, 1.5]}
20+
additive_gaussian_noise: {std: [0, 10]}
21+
additive_speckle_noise: {intensity: 5}
22+
add_shade:
23+
amplitude: [-0.5, 0.5]
24+
kernel_size_interval: [100, 150]
25+
motion_blur: {max_ksize: 3}
26+
homographic:
27+
enable: false # not implemented
28+
gcn_mask:
29+
enable: false
30+
primitives: [
31+
'erode', 'dilate', 'random_region_zero', 'random_block_zero',
32+
'random_block_one']
33+
params:
34+
erode: {kernel_size: 10}
35+
dilate: {kernel_size: 10}
36+
random_region_zero:
37+
scale_x: 0.3
38+
scale_y: 0.3
39+
random_block_zero:
40+
num: 5
41+
size: 10
42+
random_block_one:
43+
num: 5
44+
size: 10
45+
warped_pair:
46+
enable: false
47+
params:
48+
translation: true
49+
rotation: true
50+
scaling: true
51+
perspective: true
52+
scaling_amplitude: 0.2
53+
perspective_amplitude_x: 0.2
54+
perspective_amplitude_y: 0.2
55+
patch_ratio: 0.85
56+
max_angle: 1.57
57+
allow_artifacts: true
58+
valid_border_margin: 3
59+
model:
60+
superpoint:
61+
cell : 8
62+
detection_threshold: 0.2
63+
mask_rcnn:
64+
add_maskrcnn: true
65+
trainable_layers: 5 # backbone trainable layers
66+
fix_backbone: true
67+
backbone_type: 'resnet50'
68+
image_mean: [0.45, 0.45, 0.45]
69+
image_std: [0.225, 0.225, 0.225]
70+
gcn:
71+
descriptor_dim: 256
72+
points_encoder_dims: [2, 4, 8, 16]
73+
hidden_dim: 512
74+
dropout: 0
75+
alpha: 0.2
76+
nheads: 4
77+
nout: 2048
78+
train:
79+
batch_szie: 16
80+
positive_margin: 1
81+
negative_margin: 0.2
82+
lambda_d: 1
83+
epochs : 100
84+
lr : 0.000001
85+
momentum : 0
86+
w_decay : 0.0001
87+
milestones : [1000, 2000, 5000, 10000, 15000] # iter
88+
gamma : 0.3
89+
checkpoint: 1000
90+
weight_lambda: [0.1, 0.1]

‎config/train_maskrcnn_coco.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
data:
2+
name: 'coco'
3+
nclass: 81
4+
# TRAIN: 'coco_2014_train'
5+
TRAIN: 'coco_2014_train'
6+
VAL: 'coco_2014_minival'
7+
TEST: 'coco_2014_valminusminival'
8+
normal_size: [320, 320] # min_size, max_size
9+
cache_in_memory: false
10+
validation_size: 96
11+
model:
12+
maskrcnn:
13+
trainable_layers: 5 # backbone trainable layers
14+
backbone_type: 'resnet50'
15+
image_mean: [0.45, 0.45, 0.45]
16+
image_std: [0.225, 0.225, 0.225]
17+
batch_size : 8
18+
epochs : 10
19+
lr : 0.00001
20+
momentum : 0
21+
w_decay : 0.0001
22+
milestones : [10000, 20000, 50000, 100000, 150000] # iter
23+
gamma : 0.3
24+
dataset_size : 10000
25+
checkpoint: 1000

0 commit comments

Comments
 (0)