Gibson Data Structure for Reproduce

[Tllokn]

Hello,
Thanks for your great work. I am trying to reproduce the work to gibson datasets.
I followed #8 :

1. add a parameter "--task_config tasks/objectnav_gibson.yaml"
2. replace the line in the file agents/sem_exp.py from ObjectGoal_Env21 to ObjectGoal_Env
3. I also Force the task only work on the Darden scene, since I am sure I have all assets of this scene. I changed code: in envs/habitat/__init__.py

    if "*" in basic_config.DATASET.CONTENT_SCENES:
        content_dir = os.path.join("data/datasets/objectnav/gibson/v1.1/" + args.split, "content")
        scenes = _get_scenes_from_folder(content_dir)
        # force it to be only Darden, since we only have this in scene dataset.
        scenes = ['Darden.glb']

4. To match the dataset, I changed the objectnav_gibson.yaml with:

DATASET:
  TYPE: PointNav-v1
  SPLIT: train
  DATA_PATH: "data/datasets/objectnav/gibson/v1.1/{split}/{split}.json.gz"
  EPISODES_DIR: "data/datasets/objectnav/gibson/v1.1/{split}/"
  SCENES_DIR: "/data/scene_datasets"

With all these changes, I got an error:

Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/L3MVN/envs/habitat/utils/vector_env.py", line 176, in _worker_env
    env = env_fn(*env_fn_args)
  File "/L3MVN/envs/habitat/__init__.py", line 24, in make_env_fn
    env = Sem_Exp_Env_Agent(args=args, rank=rank,
  File "/L3MVN/agents/sem_exp.py", line 32, in __init__
    super().__init__(args, rank, config_env, dataset)
  File "/L3MVN/envs/habitat/objectgoal_env.py", line 26, in __init__
    super().__init__(config_env, dataset)
  File "/habitat-lab/habitat/core/env.py", line 377, in __init__
    self._env = Env(config, dataset)
  File "/habitat-lab/habitat/core/env.py", line 107, in __init__
    self._sim = make_sim(
  File "/habitat-lab/habitat/sims/registration.py", line 19, in make_sim
    return _sim(**kwargs)
  File "/habitat-lab/habitat/sims/habitat_simulator/habitat_simulator.py", line 276, in __init__
    super().__init__(self.sim_config)
  File "<attrs generated init habitat_sim.simulator.Simulator>", line 11, in __init__
    self.__attrs_post_init__()
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 122, in __attrs_post_init__
    self.__set_from_config(self.config)
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 260, in __set_from_config
    self._config_backend(config)
  File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 204, in _config_backend
    super().__init__(config.sim_cfg, config.metadata_mediator)
AssertionError: ESP_CHECK failed: Missing (at least) one of scene dataset attributes, stage attributes, or dataset scene attributes for scene '/data/scene_datasets/gibson_semantic//Darden.glb'.  Likely an invalid scene name.

I guess this error is related to how I organize the data of Gibson, since I have seen similar problem when I didn't put data of hm3d correctly. But given these 2 dataset use different data format, I do not have a concrete idea on how to solving this problem. Can you share yours? My data structure looks like below

My data folder looks like this:
Data:

Data
|-- datasets
|-- matterport_category_mappings.tsv
|-- object_norm_inv_perplexity.npy
|-- objectgoal_hm3d
|-- scene_datasets
`-- versioned_data

datasets

datasets
|-- objectnav
|   `-- gibson
|       `-- v1.1
|           |-- train
|           |   |-- content
|           |   |   |-- Allensville.glb.json.gz
|           |   |   |-- Beechwood.glb.json.gz
|           |   |   |-- Benevolence.glb.json.gz
|           |   |   |-- Coffeen.glb.json.gz
|           |   |   |-- Cosmos.glb.json.gz
|           |   |   |-- Forkland.glb.json.gz
|           |   |   |-- Hanson.glb.json.gz
|           |   |   |-- Hiteman.glb.json.gz
|           |   |   |-- Klickitat.glb.json.gz
|           |   |   |-- Lakeville.glb.json.gz
|           |   |   |-- Leonardo.glb.json.gz
|           |   |   |-- Lindenwood.glb.json.gz
|           |   |   |-- Marstons.glb.json.gz
|           |   |   |-- Merom.glb.json.gz
|           |   |   |-- Mifflinburg.glb.json.gz
|           |   |   |-- Newfields.glb.json.gz
|           |   |   |-- Onaga.glb.json.gz
|           |   |   |-- Pinesdale.glb.json.gz
|           |   |   |-- Pomaria.glb.json.gz
|           |   |   |-- Ranchester.glb.json.gz
|           |   |   |-- Shelbyville.glb.json.gz
|           |   |   |-- Stockman.glb.json.gz
|           |   |   |-- Tolstoy.glb.json.gz
|           |   |   |-- Wainscott.glb.json.gz
|           |   |   `-- Woodbine.glb.json.gz
|           |   |-- train.json.gz
|           |   `-- train_info.pbz2
|           `-- val
|               |-- content
|               |   |-- Collierville.glb.json.gz
|               |   |-- Collierville_episodes.json.gz
|               |   |-- Corozal.glb.json.gz
|               |   |-- Corozal_episodes.json.gz
|               |   |-- Darden.glb.json.gz
|               |   |-- Darden_episodes.json.gz
|               |   |-- Markleeville.glb.json.gz
|               |   |-- Markleeville_episodes.json.gz
|               |   |-- Wiconisco.glb.json.gz
|               |   `-- Wiconisco_episodes.json.gz
|               |-- val.json.gz
|               `-- val_info.pbz2
`-- objectnav_gibson_v1.1.zip

scene_datasets

scene_datasets
|-- gibson_semantic
|   |-- Ackermanville.glb
|   |-- Ackermanville.navmesh
|   |-- Darden.glb
|   `-- Darden.navmesh
`-- hm3d_v0.2
    |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |-- hm3d_annotated_basis.scene_dataset_config.json
    `-- val

versioned_data

versioned_data
`-- hm3d-0.2
    |-- hm3d
    |   |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |   |-- hm3d_annotated_basis.scene_dataset_config.json
    |   `-- val
    |-- val-configs-files.json.gz
    |-- val-habitat-files.json.gz
    |-- val-semantic-annots-files.json.gz
    `-- val-semantic-configs-files.json.gz

[weizhenFrank]

I solved this problem by downloading the gibson with semantic annotation.

https://github.com/StanfordVL/3DSceneGraph/tree/master

According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.

And we need habitat-sim v-0.1.4 to install
https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4

Also, by facebookresearch/habitat-sim#648

we need python setup.py build --build-datatool
then we copy newest tools/gen_gibson_semantics.sh

then we run in the v0.1.4 habitat-sim,
tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

This way help me to get the gibson with semantic info.

[hhhuxy]

I solved this problem by downloading the gibson with semantic annotation.

https://github.com/StanfordVL/3DSceneGraph/tree/master

According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.

And we need habitat-sim v-0.1.4 to install https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4

Also, by facebookresearch/habitat-sim#648

we need python setup.py build --build-datatool then we copy newest tools/gen_gibson_semantics.sh

then we run in the v0.1.4 habitat-sim, tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

This way help me to get the gibson with semantic info.

@weizhenFrank Thank you for your advice, I follow the above instructions, but I am still confused.

When I download gibson scene dataset for habitat, there are only *.ply and *.navmesh files in the folder.
In tools/gen_gibson_semantics.sh L24, "${TOOLS_DIR}"/../build/utils/datatool/datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply, however, there is no mesh.obj file in my gibson dataset thus leads to errors.

Did I downloaded the wrong link for gibson? Could u please share which part of gibson dataset u downloaded, or how to obtain mesh.obj file?

Thank you in advance.

[luyuanOo]

I solved this problem by downloading the gibson with semantic annotation.
https://github.com/StanfordVL/3DSceneGraph/tree/master
According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.
And we need habitat-sim v-0.1.4 to install https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4
Also, by facebookresearch/habitat-sim#648
we need python setup.py build --build-datatool then we copy newest tools/gen_gibson_semantics.sh
then we run in the v0.1.4 habitat-sim, tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic
This way help me to get the gibson with semantic info.

@weizhenFrank Thank you for your advice, I follow the above instructions, but I am still confused.

When I download gibson scene dataset for habitat, there are only *.ply and *.navmesh files in the folder. In tools/gen_gibson_semantics.sh L24, "${TOOLS_DIR}"/../build/utils/datatool/datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply, however, there is no mesh.obj file in my gibson dataset thus leads to errors.

Did I downloaded the wrong link for gibson? Could u please share which part of gibson dataset u downloaded, or how to obtain mesh.obj file?

Thank you in advance.

你得去这里https://docs.google.com/forms/d/e/1FAIpQLScWlx5Z1DM1M-wTSXaa6zV8lTFkPmTHW1LqMsoCBDWsTDjBkQ/viewform下载"Tiny" Partition (8.02 GB): gibson_tiny.tar.gz。

然后在https://docs.google.com/forms/d/e/1FAIpQLScnlTFPUYtBqlN8rgj_1J3zJm44bIhmIx8gDhOqiJyTwja8vw/viewform下载：3DSceneGraph_tiny.zip。

最后使用这个命令tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

补充：
gen_gibson_semantics.sh这个文件可以在https://gitlab.ciirc.cvut.cz/steidsta/aag-habitat-sim/-/blob/master/tools/gen_gibson_semantics.sh找到

[womeizhaol]

I solved this problem by downloading the gibson with semantic annotation.
https://github.com/StanfordVL/3DSceneGraph/tree/master
According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.
And we need habitat-sim v-0.1.4 to install https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4
Also, by facebookresearch/habitat-sim#648
we need python setup.py build --build-datatool then we copy newest tools/gen_gibson_semantics.sh
then we run in the v0.1.4 habitat-sim, tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic
This way help me to get the gibson with semantic info.

@weizhenFrank Thank you for your advice, I follow the above instructions, but I am still confused.
When I download gibson scene dataset for habitat, there are only *.ply and *.navmesh files in the folder. In tools/gen_gibson_semantics.sh L24, "${TOOLS_DIR}"/../build/utils/datatool/datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply, however, there is no mesh.obj file in my gibson dataset thus leads to errors.
Did I downloaded the wrong link for gibson? Could u please share which part of gibson dataset u downloaded, or how to obtain mesh.obj file?
Thank you in advance.

你得去这里https://docs.google.com/forms/d/e/1FAIpQLScWlx5Z1DM1M-wTSXaa6zV8lTFkPmTHW1LqMsoCBDWsTDjBkQ/viewform下载"Tiny" Partition (8.02 GB): gibson_tiny.tar.gz。

然后在https://docs.google.com/forms/d/e/1FAIpQLScnlTFPUYtBqlN8rgj_1J3zJm44bIhmIx8gDhOqiJyTwja8vw/viewform下载：3DSceneGraph_tiny.zip。

最后使用这个命令tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic

补充： gen_gibson_semantics.sh这个文件可以在https://gitlab.ciirc.cvut.cz/steidsta/aag-habitat-sim/-/blob/master/tools/gen_gibson_semantics.sh找到

我想使用让Vlfm在gibson数据集上测评，但我下载的gibson（10.8g）只有glb和nvmesh，我该如何下载带有语义的，上面的链接失效了，能在分享一下吗

[womeizhaol]

I solved this problem by downloading the gibson with semantic annotation.
https://github.com/StanfordVL/3DSceneGraph/tree/master
According to https://github.com/facebookresearch/habitat-sim/blob/main/DATASETS.md#gibson-and-3dscenegraph-datasets.
And we need habitat-sim v-0.1.4 to install https://github.com/facebookresearch/habitat-sim/releases/tag/v0.1.4
Also, by facebookresearch/habitat-sim#648
we need python setup.py build --build-datatool then we copy newest tools/gen_gibson_semantics.sh
then we run in the v0.1.4 habitat-sim, tools/gen_gibson_semantics.sh ~/projects/L3MVN/data/gibson_tiny_data_semantics/automated_graph ~/projects/L3MVN/data/gibson_tiny ~/projects/L3MVN/data/scene_datasets/gibson/gibson_semantic
This way help me to get the gibson with semantic info.

@weizhenFrank Thank you for your advice, I follow the above instructions, but I am still confused.

When I download gibson scene dataset for habitat, there are only *.ply and *.navmesh files in the folder. In tools/gen_gibson_semantics.sh L24, "${TOOLS_DIR}"/../build/utils/datatool/datatool create_gibson_semantic_mesh "${OBJ_PATH}"/"${scene}"/mesh.obj "${OUT_PATH}"/"${scene}".ids "${OUT_PATH}"/"${scene}"_semantic.ply, however, there is no mesh.obj file in my gibson dataset thus leads to errors.

Did I downloaded the wrong link for gibson? Could u please share which part of gibson dataset u downloaded, or how to obtain mesh.obj file?

Thank you in advance.

我想使用让Vlfm在gibson数据集上测评，但我下载的gibson（10.8g）只有glb和nvmesh，我该如何下载带有语义的，上面的链接失效了，能在分享一下吗

[Yunbo-max]

您好， 感谢您出色的工作。我正在尝试将这项工作应用于 Gibson 数据集。 我按照#8 的步骤操作：

1. 添加参数“--task_config tasks/objectnav_gibson.yaml”

2. 将 agents/sem_exp.py 文件中的ObjectGoal_Env21行替换为ObjectGoal_Env

3. 我还强制任务只在_Darden_场景上运行，因为我确定我拥有该场景的所有资源。我修改了代码：在envs/habitat/__init__.py

   if "*" in basic_config.DATASET.CONTENT_SCENES:
   content_dir = os.path.join("data/datasets/objectnav/gibson/v1.1/" + args.split, "content")
   scenes = _get_scenes_from_folder(content_dir)
   # force it to be only Darden, since we only have this in scene dataset.
   scenes = ['Darden.glb']

4. 为了匹配数据集，我修改了 objectnav_gibson.yaml 文件，内容如下：

DATASET:
TYPE: PointNav-v1
SPLIT: train
DATA_PATH: "data/datasets/objectnav/gibson/v1.1/{split}/{split}.json.gz"
EPISODES_DIR: "data/datasets/objectnav/gibson/v1.1/{split}/"
SCENES_DIR: "/data/scene_datasets"
修改这些内容后，我遇到了一个错误：

Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/L3MVN/envs/habitat/utils/vector_env.py", line 176, in _worker_env
env = env_fn(*env_fn_args)
File "/L3MVN/envs/habitat/init.py", line 24, in make_env_fn
env = Sem_Exp_Env_Agent(args=args, rank=rank,
File "/L3MVN/agents/sem_exp.py", line 32, in init
super().init(args, rank, config_env, dataset)
File "/L3MVN/envs/habitat/objectgoal_env.py", line 26, in init
super().init(config_env, dataset)
File "/habitat-lab/habitat/core/env.py", line 377, in init
self._env = Env(config, dataset)
File "/habitat-lab/habitat/core/env.py", line 107, in init
self._sim = make_sim(
File "/habitat-lab/habitat/sims/registration.py", line 19, in make_sim
return _sim(**kwargs)
File "/habitat-lab/habitat/sims/habitat_simulator/habitat_simulator.py", line 276, in init
super().init(self.sim_config)
File "", line 11, in init
self.attrs_post_init()
File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 122, in attrs_post_init
self.__set_from_config(self.config)
File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 260, in __set_from_config
self._config_backend(config)
File "/usr/local/lib/python3.8/dist-packages/habitat_sim-0.2.1-py3.8-linux-x86_64.egg/habitat_sim/simulator.py", line 204, in _config_backend
super().init(config.sim_cfg, config.metadata_mediator)
AssertionError: ESP_CHECK failed: Missing (at least) one of scene dataset attributes, stage attributes, or dataset scene attributes for scene '/data/scene_datasets/gibson_semantic//Darden.glb'. Likely an invalid scene name.
我猜想这个错误与我组织Gibson数据的方式有关，因为我之前在hm3d数据集中没有正确导入数据时也遇到过类似的问题。但由于这两个数据集的数据格式不同，我目前还没有具体的解决方法。您能分享一下您的经验吗？我的数据结构如下所示。

我的数据文件夹结构如下： 数据：

Data
|-- datasets
|-- matterport_category_mappings.tsv
|-- object_norm_inv_perplexity.npy
|-- objectgoal_hm3d
|-- scene_datasets
`-- versioned_data

数据集

datasets
|-- objectnav
|   `-- gibson
|       `-- v1.1
|           |-- train
|           |   |-- content
|           |   |   |-- Allensville.glb.json.gz
|           |   |   |-- Beechwood.glb.json.gz
|           |   |   |-- Benevolence.glb.json.gz
|           |   |   |-- Coffeen.glb.json.gz
|           |   |   |-- Cosmos.glb.json.gz
|           |   |   |-- Forkland.glb.json.gz
|           |   |   |-- Hanson.glb.json.gz
|           |   |   |-- Hiteman.glb.json.gz
|           |   |   |-- Klickitat.glb.json.gz
|           |   |   |-- Lakeville.glb.json.gz
|           |   |   |-- Leonardo.glb.json.gz
|           |   |   |-- Lindenwood.glb.json.gz
|           |   |   |-- Marstons.glb.json.gz
|           |   |   |-- Merom.glb.json.gz
|           |   |   |-- Mifflinburg.glb.json.gz
|           |   |   |-- Newfields.glb.json.gz
|           |   |   |-- Onaga.glb.json.gz
|           |   |   |-- Pinesdale.glb.json.gz
|           |   |   |-- Pomaria.glb.json.gz
|           |   |   |-- Ranchester.glb.json.gz
|           |   |   |-- Shelbyville.glb.json.gz
|           |   |   |-- Stockman.glb.json.gz
|           |   |   |-- Tolstoy.glb.json.gz
|           |   |   |-- Wainscott.glb.json.gz
|           |   |   `-- Woodbine.glb.json.gz
|           |   |-- train.json.gz
|           |   `-- train_info.pbz2
|           `-- val
|               |-- content
|               |   |-- Collierville.glb.json.gz
|               |   |-- Collierville_episodes.json.gz
|               |   |-- Corozal.glb.json.gz
|               |   |-- Corozal_episodes.json.gz
|               |   |-- Darden.glb.json.gz
|               |   |-- Darden_episodes.json.gz
|               |   |-- Markleeville.glb.json.gz
|               |   |-- Markleeville_episodes.json.gz
|               |   |-- Wiconisco.glb.json.gz
|               |   `-- Wiconisco_episodes.json.gz
|               |-- val.json.gz
|               `-- val_info.pbz2
`-- objectnav_gibson_v1.1.zip

场景数据集

scene_datasets
|-- gibson_semantic
|   |-- Ackermanville.glb
|   |-- Ackermanville.navmesh
|   |-- Darden.glb
|   `-- Darden.navmesh
`-- hm3d_v0.2
    |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |-- hm3d_annotated_basis.scene_dataset_config.json
    `-- val

版本化数据

versioned_data
`-- hm3d-0.2
    |-- hm3d
    |   |-- hm3d -> /home/my-home-robot/data/versioned_data/hm3d-0.2/hm3d
    |   |-- hm3d_annotated_basis.scene_dataset_config.json
    |   `-- val
    |-- val-configs-files.json.gz
    |-- val-habitat-files.json.gz
    |-- val-semantic-annots-files.json.gz
    `-- val-semantic-configs-files.json.gz

Hi, do you know how to downlaod thise > Data

|-- datasets
|-- matterport_category_mappings.tsv
|-- object_norm_inv_perplexity.npy
|-- objectgoal_hm3d
|-- scene_datasets
`-- versioned_data and especially the object_norm_inv_perplexity.npy i can not find it!