Thank you for your great work and for releasing the RealVLG-R1 code and dataset. I have been reading the paper and the GitHub repository, and I have a few questions about the training and inference setup.
In the paper, RealVLG-R1 is described as a unified framework for bounding box prediction, segmentation, grasp pose prediction, and contact point prediction. However, in the released training scripts, I noticed that each task seems to have its own prompt template, reward function, and training script.
(1) Could you please clarify whether the models reported in the paper are trained separately for each task, or whether there is also a single multi-task model trained jointly on all tasks?
(2) I find that there is "data.train_files=${DATA_PATH}/data/train_gs_kn_10p.parquet" in your training script. So how could I obtain the "train_gs_kn_10p.parquet" file? I downloaded the Cornell_VLG.zip and VMRD_VLG.zip, and could not find the file for training.
(3) In addition, I would like to ask whether you can release the following resources:
- The 7B RealVLG-R1 checkpoints reported in the paper.
- The inference scripts for the Grasp and Contact tasks.
Currently, I can find the training scripts for grasp and contact, but I am not sure how to perform inference and evaluation for these two tasks using the released code.
Hope for your reply!
Thank you for your great work and for releasing the RealVLG-R1 code and dataset. I have been reading the paper and the GitHub repository, and I have a few questions about the training and inference setup.
In the paper, RealVLG-R1 is described as a unified framework for bounding box prediction, segmentation, grasp pose prediction, and contact point prediction. However, in the released training scripts, I noticed that each task seems to have its own prompt template, reward function, and training script.
(1) Could you please clarify whether the models reported in the paper are trained separately for each task, or whether there is also a single multi-task model trained jointly on all tasks?
(2) I find that there is "data.train_files=${DATA_PATH}/data/train_gs_kn_10p.parquet" in your training script. So how could I obtain the "train_gs_kn_10p.parquet" file? I downloaded the Cornell_VLG.zip and VMRD_VLG.zip, and could not find the file for training.
(3) In addition, I would like to ask whether you can release the following resources:
Currently, I can find the training scripts for grasp and contact, but I am not sure how to perform inference and evaluation for these two tasks using the released code.
Hope for your reply!