Hi! Thank you for open-sourcing WorldVLN — the code and the weights are very helpful to us.
We are trying to reproduce the IndoorUAV-VLA results in Table 4 of the paper (WorldVLN 41.76% SR / 13.48% NDTW). We set up the full evaluation pipeline following the original protocol (Habitat simulator, SR/NDTW metrics), but we were not able to reach the reported numbers.
We noticed that the weights released on Hugging Face appear to correspond to the UAV-Flow training (the source checkpoint noted in WorldVLN_backbone/README.md is rluavflowcheckpoint_partialfreeze_stageb_only/... ), and we could not find the IndoorUAV counterpart on HF / ModelScope / GitHub.
So we would like to ask: which weight set was used for the IndoorUAV evaluation? If possible, could you share it, or let us know how to obtain it? If the weights behind the Table 4 IndoorUAV results differ from what we downloaded, any clarification would be greatly appreciated.
Looking forward to your reply. We are also happy to share our evaluation setup and results. Thank you!
Hi! Thank you for open-sourcing WorldVLN — the code and the weights are very helpful to us.
We are trying to reproduce the IndoorUAV-VLA results in Table 4 of the paper (WorldVLN 41.76% SR / 13.48% NDTW). We set up the full evaluation pipeline following the original protocol (Habitat simulator, SR/NDTW metrics), but we were not able to reach the reported numbers.
We noticed that the weights released on Hugging Face appear to correspond to the UAV-Flow training (the source checkpoint noted in
WorldVLN_backbone/README.mdisrluavflowcheckpoint_partialfreeze_stageb_only/...), and we could not find the IndoorUAV counterpart on HF / ModelScope / GitHub.So we would like to ask: which weight set was used for the IndoorUAV evaluation? If possible, could you share it, or let us know how to obtain it? If the weights behind the Table 4 IndoorUAV results differ from what we downloaded, any clarification would be greatly appreciated.
Looking forward to your reply. We are also happy to share our evaluation setup and results. Thank you!