Enable resuming with weights.pth saved by visual prompting

[sungchul1]

Summary

#2274 (comment)
For this, it was needed to save metas for pytorch lightning in weights.pth.
It was implemented by loading best checkpoint saved in Trainer.checkpoint_callback and saving metas in weights.pth.

How to test

$ tox -vv -e unittest-all-py310-pt1 -- tests/unit/algorithms/visual_prompting/adapters/pytorch_lightning/config/test_visual_prompting_config.py
=============================================================== 7 passed, 1 warning in 0.72s ===============================================================
Name                                                                                                                Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------------------------------------------------
src/otx/algorithms/visual_prompting/adapters/pytorch_lightning/config/visual_prompting_config.py                       43      7    84%   56, 102-103, 106, 111-113


$ tox -vv -e unittest-all-py310-pt1 --  tests/unit/algorithms/visual_prompting/tasks/test_inference.py
============================================================= 31 passed, 17 warnings in 24.33s =============================================================
Name                                                                                                                Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------------------------------------------------
src/otx/algorithms/visual_prompting/tasks/inference.py                                                                213     28    87%   96, 115-123, 178, 218-219, 224, 337, 347, 352, 387, 434, 438-447, 454-455, 459-460


$ tox -vv -e unittest-all-py310-pt1 --  tests/unit/algorithms/visual_prompting/tasks/test_train.py
============================================================== 1 passed, 10 warnings in 2.22s ==============================================================
Name                                                                                                                Stmts   Miss  Cover   Missing
-------------------------------------------------------------------------------------------------------------------------------------------------
src/otx/algorithms/visual_prompting/tasks/train.py                                                                     41      4    90%   88-89, 96-97


$ tox -vv -e tests-all-py310-pt1 -- tests/integration/cli/visual_prompting/test_visual_prompting.py
=================================================== 8 passed, 2 skipped, 2 warnings in 108.48s (0:01:48) ===================================================

$ tox -vv -e tests-all-py310-pt1 -- tests/e2e/cli/visual_prompting/test_visual_prompting.py
======================================================== 10 passed, 2 warnings in 274.07s (0:04:34) ========================================================

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[eunwoosh]

Thanks for your work! I left some questions. Please take a look.