Skip to content

Support backwards compatible operation by default#2065

Merged
pixelb merged 7 commits intofacebookresearch:mainfrom
pixelb:version_base
Mar 28, 2022
Merged

Support backwards compatible operation by default#2065
pixelb merged 7 commits intofacebookresearch:mainfrom
pixelb:version_base

Conversation

@pixelb
Copy link
Contributor

@pixelb pixelb commented Mar 3, 2022

Add the "version_base" parameter to hydra.main,
which acts as central way to select the defaults for that version.

Without a specified "version_base" the user will get a warning,
and we'll default to the currently supported minimum version defaults
(currently those of version 1.1).

Specifying a "version_base" will suffice to select appropriate defaults,
rather than requiring one explicitly set other config items like
config_path, or hydra.job.chdir.

New hydra uses should use version_base=None to get
the default set of the currently installed version.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 3, 2022
@pixelb
Copy link
Contributor Author

pixelb commented Mar 3, 2022

TODO:

  • tests
  • docs
  • split default config changes from main diff
  • init version_base from all launchers

@pixelb pixelb added the In progress Work in progress label Mar 3, 2022
@Jasha10
Copy link
Collaborator

Jasha10 commented Mar 3, 2022

Let me outline an argument in favor of using hydra's Singleton class to encapsulate the version base (instead of using global __version_base__):

Remote launcher plugins need to serialize Hydra's global state so that the remote process that will run the job can use the same global state. For example, the Hydra submitit launcher uses this pattern:

Since the final config is composed on the remote process, that remote process must have access to Hydra's global state. I believe that this should include the version base, as the composition process will probably depend on the version base (e.g. PR #2056 removes deprecated functionality that affects config composition, and so the version base will play a role in finalizing that PR). The Singleton.get_state and Singleton.set_state methods allow simultaneously serializing or restoring the states of all of Hydra's singleton subclasses (e.g. ConfigStore, Plugins, GlobalHydra, etc) using a single access point.

@pixelb pixelb force-pushed the version_base branch 2 times, most recently from cfb28f5 to 8cdef72 Compare March 22, 2022 01:46
@pixelb pixelb removed the In progress Work in progress label Mar 22, 2022
@pixelb pixelb force-pushed the version_base branch 4 times, most recently from c132bad to cd48ce9 Compare March 23, 2022 14:55
@pixelb pixelb added this to the Hydra 1.2.0 milestone Mar 24, 2022
@pixelb pixelb added the enhancement Enhanvement request label Mar 24, 2022
@pixelb pixelb self-assigned this Mar 24, 2022
Add the "version_base" parameter to hydra.main,
which acts as central way to select the defaults for that version.

Without a specified "version_base" the user will get a warning,
and we'll default to the currently supported minimum version defaults
(currently those of version 1.1).

Specifying a "version_base" will suffice to select appropriate defaults,
rather than requiring one explicitly set other config items like
config_path, or hydra.job.chdir.

New hydra uses should use version_base=None to get
the default set of the currently installed version.
pixelb added 6 commits March 25, 2022 18:43
Update the website/docs/upgrades/*/hydra_main_config_path.md docs,
to indicate the new version_base default setting for config_path.
  $ git grep -l hydra.main.*config_path=None | grep -Ev '(docs/upgrades|versioned_docs)' |
    xargs sed -i 's/hydra.main(config_path=None/hydra.main(version_base=None/'

  $ git grep -l initialize.*config_path=None | grep -Ev '(docs/upgrades|versioned_docs)' |
    xargs sed -i 's/initialize(config_path=None)/initialize(version_base=None)/'
  $ git grep hydra.main.*config_path= | grep -v version_base | grep -Ev '(docs/upgrades|versioned_docs)' |
    cut -d: -f1 | uniq | xargs sed -i 's/hydra\.main(config_path=/hydra.main(version_base=None, config_path=/'

Also various manual additions
Also adjust docs to remove the implication we'll break behavior in hydra 1.3
@jieru-hu
Copy link
Contributor

lg! should we add some notes on version_base in the developer guide as well? This is helpful to know for introducing breaking changes.

Copy link
Collaborator

@Jasha10 Jasha10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@pixelb pixelb merged commit f999139 into facebookresearch:main Mar 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. enhancement Enhanvement request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants