Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,7 @@
("User Guide", "usage"),
("API", "api"),
("Examples", "examples/index"),
("Extensions", "extensions"),
("Contributing", "contributing"),
("Changelog", "progress"),
],
Expand Down
66 changes: 0 additions & 66 deletions doc/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,70 +29,4 @@ In particular, a few ways to contribute to openml-python are:

.. _extensions:

Connecting new machine learning libraries
=========================================

Content of the Library
~~~~~~~~~~~~~~~~~~~~~~

To leverage support from the community and to tap in the potential of OpenML, interfacing
with popular machine learning libraries is essential. However, the OpenML-Python team does
not have the capacity to develop and maintain such interfaces on its own. For this, we
have built an extension interface to allows others to contribute back. Building a suitable
extension for therefore requires an understanding of the current OpenML-Python support.

The :ref:`sphx_glr_examples_20_basic_simple_flows_and_runs_tutorial.py` tutorial
shows how scikit-learn currently works with OpenML-Python as an extension. The *sklearn*
extension packaged with the `openml-python <https://github.com/openml/openml-python>`_
repository can be used as a template/benchmark to build the new extension.


API
+++
* The extension scripts must import the `openml` package and be able to interface with
any function from the OpenML-Python :ref:`api`.
* The extension has to be defined as a Python class and must inherit from
:class:`openml.extensions.Extension`.
* This class needs to have all the functions from `class Extension` overloaded as required.
* The redefined functions should have adequate and appropriate docstrings. The
`Sklearn Extension API :class:`openml.extensions.sklearn.SklearnExtension.html`
is a good benchmark to follow.


Interfacing with OpenML-Python
++++++++++++++++++++++++++++++
Once the new extension class has been defined, the openml-python module to
:meth:`openml.extensions.register_extension` must be called to allow OpenML-Python to
interface the new extension.


Hosting the library
~~~~~~~~~~~~~~~~~~~

Each extension created should be a stand-alone repository, compatible with the
`OpenML-Python repository <https://github.com/openml/openml-python>`_.
The extension repository should work off-the-shelf with *OpenML-Python* installed.

Create a `public Github repo <https://docs.github.com/en/github/getting-started-with-github/create-a-repo>`_
with the following directory structure:

::

| [repo name]
| |-- [extension name]
| | |-- __init__.py
| | |-- extension.py
| | |-- config.py (optionally)



Recommended
~~~~~~~~~~~
* Test cases to keep the extension up to date with the `openml-python` upstream changes.
* Documentation of the extension API, especially if any new functionality added to OpenML-Python's
extension design.
* Examples to show how the new extension interfaces and works with OpenML-Python.
* Create a PR to add the new extension to the OpenML-Python API documentation.


Happy contributing!
87 changes: 87 additions & 0 deletions doc/extensions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
:orphan:

.. _extensions:

==========
Extensions
==========

OpenML-Python provides an extension interface to connect other machine learning libraries than
scikit-learn to OpenML. Please check the :ref:`api_extensions` and use the
scikit-learn extension in :class:`openml.extensions.sklearn.SklearnExtension` as a starting point.

List of extensions
==================

Here is a list of currently maintained OpenML extensions:

* :class:`openml.extensions.sklearn.SklearnExtension`
* `openml-keras <https://github.com/openml/openml-keras>`_
* `openml-pytorch <https://github.com/openml/openml-pytorch>`_
* `openml-tensorflow (for tensorflow 2+) <https://github.com/openml/openml-tensorflow>`_


Connecting new machine learning libraries
=========================================

Content of the Library
~~~~~~~~~~~~~~~~~~~~~~

To leverage support from the community and to tap in the potential of OpenML, interfacing
with popular machine learning libraries is essential. However, the OpenML-Python team does
not have the capacity to develop and maintain such interfaces on its own. For this, we
have built an extension interface to allows others to contribute back. Building a suitable
extension for therefore requires an understanding of the current OpenML-Python support.

The :ref:`sphx_glr_examples_20_basic_simple_flows_and_runs_tutorial.py` tutorial
shows how scikit-learn currently works with OpenML-Python as an extension. The *sklearn*
extension packaged with the `openml-python <https://github.com/openml/openml-python>`_
repository can be used as a template/benchmark to build the new extension.


API
+++
* The extension scripts must import the `openml` package and be able to interface with
any function from the OpenML-Python :ref:`api`.
* The extension has to be defined as a Python class and must inherit from
:class:`openml.extensions.Extension`.
* This class needs to have all the functions from `class Extension` overloaded as required.
* The redefined functions should have adequate and appropriate docstrings. The
`Sklearn Extension API :class:`openml.extensions.sklearn.SklearnExtension.html`
is a good benchmark to follow.


Interfacing with OpenML-Python
++++++++++++++++++++++++++++++
Once the new extension class has been defined, the openml-python module to
:meth:`openml.extensions.register_extension` must be called to allow OpenML-Python to
interface the new extension.


Hosting the library
~~~~~~~~~~~~~~~~~~~

Each extension created should be a stand-alone repository, compatible with the
`OpenML-Python repository <https://github.com/openml/openml-python>`_.
The extension repository should work off-the-shelf with *OpenML-Python* installed.

Create a `public Github repo <https://docs.github.com/en/github/getting-started-with-github/create-a-repo>`_
with the following directory structure:

::

| [repo name]
| |-- [extension name]
| | |-- __init__.py
| | |-- extension.py
| | |-- config.py (optionally)

Recommended
~~~~~~~~~~~
* Test cases to keep the extension up to date with the `openml-python` upstream changes.
* Documentation of the extension API, especially if any new functionality added to OpenML-Python's
extension design.
* Examples to show how the new extension interfaces and works with OpenML-Python.
* Create a PR to add the new extension to the OpenML-Python API documentation.

Happy contributing!
1 change: 1 addition & 0 deletions doc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ Content
* :ref:`usage`
* :ref:`api`
* :ref:`sphx_glr_examples`
* :ref:`extensions`
* :ref:`contributing`
* :ref:`progress`

Expand Down
18 changes: 3 additions & 15 deletions doc/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -155,19 +155,7 @@ obtained on. Learn how to share your datasets in the following tutorial:
Extending OpenML-Python
***********************

OpenML-Python provides an extension interface to connect other machine learning libraries than
scikit-learn to OpenML. Please check the :ref:`api_extensions` and use the
scikit-learn extension in :class:`openml.extensions.sklearn.SklearnExtension` as a starting point.

Runtime measurement is incorporated in the OpenML sklearn-extension. Example usage and potential
usage for Hyperparameter Optimisation can be found in the example tutorial:

* :ref:`sphx_glr_examples_30_extended_fetch_runtimes_tutorial.py`


Here is a list of currently maintained OpenML extensions:

* `openml-keras <https://github.com/openml/openml-keras>`_
* `openml-pytorch <https://github.com/openml/openml-pytorch>`_
* `openml-tensorflow(for tensorflow 2+) <https://github.com/openml/openml-tensorflow>`_
OpenML-Python provides an extension interface to connect machine learning libraries directly to
the API and ships a ``scikit-learn`` extension. You can find more information in the Section
:ref:`extensions`'