[Feature Request] Multiple backend C++ API

[njzjz]

Summary

Propose a multiple backend C++ API. The backend can be switched according to the file format.

Detailed Description

Backend independent library

When @wanghan-iapcm designed v1, libdeepmd.so has been backend-independent, so we can just reuse it.

Factory method pattern

Take the example of DeepPot. Move the current DeepPot to DeepPotTf. Create a base class DeepPotBase, with two subclasses inherited, DeepPotTf and DeepPotPt. Create a wrapper class, DeepPot, to keep the same API as the current one, so no downstream codes need to change.

The wrapper class may look like:

enum Backend {TENSORFLOW, PYTORCH, UNKNOWN};

class DeepPot {
  public:
  DeepPot(const std::string& model,
                 const int& gpu_rank,
                 const std::string& file_content)
    Backend backend = detect_backend(model, file_content);
    if (Backend::TENSORFLOW == backend) {
        dp = std::make_unique<DeepPotTf>(model, gpu_rank, file_content);
    } else if (ProductId::PYTORCH == backend) {
        dp = std::make_unique<DeepPotPt>(model, gpu_rank, file_content);
    } else {
        throw deepmd::exception("Unknown file type");
    }
  }
  void compute(
      std::vector<double> &ener,
      std::vector<std::vector<VALUETYPE>> &force,
      std::vector<std::vector<VALUETYPE>> &virial,
      const std::vector<VALUETYPE> &coord,
      const std::vector<int> &atype,
      const std::vector<VALUETYPE> &box,
      const int nghost,
      const InputNlist &lmp_list,
      const int &ago,
      const std::vector<VALUETYPE> &fparam = std::vector<VALUETYPE>(),
      const std::vector<VALUETYPE> &aparam = std::vector<VALUETYPE>())
  {
    dp->compute(ener, force, virial, coord, atype, box, nghost, lmp_list, ago, fparam, aparam);
  }
  private:
    std::unique_ptr<DeepPotBase> dp;
};

The list of public APIs can be found in deepmd.hpp and c_api.cc. Not implemented methods can throw an exception.

Backend detection

The backend can be detected from the model file described below:

• PyTorch: in zip files. Zip files start with PK.
• TensorFlow: It's not easy to detect, but ProtoBuf should throw an error when parsing it.

If a backend is not enabled, we should skip it.
If a backend fails to read the file, continue to the next backend (assume there is a list).
If all backends fail, throw the error, with errors of each backend.

DeepPotModelDevi

DeepPotModelDevi may need to be refactored. The current implementation requires 4 models to come from the same backend, while they may come from different backends.

Further Information, Files, and Links

No response