Trt refactor remove mixin#420
Conversation
Trt refactor remove exporters
…ild trt engines from onnx model
This PR is in charge of adding a trt-config class. This class is responsible to build the trt engine given an onnx path and various trt-flags. As the same model might need different trt-configurations depending on which precision is used, a registry is used to collect all the model configuration. Based on the provided key the get_config method will return the appropriate model configuration to use. Each configuration is a dataclass containing: the needed trt flags a from_model factory method to feed all needed parameters to the config class an get_input_profile method that return the max and mix input supported by the build engine Engine classes are changed to: use trt-config class instead of mixin classes
|
merge 8 description This PR is in charge of removing onnx-exporter class as it is not needed. |
|
merge 9 description This PR is in charge of adding a trt-config class. As the same model might need different trt-configurations depending on which precision is used, a registry is used to collect all the model configuration. @timudk here are the main changes
Engine classes are changed to:
|
Trt refactor trt manager
|
merge 10 description This PR implements the changes on trt-manager to use trt-config classes instead of exporters and mixin. for each porvided model a trt-config is provided by _get_trt_configs method. @timudk this PR greatly simplify this class by reducing code lines by half. |
|
merge 11 description This PR change the CLI script to support the new TRT interface.
a different set of inputs are used:
The following are additional inputs args that can be provided:
Note that:
|
This is the main PR used to refactor TRT inference code to: