feat: Adding support for native int64

[narendasan]

Signed-off-by: Naren Dasan naren@narendasan.com
Signed-off-by: Naren Dasan narens@nvidia.com

Description

Adds support for int64 as a native type in TensorRT

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• New feature (non-breaking change which adds functionality)

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp b/tmp/changes.txt
index 503b88e..3ca5780 100644
--- a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp
+++ b/tmp/changes.txt
@@ -164,8 +164,8 @@ nvinfer1::Dims unsqueezeDims(const nvinfer1::Dims& d, int pos, int val, bool use
  // Acceptable range for pos is [-d.nbDims - 1, d.nbDims]
  TORCHTRT_ASSERT(
      pos >= (-d.nbDims - 1) && pos <= d.nbDims,
-      "ERROR: Index to unsqueeze is out of bounds. " << "Expected value in range [" << (-d.nbDims - 1) << ", "
-                                                     << d.nbDims << "], but got " << pos);
+      "ERROR: Index to unsqueeze is out of bounds. "
+          << "Expected value in range [" << (-d.nbDims - 1) << ", " << d.nbDims << "], but got " << pos);

  // Unsqueeze with negative dimensions creates a new dimension at that index
  pos = (pos < 0) ? (pos + d.nbDims + 1) : pos;
ERROR: Some files do not conform to style guidelines

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp b/tmp/changes.txt
index 503b88e..3ca5780 100644
--- a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp
+++ b/tmp/changes.txt
@@ -164,8 +164,8 @@ nvinfer1::Dims unsqueezeDims(const nvinfer1::Dims& d, int pos, int val, bool use
  // Acceptable range for pos is [-d.nbDims - 1, d.nbDims]
  TORCHTRT_ASSERT(
      pos >= (-d.nbDims - 1) && pos <= d.nbDims,
-      "ERROR: Index to unsqueeze is out of bounds. " << "Expected value in range [" << (-d.nbDims - 1) << ", "
-                                                     << d.nbDims << "], but got " << pos);
+      "ERROR: Index to unsqueeze is out of bounds. "
+          << "Expected value in range [" << (-d.nbDims - 1) << ", " << d.nbDims << "], but got " << pos);

  // Unsqueeze with negative dimensions creates a new dimension at that index
  pos = (pos < 0) ? (pos + d.nbDims + 1) : pos;
ERROR: Some files do not conform to style guidelines

[github-actions]

There are some changes that do not conform to C++ style guidelines:

diff --git a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp b/tmp/changes.txt
index 503b88e..3ca5780 100644
--- a/home/runner/work/TensorRT/TensorRT/core/util/trt_util.cpp
+++ b/tmp/changes.txt
@@ -164,8 +164,8 @@ nvinfer1::Dims unsqueezeDims(const nvinfer1::Dims& d, int pos, int val, bool use
  // Acceptable range for pos is [-d.nbDims - 1, d.nbDims]
  TORCHTRT_ASSERT(
      pos >= (-d.nbDims - 1) && pos <= d.nbDims,
-      "ERROR: Index to unsqueeze is out of bounds. " << "Expected value in range [" << (-d.nbDims - 1) << ", "
-                                                     << d.nbDims << "], but got " << pos);
+      "ERROR: Index to unsqueeze is out of bounds. "
+          << "Expected value in range [" << (-d.nbDims - 1) << ", " << d.nbDims << "], but got " << pos);

  // Unsqueeze with negative dimensions creates a new dimension at that index
  pos = (pos < 0) ? (pos + d.nbDims + 1) : pos;
ERROR: Some files do not conform to style guidelines

[gs-olive]

Overall looks good! May need modification to this file as well:

TensorRT/py/torch_tensorrt/dynamo/conversion/converter_utils.py

Lines 300 to 315 in 1a4ffe4

	if (
	isinstance(input_val, torch.Tensor)
	and ctx.compilation_settings.truncate_long_and_double
	):
	if input_val.dtype == torch.int64:
	input_val = input_val.to(torch.int32)
	elif input_val.dtype == torch.float64:
	input_val = input_val.to(torch.float32)
	elif (
	isinstance(input_val, np.ndarray)
	and ctx.compilation_settings.truncate_long_and_double
	):
	if input_val.dtype == np.int64:
	input_val = input_val.astype(np.int32)
	elif input_val.dtype == np.float64:
	input_val = input_val.astype(np.float32)

[peri044]

LGTM. Minor changes

[gs-olive]

Looks good to me

[gs-olive]

Any reason for not using logger here?

[narendasan]

This is apparently the recommended way to handle deprecation warnings, iirc I configured the logger to pull these messages in in an earlier PR

[gs-olive]

• Verified functional compilation on Stable Diffusion example