Skip to content

RuntimeError: failed to load ONNX file: weights/sam_default_prompt_mask.onnx #6

Description

@onefish51

很棒的工作!我在尝试时遇到下面的问题,有点奇怪。
原始的error log如下:

[1692782200.924099] [d72ed04e9e58:5116 :f]        vfs_fuse.c:281  UCX  ERROR inotify_add_watch(/tmp) failed: No space left on device
['/app/sam/segment_anything_tensorrt/scripts', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/dist-packages', '/app/sam/segment-anything', '/usr/lib/python3/dist-packages', '.', '..']
os.path.exists(f) is  True
[08/23/2023-09:16:43] [TRT] [I] [MemUsageChange] Init CUDA: CPU +15, GPU +0, now: CPU 116, GPU 19090 (MiB)
[08/23/2023-09:16:48] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +424, GPU +74, now: CPU 617, GPU 19164 (MiB)
workspace:  6
/app/sam/segment_anything_tensorrt/scripts/onnx2trt.py:323: DeprecationWarning: Use set_memory_pool_limit instead.
  config.max_workspace_size = workspace * 1 << 30
weights/sam_default_prompt_mask.onnx
[08/23/2023-09:16:48] [TRT] [I] ----------------------------------------------------------------
[08/23/2023-09:16:48] [TRT] [I] Input filename:   weights/sam_default_prompt_mask.onnx
[08/23/2023-09:16:48] [TRT] [I] ONNX IR version:  0.0.8
[08/23/2023-09:16:48] [TRT] [I] Opset version:    17
[08/23/2023-09:16:48] [TRT] [I] Producer name:    pytorch
[08/23/2023-09:16:48] [TRT] [I] Producer version: 2.1.0
[08/23/2023-09:16:48] [TRT] [I] Domain:           
[08/23/2023-09:16:48] [TRT] [I] Model version:    0
[08/23/2023-09:16:48] [TRT] [I] Doc string:       
[08/23/2023-09:16:48] [TRT] [I] ----------------------------------------------------------------
[08/23/2023-09:16:48] [TRT] [W] onnx2trt_utils.cpp:374: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/23/2023-09:16:48] [TRT] [E] [graph.cpp::symbolicExecute::539] Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
[08/23/2023-09:16:48] [TRT] [E] ModelImporter.cpp:771: While parsing node number 146 [Tile -> "/Tile_output_0"]:
[08/23/2023-09:16:48] [TRT] [E] ModelImporter.cpp:772: --- Begin node ---
[08/23/2023-09:16:48] [TRT] [E] ModelImporter.cpp:773: input: "/Unsqueeze_3_output_0"
input: "/Reshape_2_output_0"
output: "/Tile_output_0"
name: "/Tile"
op_type: "Tile"

[08/23/2023-09:16:48] [TRT] [E] ModelImporter.cpp:774: --- End node ---
[08/23/2023-09:16:48] [TRT] [E] ModelImporter.cpp:777: ERROR: ModelImporter.cpp:195 In function parseGraph:
[6] Invalid Node - /Tile
[graph.cpp::symbolicExecute::539] Error Code 4: Internal Error (/OneHot: an IIOneHotLayer cannot be used to compute a shape tensor)
Traceback (most recent call last):
  File "/app/sam/segment_anything_tensorrt/scripts/onnx2trt.py", line 376, in <module>
    export_engine_prompt_encoder_and_mask_decoder(args.sam_onnx_path)
  File "/app/sam/segment_anything_tensorrt/scripts/onnx2trt.py", line 329, in export_engine_prompt_encoder_and_mask_decoder
    raise RuntimeError(f'failed to load ONNX file: {onnx}')
RuntimeError: failed to load ONNX file: weights/sam_default_prompt_mask.onnx

我是在docker环境用运行的,基于nvcr.io/nvidia/pytorch:23.07-py3镜像,其中sam_default_embedding.engine都可以正常导出,导出的sam_default_embedding.onnx 和 sam_default_prompt_mask.onnx 也经过sam_onnx_inference.py测试是可以正常预测的。但是在将sam_default_prompt_mask.onnx转为tensorRT engine时报错了。

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions