Skip to content

RefineDet-Mxnet can not evaluate or demo #7

@AaronYoooooooog

Description

@AaronYoooooooog

I can use vgg16_reduced or ssd_vgg16_reduced pretrained model for training, but after trained for several epoch, the trained model cannot be evaluate or demo. The report error as follow:

root@5cb9ced7388e:/mnt/Git/mxnet-ssd# python evaluate_pig.py --network vgg16_reduced --prefix ./checkpoint/refinedet_vgg16_reduced_320 --data-shape 320 --epoch 15
Using mxnet as:
<module 'mxnet' from '/mxnet/python/mxnet/init.pyc'>
Warning: using pre-installed version of mxnet may cause unexpected error...
(export MXNET_EXAMPLE_SSD_DISABLE_PRE_INSTALLED=1) to prevent loading pre-installed mxnet.
[03:21:03] src/io/iter_image_det_recordio.cc:280: ImageDetRecordIOParser: /mnt/Git/mxnet-ssd/data/val.rec, use 8 threads for decoding..
[03:21:03] src/io/iter_image_det_recordio.cc:333: ImageDetRecordIOParser: /mnt/Git/mxnet-ssd/data/val.rec, label padding width: 116
model_prefix ./checkpoint/refinedet_vgg16_reduced_320
epoch 15
[03:21:03] /mxnet/dmlc-core/include/dmlc/./logging.h:308: [03:21:03] src/operator/custom/custom.cc:78: Check failed: Registry::Get()->Find(params.op_type) != nullptr Cannot find custom operator modify_label

Stack trace returned 10 entries:
[bt] (0) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7ffa1e95df8c]
[bt] (1) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet2op6custom10AttrParserEPN4nnvm9NodeAttrsE+0xc4a) [0x7ffa1eb3f23a]
[bt] (2) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet17UpgradeJSON_ParseEN4nnvm5GraphE+0x52e) [0x7ffa2096776e]
[bt] (3) /mxnet/python/mxnet/../../lib/libmxnet.so(ZNSt17_Function_handlerIFN4nnvm5GraphES1_EPS2_E9_M_invokeERKSt9_Any_dataOS1+0x111) [0x7ffa2096cd21]
[bt] (4) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet18LoadLegacyJSONPassEN4nnvm5GraphE+0x1803) [0x7ffa2095d6f3]
[bt] (5) /mxnet/python/mxnet/../../lib/libmxnet.so(ZNSt17_Function_handlerIFN4nnvm5GraphES1_EPS2_E9_M_invokeERKSt9_Any_dataOS1+0x111) [0x7ffa2096cd21]
[bt] (6) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x32c) [0x7ffa2210a85c]
[bt] (7) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7ffa20982aaa]
[bt] (8) /mxnet/python/mxnet/../../lib/libmxnet.so(MXSymbolCreateFromFile+0x7d1) [0x7ffa20dd0f81]
[bt] (9) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7ffa2abf1e40]

Traceback (most recent call last):
File "evaluate_pig.py", line 90, in
voc07_metric=args.use_voc07_metric, frequent=args.frequent)
File "/mnt/Git/mxnet-ssd/evaluate/evaluate_net.py", line 74, in evaluate_net
load_net, args, auxs = mx.model.load_checkpoint(model_prefix, epoch)
File "/mxnet/python/mxnet/model.py", line 394, in load_checkpoint
symbol = sym.load('%s-symbol.json' % prefix)
File "/mxnet/python/mxnet/symbol/symbol.py", line 2494, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "/mxnet/python/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [03:21:03] src/operator/custom/custom.cc:78: Check failed: Registry::Get()->Find(params.op_type) != nullptr Cannot find custom operator modify_label

Stack trace returned 10 entries:
[bt] (0) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7ffa1e95df8c]
[bt] (1) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet2op6custom10AttrParserEPN4nnvm9NodeAttrsE+0xc4a) [0x7ffa1eb3f23a]
[bt] (2) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet17UpgradeJSON_ParseEN4nnvm5GraphE+0x52e) [0x7ffa2096776e]
[bt] (3) /mxnet/python/mxnet/../../lib/libmxnet.so(ZNSt17_Function_handlerIFN4nnvm5GraphES1_EPS2_E9_M_invokeERKSt9_Any_dataOS1+0x111) [0x7ffa2096cd21]
[bt] (4) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN5mxnet18LoadLegacyJSONPassEN4nnvm5GraphE+0x1803) [0x7ffa2095d6f3]
[bt] (5) /mxnet/python/mxnet/../../lib/libmxnet.so(ZNSt17_Function_handlerIFN4nnvm5GraphES1_EPS2_E9_M_invokeERKSt9_Any_dataOS1+0x111) [0x7ffa2096cd21]
[bt] (6) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x32c) [0x7ffa2210a85c]
[bt] (7) /mxnet/python/mxnet/../../lib/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7ffa20982aaa]
[bt] (8) /mxnet/python/mxnet/../../lib/libmxnet.so(MXSymbolCreateFromFile+0x7d1) [0x7ffa20dd0f81]
[bt] (9) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7ffa2abf1e40]

how to solve this problem?
@MTCloudVision

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions