This repository was archived by the owner on Nov 17, 2023. It is now read-only.
Repository navigation
This repository was archived by the owner on Nov 17, 2023. It is now read-only.
MXNet: Operator softmax registered in backend is known as softmax in Python #21189
Description
Description
Trying to pass class_weights to a SoftmaxCrossEntropy loss function leads to the following error:
Error Message
Traceback (most recent call last):
File "mx.py", line 104, in <module>
outputs = [network(data_slice) for data_slice in data]
File "mx.py", line 104, in <listcomp>
outputs = [network(data_slice) for data_slice in data]
File "/opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/mxnet/gluon/block.py", line 825, in __call__
out = self.forward(*args)
File "/home/ubuntu/deepspace/deeplearning/nn/mxconv/mxnet.py", line 37, in forward
return nd.softmax(x)
File "<string>", line 78, in softmax
File "/opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/mxnet/ndarray/register.py", line 92, in _verify_all_legacy_ndarrays
raise TypeError('Operator `{}` registered in backend is known as `{}` in Python. '
TypeError: Operator `softmax` registered in backend is known as `softmax` in Python. This is a legacy operator which can only accept legacy ndarrays, while received an MXNet numpy ndarray. Please call `as_nd_ndarray()` upon the numpy ndarray to convert it to a legacy ndarray, and then feed the converted array to this operator.
To Reproduce
Just create a network, which should categorize 5 different categories. Since the dataset can contain some unbalance in the data, you need to create the following class_weight array:
mx.npx.set_np()
loss_function = gluon.loss.SoftmaxCrossEntropyLoss()
weights = mx.np.array([0.05, 0.71, 0.21, 0.02, 0.01]).as_nd_ndarray()
then pass the weights to the loss_function above like:
with mx.autograd.record():
outputs = [networ(data_slice) for data_slice in data]
losses = [loss_function(o, l, weights) for o, l in zip(outputs, labels)
What have you tried to solve it?
- I added ad_nd_ndarray() to the weights variable
- I tried to create a "conventional" array using numpy and not mx
Environment
We recommend using our script for collecting the diagnostic information with the following command
curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python3
Environment Information
----------Python Info----------
Version : 3.8.15
Compiler : GCC 10.4.0
Build : ('default', 'Nov 22 2022 08:49:35')
Arch : ('64bit', 'ELF')
------------Pip Info-----------
Version : 22.3.1
Directory : /opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/pip
----------MXNet Info-----------
Version : 1.9.1
Directory : /opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/mxnet
Commit hash file "/opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/mxnet/COMMIT_HASH" not found. Not installed from pre-built package or built from source.
Library : ['/opt/anaconda/anaconda3/envs/dl/lib/python3.8/site-packages/mxnet/libmxnet.so']
Build features:
✔ CUDA
✔ CUDNN
✔ NCCL
✔ CUDA_RTC
✖ TENSORRT
✔ CPU_SSE
✔ CPU_SSE2
✔ CPU_SSE3
✖ CPU_SSE4_1
✖ CPU_SSE4_2
✖ CPU_SSE4A
✖ CPU_AVX
✖ CPU_AVX2
✔ OPENMP
✖ SSE
✖ F16C
✖ JEMALLOC
✔ BLAS_OPEN
✖ BLAS_ATLAS
✖ BLAS_MKL
✖ BLAS_APPLE
✔ LAPACK
✔ MKLDNN
✔ OPENCV
✖ CAFFE
✖ PROFILER
✔ DIST_KVSTORE
✖ CXX14
✖ INT64_TENSOR_SIZE
✔ SIGNAL_HANDLER
✖ DEBUG
✖ TVM_OP
----------System Info----------
Platform : Linux-5.10.0-20-amd64-x86_64-with-glibc2.10
system : Linux
node : freiraum
release : 5.10.0-20-amd64
version : #1 SMP Debian 5.10.158-2 (2022-12-13)
----------Hardware Info----------
machine : x86_64
processor :
Architektur: x86_64
CPU Operationsmodus: 32-bit, 64-bit
Byte-Reihenfolge: Little Endian
Adressgrößen: 46 bits physical, 57 bits virtual
CPU(s): 80
Liste der Online-CPU(s): 0-79
Thread(s) pro Kern: 2
Kern(e) pro Sockel: 20
Sockel: 2
NUMA-Knoten: 2
Anbieterkennung: GenuineIntel
Prozessorfamilie: 6
Modell: 106
Modellname: Intel(R) Xeon(R) Silver 4316 CPU @ 2.30GHz
Stepping: 6
CPU MHz: 800.816
BogoMIPS: 4600.00
Virtualisierung: VT-x
L1d Cache: 1,9 MiB
L1i Cache: 1,3 MiB
L2 Cache: 50 MiB
L3 Cache: 60 MiB
NUMA-Knoten0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,
56,58,60,62,64,66,68,70,72,74,76,78
NUMA-Knoten1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,
57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequen
ce
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Markierungen: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush
dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_
tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmp
erf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtp
r pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave
avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single
intel_ppin ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority
ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a
avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd
sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc c
qm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts a
vx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_b
italg tme avx512_vpopcntdq rdpid fsrm md_clear pconfig flush_l1d arch_capabilit
ies
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/mxnet, DNS: 0.0029 sec, LOAD: 0.5381 sec.
Error open Gluon Tutorial(en): http://gluon.mxnet.io, HTTP Error 404: Not Found, DNS finished in 0.05452752113342285 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131)>, DNS finished in 0.013855934143066406 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0075 sec, LOAD: 0.7042 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0011 sec, LOAD: 0.4958 sec.
Error open Conda: https://repo.continuum.io/pkgs/free/, HTTP Error 403: Forbidden, DNS finished in 0.0060999393463134766 sec.
----------Environment----------
Activity