Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error when running the ipynb file. #16

Open
fivejjs opened this issue May 14, 2016 · 15 comments
Open

error when running the ipynb file. #16

fivejjs opened this issue May 14, 2016 · 15 comments

Comments

@fivejjs
Copy link

fivejjs commented May 14, 2016

if not os.path.exists('deep3d-0050.params'):
    urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

The error gives:

---------------------------------------------------------------------------
MXNetError                                Traceback (most recent call last)
<ipython-input-7-9deaeed89c01> in <module>()
      1 if not os.path.exists('deep3d-0050.params'):
      2     urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
----> 3 model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load(prefix, epoch, ctx, **kwargs)
    832         - ``prefix-epoch.params`` will be saved for parameters.
    833         """
--> 834         symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
    835         return FeedForward(symbol, ctx=ctx,
    836                            arg_params=arg_params, aux_params=aux_params,

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load_checkpoint(prefix, epoch)
    361     - parameters will be loaded from ``prefix-epoch.params``.
    362     """
--> 363     symbol = sym.load('%s-symbol.json' % prefix)
    364     save_dict = nd.load('%s-%04d.params' % (prefix, epoch))
    365     arg_params = {}

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.pyc in load(fname)
    932         raise TypeError('fname need to be string')
    933     handle = SymbolHandle()
--> 934     check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
    935     return Symbol(handle)
    936 

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.pyc in check_call(ret)
     75     """
     76     if ret != 0:
---> 77         raise MXNetError(py_str(_LIB.MXGetLastError()))
     78 
     79 def c_str(string):

MXNetError: Failed loading Op softmax of type SoftmaxActivation: Cannot find argument 'type', Possible Arguments:
----------------
mode : {'channel', 'instance'},optional, default='instance'
    Softmax Mode. If set to instance, this operator will compute a softmax for each instance in the batch; this is the default mode. If set to channel, this operator will compute a num_channel-class softmax at each position of each instance; this can be used for fully convolutional network, image segmentation, etc.```

It looks like:
```sym = symbol.load('deep3d-symbol.json')

How to fix it?
Thanks

@Feng137
Copy link

Feng137 commented May 17, 2016

@fivejjs
I have met this problem too. It is because that the "MXNET" you download is the latest version, and the version the author use is a previous version.

In the latest version, when u want to use SoftmaxActivation, u should replace the "type" into "mode" in the json file.
But after u do this, u will meet another CudnnBatchNorm problem. Then u should change all "CudnnBatchNorm" to "BatchNorm" in the json file, and modify the model.aux_params by replace "moving_inv_var" into "moving_var". Then it will be all ok to run.

The other easy solution is u download a previous version of MXNET for example "tag 0321"

@lasandcris
Copy link

@fivejjs
However, if you compile mxnet using Cuda7.0 and cudnn 4 or >, then you shouldn't have the cudnn problem either, even using the latest mxnet.

@azrael0fog
have you managed to run the convert_movie.py? mine runs until it hits line 285 in data.py and complaining of Attribute Error: 'NoneType' object has no attribute split. As if there is no data in the list.

@n1ckfg
Copy link

n1ckfg commented Jun 1, 2016

Couple questions--where can I find model.aux_params? And where do I get the .ctx file that convert_movie.py needs as an argument?

I'm encountering this error too...Ubuntu 14.04 with CUDA 7 / CUDNN 4 and CUDA 7.5 / CUDNN 5. Tried search-and-replace in json file, building from tag 0321, no luck.

@dineshreddy91
Copy link

i am having the same issue... Ubuntu 16.04 CUDA 7.5 / CUDNN 4 and i keep getting the same error:

ValueError: Find name bn_pool3_moving_inv_var that is not in the auxiliary states

i tried changing the deep3d-0050.params file from moving_inv_var to moving_var. Then the model doesnot load. Can you specify the exact location to change the moving_inv_var file.

@Feng137
Copy link

Feng137 commented Jun 7, 2016

@dineshreddy91
I don't know whether the CUDA 7.5 and CUDNN 4 is match.
When using CUDA 7.5, I use CUDNN 5; when using CUDA 7, I use CUDNN 4. Both works.

Though both works, when using CUDA 7.5/CUDNN 5, the result picture is not the same as the author shows. I guess the author is using CUDA 7/CUDNN 4.

I am not sure, but I guess there may some difference of CUDA 7.5 and CUDA 7 when calculating.

Highly recommanded changing the version of CUDA into version7.

@ashishgupta-skg
Copy link

ashishgupta-skg commented Aug 2, 2016

@azrael0fog
I am not getting how are you modifying model.aux_params. Please can you explain it. Where is the file and how to modify it.

@fivejjs
Copy link
Author

fivejjs commented Aug 2, 2016

Still no fix?

@ashishgupta-skg
Copy link

ashishgupta-skg commented Aug 2, 2016

Can you suggest any way to modify deep3d-0050.params. I am really struggling with it.

@Greenleaf88
Copy link

I somewhat understand his meaning, model.aux_params is not a file, it's a variable, the modification should be done in the program.

@Greenleaf88
Copy link

@azrael0fog
I have done all those modifications:
In the latest version, when u want to use SoftmaxActivation, u should replace the "type" into "mode" in the json file.
But after u do this, u will meet another CudnnBatchNorm problem. Then u should change all "CudnnBatchNorm" to "BatchNorm" in the json file, and modify the model.aux_params by replace "moving_inv_var" into "moving_var". Then it will be all ok to run.

But still got this error:
mxnet.base.MXNetError: [19:49:17] src/operator/cudnn_batch_norm.cc:20: CuDNNBatchNorm is merged into BatchNorm for cudnn version above v5.Use the later instead.

Any advise?

@Greenleaf88
Copy link

degrade the cudnn version from v5 to v4, it works.

@gudongxing
Copy link

@azrael0fog I am a new comer to mxnet .I want to use deep3d to make a few pictures. I want to know which file need to modify to aviod the CudnnBatchNorm problem,.I got in the trouble for a few days. I need your help Thanks a lot.

@WaratenZ
Copy link

The solution by @azrael0fog worked in my case.
I am using mxnet(v0.9.3) and CUDA 8.0 with cudnn downgraded to v4. Modify line 962 in "deep3d-symbol.json", which I believe defines the structure of the network, as following:
FROM

{
      "op": "SoftmaxActivation", 
      "param": {"type": "channel"}, 
      "name": "softmax", 
      "inputs": [[105, 0]], 
      "backward_source_id": -1
    }

TO

{
      "op": "SoftmaxActivation", 
      "param": {"mode": "channel"}, 
      "name": "softmax", 
      "inputs": [[105, 0]], 
      "backward_source_id": -1
    }

However, the difference with @azrael0fog's case is that I did not have to change "CudnnBatchNorm" and "model.aux_params" at all. Hope this works in yours.

@yishi2013
Copy link

@Greenleaf88 , do you know which file(s) need to change model.aux_params? I didn't find any file include model.aux_params. If possible, can you list the files' name. Thanks.

@yishi2013
Copy link

I find how to change model.aux_params from #23. Thanks @zedomel

model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

model.aux_params['bn_pool3_moving_var'] = model.aux_params['bn_pool3_moving_inv_var']
model.aux_params['bn_pool2_moving_var'] = model.aux_params['bn_pool2_moving_inv_var']
model.aux_params['bn_pool4_moving_var'] = model.aux_params['bn_pool4_moving_inv_var']
model.aux_params['bn_pool1_moving_var'] = model.aux_params['bn_pool1_moving_inv_var']
del model.aux_params['bn_pool1_moving_inv_var']
del model.aux_params['bn_pool2_moving_inv_var']
del model.aux_params['bn_pool3_moving_inv_var']
del model.aux_params['bn_pool4_moving_inv_var']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants