Navigation

    Gpushare.com

    • Register
    • Login
    • Search
    • Popular
    • Categories
    • Recent
    • Tags

    代码报错: Blas GEMM launch failed :【官方悬赏20元代金券】

    机器学习
    2
    2
    106
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 1
      182****6091 last edited by Alice_恒源云

      大佬们谁能告诉我,代码一点击训练就会报Blas GEMM launch failed的报错,而且从pysmi来看显存没有进程在用
      Caused by op ‘gradients/tower_1/predict_same/predict/fc/fc_2/MatMul_grad/MatMul_1’, defined at:
      File “train_siamese.py”, line 29, in <module>
      net.model()
      File “/hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master/siamese_single_net.py”, line 88, in model
      grad = self._opt.compute_gradients(loss, var_list=self.get_variables())
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/optimizer.py”, line 512, in compute_gradients
      colocate_gradients_with_ops=colocate_gradients_with_ops)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py”, line 664, in gradients
      unconnected_gradients)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py”, line 965, in _GradientsHelper
      lambda: grad_fn(op, *out_grads))
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py”, line 420, in _MaybeCompile
      return grad_fn() # Exit early
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gradients_impl.py”, line 965, in <lambda>
      lambda: grad_fn(op, *out_grads))
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_grad.py”, line 1132, in _MatMulGrad
      grad_b = gen_math_ops.mat_mul(a, grad, transpose_a=True)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py”, line 5333, in mat_mul
      name=name)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py”, line 788, in _apply_op_helper
      op_def=op_def)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py”, line 507, in new_func
      return func(*args, **kwargs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py”, line 3300, in create_op
      op_def=op_def)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py”, line 1801, in init
      self._traceback = tf_stack.extract_stack()

      …which was originally created as op ‘tower_1/predict_same/predict/fc/fc_2/MatMul’, defined at:
      File “train_siamese.py”, line 29, in <module>
      net.model()
      File “/hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master/siamese_single_net.py”, line 79, in model
      out_ab = self.predict(feat_ab, name=‘predict’)
      File “/hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master/siamese_single_net.py”, line 138, in predict
      out = layers.stack(feat_ab, layers.fully_connected, [1024,256], scope=‘fc’)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py”, line 3057, in stack
      outputs = layer(outputs, *layer_args, **kwargs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/contrib/framework/python/ops/arg_scope.py”, line 182, in func_with_args
      return func(*args, **current_args)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/contrib/layers/python/layers/layers.py”, line 1855, in fully_connected
      outputs = layer.apply(inputs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 1227, in apply
      return self.call(inputs, *args, **kwargs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/layers/base.py”, line 530, in call
      outputs = super(Layer, self).call(inputs, *args, **kwargs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 554, in call
      outputs = self.call(inputs, *args, **kwargs)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/layers/core.py”, line 975, in call
      outputs = gen_math_ops.mat_mul(inputs, self.kernel)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py”, line 5333, in mat_mul
      name=name)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/op_def_library.py”, line 788, in _apply_op_helper
      op_def=op_def)
      File “/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py”, line 507, in new_func
      return func(*args, **kwargs)

      InternalError (see above for traceback): Blas GEMM launch failed : a.shape=(128, 1024), b.shape=(128, 256), m=1024, n=256, k=128
      [[node gradients/tower_1/predict_same/predict/fc/fc_2/MatMul_grad/MatMul_1 (defined at /hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master/siamese_single_net.py:88) ]]
      [[node Adam/update (defined at /hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master/siamese_single_net.py:98) ]]

      root@Ia3646c1ff00501b77:/hy-tmp/MultiBranch_CNNCopyMove_Disambiguation-master#

      1 Reply Last reply Reply Quote 0
      • Alice_恒源云
        Alice_恒源云 last edited by

        你好呀,GPUSHARE为大家打造了专属的交流群,欢迎你加入哦,如果是认证学生,首次入群还可以领取50元代金券

        你提出的技术问题、分享的训练心得,都会得到大家的关注和讨论哈

        屏幕快照 2022-04-01 上午11.09.36.png

        如何入群:联系平台客服,发送用户ID即可

        客服企微 5.12更新副本 -200.png

        1 Reply Last reply Reply Quote 0
        • First post
          Last post