我有一个简单的Convolution1D模型,我已成功训练
model = Sequential()
model.add(Embedding(input_dim=vocabsize, output_dim=32,
input_length=STR_MAX_LEN, dropout=0.2))
model.add(Dropout(0.2))
model.add(Convolution1D(64, 5, activation='relu', border_mode='same'))
model.add(Dropout(0.2))
model.add(MaxPooling1D())
model.add(Flatten())
model.add(Dense(100, activation='relu'))
model.add(Dropout(0.7))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss="binary_crossentropy", optimizer=Adam(), metrics=['accuracy'])
model.summary()
模型摘要如下
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
embedding_1 (Embedding) (None, 500, 32) 160000 embedding_input_1[0][0]
____________________________________________________________________________________________________
dropout_1 (Dropout) (None, 500, 32) 0 embedding_1[0][0]
____________________________________________________________________________________________________
convolution1d_1 (Convolution1D) (None, 500, 64) 10304 dropout_1[0][0]
____________________________________________________________________________________________________
dropout_2 (Dropout) (None, 500, 64) 0 convolution1d_1[0][0]
____________________________________________________________________________________________________
maxpooling1d_1 (MaxPooling1D) (None, 250, 64) 0 dropout_2[0][0]
____________________________________________________________________________________________________
flatten_1 (Flatten) (None, 16000) 0 maxpooling1d_1[0][0]
____________________________________________________________________________________________________
dense_1 (Dense) (None, 100) 1600100 flatten_1[0][0]
____________________________________________________________________________________________________
dropout_3 (Dropout) (None, 100) 0 dense_1[0][0]
____________________________________________________________________________________________________
dense_2 (Dense) (None, 1) 101 dropout_3[0][0]
====================================================================================================
Total params: 1770505
____________________________________________________________________________________________________
我有一个文本,我需要运行预测。
text = "dont know what could have saved limp dispiriting yam but it definitely wasnt a lukewarm mushroom as murky and appealing as bong water"
textWordsArray = np.array(text.split())
textIdxArrayPadded =
sequence.pad_sequences(textWordsIdxArray,maxlen=STR_MAX_LEN, value=0)
textIdxArrayPadded
文本输入的结构
array([[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 5363, 121, 48, 97,
25, 1891, 8849, 51645, 19831, 18, 9, 404, 15422, 3, 15610, 27479, 14,
7217, 2, 2273, 14, 36597, 1090]], dtype=int32)
但是,当我运行预测时,我收到以下错误。
预测= model.predict(textIdxArrayPadded,batch_size = 1,verbose = 1)
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-70-818365da75ca> in <module>()
----> 1 prediction = model.predict(textIdxArrayPadded, batch_size=1,verbose=1)
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/models.pyc in predict(self, x, batch_size, verbose)
669 if self.model is None:
670 self.build()
--> 671 return self.model.predict(x, batch_size=batch_size, verbose=verbose)
672
673 def predict_on_batch(self, x):
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in predict(self, x, batch_size, verbose)
1177 f = self.predict_function
1178 return self._predict_loop(f, ins,
-> 1179 batch_size=batch_size, verbose=verbose)
1180
1181 def train_on_batch(self, x, y,
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/engine/training.pyc in _predict_loop(self, f, ins, batch_size, verbose)
876 ins_batch = slice_X(ins, batch_ids)
877
--> 878 batch_outs = f(ins_batch)
879 if type(batch_outs) != list:
880 batch_outs = [batch_outs]
/home/ubuntu/anaconda2/lib/python2.7/site-packages/keras/backend/theano_backend.pyc in __call__(self, inputs)
715 def __call__(self, inputs):
716 assert type(inputs) in {list, tuple}
--> 717 return self.function(*inputs)
718
719
/home/ubuntu/anaconda2/lib/python2.7/site-packages/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
869 node=self.fn.nodes[self.fn.position_of_error],
870 thunk=thunk,
--> 871 storage_map=getattr(self.fn, 'storage_map', None))
872 else:
873 # old-style linkers raise their own exceptions
/home/ubuntu/anaconda2/lib/python2.7/site-packages/theano/gof/link.pyc in raise_with_op(node, thunk, exc_info, storage_map)
312 # extra long error message in that case.
313 pass
--> 314 reraise(exc_type, exc_value, exc_trace)
315
316
/home/ubuntu/anaconda2/lib/python2.7/site-packages/theano/compile/function_module.pyc in __call__(self, *args, **kwargs)
857 t0_fn = time.time()
858 try:
--> 859 outputs = self.fn()
860 except Exception:
861 if hasattr(self.fn, 'position_of_error'):
IndexError: One of the index value is out of bound. Error code: 65535.\n
Apply node that caused the error: GpuAdvancedSubtensor1(GpuElemwise{Composite{Switch(i0, (i1 * i2 * i3), i2)},no_inplace}.0, Elemwise{Cast{int64}}.0)
Toposort index: 38
Inputs types: [CudaNdarrayType(float32, matrix), TensorType(int64, vector)]
Inputs shapes: [(5000, 32), (500,)]
Inputs strides: [(32, 1), (8,)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[GpuReshape{3}(GpuAdvancedSubtensor1.0, MakeVector{dtype='int64'}.0)]]
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
答案 0 :(得分:0)
我的嵌入仅限于vocabsize,但是我忘了将id这个词限制为vocabsize
我在另一个论坛上回答了这个问题,并在此处发布了作者的解决方案。
@niazangels Niyas Mohammed看起来你忘了限制 编码测试输入时,词汇量为5000!
将词汇量限制为5000
textWordsIdxArray = [np.array([i if i < vocabsize -1 else vocabsize -1 for i in s]) for s in textWordsIdxArray]