我正在尝试学习在tensorflow中实现RNN . 我相信我已经成功实现了它，但是损失在0.36左右停止移动，并且推断最终会在大多数时间输出错误的序列 . 我不确定bug在哪里或者某处是否存在实现错误 .
# generate toy data - sequence is start-int-int-end
inp_dat = tf.constant(np.array([[0, 1, 1], [0, 2, 2], [0, 3, 3]]).reshape((3, 3, 1)), dtype=tf.float32)
out_dat = tf.constant(np.array([[1, 1, 5], [2, 2, 5], [3, 3, 5]]), dtype=tf.int32)
mask = np.full((3,), 3)
# define the network
cell = tf.nn.rnn_cell.BasicLSTMCell(64)
cell = tf.contrib.rnn.OutputProjectionWrapper(cell, 6)
rnn_outs, _ = tf.nn.dynamic_rnn(cell, inp_dat, mask,
initial_state=cell.zero_state(3, tf.float32))
loss = tf.contrib.seq2seq.sequence_loss(logits=rnn_outs, targets=out_dat,
weights=tf.constant(np.ones((3, 3), dtype=np.float32)))
tf.losses.add_loss(loss)
l = tf.losses.get_total_loss(add_regularization_losses=False)
opt = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
for i in range(int(10000)):
_, tl = sess.run([opt, l])
if not i % 1000:
print(tl)
这输出：
1.80929
0.595527
0.426493
0.385593
0.375896
0.372292
0.370524
0.369503
0.368848
0.368397
然后，进行推理：
start_tokens = tf.fill([1], 0)
toks = np.array([0, 1, 2, 3, 4, 5], dtype=np.float32).reshape(6, 1)
holder = tf.placeholder(tf.float32, shape=[1, 1])
pred_helper = tf.contrib.seq2seq.GreedyEmbeddingHelper(toks, start_tokens=start_tokens, end_token=5)
_, init_state = cell(holder, cell.zero_state(1, tf.float32))
decoder = tf.contrib.seq2seq.BasicDecoder(cell=cell, helper=pred_helper,
initial_state=init_state)
outputs = tf.contrib.seq2seq.dynamic_decode(decoder=decoder,output_time_major=False,
impute_finished=True, maximum_iterations=3)
当喂食 Value 时：
print(sess.run(outputs, feed_dict={holder: [[3]]})[0][1]) # [[5]]
print(sess.run(outputs, feed_dict={holder: [[2]]})[0][1]) # [[5]]
print(sess.run(outputs, feed_dict={holder: [[1]]})[0][1]) # [[1, 5]]
我觉得问题在于训练（因为那里的损失停止减少），但我也不确定我的推理是否正确编码 . 任何帮助表示赞赏！
编辑：我注意到顺序中的第1位，第2位和第3位都是5位，这可能会导致性能不佳？