Keras详细培训进度条在每个批次问题上写一个新行

Question

在Keras运行Dense前馈神经网络。 两个输出有class_weights，第三个输出有sample_weights。出于某种原因，它打印出计算的每个批次的进度详细显示，而不是更新与其应该在同一行上的打印... 这件事发生在你身上吗？ 它是如何修复的？ 来自shell：

42336/747322 [====>.........................] - ETA: 79s - loss: 20.7154 - x1_loss: 9.5913 - x2_loss: 10.0536 - x3_loss: 1.0705 - x1_acc: 0.6930 - x2_acc: 0.4433 - x3_acc: 0.6821
143360/747322 [====>.........................] - ETA: 78s - loss: 20.7387 - x1_loss: 9.6131 - x2_loss: 10.0555 - x3_loss: 1.0702 - x1_acc: 0.6930 - x2_acc: 0.4432 - x3_acc: 0.6820
144384/747322 [====>.........................] - ETA: 78s - loss: 20.7362 - x1_loss: 9.6067 - x2_loss: 10.0608 - x3_loss: 1.0687 - x1_acc: 0.6930 - x2_acc: 0.4429 - x3_acc: 0.6817
145408/747322 [====>.........................] - ETA: 78s - loss: 20.7257 - x1_loss: 9.5985 - x2_loss: 10.0571 - x3_loss: 1.0702 - x1_acc: 0.6929 - x2_acc: 0.4428 - x3_acc: 0.6815
146432/747322 [====>.........................] - ETA: 78s - loss: 20.7145 - x1_loss: 9.5849 - x2_loss: 10.0605 - x3_loss: 1.0691 - x1_acc: 0.6932 - x2_acc: 0.4429 - x3_acc: 0.6816
147456/747322 [====>.........................] - ETA: 78s - loss: 20.7208 - x1_loss: 9.5859 - x2_loss: 10.0662 - x3_loss: 1.0688 - x1_acc: 0.6931 - x2_acc: 0.4429 - x3_acc: 0.6815
148480/747322 [====>.........................] - ETA: 78s - loss: 20.7078 - x1_loss: 9.5762 - x2_loss: 10.0636 - x3_loss: 1.0680 - x1_acc: 0.6932 - x2_acc: 0.4430 - x3_acc: 0.6815
149504/747322 [=====>........................] - ETA: 77s - loss: 20.6987 - x1_loss: 9.5749 - x2_loss: 10.0555 - x3_loss: 1.0683 - x1_acc: 0.6931 - x2_acc: 0.4430 - x3_acc: 0.6817
150528/747322 [=====>........................] - ETA: 77s - loss: 20.9883 - x1_loss: 9.5688 - x2_loss: 10.3509 - x3_loss: 1.0686 - x1_acc: 0.6928 - x2_acc: 0.4428 - x3_acc: 0.6819
151552/747322 [=====>........................] - ETA: 77s - loss: 20.9721 - x1_loss: 9.5606 - x2_loss: 10.3435 - x3_loss: 1.0679 - x1_acc: 0.6927 - x2_acc: 0.4426 - x3_acc: 0.6821
152576/747322 [=====>........................] - ETA: 77s - loss: 20.9585 - x1_loss: 9.5558 - x2_loss: 10.3355 - x3_loss: 1.0672 - x1_acc: 0.6926 - x2_acc: 0.4425 - x3_acc: 0.6822
153600/747322 [=====>........................] - ETA: 77s - loss: 20.9409 - x1_loss: 9.5447 - x2_loss: 10.3300 - x3_loss: 1.0662 - x1_acc: 0.6925 - x2_acc: 0.4426 - x3_acc: 0.6822
154624/747322 [=====>........................] - ETA: 77s - loss: 20.9254 - x1_loss: 9.5341 - x2_loss: 10.3250 - x3_loss: 1.0663 - x1_acc: 0.6924 - x2_acc: 0.4425 - x3_acc: 0.6825
155648/747322 [=====>........................] - ETA: 77s - loss: 20.9189 - x1_loss: 9.5270 - x2_loss: 10.3249 - x3_loss: 1.0670 - x1_acc: 0.6925 - x2_acc: 0.4425 - x3_acc: 0.6825
156672/747322 [=====>........................] - ETA: 76s - loss: 20.9069 - x1_loss: 9.5155 - x2_loss: 10.3256 - x3_loss: 1.0658 - x1_acc: 0.6927 - x2_acc: 0.4423 - x3_acc: 0.6827
157696/747322 [=====>........................] - ETA: 76s - loss: 20.9275 - x1_loss: 9.5461 - x2_loss: 10.3163 - x3_loss: 1.0651 - x1_acc: 0.6927 - x2_acc: 0.4422 - x3_acc: 0.6828
158720/747322 [=====>........................] - ETA: 76s - loss: 21.4809 - x1_loss: 10.1018 - x2_loss: 10.3133 - x3_loss: 1.0659 - x1_acc: 0.6928 - x2_acc: 0.4422 - x3_acc: 0.6829
159744/747322 [=====>........................] - ETA: 76s - loss: 21.4617 - x1_loss: 10.0871 - x2_loss: 10.3093 - x3_loss: 1.0653 - x1_acc: 0.6928 - x2_acc: 0.4421 - x3_acc: 0.6830
160768/747322 [=====>........................] - ETA: 76s - loss: 21.5462 - x1_loss: 10.1705 - x2_loss: 10.3105 - x3_loss: 1.0652 - x1_acc: 0.6928 - x2_acc: 0.4420 - x3_acc: 0.6832
161792/747322 [=====>........................] - ETA: 76s - loss: 21.5642 - x1_loss: 10.1849 - x2_loss: 10.3138 - x3_loss: 1.0655 - x1_acc: 0.6928 - x2_acc: 0.4418 - x3_acc: 0.6832
162816/747322 [=====>........................] - ETA: 76s - loss: 21.5508 - x1_loss: 10.1739 - x2_loss: 10.3118 - x3_loss: 1.0651 - x1_acc: 0.6928 - x2_acc: 0.4418 - x3_acc: 0.6833
163840/747322 [=====>........................] - ETA: 76s - loss: 21.5323 - x1_loss: 10.1606 - x2_loss: 10.3057 - x3_loss: 1.0659 - x1_acc: 0.6927 - x2_acc: 0.4419 - x3_acc: 0.6833
164864/747322 [=====>........................] - ETA: 75s - loss: 21.5282 - x1_loss: 10.1607 - x2_loss: 10.3016 - x3_loss: 1.0659 - x1_acc: 0.6926 - x2_acc: 0.4418 - x3_acc: 0.6834
165888/747322 [=====>........................] - ETA: 75s - loss: 21.5321 - x1_loss: 10.1696 - x2_loss: 10.2963 - x3_loss: 1.0662 - x1_acc: 0.6927 - x2_acc: 0.4417 - x3_acc: 0.6834
166912/747322 [=====>........................] - ETA: 75s - loss: 21.5131 - x1_loss: 10.1554 - x2_loss: 10.2912 - x3_loss: 1.0664 - x1_acc: 0.6927 - x2_acc: 0.4416 - x3_acc: 0.6833
167936/747322 [=====>........................] - ETA: 75s - loss: 21.5211 - x1_loss: 10.1649 - x2_loss: 10.2886 - x3_loss: 1.0676 - x1_acc: 0.6929 - x2_acc: 0.4415 - x3_acc: 0.6835
168960/747322 [=====>........................] - ETA: 75s - loss: 21.5049 - x1_loss: 10.1504 - x2_loss: 10.2870 - x3_loss: 1.0676 - x1_acc: 0.6930 - x2_acc: 0.4414 - x3_acc: 0.6835
169984/747322 [=====>........................] - ETA: 75s - loss: 21.5171 - x1_loss: 10.1684 - x2_loss: 10.2818 - x3_loss: 1.0670 - x1_acc: 0.6931 - x2_acc: 0.4414 - x3_acc: 0.6832
171008/747322 [=====>........................] - ETA: 75s - loss: 21.5036 - x1_loss: 10.1541 - x2_loss: 10.2816 - x3_loss: 1.0678 - x1_acc: 0.6931 - x2_acc: 0.4413 - x3_acc: 0.6828
172032/747322 [=====>........................] - ETA: 75s - loss: 21.4870 - x1_loss: 10.1377 - x2_loss: 10.2816 - x3_loss: 1.0677 - x1_acc: 0.6931 - x2_acc: 0.4413 - x3_acc: 0.6827
173056/747322 [=====>........................] - ETA: 75s - loss: 21.4729 - x1_loss: 10.1210 - x2_loss: 10.2836 - x3_loss: 1.0683 - x1_acc: 0.6931 - x2_acc: 0.4413 - x3_acc: 0.6824
174080/747322 [=====>........................] - ETA: 74s - loss: 21.4512 - x1_loss: 10.1085 - x2_loss: 10.2742 - x3_loss: 1.0685 - x1_acc: 0.6931 - x2_acc: 0.4414 - x3_acc: 0.6821
175104/747322 [======>.......................] - ETA: 74s - loss: 21.4315 - x1_loss: 10.0977 - x2_loss: 10.2647 - x3_loss: 1.0690 - x1_acc: 0.6931 - x2_acc: 0.4414 - x3_acc: 0.6817
176128/747322 [======>.......................] - ETA: 74s - loss: 21.4231 - x1_loss: 10.0880 - x2_loss: 10.2656 - x3_loss: 1.0695 - x1_acc: 0.6932 - x2_acc: 0.4412 - x3_acc: 0.6813
177152/747322 [======>.......................] - ETA: 74s - loss: 21.4059 - x1_loss: 10.0732 - x2_loss: 10.2639 - x3_loss: 1.0688 - x1_acc: 0.6931 - x2_acc: 0.4412 - x3_acc: 0.6809
178176/747322 [======>.......................] - ETA: 74s - loss: 21.4289 - x1_loss: 10.0967 - x2_loss: 10.2634 - x3_loss: 1.0688 - x1_acc: 0.6930 - x2_acc: 0.4413 - x3_acc: 0.6807
179200/747322 [======>.......................] - ETA: 74s - loss: 21.4329 - x1_loss: 10.1092 - x2_loss: 10.2557 - x3_loss: 1.0681 - x1_acc: 0.6930 - x2_acc: 0.4414 - x3_acc: 0.6807
180224/747322 [======>.......................] - ETA: 74s - loss: 21.4277 - x1_loss: 10.1099 - x2_loss: 10.2503 - x3_loss: 1.0675 - x1_acc: 0.6930 - x2_acc: 0.4415 - x3_acc: 0.6807
181248/747322 [======>.......................] - ETA: 73s - loss: 21.4088 - x1_loss: 10.0975 - x2_loss: 10.2441 - x3_loss: 1.0671 - x1_acc: 0.6929 - x2_acc: 0.4416 - x3_acc: 0.6808
182272/747322 [======>.......................] - ETA: 73s - loss: 21.3909 - x1_loss: 10.0841 - x2_loss: 10.2405 - x3_loss: 1.0663 - x1_acc: 0.6929 - x2_acc: 0.4415 - x3_acc: 0.6811
183296/747322 [======>.......................] - ETA: 73s - loss: 21.3775 - x1_loss: 10.0699 - x2_loss: 10.2416 - x3_loss: 1.0660 - x1_acc: 0.6927 - x2_acc: 0.4415 - x3_acc: 0.6813
184320/747322 [======>.......................] - ETA: 73s - loss: 21.3682 - x1_loss: 10.0664 - x2_loss: 10.2355 - x3_loss: 1.0662 - x1_acc: 0.6928 - x2_acc: 0.4417 - x3_acc: 0.6818
185344/747322 [======>.......................] - ETA: 73s - loss: 21.4162 - x1_loss: 10.1213 - x2_loss: 10.2291 - x3_loss: 1.0658 - x1_acc: 0.6927 - x2_acc: 0.4417 - x3_acc: 0.6821
186368/747322 [======>.......................] - ETA: 73s - loss: 21.3981 - x1_loss: 10.1050 - x2_loss: 10.2259 - x3_loss: 1.0672 - x1_acc: 0.6928 - x2_acc: 0.4418 - x3_acc: 0.6825
187392/747322 [======>.......................] - ETA: 73s - loss: 21.3793 - x1_loss: 10.0909 - x2_loss: 10.2212 - x3_loss: 1.0673 - x1_acc: 0.6928 - x2_acc: 0.4417 - x3_acc: 0.6827
188416/747322 [======>.......................] - ETA: 73s - loss: 21.3614 - x1_loss: 10.0784 - x2_loss: 10.2163 - x3_loss: 1.0668 - x1_acc: 0.6930 - x2_acc: 0.4418 - x3_acc: 0.6830
189440/747322 [======>.......................] - ETA: 72s - loss: 21.3736 - x1_loss: 10.0909 - x2_loss: 10.2169 - x3_loss: 1.0659 - x1_acc: 0.6930 - x2_acc: 0.4417 - x3_acc: 0.6833
190464/747322 [======>.......................] - ETA: 72s - loss: 21.4615 - x1_loss: 10.0802 - x2_loss: 10.3165 - x3_loss: 1.0648 - x1_acc: 0.6930 - x2_acc: 0.4418 - x3_acc: 0.6836
191488/747322 [======>.......................] - ETA: 72s - loss: 21.4493 - x1_loss: 10.0653 - x2_loss: 10.3194 - x3_loss: 1.0646 - x1_acc: 0.6930 - x2_acc: 0.4417 - x3_acc: 0.6837
192512/747322 [======>.......................] - ETA: 72s - loss: 21.4863 - x1_loss: 10.0997 - x2_loss: 10.3207 - x3_loss: 1.0659 - x1_acc: 0.6927 - x2_acc: 0.4416 - x3_acc: 0.6837
193536/747322 [======>.......................] - ETA: 72s - loss: 21.4750 - x1_loss: 10.0895 - x2_loss: 10.3198 - x3_loss: 1.0657 - x1_acc: 0.6929 - x2_acc: 0.4416 - x3_acc: 0.6839
194560/747322 [======>.......................] - ETA: 72s - loss: 21.4577 - x1_loss: 10.0755 - x2_loss: 10.3168 - x3_loss: 1.0654 - x1_acc: 0.6929 - x2_acc: 0.4416 - x3_acc: 0.6839
195584/747322 [======>.......................] - ETA: 72s - loss: 21.4429 - x1_loss: 10.0627 - x2_loss: 10.3148 - x3_loss: 1.0655 - x1_acc: 0.6929 - x2_acc: 0.4417 - x3_acc: 0.6838
196608/747322 [======>.......................] - ETA: 71s - loss: 21.4307 - x1_loss: 10.0558 - x2_loss: 10.3089 - x3_loss: 1.0660 - x1_acc: 0.6929 - x2_acc: 0.4418 - x3_acc: 0.6834
197632/747322 [======>.......................] - ETA: 71s - loss: 21.4446 - x1_loss: 10.0669 - x2_loss: 10.3107 - x3_loss: 1.0670 - x1_acc: 0.6929 - x2_acc: 0.4418 - x3_acc: 0.6830
198656/747322 [======>.......................] - ETA: 71s - loss: 21.4287 - x1_loss: 10.0552 - x2_loss: 10.3071 - x3_loss: 1.0665 - x1_acc: 0.6930 - x2_acc: 0.4418 - x3_acc: 0.6827
199680/747322 [=======>......................] - ETA: 71s - loss: 21.4168 - x1_loss: 10.0474 - x2_loss: 10.3034 - x3_loss: 1.0660 - x1_acc: 0.6931 - x2_acc: 0.4417 - x3_acc: 0.6823
200704/747322 [=======>......................] - ETA: 71s - loss: 21.4064 - x1_loss: 10.0385 - x2_loss: 10.3015 - x3_loss: 1.0664 - x1_acc: 0.6931 - x2_acc: 0.4417 - x3_acc: 0.6819
201728/747322 [=======>......................] - ETA: 71s - loss: 21.3954 - x1_loss: 10.0320 - x2_loss: 10.2974 - x3_loss: 1.0659 - x1_acc: 0.6931 - x2_acc: 0.4416 - x3_acc: 0.6817
202752/747322 [=======>......................] - ETA: 71s - loss: 21.3870 - x1_loss: 10.0243 - x2_loss: 10.2965 - x3_loss: 1.0662 - x1_acc: 0.6931 - x2_acc: 0.4415 - x3_acc: 0.6816
203776/747322 [=======>......................] - ETA: 70s - loss: 21.3782 - x1_loss: 10.0155 - x2_loss: 10.2954 - x3_loss: 1.0673 - x1_acc: 0.6929 - 

etc...

Answer 1

这似乎是Keras的一贯问题。我试图找到行

sys.stdout.write('\b' * prev_total_width)

sys.stdout.write('\r')

在Keras / utils / generic_utils.py文件中，它们（截至当前版本）相应地在258和259处。我评价像258，但这似乎不能解决问题。我确实设法通过评论该行来使进度条工作：

第303行：sys.stdout.write(info)

似乎信息使得终端的栏太长，所以它会突破到新的一行。

所以我终于解决了这个问题。看起来它最后很简单......

  

让终端更宽......

注意：在Linux Ubuntu 16.04上测试| Keras版本2.0.5

Answer 2

之前曾提到过，但我会将其改写为对未来的用户更加明显。

您的终端太窄，无法打印所有这些值 - 只需将width构造函数的Progbar参数设置为较小的数字，或者删除/重命名一些提供的值。

Answer 3

@ user11353683和@Panos的解决方案可以完成，以便不修改源代码（这可能会造成将来的问题）：只需安装ipykernel并将其导入您的代码中即可：

pip install ipykernel 然后 import ipykernel

实际上，在Keras的generic_utils.py文件中，探测行为：

            if self._dynamic_display:
                sys.stdout.write('\b' * prev_total_width)
                sys.stdout.write('\r')
            else:
                sys.stdout.write('\n')

并且值self._dynamic_display已启动，例如：

        self._dynamic_display = ((hasattr(sys.stdout, 'isatty') and
                                  sys.stdout.isatty()) or
                                 'ipykernel' in sys.

因此，加载ipykernel会将其添加到sys.modules中，并为我解决了该问题。

Answer 4

导入库：

from tqdm.keras import TqdmCallback

使用tqdm库指定：

model.fit(xs, ys, epochs=10000, verbose=0, callbacks=[TqdmCallback(verbose=1)])

结果：

100%|██████████████| 10000/10000 [01:34<00:00, 105.57epoch/s, loss=1.56e+3]
 29%|███▌          |  2891/10000 [00:28<01:05, 108.02epoch/s, loss=1.57e+3]

对于我来说，verbose = 1是对我真正有效的。我的答案基于casper.dcl's answer。

Answer 5

我有类似的问题，但没有时间进一步调查。这个问题似乎与keras的generic_utils.py中的类Progbar有关，请参阅link，也许Python＆gt; = 3.3。

在类的更新功能中可以找到以下行：

第107行：sys.stdout.write('\b' * prev_total_width)
第108行：sys.stdout.write('\r')

我只是删除了第107行作为快速修复，因此我没有退回前一行然后执行转移到行的开头，而是仅执行移位。我想有一些比改变源代码更好的方法。

Answer 6

androst的变通办法（2017年1月4日）对我不起作用。但是，我发现当我运行我的代码时，androst从generic_utils.py引用的代码行从未执行过，因为拥抱“ if-clause”的条件始终为False。当我对if子句进行注释时，将相应的变量（手动）设置为“ True”，就可以了。

这就是我所做的更改（对我来说：generic_utils.py的第311-314行）：

    #self._dynamic_display = ((hasattr(sys.stdout, 'isatty') and
    #                          sys.stdout.isatty()) or
    #                         'ipykernel' in sys.modules)
    self._dynamic_display = True   # inserted to overwrite the above (workaround by KS)

然后进度条运行良好：-）

Answer 7

tqdm（版本> = 4.41.0）刚刚添加了对keras的内置支持，因此您可以改用它：

from tqdm.keras import TqdmCallback
...
model.fit(..., verbose=0, callbacks=[TqdmCallback(verbose=2)])

这将关闭keras的进度（verbose=0），而改用tqdm。对于回调，verbose=2表示时期和批次的单独进度条。 1表示完成后清除批处理栏。 0表示仅显示纪元（从不显示批处理条）。

如果有问题，请通过https://github.com/tqdm/tqdm/issues

Answer 8

在 Keras 文件 id time frac_day frac_week frac_month 1 4/01/2019 08:00:00 0.333 0.047 0.011 2 4/02/2019 12:00:00 0.5 0.214 0.050 3 4/03/2019 18:00:00 0.75 0.393 0.092 的第 354-359 行中：

keras/utils/generic_utils.py

我将 if self._dynamic_display: sys.stdout.write('\b' * prev_total_width) sys.stdout.write('\r') else: sys.stdout.write('\n') 替换为：sys.stdout.write('\n')。 为我工作。