如何从网格搜索结果中确定epoch超参数

时间:2016-11-29 10:59:30

标签: r deep-learning h2o grid-search hyperparameters

我运行了一个网格搜索,其中epochs是超参数之一。选择最佳模型后,如何确定为此特定模型选择的纪元?

以下是该模型的摘要     型号细节:     ==============

H2OBinomialModel: deeplearning
Model ID:  dl_grid_model_19 
Status of Neuron Layers: predicting Churn, 2-class classification, bernoulli distribution, CrossEntropy loss, 4,226 weights/biases, 44.1 KB, 47,520 training samples, mini-batch size 1
  layer units             type dropout       l1       l2 mean_rate rate_rms momentum mean_weight weight_rms
1     1    30            Input  0.00 %                                                                     
2     2    32 RectifierDropout 20.00 % 0.000010 0.000010  0.009995 0.000000 0.501901   -0.011006   0.210611
3     3    32 RectifierDropout 20.00 % 0.000010 0.000010  0.009995 0.000000 0.501901   -0.035854   0.191687
4     4    32 RectifierDropout 20.00 % 0.000010 0.000010  0.009995 0.000000 0.501901   -0.029072   0.185352
5     5    32 RectifierDropout 20.00 % 0.000010 0.000010  0.009995 0.000000 0.501901   -0.057359   0.186863
6     6     2          Softmax         0.000010 0.000010  0.009995 0.000000 0.501901    0.122655   0.406789
  mean_bias bias_rms
1                   
2  0.401924 0.136989
3  0.938406 0.041128
4  0.950918 0.043826
5  0.915588 0.060796
6  0.019925 0.175195


H2OBinomialMetrics: deeplearning
** Reported on training data. **
** Metrics reported on full training frame **

MSE:  0.1946901
RMSE:  0.441237
LogLoss:  0.5731371
Mean Per-Class Error:  0.194215
AUC:  0.8767996
Gini:  0.7535992

Confusion Matrix for F1-optimal threshold:
         No  Yes    Error       Rate
No     1755  614 0.259181  =614/2369
Yes     308 2075 0.129249  =308/2383
Totals 2063 2689 0.194024  =922/4752

Maximum Metrics: Maximum metrics at their respective thresholds
                        metric threshold    value idx
1                       max f1  0.216316 0.818218 266
2                       max f2  0.058723 0.889206 348
3                 max f0point5  0.306487 0.801744 216
4                 max accuracy  0.217122 0.805976 265
5                max precision  0.730797 1.000000   0
6                   max recall  0.006754 1.000000 398
7              max specificity  0.730797 1.000000   0
8             max absolute_mcc  0.216316 0.616944 266
9   max min_per_class_accuracy  0.257957 0.795636 242
10 max mean_per_class_accuracy  0.217122 0.805792 265

Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`
H2OBinomialMetrics: deeplearning
** Reported on validation data. **
** Metrics reported on full validation frame **

MSE:  0.1418929
RMSE:  0.3766867
LogLoss:  0.4374728
Mean Per-Class Error:  0.2603761
AUC:  0.8306744
Gini:  0.6613489

Confusion Matrix for F1-optimal threshold:
         No Yes    Error       Rate
No     1075 201 0.157524  =201/1276
Yes     162 284 0.363229   =162/446
Totals 1237 485 0.210801  =363/1722

Maximum Metrics: Maximum metrics at their respective thresholds
                        metric threshold    value idx
1                       max f1  0.323830 0.610097 183
2                       max f2  0.087110 0.740000 319
3                 max f0point5  0.514027 0.608666  94
4                 max accuracy  0.514027 0.800232  94
5                max precision  0.668538 0.875000  21
6                   max recall  0.011443 1.000000 389
7              max specificity  0.717464 0.999216   0
8             max absolute_mcc  0.323830 0.466764 183
9   max min_per_class_accuracy  0.229876 0.746082 238
10 max mean_per_class_accuracy  0.173814 0.753367 273

Gains/Lift Table: Extract with `h2o.gainsLift(<model>, <data>)` or `h2o.gainsLift(<model>, valid=<T/F>, xval=<T/F>)`

1 个答案:

答案 0 :(得分:2)

要了解模型使用了多少个纪元,最好的方法是查看分数历史记录。例如。对于模型 <?xml version="1.0" encoding="utf-8"?> <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:id="@+id/activity_main" android:layout_width="match_parent" android:layout_height="match_parent" android:paddingLeft="@dimen/activity_horizontal_margin" android:paddingRight="@dimen/activity_horizontal_margin" android:paddingTop="@dimen/activity_vertical_margin" android:paddingBottom="@dimen/activity_vertical_margin" tools:context="com.example.alime.beispieltextview1.MainActivity"> <LinearLayout android:orientation="vertical" android:layout_width="match_parent" android:layout_height="match_parent"> <LinearLayout android:orientation="vertical" android:layout_width="match_parent" android:layout_height="150dp"> <TextView android:id="@+id/text1" android:layout_width="80dp" android:layout_height="30dp" android:layout_alignParentStart="true" android:layout_below="@+id/frage1" android:layout_marginTop="14dp" android:text="Zutat: *" android:textColor="@android:color/background_dark" android:textSize="18sp" android:textStyle="normal|bold" /> <AutoCompleteTextView android:id="@+id/zutatensuche" android:layout_width="260dp" android:layout_height="40dp" android:layout_alignParentStart="true" android:layout_below="@+id/text1" android:background="@android:drawable/editbox_dropdown_light_frame" android:ems="10" android:inputType="textPersonName" android:singleLine="true"/> <Button android:id="@+id/button" android:layout_width="60dp" android:layout_height="45dp" android:layout_alignBaseline="@+id/zutatensuche" android:layout_alignBottom="@+id/zutatensuche" android:layout_alignEnd="@+id/suchbutton" android:text="+" android:background="#FF2FC10F"/> </LinearLayout> <LinearLayout android:orientation="vertical" android:layout_width="match_parent" android:layout_height="108dp" android:id="@+id/layoutlist"> <TextView android:id="@+id/zutat1" android:layout_width="150dp" android:layout_margin="10dp" android:layout_height="30dp" /> <TextView android:layout_height="30dp" android:id="@+id/zutat2" android:layout_width="150dp" android:layout_margin="10dp" /> <TextView android:layout_height="30dp" android:id="@+id/zutat3" android:layout_width="150dp" android:layout_margin="10dp" /> <TextView android:layout_height="30dp" android:id="@+id/zutat4" android:layout_width="150dp" android:layout_margin="10dp" /> <TextView android:layout_height="30dp" android:id="@+id/zutat5" android:layout_width="150dp" android:layout_margin="10dp" /> <TextView android:layout_height="30dp" android:id="@+id/zutat6" android:layout_width="150dp" android:layout_margin="10dp" /> </LinearLayout> <Button android:text="Button" android:layout_width="match_parent" android:layout_height="wrap_content" android:id="@+id/button8" /> </LinearLayout> </RelativeLayout> public class MainActivity extends AppCompatActivity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); /*Die eingegebenen Zutaten werden nach betätigen der Button aufgelistet*/ Button zutatbutton = (Button) findViewById(R.id.button); zutatbutton.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View view) { EditText e = (EditText) findViewById(R.id.zutatensuche); String text = e.getText().toString(); TextView t1 = (TextView) findViewById(R.id.zutat1); TextView t2 = (TextView) findViewById(R.id.zutat2); TextView t3 = (TextView) findViewById(R.id.zutat3); TextView t4 = (TextView) findViewById(R.id.zutat4); TextView t5 = (TextView) findViewById(R.id.zutat5); TextView t6 = (TextView) findViewById(R.id.zutat6); if (t1.getText().toString().trim().isEmpty()) { t1.setText(text); } else if (t2.getText().toString().trim().isEmpty()) { String text2 = e.getText().toString(); t2.setText(text2); } else if (t3.getText().toString().trim().isEmpty()) { String text3 = e.getText().toString(); t3.setText(text3); } else if (t4.getText().toString().trim().isEmpty()) { String text4 = e.getText().toString(); t4.setText(text4); } else if (t5.getText().toString().trim().isEmpty()) { String text5 = e.getText().toString(); t5.setText(text5); } else if (t6.getText().toString().trim().isEmpty()) { String text6 = e.getText().toString(); t6.setText(text6); } e.setText(""); } }); } }

m

(或者对于图形版本,绘制模型:h2o.scoreHistory(m)

这可能是太多的信息,所以减少它只显示时代:

plot(m)

(我刚才注意到h2o.scoreHistory(m)[,c("epochs")] 也会奏效。)

显示返回的最终模型的纪元,包括:

h2o.scoreHistory(m)$epochs

顺便说一句,如果您刚刚打印了网格对象,那么您应该将时期视为其中一个列,如果它是您的超参数之一。

回答你没有问过的问题:看看提前停止,这将使你不必提前猜测你需要多少个纪元,因此也可以节省你的网格搜索中有一个超参数。

您还可以简单地制作具有您正在考虑的最高纪元值的模型,并查看得分历史记录,以获得您感兴趣的每个其他纪元值的得分。