我对Tensorflow中Gradient Descent Optimizer的学习速度感到困惑，
假设我试图从这些数据中预测下一个值：
x_data = [5,10,15,20,25,30,35,40]
y_data = [2,4,6,8,10,12,14,16]
如果我选择学习率为0.01，这是我的计划：
import tensorflow as tf
tf.set_random_seed(777)
#x_data=[5,10,15,20,25,30,35,40]
#y_data=[2,4,6,8,10,12,14,16,18]
x_data = [5,10,15,20,25,30,35,40]
y_data = [2,4,6,8,10,12,14,16]
one=tf.Variable(tf.random_normal([1]))
two=tf.Variable(tf.random_normal([1]))
hypo=x_data*one+two
cost=tf.reduce_mean(tf.square(hypo-y_data))
train=tf.train.GradientDescentOptimizer(0.01).minimize(cost)
ina=tf.global_variables_initializer()
with tf.Session() as tt:
tt.run(ina)
for i in range(3000):
a,b,c,d=tt.run([train,cost,one,two])
if i%10==0:
print(c,d)
然后我得到这个输出并且它进入inf（这是我的第二个混淆为什么它进入无限？）
[-20.48267746] [-1.6179111]
[ -1.06335529e+12] [ -3.75422935e+10]
[ -5.40660918e+22] [ -1.90883086e+21]
[ -2.74898110e+33] [ -9.70541703e+31]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
[ nan] [ nan]
.... ....
.... ....
但是，如果我选择learning rate as 0.001，那么我得到正确的输出：
[-0.06046534] [-0.90016752]
[ 0.43103883] [-0.87918627]
[ 0.43091267] [-0.87557721]
[ 0.4307858] [-0.87198305]
[ 0.43065941] [-0.86840361]
[ 0.43053356] [-0.8648389]
[ 0.43040821] [-0.86128885]
[ 0.43028343] [-0.85775328]
[ 0.43015912] [-0.85423231]
[ 0.43003532] [-0.85072571]
[ 0.429912] [-0.84723359]
[ 0.42978922] [-0.84375578]
[ 0.42966694] [-0.84029222]
[ 0.42954516] [-0.83684289]
[ 0.42942387] [-0.8334077]
[ 0.42930311] [-0.82998663]
[ 0.4291828] [-0.82657957]
[ 0.42906302] [-0.82318658]
[ 0.42894369] [-0.81980747]
[ 0.4288249] [-0.81644231]
[ 0.42870659] [-0.81309086]
[ 0.42858875] [-0.80975318]
[ 0.42847139] [-0.80642921]
[ 0.42835453] [-0.80311882]
[ 0.42823812] [-0.79982209]
[ 0.42812222] [-0.79653889]
[ 0.42800677] [-0.7932691]
[ 0.42789182] [-0.79001278]
[ 0.42777732] [-0.78676981]
[ 0.42766327] [-0.78354019]
[ 0.42754975] [-0.78032386]
[ 0.42743665] [-0.77712065]
[ 0.42732403] [-0.77393067]
[ 0.42721185] [-0.77075368]
[ 0.42710015] [-0.76758981]
[ 0.4269889] [-0.76443887]
[ 0.42687812] [-0.76130092]
[ 0.42676777] [-0.75817585]
[ 0.42665792] [-0.75506359]
[ 0.42654848] [-0.75196409]
[ 0.42643949] [-0.74887735]
[ 0.42633098] [-0.74580324]
[ 0.42622289] [-0.74274176]
[ 0.42611524] [-0.73969287]
[ 0.42600802] [-0.73665649]
[ 0.42590126] [-0.73363262]
[ 0.42579496] [-0.73062116]
[ 0.42568904] [-0.72762191]
[ 0.42558363] [-0.72463512]
[ 0.42547861] [-0.72166055]
[ 0.425374] [-0.7186982]
[ 0.42526984] [-0.71574789]
[ 0.4251661] [-0.71280998]
[ 0.42506284] [-0.70988399]
[ 0.42495993] [-0.70696992]
[ 0.42485747] [-0.70406777]
[ 0.42475539] [-0.70117754]
[ 0.42465383] [-0.69829923]
[ 0.42455259] [-0.69543284]
[ 0.42445183] [-0.69257832]
[ 0.42435145] [-0.68973517]
[ 0.4242515] [-0.68690395]
[ 0.42415196] [-0.68408424]
[ 0.4240528] [-0.6812762]
[ 0.42395407] [-0.67847955]
[ 0.42385572] [-0.67569441]
[ 0.42375779] [-0.6729207]
[ 0.42366028] [-0.67015845]
[ 0.42356315] [-0.66740751]
[ 0.42346644] [-0.66466784]
[ 0.42337012] [-0.66193944]
[ 0.42327416] [-0.65922225]
[ 0.42317864] [-0.65651619]
[ 0.42308348] [-0.65382123]
[ 0.42298874] [-0.65113741]
[ 0.42289436] [-0.6484645]
[ 0.42280039] [-0.64580262]
[ 0.42270681] [-0.6431517]
[ 0.42261356] [-0.64051157]
[ 0.42252076] [-0.63788235]
[ 0.42242831] [-0.63526386]
[ 0.42233622] [-0.6326561]
[ 0.42224455] [-0.63005906]
[ 0.42215326] [-0.62747276]
[ 0.42206231] [-0.62489706]
[ 0.42197174] [-0.62233192]
[ 0.42188156] [-0.61977726]
[ 0.42179173] [-0.61723322]
[ 0.42170227] [-0.61469954]
[ 0.42161322] [-0.6121763]
[ 0.42152449] [-0.60966337]
[ 0.4214361] [-0.60716075]
[ 0.42134812] [-0.60466844]
[ 0.42126048] [-0.60218632]
[ 0.42117321] [-0.5997144]
[ 0.42108631] [-0.59725261]
[ 0.42099974] [-0.59480095]
[ 0.42091355] [-0.5923593]
[ 0.42082772] [-0.58992773]
[ 0.42074221] [-0.58750612]
[ 0.42065707] [-0.58509439]
[ 0.42057225] [-0.58269262]
[ 0.42048782] [-0.58030075]
[ 0.42040369] [-0.57791865]
[ 0.42031991] [-0.57554632]
[ 0.42023653] [-0.57318377]
[ 0.42015347] [-0.57083094]
[ 0.42007077] [-0.5684877]
[ 0.41998836] [-0.56615406]
[ 0.41990632] [-0.56383008]
[ 0.41982457] [-0.56151563]
[ 0.41974318] [-0.55921066]
[ 0.41966218] [-0.55691516]
[ 0.41958147] [-0.55462909]
[ 0.4195011] [-0.55235237]
[ 0.41942102] [-0.55008501]
[ 0.41934133] [-0.54782701]
[ 0.41926193] [-0.54557824]
[ 0.41918284] [-0.54333872]
[ 0.4191041] [-0.54110831]
[ 0.41902569] [-0.53888714]
[ 0.41894755] [-0.5366751]
[ 0.41886982] [-0.53447211]
[ 0.41879237] [-0.53227806]
[ 0.41871524] [-0.53009313]
[ 0.41863838] [-0.52791727]
[ 0.41856188] [-0.52575034]
[ 0.41848567] [-0.52359205]
[ 0.41840979] [-0.52144271]
[ 0.41833425] [-0.51930231]
[ 0.41825897] [-0.51717055]
[ 0.41818401] [-0.51504761]
[ 0.41810936] [-0.51293337]
[ 0.41803503] [-0.51082784]
[ 0.417961] [-0.50873089]
[ 0.41788727] [-0.50664258]
[ 0.41781384] [-0.50456285]
[ 0.4177407] [-0.50249171]
[ 0.4176679] [-0.50042903]
[ 0.41759539] [-0.49837482]
[ 0.41752315] [-0.49632904]
[ 0.41745123] [-0.49429163]
[ 0.41737959] [-0.4922626]
[ 0.41730824] [-0.49024191]
[ 0.41723716] [-0.48822951]
[ 0.41716644] [-0.4862254]
[ 0.41709596] [-0.48422945]
[ 0.41702577] [-0.48224172]
[ 0.41695589] [-0.48026216]
[ 0.4168863] [-0.47829071]
[ 0.41681695] [-0.47632736]
[ 0.41674796] [-0.47437206]
[ 0.41667923] [-0.47242478]
[ 0.41661072] [-0.47048554]
[ 0.41654253] [-0.46855426]
[ 0.41647464] [-0.46663091]
[ 0.41640702] [-0.46471542]
[ 0.41633967] [-0.4628078]
[ 0.41627261] [-0.46090803]
[ 0.41620579] [-0.45901603]
[ 0.41613927] [-0.4571318]
[ 0.41607302] [-0.4552553]
[ 0.41600704] [-0.45338652]
[ 0.41594133] [-0.45152542]
[ 0.41587588] [-0.44967195]
[ 0.41581073] [-0.44782609]
[ 0.41574579] [-0.44598779]
[ 0.41568118] [-0.44415703]
[ 0.41561681] [-0.44233382]
[ 0.41555271] [-0.44051811]
[ 0.41548887] [-0.43870986]
[ 0.4154253] [-0.43690899]
[ 0.41536197] [-0.43511549]
[ 0.41529894] [-0.43332937]
[ 0.41523612] [-0.43155059]
[ 0.41517356] [-0.42977911]
[ 0.41511127] [-0.4280149]
[ 0.41504925] [-0.42625797]
[ 0.41498747] [-0.42450821]
[ 0.41492593] [-0.42276564]
[ 0.41486469] [-0.42103022]
[ 0.41480365] [-0.41930193]
[ 0.41474292] [-0.41758072]
[ 0.41468239] [-0.41586661]
[ 0.4146221] [-0.41415951]
[ 0.41456211] [-0.41245943]
[ 0.41450229] [-0.4107663]
[ 0.41444278] [-0.40908015]
[ 0.4143835] [-0.40740094]
[ 0.41432443] [-0.40572858]
[ 0.41426563] [-0.40406311]
[ 0.4142071] [-0.40240449]
[ 0.41414878] [-0.40075263]
[ 0.41409069] [-0.39910758]
[ 0.41403285] [-0.39746928]
[ 0.41397524] [-0.39583766]
[ 0.41391787] [-0.39421278]
[ 0.41386077] [-0.39259458]
[ 0.41380385] [-0.39098299]
[ 0.41374719] [-0.38937804]
[ 0.41369078] [-0.38777968]
[ 0.41363457] [-0.38618785]
[ 0.41357857] [-0.38460258]
[ 0.41352287] [-0.38302383]
[ 0.41346738] [-0.38145158]
[ 0.41341206] [-0.37988576]
[ 0.41335702] [-0.37832636]
[ 0.41330215] [-0.37677333]
[ 0.41324756] [-0.37522671]
[ 0.41319317] [-0.37368643]
[ 0.41313902] [-0.37215248]
[ 0.41308507] [-0.37062484]
[ 0.41303137] [-0.36910346]
[ 0.4129779] [-0.36758831]
[ 0.41292462] [-0.36607942]
[ 0.41287157] [-0.36457673]
[ 0.41281876] [-0.36308014]
[ 0.41276613] [-0.36158973]
[ 0.41271371] [-0.36010543]
[ 0.41266152] [-0.35862723]
[ 0.41260952] [-0.357155]
[ 0.41255775] [-0.35568899]
[ 0.41250622] [-0.35422897]
[ 0.41245487] [-0.35277492]
[ 0.41240376] [-0.35132682]
[ 0.41235286] [-0.34988469]
[ 0.41230217] [-0.34844851]
[ 0.41225165] [-0.34701818]
[ 0.41220134] [-0.34559363]
[ 0.41215128] [-0.34417504]
[ 0.41210139] [-0.34276217]
[ 0.41205171] [-0.3413552]
[ 0.41200227] [-0.33995393]
[ 0.41195297] [-0.33855847]
[ 0.41190392] [-0.33716872]
[ 0.41185504] [-0.33578467]
[ 0.41180637] [-0.33440632]
[ 0.41175792] [-0.33303359]
[ 0.41170964] [-0.3316665]
[ 0.4116616] [-0.33030504]
[ 0.4116137] [-0.32894915]
[ 0.41156605] [-0.32759884]
[ 0.41151857] [-0.32625404]
[ 0.41147125] [-0.32491481]
[ 0.41142419] [-0.32358104]
[ 0.41137731] [-0.32225275]
[ 0.41133058] [-0.32092994]
[ 0.41128409] [-0.31961253]
[ 0.41123778] [-0.31830055]
[ 0.41119161] [-0.31699392]
[ 0.41114569] [-0.31569266]
[ 0.41109994] [-0.31439677]
[ 0.41105434] [-0.31310621]
[ 0.41100898] [-0.31182092]
[ 0.4109638] [-0.31054091]
[ 0.41091877] [-0.30926618]
[ 0.41087398] [-0.30799666]
[ 0.41082937] [-0.30673239]
[ 0.4107849] [-0.30547327]
[ 0.41074061] [-0.30421934]
[ 0.41069651] [-0.30297056]
[ 0.41065264] [-0.30172691]
[ 0.41060886] [-0.30048832]
[ 0.41056535] [-0.29925483]
[ 0.41052195] [-0.29802641]
[ 0.4104788] [-0.29680306]
[ 0.41043577] [-0.29558468]
[ 0.41039294] [-0.29437134]
[ 0.41035026] [-0.293163]
[ 0.41030779] [-0.29195961]
[ 0.41026548] [-0.29076111]
[ 0.41022334] [-0.28956759]
[ 0.41018137] [-0.28837892]
[ 0.41013956] [-0.28719518]
[ 0.41009796] [-0.28601629]
[ 0.41005653] [-0.28484219]
[ 0.41001526] [-0.28367293]
[ 0.40997413] [-0.28250849]
[ 0.40993318] [-0.28134882]
[ 0.40989238] [-0.28019392]
[ 0.40985179] [-0.27904376]
[ 0.40981135] [-0.27789828]
[ 0.40977108] [-0.27675754]
[ 0.40973094] [-0.27562147]
[ 0.40969101] [-0.27449009]
[ 0.40965122] [-0.27336332]
[ 0.40961161] [-0.2722412]
[ 0.40957215] [-0.27112368]
[ 0.40953287] [-0.27001071]
[ 0.40949374] [-0.26890236]
[ 0.40945476] [-0.26779851]
[ 0.40941596] [-0.26669925]
[ 0.40937731] [-0.26560447]
[ 0.4093388] [-0.26451415]
[ 0.40930048] [-0.2634283]
[ 0.4092623] [-0.26234692]
[ 0.40922427] [-0.26127002]
[ 0.40918639] [-0.26019755]
[ 0.40914869] [-0.25912943]
[ 0.40911114] [-0.25806573]
再次，如果我选择学习率0.0001，那么我没有得到正确的输出：
[ 1.98175597] [-0.82839316]
[ 0.82685816] [-0.86880374]
[ 0.53213042] [-0.87884581]
[ 0.45690936] [-0.88113832]
[ 0.43770415] [-0.88145328]
[ 0.43279362] [-0.88126367]
[ 0.43153098] [-0.88094544]
[ 0.43119925] [-0.88059455]
[ 0.43110508] [-0.88023537]
[ 0.43107152] [-0.87987429]
[ 0.43105346] [-0.87951249]
[ 0.43103933] [-0.87915069]
[ 0.43102625] [-0.87878931]
[ 0.43101344] [-0.8784281]
[ 0.43100062] [-0.8780669]
[ 0.43098781] [-0.87770569]
[ 0.43097505] [-0.87734485]
[ 0.43096232] [-0.87698424]
[ 0.4309496] [-0.87662363]
[ 0.43093687] [-0.87626302]
[ 0.43092415] [-0.87590277]
[ 0.43091145] [-0.87554276]
[ 0.43089876] [-0.87518275]
[ 0.43088603] [-0.87482274]
[ 0.43087333] [-0.87446308]
[ 0.43086067] [-0.87410367]
[ 0.43084797] [-0.87374425]
[ 0.43083528] [-0.87338483]
[ 0.43082261] [-0.87302572]
[ 0.43080994] [-0.8726669]
[ 0.43079728] [-0.87230808]
[ 0.43078461] [-0.87194926]
[ 0.43077198] [-0.87159073]
[ 0.43075931] [-0.87123251]
[ 0.43074667] [-0.87087429]
[ 0.43073404] [-0.87051606]
[ 0.43072137] [-0.87015808]
[ 0.43070877] [-0.86980045]
[ 0.43069616] [-0.86944282]
[ 0.43068352] [-0.86908519]
[ 0.43067092] [-0.8687278]
[ 0.43065831] [-0.86837077]
[ 0.4306457] [-0.86801374]
[ 0.4306331] [-0.86765671]
[ 0.43062052] [-0.86729985]
[ 0.43060791] [-0.86694342]
[ 0.43059534] [-0.86658698]
[ 0.43058276] [-0.86623055]
[ 0.43057019] [-0.86587429]
[ 0.43055761] [-0.86551845]
[ 0.43054506] [-0.86516261]
[ 0.43053252] [-0.86480677]
[ 0.43051994] [-0.86445105]
[ 0.43050742] [-0.86409581]
[ 0.43049487] [-0.86374056]
[ 0.43048233] [-0.86338532]
[ 0.43046981] [-0.86303014]
[ 0.43045726] [-0.86267549]
[ 0.43044475] [-0.86232084]
[ 0.43043223] [-0.86196619]
[ 0.43041971] [-0.86161155]
[ 0.4304072] [-0.86125749]
[ 0.43039468] [-0.86090344]
[ 0.43038216] [-0.86054939]
[ 0.43036965] [-0.86019534]
[ 0.43035713] [-0.85984182]
[ 0.43034461] [-0.85948837]
[ 0.43033212] [-0.85913491]
[ 0.43031967] [-0.85878146]
[ 0.43030721] [-0.85842848]
[ 0.43029472] [-0.85807562]
[ 0.43028226] [-0.85772276]
[ 0.43026984] [-0.8573699]
[ 0.43025738] [-0.85701746]
[ 0.43024495] [-0.85666519]
[ 0.43023252] [-0.85631293]
[ 0.4302201] [-0.85596067]
[ 0.43020767] [-0.85560876]
[ 0.43019524] [-0.85525709]
[ 0.43018284] [-0.85490543]
[ 0.43017042] [-0.85455376]
[ 0.43015802] [-0.85420239]
[ 0.43014562] [-0.85385132]
[ 0.43013322] [-0.85350025]
[ 0.43012086] [-0.85314918]
[ 0.43010846] [-0.85279834]
[ 0.43009609] [-0.85244787]
[ 0.43008372] [-0.85209739]
[ 0.43007135] [-0.85174692]
[ 0.43005899] [-0.85139656]
[ 0.43004665] [-0.85104668]
[ 0.43003428] [-0.8506968]
[ 0.43002194] [-0.85034692]
[ 0.43000957] [-0.8499971]
[ 0.42999727] [-0.84964782]
[ 0.42998493] [-0.84929854]
[ 0.42997259] [-0.84894925]
[ 0.42996028] [-0.84859997]
[ 0.42994797] [-0.84825122]
[ 0.42993566] [-0.84790254]
[ 0.42992336] [-0.84755385]
[ 0.42991105] [-0.84720516]
[ 0.42989877] [-0.84685695]
[ 0.42988646] [-0.84650886]
[ 0.42987418] [-0.84616077]
[ 0.4298619] [-0.84581268]
[ 0.42984962] [-0.84546494]
[ 0.42983735] [-0.84511745]
[ 0.4298251] [-0.84476995]
[ 0.42981282] [-0.84442246]
[ 0.42980057] [-0.84407526]
[ 0.42978832] [-0.84372836]
[ 0.42977607] [-0.84338146]
[ 0.42976385] [-0.84303457]
[ 0.4297516] [-0.84268785]
[ 0.42973939] [-0.84234154]
[ 0.42972714] [-0.84199524]
[ 0.42971492] [-0.84164894]
[ 0.4297027] [-0.84130269]
[ 0.42969048] [-0.84095699]
[ 0.42967826] [-0.84061128]
[ 0.42966604] [-0.84026557]
[ 0.42965382] [-0.83991987]
[ 0.4296416] [-0.83957469]
[ 0.42962939] [-0.83922958]
[ 0.42961717] [-0.83888447]
[ 0.42960498] [-0.83853936]
[ 0.42959282] [-0.83819467]
[ 0.42958066] [-0.83785015]
[ 0.4295685] [-0.83750564]
[ 0.42955634] [-0.83716112]
[ 0.42954418] [-0.83681691]
[ 0.42953205] [-0.83647299]
[ 0.42951992] [-0.83612907]
[ 0.42950779] [-0.83578515]
[ 0.42949563] [-0.83544135]
[ 0.42948353] [-0.83509803]
[ 0.4294714] [-0.83475471]
[ 0.4294593] [-0.83441138]
[ 0.42944717] [-0.83406806]
[ 0.42943507] [-0.83372533]
[ 0.42942297] [-0.83338261]
[ 0.4294109] [-0.83303988]
[ 0.42939878] [-0.83269715]
[ 0.42938671] [-0.8323549]
[ 0.42937461] [-0.83201277]
[ 0.42936257] [-0.83167064]
[ 0.42935047] [-0.83132851]
[ 0.42933843] [-0.83098674]
[ 0.42932636] [-0.8306452]
[ 0.42931429] [-0.83030367]
[ 0.42930225] [-0.82996213]
[ 0.42929021] [-0.82962078]
[ 0.42927817] [-0.82927984]
[ 0.42926612] [-0.8289389]
[ 0.42925411] [-0.82859796]
[ 0.42924204] [-0.82825708]
[ 0.42923003] [-0.82791674]
[ 0.42921802] [-0.8275764]
[ 0.42920604] [-0.82723606]
[ 0.42919403] [-0.82689571]
[ 0.42918202] [-0.82655585]
[ 0.42917004] [-0.8262161]
[ 0.42915803] [-0.82587636]
[ 0.42914605] [-0.82553661]
[ 0.42913407] [-0.82519722]
[ 0.42912209] [-0.82485807]
[ 0.42911011] [-0.82451892]
[ 0.42909813] [-0.82417977]
[ 0.42908618] [-0.8238408]
[ 0.42907423] [-0.82350224]
[ 0.42906228] [-0.82316369]
[ 0.42905033] [-0.82282513]
[ 0.42903838] [-0.82248658]
[ 0.42902645] [-0.82214862]
[ 0.42901453] [-0.82181066]
[ 0.42900261] [-0.8214727]
[ 0.42899066] [-0.82113475]
[ 0.42897874] [-0.8207972]
[ 0.42896682] [-0.82045984]
[ 0.4289549] [-0.82012248]
[ 0.42894298] [-0.81978512]
[ 0.42893106] [-0.81944799]
[ 0.42891914] [-0.81911123]
[ 0.42890722] [-0.81877446]
[ 0.42889529] [-0.8184377]
[ 0.4288834] [-0.81810099]
[ 0.42887154] [-0.81776482]
[ 0.42885965] [-0.81742865]
[ 0.42884779] [-0.81709248]
[ 0.42883593] [-0.81675631]
[ 0.4288241] [-0.81642061]
[ 0.42881224] [-0.81608504]
[ 0.4288004] [-0.81574947]
[ 0.42878857] [-0.81541389]
[ 0.42877674] [-0.81507862]
[ 0.42876491] [-0.81474364]
[ 0.42875308] [-0.81440866]
[ 0.42874125] [-0.81407368]
[ 0.42872944] [-0.81373882]
[ 0.42871761] [-0.81340444]
[ 0.42870581] [-0.81307006]
[ 0.42869401] [-0.81273568]
[ 0.42868224] [-0.81240129]
[ 0.42867044] [-0.81206739]
[ 0.42865866] [-0.8117336]
[ 0.42864686] [-0.81139982]
[ 0.42863509] [-0.81106603]
[ 0.42862332] [-0.81073254]
[ 0.42861155] [-0.81039935]
[ 0.4285998] [-0.81006616]
[ 0.42858803] [-0.80973297]
[ 0.42857626] [-0.80939984]
[ 0.42856455] [-0.80906725]
[ 0.42855281] [-0.80873466]
[ 0.42854106] [-0.80840206]
[ 0.42852932] [-0.80806947]
[ 0.42851761] [-0.80773735]
[ 0.4285059] [-0.80740535]
[ 0.42849416] [-0.80707335]
[ 0.42848244] [-0.80674136]
[ 0.42847073] [-0.8064096]
[ 0.42845905] [-0.8060782]
[ 0.42844734] [-0.80574679]
[ 0.42843565] [-0.80541539]
[ 0.42842394] [-0.80508399]
[ 0.42841226] [-0.80475318]
[ 0.42840061] [-0.80442238]
[ 0.42838892] [-0.80409157]
[ 0.42837724] [-0.80376077]
[ 0.42836559] [-0.80343038]
[ 0.42835391] [-0.80310017]
[ 0.42834228] [-0.80276996]
[ 0.4283306] [-0.80243975]
[ 0.42831898] [-0.80210972]
[ 0.42830732] [-0.8017801]
[ 0.4282957] [-0.80145049]
[ 0.42828405] [-0.80112088]
[ 0.42827243] [-0.80079126]
[ 0.4282608] [-0.80046219]
[ 0.42824918] [-0.80013317]
[ 0.42823756] [-0.79980415]
[ 0.42822593] [-0.79947513]
[ 0.42821431] [-0.79914641]
[ 0.42820269] [-0.79881799]
[ 0.42819107] [-0.79848957]
[ 0.42817944] [-0.79816115]
[ 0.42816782] [-0.79783273]
[ 0.42815623] [-0.7975049]
[ 0.42814466] [-0.79717708]
[ 0.4281331] [-0.79684925]
[ 0.42812154] [-0.79652143]
[ 0.42810997] [-0.79619396]
[ 0.42809841] [-0.79586673]
[ 0.42808688] [-0.7955395]
[ 0.42807531] [-0.79521227]
[ 0.42806378] [-0.79488516]
[ 0.42805225] [-0.79455853]
[ 0.42804071] [-0.79423189]
[ 0.42802918] [-0.79390526]
[ 0.42801768] [-0.79357862]
[ 0.42800614] [-0.79325241]
[ 0.42799467] [-0.79292637]
[ 0.42798313] [-0.79260033]
[ 0.42797163] [-0.7922743]
[ 0.42796013] [-0.79194844]
[ 0.42794862] [-0.791623]
[ 0.42793715] [-0.79129755]
[ 0.42792565] [-0.79097211]
[ 0.42791417] [-0.79064667]
[ 0.4279027] [-0.79032171]
[ 0.42789125] [-0.78999686]
[ 0.42787978] [-0.78967202]
[ 0.42786831] [-0.78934717]
[ 0.42785686] [-0.78902256]
[ 0.42784542] [-0.78869832]
[ 0.42783397] [-0.78837407]
[ 0.42782253] [-0.78804982]
[ 0.42781109] [-0.78772557]
[ 0.42779964] [-0.78740185]
[ 0.42778823] [-0.7870782]
[ 0.42777681] [-0.78675455]
[ 0.42776537] [-0.7864309]
[ 0.42775396] [-0.78610754]
[ 0.42774257] [-0.78578448]
[ 0.42773116] [-0.78546143]
[ 0.42771974] [-0.78513837]
[ 0.42770836] [-0.78481531]
[ 0.42769697] [-0.78449285]
[ 0.42768559] [-0.78417039]
[ 0.42767423] [-0.78384793]
[ 0.42766282] [-0.78352547]
[ 0.42765146] [-0.7832033]
[ 0.42764008] [-0.78288144]
[ 0.42762873] [-0.78255957]
[ 0.42761737] [-0.78223771]
[ 0.42760602] [-0.78191584]
[ 0.42759466] [-0.78159457]
[ 0.42758334] [-0.78127331]
[ 0.42757198] [-0.78095204]
所以我的问题是，我怎么知道哪个学习率最适合我的等式和我的预测？我将如何选择正确的学习率？
提前谢谢。
答案 0 :(得分：3)
实际上很有趣，因为我正在写一本关于深度学习的书，而我写的最后一章涉及这个确切的问题。您观察的是三种情况：
1）学习率太大：当更新权重时所采取的步骤（-lambda *成本函数的梯度）太大，因此不是接近成本函数的最小值而是远离它们，因此在某个时刻，数字变得如此之大，以至于Python给你带来了南非。
2）学习率较低，一切似乎都很好。你很好地走向最低
3）学习率更低，只需要花费很长时间才能达到最低限度。正如您从数字中看到的那样，成本函数正在下降，但速度非常慢。
确实没有办法知道正确的学习率是多少。这里有一些提示
1）规范化您的输入以使它们不会太大（例如，您可以将它们除以它们的总和） 2）绘制成本函数与迭代的关系，并尝试不同的学习率。您应该看到成本函数减少并达到稳定水平。然后你知道你正朝着正确的方向前进。
还有更复杂的算法意味着在此过程中会改变学习速度，但我会坚持你在开始时尝试的内容。
但实际上，绘制成本函数与迭代（或时期）的关系为您提供了一个很好的工具来检查学习率是否合适。
希望有所帮助，翁贝托
答案 1 :(得分：0)
所以你应该对学习率有一点了解。因此，当您执行渐变下降时，您希望在渐变的每个步骤中获得局部最小值。因此，学习率可以让您决定在成本最低的最小值上采取多大的步骤。
如果你的学习速度很快，你会朝着最小的方向迈出一大步，但是你可以超越并最终超越最小值，这反过来也无助于找到最小值。
如果学习率很低，你会花很多时间达到最低点，而且也不会有效。
在实际操作中，我从各种学习率中选择，以检查哪一个在计算上表现更好。