我有两个数据框(df_1和df_2)和一些变量(A,B,C):
df_1 = pd.DataFrame({'O' : [1,2,3], 'M' : [2,8,3]})
df_2 = pd.DataFrame({'O' : [1,1,1, 2,2,2, 3,3,3],
'M' : [9,2,4, 6,7,8, 5,3,4],
'X' : [2,4,6, 4,8,7, 3,1,9],
'Y' : [3,6,1, 4,6,5, 1,0,7],
'Z' : [2,4,8, 3,5,4, 7,5,1]})
下面我有一种算法,该算法使用A,B,C为df_2中的每一行计算得分(S)。它在df_2中找到得分最高(S)的行。它将df_2和df_1中得分最高的行进行比较,并计算出p_hat(衡量它们之间的相似性):
M_G = df_1.M
df_1 = df_1.set_index('O')
A = 1
B = 1
C = 1
# Score
df_2['S'] = df_2['X']*A + df_2['Y']*B + df_2['Z']*C
# Top score
df_Sort = df_2.sort_values(['S', 'X', 'M'], ascending=[False, True, True])
df_O = df_Sort.set_index('O')
M_Top = df_O[~df_O.index.duplicated(keep='first')].M
M_Top = M_Top.sort_index()
# Compare the top scoring row for each O to df_1
df_1_M = df_1.M
df_1_M = df_1_M.sort_index()
df_1_R = df_1_M.reindex(M_Top.index)
T_N_T = M_Top == df_1_R
# Record the results for the given values of A,B,C
df_Res = pd.DataFrame({'it_is':T_N_T}) # is this row of df_1 the same as this row of M_Top?
# p_hat = TP / (TP + FP)
p_hat = df_Res.sum() / len(df_Res.index)
对于示例中的A,B,C值,其p_hat = 0.333。我想找到给出p_hat的最大可能值的A,B,C值。我想使用优化算法来确保获得最大值。
该图适用于C = 2:
答案 0 :(得分:0)
我想您可以使用Optuna软件包,它非常易于使用。您定义一个目标函数,在该目标函数中以所需的方式计算要优化的变量,并为其创建一个study
对象,然后Optuna基本上完成其余的工作。
简单的2D功能的小示例:
import optuna
def optimize_me(trial):
x = trial.suggest_uniform('x', -10, 10)
y = trial.suggest_uniform('y', -10, 10)
return ((y - 1) ** 2) + ((x + 2) ** 2)
study = optuna.create_study()
study.optimize(optimize_me, n_trials = 10)
您可以了解有关Optuna here
的更多信息我主要用它来优化RNN的超级参数,这是一个非常强大的软件包。
答案 1 :(得分:0)
我找到了一种使用全局蛮力优化的方法:
from scipy.optimize import brute
df_1 = pd.DataFrame({'O' : [1,2,3], 'M' : [2,8,3]})
df_2 = pd.DataFrame({'O' : [1,1,1, 2,2,2, 3,3,3],
'M' : [9,2,4, 6,7,8, 5,3,4],
'X' : [2,4,6, 4,8,7, 3,1,9],
'Y' : [3,6,1, 4,6,5, 1,0,7],
'Z' : [2,4,8, 3,5,4, 7,5,1]})
# Range
min_ = -2
max_ = 2
step = .5
ran_ge = slice(min_, max_+step, step)
ranges = (ran_ge,ran_ge,ran_ge)
# Params
params = (df_1, df_2)
# Index
df_1 = df_1.set_index('O')
df_1_M = df_1.M
df_1_M = df_1_M.sort_index()
# Fun
def fun(z, *params):
A,B,C = z
# Score
df_2['S'] = df_2['X']*A + df_2['Y']*B + df_2['Z']*C
# Top score
df_Sort = df_2.sort_values(['S', 'X', 'M'], ascending=[False, True, True])
df_O = df_Sort.set_index('O')
M_Top = df_O[~df_O.index.duplicated(keep='first')].M
M_Top = M_Top.sort_index()
# Compare the top scoring row for each O to df_1
df_1_R = df_1_M.reindex(M_Top.index) # Nan
T_N_T = M_Top == df_1_R
# Record the results for the given values of A,B,C
df_Res = pd.DataFrame({'it_is':T_N_T}) # is this row of df_1 the same as this row of M_Top?
# p_hat = TP / (TP + FP)
p_hat = df_Res.sum() / len(df_Res.index)
return -p_hat
# Brute
resbrute = brute(fun,
ranges,
args=params,
full_output=True,
finish=None)
print('Global maximum ', resbrute[0])
print('Function value at global maximum ',-resbrute[1])
它给出:
Global maximum [-2. 0.5 1.5]
Function value at global maximum 0.6666666666666666
答案 2 :(得分:0)
或使用全局,渐进式优化:(代码由@Aviv Yaniv固定)
from scipy.optimize import differential_evolution
df_1 = pd.DataFrame({'O' : [1,2,3], 'M' : [2,8,3]})
df_2 = pd.DataFrame({'O' : [1,1,1, 2,2,2, 3,3,3],
'M' : [9,2,4, 6,7,8, 5,3,4],
'X' : [2,4,6, 4,8,7, 3,1,9],
'Y' : [3,6,1, 4,6,5, 1,0,7],
'Z' : [2,4,8, 3,5,4, 7,5,1]})
# Index
df_1 = df_1.set_index('O')
df_1_M = df_1.M
df_1_M = df_1_M.sort_index()
# Fun
def fun(z, *params):
A,B,C = z
# Score
df_2['S'] = df_2['X']*A + df_2['Y']*B + df_2['Z']*C
# Top score
df_Sort = df_2.sort_values(['S', 'X', 'M'], ascending=[False, True, True])
df_O = df_Sort.set_index('O')
M_Top = df_O[~df_O.index.duplicated(keep='first')].M
M_Top = M_Top.sort_index()
# Compare the top scoring row for each O to df_1
df_1_R = df_1_M.reindex(M_Top.index) # Nan
T_N_T = M_Top == df_1_R
# Record the results for the given values of A,B,C
df_Res = pd.DataFrame({'it_is':T_N_T}) # is this row of df_1 the same as this row of M_Top?
# p_hat = TP / (TP + FP)
p_hat = df_Res.sum() / len(df_Res.index)
print(z)
return -p_hat[0]
# Bounds
min_ = -2
max_ = 2
ran_ge = (min_, max_)
bounds = [ran_ge,ran_ge,ran_ge]
# Params
params = (df_1, df_2)
# DE
DE = differential_evolution(fun, bounds, args=params)
print('Function value at global maximum ',-DE.fun)