Question

我正在计算数据帧内不同子段的Z得分和P值。

数据框有两列，这是我数据框中的前5个值：

df[["Engagement_score", "Performance"]].head()
   Engagement_score  Performance
0    6                 0.0
1    5                 0.0
2    7                 66.3
3    3                 0.0
4    11                0.0

以下是参与度得分的分布：

以下是效果的分布：

我按照参与度得分对数据框进行分组，然后为这些组计算这三个统计数据：

1）平均绩效得分（sub_average）和该组中的值数（sub_bookings）

2）其余组的平均表现得分（rest_average）和其余组的值数量（rest_bookings）

针对总体数据框架计算总体绩效得分和总体预订。

这是我的代码。

def stats_comparison(i):
    df.groupby(i)['Performance'].agg({
    'average': 'mean',
    'bookings': 'count'
    }).reset_index()
    cat = df.groupby(i)['Performance']\
        .agg({
            'sub_average': 'mean',
            'sub_bookings': 'count'
       }).reset_index()
    cat['overall_average'] = df['Performance'].mean()
    cat['overall_bookings'] = df['Performance'].count()
    cat['rest_bookings'] = cat['overall_bookings'] - cat['sub_bookings']
    cat['rest_average'] = (cat['overall_bookings']*cat['overall_average'] \
                     - cat['sub_bookings']*cat['sub_average'])/cat['rest_bookings']
    cat['z_score'] = (cat['sub_average']-cat['rest_average'])/\
        np.sqrt(cat['overall_average']*(1-cat['overall_average'])
            *(1/cat['sub_bookings']+1/cat['rest_bookings'])) 
    cat['prob'] = np.around(stats.norm.cdf(cat.z_score), decimals = 10) # this is the p value
    cat['significant'] = [(lambda x: 1 if x > 0.9 else -1 if x < 0.1 else 0)(i) for i in cat['prob']] 
    # if the p value is less than 0.1 then I can confidently say that the 2 samples are different. 
    print(cat)

stats_comparison('Engagement_score')

执行代码时，我得到以下输出：

     Engagement_score  sub_average  sub_bookings  overall_average  \
0                3       57.281118          1234        34.405373   
1                 4    56.165374           722        34.405373   
2                 5    52.896404           890        34.405373   
3                 6    50.275880           966        34.405373   
4                 7    43.475344          1018        34.405373   
5                 8    37.693290          1222        34.405373   
6                 9    30.418053          1695        34.405373   
7                10    16.458142          2874        34.405373   
8                11    25.604145          1375        34.405373   
9                12    10.910013           789        34.405373   

   overall_bookings  rest_bookings  rest_average  z_score  prob  significant  
0             12785          11551     31.961544      NaN   NaN            0  
1             12785          12063     33.102984      NaN   NaN            0  
2             12785          11895     33.021850      NaN   NaN            0  
3             12785          11819     33.108233      NaN   NaN            0  
4             12785          11767     33.620702      NaN   NaN            0  
5             12785          11563     34.057900      NaN   NaN            0  
6             12785          11090     35.014797      NaN   NaN            0  
7             12785           9911     39.609727      NaN   NaN            0  
8             12785          11410     35.465995      NaN   NaN            0  
9             12785          11996     35.950709      NaN   NaN            0

我不知道为什么要在ZScore和P值列中获得NAN值列表。我的数据集中没有负值。

在Jupyter Notebook中运行代码时，我还会收到以下警告：

C:\Users\User\Anaconda3\lib\site-packages\ipykernel_launcher.py:4: FutureWarning: using a dict on a Series for aggregation
is deprecated and will be removed in a future version
  after removing the cwd from sys.path.
C:\Users\User\Anaconda3\lib\site-packages\ipykernel_launcher.py:8: FutureWarning: using a dict on a Series for aggregation
is deprecated and will be removed in a future version


    C:\Users\User\Anaconda3\lib\site-packages\ipykernel_launcher.py:15: RuntimeWarning: invalid value encountered in sqrt
      from ipykernel import kernelapp as app
    C:\Users\User\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in greater
      return (self.a < x) & (x < self.b)
    C:\Users\User\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in less
      return (self.a < x) & (x < self.b)
    C:\Users\User\Anaconda3\lib\site-packages\scipy\stats\_distn_infrastructure.py:1738: RuntimeWarning: invalid value encountered in greater_equal
      cond2 = (x >= self.b) & cond0

在Scipy中计算p值和Z分数时的NAN值列表

0 个答案: