Question

我正在尝试建立一个具有连续开始和结束日期的元组列表，其中所有列均具有NaN值。

在以下示例中，我的结果应类似于：

missing_dates = [('2018-10-10 20:00:00', '2018-10-10 22:00:00'),
('2018-10-11 02:00:00', '2018-10-11 03:00:00 ')]

如果存在孤立的NaN，则应在元组中重复该值。

带有用于可视化的表的字典示例。

   dicts = [
        {'datetime': '2018-10-10 18:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-10 19:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-10 19:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-10 19:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-10 20:00:00', 'variable1': np.nan, 'variable2': np.nan},
        {'datetime': '2018-10-10 21:00:00', 'variable1': np.nan, 'variable2': np.nan},
        {'datetime': '2018-10-10 22:00:00', 'variable1': np.nan, 'variable2': np.nan},
        {'datetime': '2018-10-10 23:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-10 23:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-11 00:00:00', 'variable1': 20, 'variable2': 30},
        {'datetime': '2018-10-11 01:00:00', 'variable1': np.nan, 'variable2': 30},
        {'datetime': '2018-10-11 02:00:00', 'variable1': np.nan, 'variable2': np.nan},
        {'datetime': '2018-10-11 03:00:00', 'variable1': np.nan, 'variable2': np.nan}]

表表示形式：

----------------------+-----------+-----------+
|          datetime   | variable1 | variable2 |
+---------------------+-----------+-----------+
| 2018-10-10 18:00:00 |      20.0 |     30.0  |
| 2018-10-10 19:00:00 |      20.0 |     30.0  | 
| 2018-10-10 19:00:00 |      20.0 |     30.0  |
| 2018-10-10 19:00:00 |      20.0 |     30.0  |
| 2018-10-10 20:00:00 |       NaN |     NaN   |
| 2018-10-10 21:00:00 |       NaN |     NaN   |
| 2018-10-10 22:00:00 |       NaN |     NaN   |
| 2018-10-10 23:00:00 |      20.0 |     30.0  |
| 2018-10-10 23:00:00 |      20.0 |     30.0  | 
| 2018-10-11 00:00:00 |      20.0 |     30.0  |
| 2018-10-11 01:00:00 |       NaN |     30.0  |
| 2018-10-11 02:00:00 |       NaN |     NaN   |
| 2018-10-11 03:00:00 |       NaN |     NaN   |
+---------------------+-----------+-----------+

我做了什么：

df = pd.DataFrame(example_dict)
s = dframe.set_index('datetime').isnull().all(axis=1)
df['new_col'] = s.values
dframe.datetime = pd.to_datetime(dframe.datetime)
new_df = dframe.loc[dframe['new_col'] == True]
new_df['delta'] = (new_df['datetime'] - new_df['datetime'].shift(1))

我有一个很好的带有增量的数据框，但我有点迷茫。

Answer 1

使用：

相似的解决方案，仅将遮罩反转：

#create boolean mask for not NaNs rows
mask = df.drop('datetime', axis=1).notnull().any(axis=1)
#create groups for missing rows with same values
df['g'] = mask.cumsum()

#aggregate first and last, convert to nested lists and map to tuples
L = list(map(tuple, df[~mask].groupby('g')['datetime'].agg(['first','last']).values.tolist()))
print (L)
[('2018-10-10 20:00:00', '2018-10-10 22:00:00'), 
 ('2018-10-11 02:00:00', '2018-10-11 03:00:00')]

创建在所有列中具有连续NaN值的元组列表

1 个答案: