如何根据另一列中的间隔填充熊猫中其他列的缺失值?

时间:2020-02-29 19:32:24

标签: python-3.x pandas

假设我有这个df_atm

     borough          Longitude     Latitude

0    bronx              40.79        -73.78    
1    manhattan          40.78        -73.90
2    staten island      40.84        -73.95
3    NaN                40.57        -74.11

每一行都代表ATM取款。

我希望根据“经度”和“纬度”列中的坐标为缺少的值生成值。

     borough          Longitude     Latitude

0    bronx              40.79        -73.78    
1    manhattan          40.78        -73.90
2    staten island      40.84        -73.95
3    staten island      40.57        -74.11

因为坐标[40.57,-74.11]位于史泰登岛的自治市镇内。

我生成了一个带有自治市镇坐标的字典:

borough_dict = {"Bronx" : [40.837048, -73.865433], "Brooklyn" : [40.650002, -73.949997], "Manhattan" : [40.758896, -73.985130], "Queens" : [40.742054,-73.769417], "Staten Island" : [40.579021,-74.151535]}

这是我到目前为止尝试的(代码/伪代码):

df_atm['borough'] = df_atm.apply(
lambda row: **idk what do to here** if np.isnan(row['borough']) else row['borough'],
axis=1
)

非常感谢!

2 个答案:

答案 0 :(得分:3)

尝试一下:

from math import cos, asin, sqrt
import pandas as pd

def distance(lat1, lon1, lat2, lon2):
    p = 0.017453292519943295
    a = 0.5 - cos((lat2-lat1)*p)/2 + cos(lat1*p)*cos(lat2*p) * (1-cos((lon2-lon1)*p)) / 2
    return 12742 * asin(sqrt(a))

def closest(data, v):
    return min(data, key=lambda p: distance(v[0], v[1], p[0], p[1]))


df = pd.DataFrame(
    [
     {'borough': 'bronx', 'lat': 40.79, 'long': -73.78}, 
     {'borough': 'manhattan', 'lat': 40.78, 'long': -73.90},
     {'borough': None, 'lat': 40.57, 'long': -74.11}
     ],
)


borough_dict = {"Bronx" : [40.837048, -73.865433], "Brooklyn" : [40.650002, -73.949997], "Manhattan" : [40.758896, -73.985130], "Queens" : [40.742054,-73.769417], "Staten Island" : [40.579021,-74.151535]}
boroughs = [(*value, key) for key, value in borough_dict.items()]


df['borough'] = df.apply(
lambda row: closest(boroughs, [row['lat'], row['long']])[2] if row['borough'] is None else row['borough'],
axis=1
)

print(df)

输出:

         borough    lat   long
0          bronx  40.79 -73.78
1      manhattan  40.78 -73.90
2  Staten Island  40.57 -74.11

贷记到@trincot answer

答案 1 :(得分:1)

您要加入spatial,因此请使用密切相关的GeoPandas库。我们会将原始DataFrame转换为GeoDataFrame,以便我们可以合并。另请注意,在您的示例中,您的LatitudeLongitude列未正确标记。我在这里修好了。

import pandas as pd
import geopandas as gpd

dfg = gpd.GeoDataFrame(df.copy(), geometry=gpd.points_from_xy(df.Longitude, df.Latitude))
#         borough   Latitude Longitude                    geometry
#0          bronx      40.79    -73.78  POINT (-73.78000 40.79000)
#1      manhattan      40.78    -73.90  POINT (-73.90000 40.78000)
#2  staten island      40.84    -73.95  POINT (-73.95000 40.84000)
#3            NaN      40.57    -74.11  POINT (-74.11000 40.57000)

# Shapefile from https://geo.nyu.edu/catalog/nyu-2451-34154 
# I downloaded the geojson
df_nys = gpd.read_file('nyu-2451-34154-geojson.json')

dfg.crs = df_nys.crs  # Set coordinate reference system to be the same

dfg = gpd.sjoin(dfg, df_nys[['geometry', 'boroname']], how='left', op='within')

         borough   Latitude Longitude                    geometry  index_right       boroname
0          bronx      40.79    -73.78  POINT (-73.78000 40.79000)          4.0         Queens
1      manhattan      40.78    -73.90  POINT (-73.90000 40.78000)          4.0         Queens
2  staten island      40.84    -73.95  POINT (-73.95000 40.84000)          NaN            NaN
3            NaN      40.57    -74.11  POINT (-74.11000 40.57000)          2.0  Staten Island

因此,现在您可以用'borough'填充缺失的'boroname'。但这似乎还有些其他方面没有被归类。这主要是因为您存储的纬度和经度没有足够的精度。尽管这可能是较精确的纬度/经度解决方案,但考虑到您数据中的精确度,我可能更喜欢使用@adnanmuttaleb进行距离计算。