用str.contains忽略NaN

时间:2015-02-04 00:57:17

标签: python pandas

我想找到包含字符串的行,如下所示:

DF[DF.col.str.contains("foo")]

但是,这会失败,因为有些元素是NaN:

  

ValueError:无法使用包含NA / NaN值的向量进行索引

所以我诉诸混淆

DF[DF.col.notnull()][DF.col.dropna().str.contains("foo")]

有更好的方法吗?

6 个答案:

答案 0 :(得分:159)

有一面旗帜:

In [11]: df = pd.DataFrame([["foo1"], ["foo2"], ["bar"], [np.nan]], columns=['a'])

In [12]: df.a.str.contains("foo")
Out[12]:
0     True
1     True
2    False
3      NaN
Name: a, dtype: object

In [13]: df.a.str.contains("foo", na=False)
Out[13]:
0     True
1     True
2    False
3    False
Name: a, dtype: bool

请参阅str.replace文档:

  

na:默认NaN,缺失值的填充值。


所以你可以做到以下几点:

In [21]: df.loc[df.a.str.contains("foo", na=False)]
Out[21]:
      a
0  foo1
1  foo2

答案 1 :(得分:0)

我不是100%的原因(实际上是来这里寻找答案的),但这还是可行的,不需要替换所有的nan值。

import pandas as pd
import numpy as np

df = pd.DataFrame([["foo1"], ["foo2"], ["bar"], [np.nan]], columns=['a'])

newdf = df.loc[df['a'].str.contains('foo') == True]

可以使用或不使用.loc

我不知道这为什么行得通,据我了解,当您使用方括号索引时,pandas会将方括号内的内容评估为TrueFalse。我不知道为什么在方括号“ extra boolean”中添加该短语完全没有效果。

答案 2 :(得分:0)

您还可以设置样式:

DF[DF.col.str.contains(pat = '(foo)', regex = True) ]

答案 3 :(得分:0)

library(sf)
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
testdf <- data.frame(long = c(-124.0048, -123.9844, -123.9691, -123.9604, -123.9810, -123.9612),
                     lat = c(45.04352, 45.10493, 45.20530, 45.29999, 45.34960, 45.40917))

testsf = st_as_sf(testdf, coords = c("long", "lat"))
testsf <- st_set_crs(testsf, "+proj=longlat +datum=WGS84") 
testsf 
#> Simple feature collection with 6 features and 0 fields
#> geometry type:  POINT
#> dimension:      XY
#> bbox:           xmin: -124.0048 ymin: 45.04352 xmax: -123.9604 ymax: 45.40917
#> CRS:            +proj=longlat +datum=WGS84
#>                     geometry
#> 1 POINT (-124.0048 45.04352)
#> 2 POINT (-123.9844 45.10493)
#> 3  POINT (-123.9691 45.2053)
#> 4 POINT (-123.9604 45.29999)
#> 5   POINT (-123.981 45.3496)
#> 6 POINT (-123.9612 45.40917)

utmsf <- st_transform(testsf,"+proj=utm +zone=10 +ellps=GRS80 +datum=NAD83" )
utmsf
#> Simple feature collection with 6 features and 0 fields
#> geometry type:  POINT
#> dimension:      XY
#> bbox:           xmin: 420866.4 ymin: 4988276 xmax: 424783.9 ymax: 5028855
#> CRS:            +proj=utm +zone=10 +ellps=GRS80 +datum=NAD83
#>                   geometry
#> 1 POINT (420866.4 4988276)
#> 2   POINT (422556 4995078)
#> 3 POINT (423893.4 5006214)
#> 4 POINT (424701.9 5016725)
#> 5 POINT (423153.9 5022256)
#> 6 POINT (424783.9 5028855)

答案 4 :(得分:0)

您还可以使用 query 方法通过布尔表达式查询 DataFrame 的列,如下所示:

df.query('a.str.contains("foo", na=False)')

请注意,您可能不会获得性能改进,但它更具可读性(可以说)。

答案 5 :(得分:-2)

import folium
import pandas

data= pandas.read_csv("maps.txt")

lat = list(data["latitude"])
lon = list(data["longitude"])

map= folium.Map(location=[31.5204, 74.3587], zoom_start=6, tiles="Mapbox Bright")

fg = folium.FeatureGroup(name="My Map")

for lt, ln in zip(lat, lon):
c1 = fg.add_child(folium.Marker(location=[lt, ln], popup="Hi i am a Country",icon=folium.Icon(color='green')))

child = fg.add_child(folium.Marker(location=[31.5204, 74.5387], popup="Welcome to Lahore", icon= folium.Icon(color='green')))

map.add_child(fg)

map.save("Lahore.html")


Traceback (most recent call last):
  File "C:\Users\Ryan\AppData\Local\Programs\Python\Python36-32\check2.py", line 14, in <module>
    c1 = fg.add_child(folium.Marker(location=[lt, ln], popup="Hi i am a Country",icon=folium.Icon(color='green')))
  File "C:\Users\Ryan\AppData\Local\Programs\Python\Python36-32\lib\site-packages\folium\map.py", line 647, in __init__
    self.location = _validate_coordinates(location)
  File "C:\Users\Ryan\AppData\Local\Programs\Python\Python36-32\lib\site-packages\folium\utilities.py", line 48, in _validate_coordinates
    'got:\n{!r}'.format(coordinates))
ValueError: Location values cannot contain NaNs, got:
[nan, nan]