Pandas Categories comparison

时间:2018-03-25 19:10:33

标签: python pandas dataframe comparison categories

I am creating a sample dataframe here.

df = pd.DataFrame( np.random.randn(10,2), columns=list('AB'))    

          A         B
0  0.459759         0.152645

1  0.183613         0.756527

2 -1.836027         0.032433

3  0.264336         0.170171

4 -0.276347         0.208389

5  0.677709         0.725274

6 -0.547858         0.376683

7 -0.994759        -0.750373

8  0.556593         1.282167

9 -1.444533         0.589768

df['A_rank']= pd.qcut(df['A'],[0,0.25,0.5,0.75,1],duplicates="drop")

          A         B                A_rank
0  0.459759        0.152645         (0.411, 0.678]

1  0.183613        0.756527         (-0.0464, 0.411]

2 -1.836027        0.032433         (-1.837, -0.883]

3  0.264336        0.170171         (-0.0464, 0.411]

4 -0.276347        0.208389         (-0.883, -0.0464]

5  0.677709        0.725274         (0.411, 0.678]

6 -0.547858        0.376683        (-0.883, -0.0464]

7 -0.994759        -0.750373       (-1.837, -0.883]

8  0.556593        1.282167        (0.411, 0.678]

9 -1.444533         0.589768       (-1.837, -0.883]

I want to create a new dataframe from above like the following which is basically a subset based on an A_rank.

          A         B             A_rank  A_rank_open_low    A_rank_closed_hi
0  0.459759        0.152645         (0.411, 0.678]           0.411               0.678

5  0.677709        0.725274          (0.411, 0.678]          0.411               0.678

8  0.556593         1.282167         (0.411, 0.678]            0.411               0.678

I don't want to use the integer label using qcut but wish to use the labels of the qcut output itself directly but I am unable to perform a comparison with a Category (A_rank) which is a range. The below attempts failed because of my lack of understanding of this datatype.

df2 = df[df['A_rank']=="(0.411, 0.678]"]

No error but the output was:

Empty DataFrame    
Columns: [A, B, A_rank]    
Index: []

df2 = df[df['A_rank']== pd.Categorical("(0.411, 0.678]")]

TypeError: Categoricals can only be compared if 'categories' are the same. Categories are different lengths

df2 = df[str(df['A_rank'])=="(0.411, 0.678]"] 

gave a Traceback too

I searched the documentation for categorical index and categories but couldn't find much. Please help me out.

2 个答案:

答案 0 :(得分:2)

Use Interval:

df2 = df[df['A_rank']== pd.Interval(0.411, 0.678)]

答案 1 :(得分:1)

@jezrael, has shown you how to do this properly. I'd like to show you how to "investigate" similar cases.

First check the column dtype:

In [8]: df.dtypes
Out[8]:
A          float64
B          float64
A_rank    category
dtype: object

then the cell type:

In [9]: df.iat[0, 2]
Out[9]: Interval(0.0809, 0.539, closed='right')

In [10]: type(df.iat[0, 2])
Out[10]: pandas._libs.interval.Interval