Question

如何使用Julia Dataframes进行分组和透视表？

假设我有Dataframe

using DataFrames

df =DataFrame(Location = [ "NY", "SF", "NY", "NY", "SF", "SF", "TX", "TX", "TX", "DC"],
                 Class = ["H","L","H","L","L","H", "H","L","L","M"],
                 Address = ["12 Silver","10 Fak","12 Silver","1 North","10 Fak","2 Fake", "1 Red","1 Dog","2 Fake","1 White"],
                 Score = ["4","5","3","2","1","5","4","3","2","1"])

我想做以下事情：

1）包含Location和Class的数据透视表，应输出

Class     H  L  M
Location         
DC        0  0  1
NY        2  1  0
SF        1  2  0
TX        1  2  0

2）按“位置”分组并计算该组中应输出的记录数

   Pop  
DC  1   
NY  3  
SF  3  
TX  3

Answer 1

你可以使用unstack来完成大部分工作（DataFrames没有索引，因此Class必须保留一列，而不是在pandas中它将是一个Index），这似乎成为DataFrames.jl对pivot_table：

的回答

julia> unstack(df, :Location, :Class, :Score)
WARNING: Duplicate entries in unstack.
4x4 DataFrames.DataFrame
| Row | Class | H   | L   | M   |
|-----|-------|-----|-----|-----|
| 1   | "DC"  | NA  | NA  | "1" |
| 2   | "NY"  | "3" | "2" | NA  |
| 3   | "SF"  | "5" | "1" | NA  |
| 4   | "TX"  | "4" | "2" | NA  |

我不确定你fillna在这里的情况（unstack没有这个选项）......

您可以使用by nrows（行数）方法执行groupby：

julia> by(df, :Location, nrow)
4x2 DataFrames.DataFrame
| Row | Location | x1 |
|-----|----------|----|
| 1   | "DC"     | 1  |
| 2   | "NY"     | 3  |
| 3   | "SF"     | 3  |
| 4   | "TX"     | 3  |

Answer 2

（1）这是我尝试创建数据透视表。我使用by（）按一列分组，然后计算函数中第二列因子的频率。

# Create pivot table from DataFrame.
#  - df : DataFrame object
#  - column1 : Column symbol used for row labels.
#  - column2 : Column symbol used for column labels.
function pivot_table(df, column1, column2)
    # For given DataArray and factor list, create single row DataFrame:
    #  ----------------------------------------
    #  |    factor1    |    factor2    | ...
    #  ----------------------------------------
    #  |freq of factor1|freq of factor1| ...
    #  ----------------------------------------
    function frequency(data, factors)
        # Convert factors to symbols.
        factor_symbols::Vector{Symbol} = map(factor -> symbol(factor), factors)

        # Convert frequency to fit the DataFrame constructor parameter type.
        frequencies::Vector{Any} = map(frequency->[frequency], map(factor -> sum(data .== factor), factors))

        DataFrame(frequencies, factor_symbols)
    end

    factors = sort(unique(df[column2]))
    by(df, column1, x -> frequency(x[column2], factors))
end

示例：

julia> pivot_table(df, :Location, :Class)
4x4 DataFrames.DataFrame
| Row | Location | H | L | M |
|-----|----------|---|---|---|
| 1   | "DC"     | 0 | 0 | 1 |
| 2   | "NY"     | 2 | 1 | 0 |
| 3   | "SF"     | 1 | 2 | 0 |
| 4   | "TX"     | 1 | 2 | 0 |

（2）你可以使用by和nrow。

julia> by(df, :Location, nrow)
4x2 DataFrames.DataFrame
| Row | Location | x1 |
|-----|----------|----|
| 1   | "DC"     | 1  |
| 2   | "NY"     | 3  |
| 3   | "SF"     | 3  |
| 4   | "TX"     | 3  |

Answer 3

包FreqTable.jl解决了这个问题：

>using FreqTables
>show(freqtable(df,:Location,:Class))

4×3 Named Array{Int64,2}
Location ╲ Class │ H  L  M
─────────────────┼────────
DC               │ 0  0  1
NY               │ 2  1  0
SF               │ 1  2  0
TX               │ 1  2  0

Answer 4

对于问题的第2部分，您可以使用匿名函数并返回DataFrame，以便命名新列，例如count：

julia> by(df, :Location, d -> DataFrame(count=nrow(d)))
4x2 DataFrames.DataFrame
| Row | Location | count |
|-----|----------|-------|
| 1   | "DC"     | 1     |
| 2   | "NY"     | 3     |
| 3   | "SF"     | 3     |
| 4   | "TX"     | 3     |

Answer 5

使用为this SO question开发的pivot (df, rowFields, colField, valuesField; <keyword arguments>)函数，您可以执行以下操作：

julia> df =DataFrame(Location = [ "NY", "SF", "NY", "NY", "SF", "SF", "TX", "TX", "TX", "DC"],
                        Class = ["H","L","H","L","L","H", "H","L","L","M"],
                        Address = ["12 Silver","10 Fak","12 Silver","1 North","10 Fak","2 Fake", "1 Red","1 Dog","2 Fake","1 White"],
                        Score = ["4","5","3","2","1","5","4","3","2","1"])

第一个问题：

julia> df_piv = pivot(df,[:Location],:Class,:Score,ops=length)
julia> [df_piv[isna(df_piv[i]), i] = 0 for i in names(df_piv)] # remove NA values across whole df
julia> df_piv
4×4 DataFrames.DataFrame
│ Row │ Location │ H │ L │ M │
├─────┼──────────┼───┼───┼───┤
│ 1   │ "DC"     │ 0 │ 0 │ 1 │
│ 2   │ "NY"     │ 2 │ 1 │ 0 │
│ 3   │ "SF"     │ 1 │ 2 │ 0 │
│ 4   │ "TX"     │ 1 │ 2 │ 0 │

第二个问题：

julia> df[:pop]="Pop" # add a dummy column with constant values
julia> pivot(df,[:Location],:pop,:Score,ops=length)
4×2 DataFrames.DataFrame
│ Row │ Location │ Pop │
├─────┼──────────┼─────┤
│ 1   │ "DC"     │ 1   │
│ 2   │ "NY"     │ 3   │
│ 3   │ "SF"     │ 3   │
│ 4   │ "TX"     │ 3   │

Julia Dataframe group by和pivot tables函数

5 个答案: