我有2个DataFrames:
city count school
0 New York 1 school_3
1 Washington 1 School_4
2 Washington 1 School_5
3 LA 1 School_1
4 LA 1 School_4
city count school
0 New York 1 School_3
1 Washington 1 School_1
2 LA 1 School_3
3 LA 2 School_4
我想得到这个结果:
city count school
0 New York 2 school_3
1 Washington 1 School_1
2 Washington 1 School_4
3 Washington 1 School_5
4 LA 1 School_1
5 LA 1 School_3
6 LA 3 School_4
以下是代码。
d1 = [{'city':'New York', 'school':'school_3', 'count':1},
{'city':'Washington', 'school':'School_4', 'count':1},
{'city':'Washington', 'school':'School_5', 'count':1},
{'city':'LA', 'school':'School_1', 'count':1},
{'city':'LA', 'school':'School_4', 'count':1}]
d2 = [{'city':'New York', 'school':'School_3', 'count':1},
{'city':'Washington', 'school':'School_1', 'count':1},
{'city':'LA', 'school':'School_3', 'count':1},
{'city':'LA', 'school':'School_4', 'count':2}]
x1 = pd.DataFrame(d1)
x2 = pd.DataFrame(d2)
#just get empty DataFrame
print pd.merge(x1, x2)
如何获得汇总结果?
答案 0 :(得分:5)
你可以这样做:
>>> pd.concat([x1, x2]).groupby(["city", "school"], as_index=False)["count"].sum()
city school count
0 LA School_1 1
1 LA School_3 1
2 LA School_4 3
3 New York School_3 1
4 New York school_3 1
5 Washington School_1 1
6 Washington School_4 1
7 Washington School_5 1
请注意,由于数据中的拼写错误(school_3
vs School_3
),纽约出现了2次。
答案 1 :(得分:1)
这是使用pandas.DataFrame.merge(...)
x1.merge(x2,on=['city', 'school', 'count'], how='outer').groupby(['city', 'school'], as_index=False)['count'].sum()
在ipython notebook %timeit
计时时,此方法比@ elyase(<1ms)
100 loops, best of 3: 6.25 ms per loop #using concat(...) with @elyase's solution
100 loops, best of 3: 5.49 ms per loop #using merge(...) in this solution
此外,如果您想使用pandas aggregate
功能,您也可以这样做:
x1.merge(x2,on=['city', 'school', 'count'], how='outer').groupby(['city', 'school'], as_index=False).agg(numpy.sum)
唯一的免责声明是使用agg(...)
是3种解决方案中最慢的。
显然,所有3都提供了正确的结果:
city school count
0 LA School_1 1
1 LA School_3 1
2 LA School_4 3
3 New York School_3 1
4 New York school_3 1
5 Washington School_1 1
6 Washington School_4 1
7 Washington School_5 1
答案 2 :(得分:0)
将熊猫作为pd导入 将numpy导入为np
pd.set_option('display.precision',16)
pd.set_option('display.max_rows',500)
pd.set_option('display.max_columns',2500)
pd.set_option('display.width',2500)
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.set_option.html
pd.describe_option()
display.chop_threshold: [default: None] [currently: None]
: float or None
if set to a float value, all float values smaller then the given threshold
will be displayed as exactly 0 by repr and friends.
display.colheader_justify: [default: right] [currently: right]
: 'left'/'right'
Controls the justification of column headers. used by DataFrameFormatter.
display.column_space: [default: 12] [currently: 12]No description available.
display.date_dayfirst: [default: False] [currently: False]
: boolean
When True, prints and parses dates with the day first, eg 20/01/2005
display.date_yearfirst: [default: False] [currently: False]
: boolean
When True, prints and parses dates with the year first, eg 2005/01/20
display.encoding: [default: UTF-8] [currently: UTF-8]
: str/unicode
Defaults to the detected encoding of the console.
Specifies the encoding to be used for strings returned by to_string,
these are generally strings meant to be displayed on the console.
display.expand_frame_repr: [default: True] [currently: True]
: boolean
Whether to print out the full DataFrame repr for wide DataFrames
across multiple lines, `max_columns` is still respected, but the output will
wrap-around across multiple "pages" if it's width exceeds `display.width`.
display.float_format: [default: None] [currently: <built-in method format of str object at 0x0000000008899AA8>]
: callable
The callable should accept a floating point number and return
a string with the desired format of the number. This is used
in some places like SeriesFormatter.
See core.format.EngFormatter for an example.
display.height: [default: 60] [currently: 60]
: int
Deprecated.
(Deprecated, use `display.height` instead.)
display.line_width: [default: 80] [currently: 80]
: int
Deprecated.
(Deprecated, use `display.width` instead.)
display.max_columns: [default: 20] [currently: 20]
: int
max_rows and max_columns are used in __repr__() methods to decide if
to_string() or info() is used to render an object to a string. In case
python/IPython is running in a terminal this can be set to 0 and pandas
will correctly auto-detect the width the terminal and swap to a smaller
format in case all columns would not fit vertically. The IPython notebook,
IPython qtconsole, or IDLE do not run in a terminal and hence it is not
possible to do correct auto-detection.
'None' value means unlimited.
display.max_colwidth: [default: 50] [currently: 50]
: int
The maximum width in characters of a column in the repr of
a pandas data structure. When the column overflows, a "..."
placeholder is embedded in the output.
display.max_info_columns: [default: 100] [currently: 100]
: int
max_info_columns is used in DataFrame.info method to decide if
per column information will be printed.
display.max_info_rows: [default: 1690785] [currently: 1690785]
: int or None
max_info_rows is the maximum number of rows for which a frame will
perform a null check on its columns when repr'ing To a console.
The default is 1,000,000 rows. So, if a DataFrame has more
1,000,000 rows there will be no null check performed on the
columns and thus the representation will take much less time to
display in an interactive session. A value of None means always
perform a null check when repr'ing.
display.max_rows: [default: 60] [currently: 60]
: int
This sets the maximum number of rows pandas should output when printing
out various output. For example, this value determines whether the repr()
for a dataframe prints out fully or just a summary repr.
'None' value means unlimited.
display.max_seq_items: [default: None] [currently: None]
: int or None
when pretty-printing a long sequence, no more then `max_seq_items`
will be printed. If items are ommitted, they will be denoted by the addition
of "..." to the resulting string.
If set to None, the number of items to be printed is unlimited.
display.mpl_style: [default: None] [currently: None]
: bool
Setting this to 'default' will modify the rcParams used by matplotlib
to give plots a more pleasing visual style by default.
Setting this to None/False restores the values to their initial value.
display.multi_sparse: [default: True] [currently: True]
: boolean
"sparsify" MultiIndex display (don't display repeated
elements in outer levels within groups)
display.notebook_repr_html: [default: True] [currently: False]
: boolean
When True, IPython notebook will use html representation for
pandas objects (if it is available).
display.pprint_nest_depth: [default: 3] [currently: 3]
: int
Controls the number of nested levels to process when pretty-printing
display.precision: [default: 7] [currently: 7]
: int
Floating point output precision (number of significant digits). This is
only a suggestion
display.width: [default: 80] [currently: 80]
: int
Width of the display in characters. In case python/IPython is running in
a terminal this can be set to None and pandas will correctly auto-detect the
width.
Note that the IPython notebook, IPython qtconsole, or IDLE do not run in a
terminal and hence it is not possible to correctly detect the width.
mode.sim_interactive: [default: False] [currently: False]
: boolean
Whether to simulate interactive mode for purposes of testing
mode.use_inf_as_null: [default: False] [currently: False]
: boolean
True means treat None, NaN, INF, -INF as null (old way),
False means None and NaN are null, but INF, -INF are not null
(new way).