时间维度的chunksize不适用于xr.open_mfdataset

时间:2018-07-06 18:46:18

标签: dask python-xarray

我希望使用xr.open_mfdataset对时间维度进行分块;但是,尽管将“时间”的块大小设置为等于12,但它仍等于1。我如何才能在“时间”维度中分块?谢谢-PG

VARS_USED = ["LANDFRAC", "PSL", "PRECC", "PRECL", "TREFHT", "ICEFRAC", "PRECSL_H218OS", "PRECSC_H218Os", "PRECRL_H218OR", "PRECRC_H218Or", "PRECSL_H2OS", "PRECSC_H2Os", "PRECRL_H2OR", "PRECRC_H2Or"]

WSOL_DATA_PATH = '/glade/p/ncgd0030/Steig/90_WSOL/b.ie12.BG1850C5CN.f19_g16.90_WSOL.001_daily.cam.h0.000[2-4]-??_avg_fc.nc'

WSOL_DATA = xr.open_mfdataset(WSOL_DATA_PATH, decode_cf=False, chunks={'time':12, 'lon':72,'lat':48})[VARS_USED]

输出:

<xarray.Dataset>
Dimensions:        (lat: 96, lon: 144, time: 36)
Coordinates:
* lon            (lon) float64 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0   ...
* time           (time) float64 380.0 409.5 439.0 469.5 500.0 530.5 561.0 ...
* lat            (lat) float64 -90.0 -88.11 -86.21 -84.32 -82.42 -80.53 ...
Data variables:
LANDFRAC       (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PSL            (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECC          (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECL          (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
TREFHT         (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
ICEFRAC        (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECSL_H218OS  (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECSC_H218Os  (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECRL_H218OR  (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECRC_H218Or  (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECSL_H2OS    (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECSC_H2Os    (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECRL_H2OR    (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
PRECRC_H2Or    (time, lat, lon) float32 dask.array<shape=(36, 96, 144), chunksize=(1, 48, 72)>
Attributes:
Conventions:               CF-1.0
source:                    CAM

1 个答案:

答案 0 :(得分:0)

目前,chunks中的

open_mfdataset仅在每个输入文件中 个块。如果每个文件仅包含一个时间点,那么设置更大的块大小将没有任何效果。

如果您希望在时间维度上使用更大的块,则需要在之后添加一个单独的.chunk()调用。例如

WSOL_DATA = WSOL_DATA.chunk({'time': 12})