python-xarray:open_dataarray HPC上的分段错误

时间:2017-12-03 04:16:58

标签: netcdf python-xarray

我很难理解为什么在我使用的HPC中使用NSStoryboard.SceneIdentifier读取NetCDF文件会产生var color = [ {pos: 0, color: "#4a55e5"}, {pos: 0.2, color: "#206bcb"}, {pos: 0.4, color: "#edffff"}, {pos: 0.6, color: "#ffaa00"}, {pos: 0.8, color: "#000200"}, {pos: 1, color: "#000764"} ]; var colorStops = {}; for(var i=0; i<color.length; i++){ let key = color[i].pos; let value = color[i].color; colorStops[key] = value; } console.log(colorStops);。但是,当我在Mac上使用open_dataarray读取文件时,它可以正常工作。

从进一步研究中看,我创建的NPac文件(子部分)似乎有问题。以下是我生成文件的步骤:

  • WaveWatchIII输出文件
  • 7月提取:Segmentation fault (core dumped)可以在HPC中打开out.nc
  • 提取变量:$ncks -d time,0,30 in.nc out.nc可以在HPC中打开out.nc
  • 提取域:$ncks -v hs in.nc out.nc无法在HPC上打开out.nc,但可以在Mac上打开out.nc。

这是我第一次看到这个问题,我相信这是由于NetCDF文件域的属性。我猜这可能与版本有关吗?我在HPC上完成了大部分工作,并使用我的Mac进行测试和理解,因此最好将其用于HPC。

可以下载NetCDF文件open_dataarray

$ncks -d longitude,100.0,290.0 -d latitude,0.0,65.0 in.nc out.nc

在HPC上:

ncdump -h ww3.Hs.July.NPac.nc
netcdf ww3.Hs.July.NPac {
dimensions:
    time = UNLIMITED ; // (31 currently)
    latitude = 66 ;
    longitude = 191 ;
variables:
    short hs(time, latitude, longitude) ;
            hs:long_name = "significant height of wind and swell waves" ;
            hs:standard_name = "sea_surface_wave_significant_height" ;
            hs:globwave_name = "significant_wave_height" ;
            hs:units = "m" ;
            hs:_FillValue = -32767s ;
            hs:scale_factor = 0.002f ;
            hs:add_offset = 0.f ;
            hs:valid_min = 0 ;
            hs:valid_max = 32000 ;
    float latitude(latitude) ;
            latitude:units = "degree_north" ;
            latitude:long_name = "latitude" ;
            latitude:standard_name = "latitude" ;
            latitude:valid_min = -90.f ;
            latitude:valid_max = 90.f ;
            latitude:axis = "Y" ;
    float longitude(longitude) ;
            longitude:units = "degree_east" ;
            longitude:long_name = "longitude" ;
            longitude:standard_name = "longitude" ;
            longitude:valid_min = -180.f ;
            longitude:valid_max = 180.f ;
            longitude:axis = "X" ;
    double time(time) ;
            time:long_name = "julian day (UT)" ;
            time:standard_name = "time" ;
            time:units = "days since 1850-01-01T00:00:00Z" ;
            time:conventions = "relative julian days with decimal part (as parts of the day )" ;
            time:axis = "T" ;

// global attributes:
            :WAVEWATCH_III_version_number = "4.18b" ;
            :WAVEWATCH_III_switches = "NC4 F90 NOGRB NOPA LRB4 SHRD PR3 UQ FLX0 LN1 ST4 STAB0 NL1 BT1 DB1 MLIM TR0 BS0 IC0 REF0 XX0 WNT1 WNX1 CRT1 CRX1 O0 O1 O2 O3 O4 O5 O6 O7 O11 O14 TRKNC" ;
            :SDS4\ namelist\ parameter\ WHITECAPWIDTH = 0.3f ;
            :product_name = "ww3.199307.nc" ;
            :area = "Indian Ocean Pacfic 1 degree" ;
            :latitude_resolution = "   1.0000000" ;
            :longitude_resolution = "   1.0000000" ;
            :southernmost_latitude = "-70.0000000" ;
            :northernmost_latitude = "65.0000000" ;
            :westernmost_longitude = "20.0000000" ;
            :easternmost_longitude = "295.0000000" ;
            :minimum_altitude = "-12000 m" ;
            :maximum_altitude = "9000 m" ;
            :altitude_resolution = "n/a" ;
            :start_date = "1993-07-01T00:00:00Z" ;
            :stop_date = "1993-07-31T00:00:00Z" ;
            :history = "Sat Dec  2 17:53:06 2017: ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.NPac.nc\nSat Dec  2 17:53:06 2017: ncks -O -v hs /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.July.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.Hs.July.nc\nSat Dec  2 17:53:05 2017: ncks -O -d time,0,30 /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.19930701_19940630.nc /projects/rsmas/kirtman/rxb826/WW3exps/IO_Pac_CCSM4/CCSM4_19930701_19940630_1a1/work/ww3.July.nc\nFri Nov  4 13:50:57 2016: ncrcat -O -o tmp.nc ww3.199307.nc ww3.199308.nc ww3.199309.nc ww3.199310.nc ww3.199311.nc ww3.199312.nc ww3.199401.nc ww3.199402.nc ww3.199403.nc ww3.199404.nc ww3.199405.nc ww3.199406.nc" ;
            :nco_openmp_thread_number = 1 ;
            :NCO = "4.3.7" ;

在我的Mac上:

$ nc-config --version
netCDF 4.2.1.1

>>> xr.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 2.6.32-431.el6.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

xarray: 0.10.0
pandas: 0.20.2
numpy: 1.13.1
scipy: 0.19.1
netCDF4: 1.2.4
h5netcdf: None
Nio: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.16.0
matplotlib: 2.0.2
cartopy: 0.15.1
seaborn: None
setuptools: 27.2.0
pip: 9.0.1
conda: 4.3.30
pytest: None
IPython: None
sphinx: None

>>> xr.open_dataarray('ww3.Hs.July.NPac.nc')
Segmentation fault (core dumped)

1 个答案:

答案 0 :(得分:0)

道歉。看起来我的ncks命令没有完成属性。与xarray没什么关系,只是奇怪我的Mac没有抛出错误。陌生人仍然给出.nc一个不同的文件名似乎解决了它

$ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 ww3.Hs.July.nc ww3.Hs.July.NPac.nc
$python
>>>import xarray as xr
>>>xr.open_dataarray('ww3.Hs.July.NPac.nc')
Segmentation fault (core dumped)

$ncks -O -d longitude,100.0,290.0 -d latitude,0.0,65.0 ww3.Hs.July.nc ww3.Hs.July.NPac1.nc
$python
>>>import xarray as xr
>>>xr.open_dataarray('ww3.Hs.July.NPac1.nc')
<xarray.DataArray 'hs' (time: 31, latitude: 66, longitude: 191)>
[390786 values with dtype=float64]
Coordinates:
  * latitude   (latitude) float32 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 ...
  * longitude  (longitude) float32 100.0 101.0 102.0 103.0 104.0 105.0 106.0 ...
  * time       (time) datetime64[ns] 1993-07-01 1993-07-02 1993-07-03 ...
Attributes:
    long_name:      significant height of wind and swell waves
    standard_name:  sea_surface_wave_significant_height
    globwave_name:  significant_wave_height
    units:          m
    valid_min:      0
    valid_max:      32000