Java NetCDF：聚合现有文件：未找到时间维度问题

时间：2017-01-05 17:48:40

标签： java netcdf netcdf4

对于我的脑筋急转弯，我已经在文档和邮件列表档案中搜索了一段时间，并且很难将我需要的步骤放在一起来处理这种聚合。

来自此处的CFSR 1小时数据文件数据：http://rda.ucar.edu/datasets/ds094.0/ 

cdas_20161215_0000_f00000_G4.grib2
cdas_20161215_0000_f00100_G4
cdas_20161215_0000_f00200_G4
cdas_20161215_0000_f00300_G4
etc...

每小时文件声明2个时间维度，一个具有边界集，另一个没有。

cdas_20161215_0000_f00300_G4.grib2
double time(time=1);
  :units = "Hour since 2016-12-15T00:00:00Z";
  :standard_name = "time";
  :long_name = "GRIB forecast or observation time";
  :calendar = "proleptic_gregorian";
  :bounds = "time_bounds";
double time_bounds(time=1, 2);
  :units = "Hour since 2016-12-15T00:00:00Z";
  :long_name = "bounds for time";
double time1(time1=1);
  :units = "Hour since 2016-12-15T00:00:00Z";
  :standard_name = "time";
  :long_name = "GRIB forecast or observation time";
  :calendar = "proleptic_gregorian";

问题在于，当我逐步完成每个数据集创建时，不同的每小时文件将交换2个时间维名称的名称。因此，AggregationExisting无法找到某些文件的维度名称“时间”，例如在0300文件中的u-component_of_wind_isobaric变量上，因为它被声明为time1。

我打电话的代码：

List<String> variableNames = Arrays.asList("u-component_of_wind_isobaric","u-component_of_wind_height_above_ground","v-component_of_wind_isobaric","v-component_of_wind_height_above_ground","Pressure_reduced_to_MSL_msl","Geopotential_height_isobaric");
NetcdfDataset netcdfDataset = new NetcdfDataset();
//here i'm trying to aggregate on a dimension called 'time'
AggregationExisting aggregationExisting = new AggregationExisting(netcdfDataset, "time", null);
aggregationExisting.addDatasetScan(null,
                   "/cfsr-gribs/201612/",
                    "G4.grib2",
                    null,
                    null,
                    NetcdfDataset.getDefaultEnhanceMode(),
                    "false",
                    null);
aggregationExisting.persistWrite();
aggregationExisting.finish(new CancelTaskImpl());
GridDataset gridDataset = new GridDataset(netcdfDataset);
writer.setRedefineMode(true);
CFGridWriter2.writeFile(gridDataset, variableNames, gridDataset.getBoundingBox(), null, 1, null, null, 1, true, writer);

时间维名称问题在2个文件中说明：

//cdas_20161215_0000_f00300_G4.grib2

float u-component_of_wind_isobaric(time1=1, isobaric3=37, lat=361, lon=720);
  :long_name = "u-component of wind @ Isobaric surface";
  :units = "m/s";
  :abbreviation = "UGRD";
  :missing_value = NaNf; // float
  :grid_mapping = "LatLon_Projection";
  :coordinates = "reftime time1 isobaric3 lat lon ";
  :Grib_Variable_Id = "VAR_0-2-2_L100";
  :Grib2_Parameter = 0, 2, 2; // int
  :Grib2_Parameter_Discipline = "Meteorological products";
  :Grib2_Parameter_Category = "Momentum";
  :Grib2_Parameter_Name = "u-component of wind";
  :Grib2_Level_Type = "Isobaric surface";
  :Grib2_Generating_Process_Type = "Forecast";


//cdas_20161215_0000_f00200_G4.grib2

float u-component_of_wind_isobaric(time=1, isobaric3=37, lat=361, lon=720);
  :long_name = "u-component of wind @ Isobaric surface";
  :units = "m/s";
  :abbreviation = "UGRD";
  :missing_value = NaNf; // float
  :grid_mapping = "LatLon_Projection";
  :coordinates = "reftime time isobaric3 lat lon ";
  :Grib_Variable_Id = "VAR_0-2-2_L100";
  :Grib2_Parameter = 0, 2, 2; // int
  :Grib2_Parameter_Discipline = "Meteorological products";
  :Grib2_Parameter_Category = "Momentum";
  :Grib2_Parameter_Name = "u-component of wind";
  :Grib2_Level_Type = "Isobaric surface";
  :Grib2_Generating_Process_Type = "Forecast";

这是我的第一个NetCDF库使用，所以我正在购买一些预处理工具来合并这些具有这个怪癖的数据集。例如，我可以将所有变量移动到相同的时间维度并重命名吗？即使是我错过的一个例子的链接也会有所帮助。否则我猜我会考虑手动标记尺寸并使用readDataSlice（）手动将数据复制到新的合并文件中。

2 个答案:

答案 0 :(得分：1)

如果您对使用非Java工具感兴趣，我建议您查看NCO。

首先，您需要将grib转换为netcdf，可能使用wgrib2实用程序（转换示例为here）或ncl_convert2nc。

其次，您可以开发一个循环浏览相关netcdf文件的简单脚本，检查public MainWindow() { InitializeComponent(); DataContext = new ViewModel(); } 是否作为维名称存在，如果存在，请将名称更改为time1。 NCO的time工具可以做到这一点：

ncrename

第三，检查以确保ncrename -d time1,time file.nc file.nc （现在所有文件中应该存在）是记录维度。如果没有，请使用NCO的time工具：

ncks

最后，使用NCO＆＃39; ncks --mk_rec_dmn time file.nc 连接记录（ncrcat）维度上的文件：

time

注意：您不必在上面的行中使用通配符，您可以只包含要连接的文件列表，例如： 

ncrcat cdas*.nc all_files.nc

答案 1 :(得分：0)

所以我收到Ucar的回复说Grib2是一个不同的野兽，目前还没有与AggregationExisting合作。他们的THREDDS服务器产品具有Grib2文件的功能，因此它是一些不同的类，例如GribCollectionImmutable。

以下是他们推荐的这种方法，对我来说非常有用：

        List<String> variableNames = Arrays.asList("u-component_of_wind_isobaric","u-component_of_wind_height_above_ground","v-component_of_wind_isobaric","v-component_of_wind_height_above_ground","Pressure_reduced_to_MSL_msl","Geopotential_height_isobaric");
        FeatureCollectionType fcType = FeatureCollectionType.GRIB2;
        Path outputPath = Paths.get("/cfsr/Netcdf4/201612/Cfsr_201612_Monthly.nc");
        String dataDir = "/cfsr-gribs/201612/";
        String spec = dataDir + ".*grib2$";
        String timePartition = "file";
        String dateFormatMark = null;
        String olderThan = null;
        Element innerNcml = null;
        String path = dataDir;
        String name = "cfsr";
        String collectionName = "cfsrCollection";

        //find and configure the folder as a grib collection
        FeatureCollectionConfig fcc = new FeatureCollectionConfig(name, path, fcType, spec,
                collectionName, dateFormatMark, olderThan, timePartition, innerNcml);

        try (GribCollectionImmutable gc = GribCdmIndex.openGribCollection(fcc, null, log)) {
            //had to breakpoint and see the dataset typenames to choose 'TP', could be different for each dataset
            GribCollectionImmutable.Dataset ds = gc.getDatasetByTypeName("TP");
            String fullCollectionIndexFilePath = dataDir + name + ".ncx3";
            // now we open the collection index file, which catalogs all of the grib
            //  records in your collection
            NetcdfDataset ncd = gc.getNetcdfDataset(ds, ds.getGroup(0), fullCollectionIndexFilePath,
                    fcc, null, log);
            try (NetcdfFileWriter writer = NetcdfFileWriter.createNew(NetcdfFileWriter.Version.netcdf4,
                    outputPath.toString(), new Nc4ChunkingDefault())) {
                GridDataset gridDataset = new GridDataset(ncd);
                for (String variableName : variableNames) {
                    GeoGrid grid = gridDataset.findGridByShortName(variableName);
                    //Check that the time dimension is the length you'd expect
                    log.info(String.format("Found grid for : %s = %s, with dimension length %s", variableName, grid != null, grid != null ? grid.getDimension(0).getLength() : 0));
                }
                writer.setRedefineMode(true);
                //write the aggregated variables to my output file
                CFGridWriter2.writeFile(gridDataset, variableNames, gridDataset.getBoundingBox(), null, 1, null, null, 1, true, writer);
            } catch (Exception exc) {
                exc.printStackTrace();
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
相关问题
最新问题