Orientdb - 使用ETL

时间:2016-08-17 08:46:32

标签: json database graph orientdb etl

OrientDB版本,操作系统或硬件。

  • V2.2.5

操作系统

  • [] Linux

预期的行为和实际行为

使用ETL导入顶点和边时,在加载一些顶点和边后会发出超时异常 顶点和边缘。

详细错误:

- Year.csv: id_year |年 1 | 2010年 2 | 2011年 3 | 2012年 4 | 2013年 5 | 2014年 6 | 2015年 7 | 2016年 8 | 2017年

- Month.csv: id_month |月|季度|年 1 |一月| JFM | 2010年 2 |二月| JFM | 2010年 3 |三月| JFM | 2010年 4 |四月| AMJ | 2010年 5 |五月| AMJ | 2010年 6 |六月| AMJ | 2010年 7 |七月| JAS | 2010年 8 |八月| JAS | 2010年 9 |九月| JAS | 2010年 10 |十月| OND | 2010年 11 |十一月| OND | 2010年 12 |十二月| OND | 2010年 13 |一月| JFM | 2011年 14 |二月| JFM | 2011年 15 |三月| JFM | 2011年 16 |四月| AMJ | 2011年 17 |五月| AMJ | 2011年 18 |六月| AMJ | 2011年 19 |七月| JAS | 2011年 20 |八月| JAS | 2011年 21 |九月| JAS | 2011年 22 |十月| OND | 2011年 23 |十一月| OND | 2011年 24 |十二月| OND | 2011年 25 |一月| JFM | 2012年 26 |二月| JFM | 2012年 27 |三月| JFM | 2012年 28 |四月| AMJ | 2012年 29 |五月| AMJ | 2012年 30 |六月| AMJ | 2012年 31 |七月| JAS | 2012年 32 |八月| JAS | 2012年 33 |九月| JAS | 2012年 34 |十月| OND | 2012年 35 |十一月| OND | 2012年 36 |十二月| OND | 2012年 37 |一月| JFM | 2013年 38 |二月| JFM | 2013年 39 |三月| JFM | 2013年 40 |四月| AMJ | 2013年 41 |五月| AMJ | 2013年 42 |六月| AMJ | 2013年 43 |七月| JAS | 2013年 44 |八月| JAS | 2013年 45 |九月| JAS | 2013年 46 |十月| OND | 2013年 47 |十一月| OND | 2013年 48 |十二月| OND | 2013年 49 |一月| JFM | 2014年 50 |二月| JFM | 2014年 51 |三月| JFM | 2014年 52 |四月| AMJ | 2014年 53 |五月| AMJ | 2014年 54 |六月| AMJ | 2014年 55 |七月| JAS | 2014年 56 |八月| JAS | 2014年 57 |九月| JAS | 2014年 58 |十月| OND | 2014年 59 |十一月| OND | 2014年 60 |十二月| OND | 2014年 61 |一月| JFM | 2015年 62 |二月| JFM | 2015年 63 |三月| JFM | 2015年 64 |四月| AMJ | 2015年 65 |五月| AMJ | 2015年 66 |六月| AMJ | 2015年 67 |七月| JAS | 2015年 68 |八月| JAS | 2015年 69 |九月| JAS | 2015年 70 |十月| OND | 2015年 71 |十一月| OND | 2015年 72 |十二月| OND | 2015年 73 |一月| JFM | 2016年 74 |二月| JFM | 2016年 75 |三月| JFM | 2016年 76 |四月| AMJ | 2016年 77 |五月| AMJ | 2016年 78 |六月| AMJ | 2016年 79 |七月| JAS | 2016年 80 |八月| JAS | 2016年 81 |九月| JAS | 2016年 82 |十月| OND | 2016年 83 |十一月| OND | 2016年 84 |十二月| OND | 2016年 85 |一月| JFM | 2017
86 |二月| JFM | 2017
87 |三月| JFM | 2017
88 |四月| AMJ | 2017
89 |五月| AMJ | 2017
90 |六月| AMJ | 2017
91 |七月| JAS | 2017
92 |八月| JAS | 2017
93 |九月| JAS | 2017
94 |十月| OND | 2017
95 |十一月| OND | 2017
96 |十二月| OND | 2017年

- Year.csv的ETL JSON:

{
    "source": {
        "file": {
            "path": "../testdb/data/year.csv"
        }
    },
    "extractor": {
        "row": {

        }
    },
    "transformers": [{
        "csv": {
            "separator": ",",
            "skipFrom": 1,
            "skipTo": 0,
            "nullValue": "Null",
            "columnsOnFirstLine": true
        }
    },
    {
        "vertex": {
            "class": "Year"
        }
    }
    ],
    "loader": {
        "orientdb": {
            "dbURL": "remote:1.2.3.4/testdb",
            "dbUser": "orientdb",
            "dbPassword": "orientdb",
            "serverUser": "orientdb",
            "serverPassword": "orientdb",
            "dbType": "graph",
            "classes": [{
                "name": "Year",
                "extends": "V"
            }],
            "indexes": [
                {"class":"Year", "fields":["year:string"], "type":"UNIQUE" }
            ]
        }
    }
}

- 月份的ETL JSON:

{
    "config": {
        "log":"debug",
        "parallel":true
    },
    "source": {
        "file": {
            "path": "../testdb/data/month.csv"
        }
    },
    "extractor": {
        "row": {
        }
    },
    "transformers": [{
        "csv": {
            "separator": ",",
            "skipFrom": 1,
            "skipTo": 0,
            "nullValue": "Null",
            "columnsOnFirstLine": true
        }
    },
    {
        "vertex": {
            "class": "Month"
        }
    },
    {
        "edge": {
            "class":"BelongsTo",
            "joinFieldName":"year",
            "lookup":"Year.year",
            "direction":"out",
            "unresolvedLinkAction":"WARNING"
        }
    }],
    "loader": {
        "orientdb": {
            "dbURL": "remote:1.2.3.4/testdb",
            "dbUser": "orientdb",
            "dbPassword": "orientdb",
            "serverUser": "orientdb",
            "serverPassword": "orientdb",
            "dbType": "graph",
            "classes": [{
                "name": "Month",
                "extends": "V"
            }]
        }
    }
}

- 正确加载了year.csv的ETL JSON。 - 为month.csv加载ETL JSON时出错如下:

C:\orientdb-community-2.2.5\bin>oetl.bat ..\testdb\etl\loadMonth.json
BEGIN ETL PROCESSOR
[file] INFO Reading from file ../testdb/data/month.csv with encoding UTF-8
Started execution with 1 worker threads
+ extracted 0 rows (0 rows/sec) - 0 rows -> loaded 0 vertices (0 vertices/sec) Total time: 1000ms 
[0 warnings, 0 errors]
+ extracted 0 rows (0 rows/sec) - 0 rows -> loaded 0 vertices (0 vertices/sec) Total time: 2s [0 
warnings, 0 errors]
+ extracted 0 rows (0 rows/sec) - 0 rows -> loaded 0 vertices (0 vertices/sec) Total time: 3s [0 
warnings, 0 errors]
[orientdb] DEBUG - OrientDBLoader: created vertex class 'Month' extends 'V'
[orientdb] DEBUG orientdb: found 0 vertices in class 'null'
+ extracted 0 rows (0 rows/sec) - 0 rows -> loaded 0 vertices (0 vertices/sec) Total time: 4s [0 
warnings, 0 errors]
+ extracted 0 rows (0 rows/sec) - 0 rows -> loaded 0 vertices (0 vertices/sec) Total time: 5s [0 
warnings, 0 errors]
Start extracting
[0:csv] DEBUG Transformer input: id_month,month,quarter,year
[0:csv] DEBUG parsing=id_month,month,quarter,year
[0:csv] DEBUG Transformer output: null
2016-08-17 12:56:22:964 WARNI Transformer [csv] returned null, skip rest of pipeline execution 
[OETLPipeline][1:csv] DEBUG Transformer input: 1,January,JFM,2010
[1:csv] DEBUG parsing=1,January,JFM,2010
[1:csv] DEBUG document={id_month:1,month:January,quarter:JFM,year:2010}
[1:csv] DEBUG Transformer output: {id_month:1,month:January,quarter:JFM,year:2010}
[1:vertex] DEBUG Transformer input: {id_month:1,month:January,quarter:JFM,year:2010}
[1:vertex] DEBUG Transformer output: v(Month)[#45:0]
[1:edge] DEBUG Transformer input: v(Month)[#45:0]
+ extracted 97 rows (97 rows/sec) - 97 rows -> loaded 0 vertices (0 vertices/sec) Total time: 6s [0 
warnings, 0 errors]
[1:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
[1:edge] DEBUG created new edge=e[#53:0][#45:0-BelongsTo->#41:0]
[1:edge] DEBUG Transformer output: v(Month)[#45:0]
[2:csv] DEBUG Transformer input: 2,February,JFM,2010
[2:csv] DEBUG parsing=2,February,JFM,2010
[2:csv] DEBUG document={id_month:2,month:February,quarter:JFM,year:2010}
[2:csv] DEBUG Transformer output: {id_month:2,month:February,quarter:JFM,year:2010}
[2:vertex] DEBUG Transformer input: {id_month:2,month:February,quarter:JFM,year:2010}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 1 vertices (1 vertices/sec) Total time: 7s [0 
warnings, 0 errors]
[2:vertex] DEBUG Transformer output: v(Month)[#46:0]
[2:edge] DEBUG Transformer input: v(Month)[#46:0]
[2:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 1 vertices (0 vertices/sec) Total time: 8s [0 

warnings, 0 errors]
[2:edge] DEBUG created new edge=e[#54:0][#46:0-BelongsTo->#41:0]
[2:edge] DEBUG Transformer output: v(Month)[#46:0]
[3:csv] DEBUG Transformer input: 3,March,JFM,2010
[3:csv] DEBUG parsing=3,March,JFM,2010
[3:csv] DEBUG document={id_month:3,month:March,quarter:JFM,year:2010}
[3:csv] DEBUG Transformer output: {id_month:3,month:March,quarter:JFM,year:2010}
[3:vertex] DEBUG Transformer input: {id_month:3,month:March,quarter:JFM,year:2010}
[3:vertex] DEBUG Transformer output: v(Month)[#45:1]
[3:edge] DEBUG Transformer input: v(Month)[#45:1]
[3:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 2 vertices (1 vertices/sec) Total time: 9s [0 

warnings, 0 errors]
[3:edge] DEBUG created new edge=e[#53:1][#45:1-BelongsTo->#41:0]
[3:edge] DEBUG Transformer output: v(Month)[#45:1]
[4:csv] DEBUG Transformer input: 4,April,AMJ,2010
[4:csv] DEBUG parsing=4,April,AMJ,2010
[4:csv] DEBUG document={id_month:4,month:April,quarter:AMJ,year:2010}
[4:csv] DEBUG Transformer output: {id_month:4,month:April,quarter:AMJ,year:2010}
[4:vertex] DEBUG Transformer input: {id_month:4,month:April,quarter:AMJ,year:2010}
[4:vertex] DEBUG Transformer output: v(Month)[#46:1]
[4:edge] DEBUG Transformer input: v(Month)[#46:1]
[4:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 3 vertices (0 vertices/sec) Total time: 

10014ms [0 warnings, 0 errors]
[4:edge] DEBUG created new edge=e[#54:1][#46:1-BelongsTo->#41:0]
[4:edge] DEBUG Transformer output: v(Month)[#46:1]
[5:csv] DEBUG Transformer input: 5,May,AMJ,2010
[5:csv] DEBUG parsing=5,May,AMJ,2010
[5:csv] DEBUG document={id_month:5,month:May,quarter:AMJ,year:2010}
[5:csv] DEBUG Transformer output: {id_month:5,month:May,quarter:AMJ,year:2010}
[5:vertex] DEBUG Transformer input: {id_month:5,month:May,quarter:AMJ,year:2010}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 4 vertices (1 vertices/sec) Total time: 
11014ms [0 warnings, 0 errors]
[5:vertex] DEBUG Transformer output: v(Month)[#45:2]
[5:edge] DEBUG Transformer input: v(Month)[#45:2]
[5:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 4 vertices (0 vertices/sec) Total time: 
12014ms [0 warnings, 0 errors]
[5:edge] DEBUG created new edge=e[#53:2][#45:2-BelongsTo->#41:0]
[5:edge] DEBUG Transformer output: v(Month)[#45:2]
[6:csv] DEBUG Transformer input: 6,June,AMJ,2010
[6:csv] DEBUG parsing=6,June,AMJ,2010
[6:csv] DEBUG document={id_month:6,month:June,quarter:AMJ,year:2010}
[6:csv] DEBUG Transformer output: {id_month:6,month:June,quarter:AMJ,year:2010}
[6:vertex] DEBUG Transformer input: {id_month:6,month:June,quarter:AMJ,year:2010}
[6:vertex] DEBUG Transformer output: v(Month)[#46:2]
[6:edge] DEBUG Transformer input: v(Month)[#46:2]
[6:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 5 vertices (1 vertices/sec) Total time: 
13014ms [0 warnings, 0 errors]
[6:edge] DEBUG created new edge=e[#54:2][#46:2-BelongsTo->#41:0]
[6:edge] DEBUG Transformer output: v(Month)[#46:2]
[7:csv] DEBUG Transformer input: 7,July,JAS,2010
[7:csv] DEBUG parsing=7,July,JAS,2010
[7:csv] DEBUG document={id_month:7,month:July,quarter:JAS,year:2010}
[7:csv] DEBUG Transformer output: {id_month:7,month:July,quarter:JAS,year:2010}
[7:vertex] DEBUG Transformer input: {id_month:7,month:July,quarter:JAS,year:2010}
[7:vertex] DEBUG Transformer output: v(Month)[#45:3]
[7:edge] DEBUG Transformer input: v(Month)[#45:3]
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 6 vertices (1 vertices/sec) Total time: 

14014ms [0 warnings, 0 errors]
[7:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
[7:edge] DEBUG created new edge=e[#53:3][#45:3-BelongsTo->#41:0]
[7:edge] DEBUG Transformer output: v(Month)[#45:3]
[8:csv] DEBUG Transformer input: 8,August,JAS,2010
[8:csv] DEBUG parsing=8,August,JAS,2010
[8:csv] DEBUG document={id_month:8,month:August,quarter:JAS,year:2010}
[8:csv] DEBUG Transformer output: {id_month:8,month:August,quarter:JAS,year:2010}
[8:vertex] DEBUG Transformer input: {id_month:8,month:August,quarter:JAS,year:2010}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 7 vertices (1 vertices/sec) Total time: 

15014ms [0 warnings, 0 errors]
[8:vertex] DEBUG Transformer output: v(Month)[#46:3]
[8:edge] DEBUG Transformer input: v(Month)[#46:3]
[8:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 7 vertices (0 vertices/sec) Total time: 

16014ms [0 warnings, 0 errors]
[8:edge] DEBUG created new edge=e[#54:3][#46:3-BelongsTo->#41:0]
[8:edge] DEBUG Transformer output: v(Month)[#46:3]
[9:csv] DEBUG Transformer input: 9,September,JAS,2010
[9:csv] DEBUG parsing=9,September,JAS,2010
[9:csv] DEBUG document={id_month:9,month:September,quarter:JAS,year:2010}
[9:csv] DEBUG Transformer output: {id_month:9,month:September,quarter:JAS,year:2010}
[9:vertex] DEBUG Transformer input: {id_month:9,month:September,quarter:JAS,year:2010}
[9:vertex] DEBUG Transformer output: v(Month)[#45:4]
[9:edge] DEBUG Transformer input: v(Month)[#45:4]
[9:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 8 vertices (1 vertices/sec) Total time: 
17014ms [0 warnings, 0 errors]
[9:edge] DEBUG created new edge=e[#53:4][#45:4-BelongsTo->#41:0]
[9:edge] DEBUG Transformer output: v(Month)[#45:4]
[10:csv] DEBUG Transformer input: 10,October,OND,2010
[10:csv] DEBUG parsing=10,October,OND,2010
[10:csv] DEBUG document={id_month:10,month:October,quarter:OND,year:2010}
[10:csv] DEBUG Transformer output: {id_month:10,month:October,quarter:OND,year:2010}
[10:vertex] DEBUG Transformer input: {id_month:10,month:October,quarter:OND,year:2010}
[10:vertex] DEBUG Transformer output: v(Month)[#46:4]
[10:edge] DEBUG Transformer input: v(Month)[#46:4]
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 9 vertices (1 vertices/sec) Total time: 
18014ms [0 warnings, 0 errors]
[10:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
[10:edge] DEBUG created new edge=e[#54:4][#46:4-BelongsTo->#41:0]
[10:edge] DEBUG Transformer output: v(Month)[#46:4]
[11:csv] DEBUG Transformer input: 11,November,OND,2010
[11:csv] DEBUG parsing=11,November,OND,2010
[11:csv] DEBUG document={id_month:11,month:November,quarter:OND,year:2010}
[11:csv] DEBUG Transformer output: {id_month:11,month:November,quarter:OND,year:2010}
[11:vertex] DEBUG Transformer input: {id_month:11,month:November,quarter:OND,year:2010}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 10 vertices (1 vertices/sec) Total time: 
19014ms [0 warnings, 0 errors]
[11:vertex] DEBUG Transformer output: v(Month)[#45:5]
[11:edge] DEBUG Transformer input: v(Month)[#45:5]
[11:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 10 vertices (0 vertices/sec) Total time: 
20014ms [0 warnings, 0 errors]
[11:edge] DEBUG created new edge=e[#53:5][#45:5-BelongsTo->#41:0]
[11:edge] DEBUG Transformer output: v(Month)[#45:5]
[12:csv] DEBUG Transformer input: 12,December,OND,2010
[12:csv] DEBUG parsing=12,December,OND,2010
[12:csv] DEBUG document={id_month:12,month:December,quarter:OND,year:2010}
[12:csv] DEBUG Transformer output: {id_month:12,month:December,quarter:OND,year:2010}
[12:vertex] DEBUG Transformer input: {id_month:12,month:December,quarter:OND,year:2010}
[12:vertex] DEBUG Transformer output: v(Month)[#46:5]
[12:edge] DEBUG Transformer input: v(Month)[#46:5]
[12:edge] DEBUG joinCurrentValue=2010, lookupResult=#41:0
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 11 vertices (1 vertices/sec) Total time: 
21014ms [0 warnings, 0 errors]
[12:edge] DEBUG created new edge=e[#54:5][#46:5-BelongsTo->#41:0]
[12:edge] DEBUG Transformer output: v(Month)[#46:5]

**<-- Skipped some rows with similar output due to space limitation-->**

[35:csv] DEBUG Transformer input: 35,November,OND,2012
[35:csv] DEBUG parsing=35,November,OND,2012
[35:csv] DEBUG document={id_month:35,month:November,quarter:OND,year:2012}
[35:csv] DEBUG Transformer output: {id_month:35,month:November,quarter:OND,year:2012}
[35:vertex] DEBUG Transformer input: {id_month:35,month:November,quarter:OND,year:2012}
[35:vertex] DEBUG Transformer output: v(Month)[#45:17]
[35:edge] DEBUG Transformer input: v(Month)[#45:17]
[35:edge] DEBUG joinCurrentValue=2012, lookupResult=#41:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 34 vertices (1 vertices/sec) Total time: 

52072ms [0 warnings, 0 errors]
[35:edge] DEBUG created new edge=e[#53:17][#45:17-BelongsTo->#41:1]
[35:edge] DEBUG Transformer output: v(Month)[#45:17]
[36:csv] DEBUG Transformer input: 36,December,OND,2012
[36:csv] DEBUG parsing=36,December,OND,2012
[36:csv] DEBUG document={id_month:36,month:December,quarter:OND,year:2012}
[36:csv] DEBUG Transformer output: {id_month:36,month:December,quarter:OND,year:2012}
[36:vertex] DEBUG Transformer input: {id_month:36,month:December,quarter:OND,year:2012}
[36:vertex] DEBUG Transformer output: v(Month)[#46:17]
[36:edge] DEBUG Transformer input: v(Month)[#46:17]
[36:edge] DEBUG joinCurrentValue=2012, lookupResult=#41:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 35 vertices (1 vertices/sec) Total time: 

53072ms [0 warnings, 0 errors]
[36:edge] DEBUG created new edge=e[#54:17][#46:17-BelongsTo->#41:1]
[36:edge] DEBUG Transformer output: v(Month)[#46:17]
[37:csv] DEBUG Transformer input: 37,January,JFM,2013
[37:csv] DEBUG parsing=37,January,JFM,2013
[37:csv] DEBUG document={id_month:37,month:January,quarter:JFM,year:2013}
[37:csv] DEBUG Transformer output: {id_month:37,month:January,quarter:JFM,year:2013}
[37:vertex] DEBUG Transformer input: {id_month:37,month:January,quarter:JFM,year:2013}
[37:vertex] DEBUG Transformer output: v(Month)[#45:18]
[37:edge] DEBUG Transformer input: v(Month)[#45:18]
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 36 vertices (1 vertices/sec) Total time: 

54072ms [0 warnings, 0 errors]
[37:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 36 vertices (0 vertices/sec) Total time: 

55074ms [0 warnings, 0 errors]
[37:edge] DEBUG created new edge=e[#53:18][#45:18-BelongsTo->#42:1]
[37:edge] DEBUG Transformer output: v(Month)[#45:18]
[38:csv] DEBUG Transformer input: 38,February,JFM,2013
[38:csv] DEBUG parsing=38,February,JFM,2013
[38:csv] DEBUG document={id_month:38,month:February,quarter:JFM,year:2013}
[38:csv] DEBUG Transformer output: {id_month:38,month:February,quarter:JFM,year:2013}
[38:vertex] DEBUG Transformer input: {id_month:38,month:February,quarter:JFM,year:2013}
[38:vertex] DEBUG Transformer output: v(Month)[#46:18]
[38:edge] DEBUG Transformer input: v(Month)[#46:18]
[38:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 37 vertices (1 vertices/sec) Total time: 

56074ms [0 warnings, 0 errors]
[38:edge] DEBUG created new edge=e[#54:18][#46:18-BelongsTo->#42:1]
[38:edge] DEBUG Transformer output: v(Month)[#46:18]
[39:csv] DEBUG Transformer input: 39,March,JFM,2013
[39:csv] DEBUG parsing=39,March,JFM,2013
[39:csv] DEBUG document={id_month:39,month:March,quarter:JFM,year:2013}
[39:csv] DEBUG Transformer output: {id_month:39,month:March,quarter:JFM,year:2013}
[39:vertex] DEBUG Transformer input: {id_month:39,month:March,quarter:JFM,year:2013}
[39:vertex] DEBUG Transformer output: v(Month)[#45:19]
[39:edge] DEBUG Transformer input: v(Month)[#45:19]
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 38 vertices (1 vertices/sec) Total time: 

57074ms [0 warnings, 0 errors]
[39:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
[39:edge] DEBUG created new edge=e[#53:19][#45:19-BelongsTo->#42:1]
[39:edge] DEBUG Transformer output: v(Month)[#45:19]
[40:csv] DEBUG Transformer input: 40,April,AMJ,2013
[40:csv] DEBUG parsing=40,April,AMJ,2013
[40:csv] DEBUG document={id_month:40,month:April,quarter:AMJ,year:2013}
[40:csv] DEBUG Transformer output: {id_month:40,month:April,quarter:AMJ,year:2013}
[40:vertex] DEBUG Transformer input: {id_month:40,month:April,quarter:AMJ,year:2013}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 39 vertices (1 vertices/sec) Total time: 

58074ms [0 warnings, 0 errors]
[40:vertex] DEBUG Transformer output: v(Month)[#46:19]
[40:edge] DEBUG Transformer input: v(Month)[#46:19]
[40:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 39 vertices (0 vertices/sec) Total time: 

59074ms [0 warnings, 0 errors]
[40:edge] DEBUG created new edge=e[#54:19][#46:19-BelongsTo->#42:1]
[40:edge] DEBUG Transformer output: v(Month)[#46:19]
[41:csv] DEBUG Transformer input: 41,May,AMJ,2013
[41:csv] DEBUG parsing=41,May,AMJ,2013
[41:csv] DEBUG document={id_month:41,month:May,quarter:AMJ,year:2013}
[41:csv] DEBUG Transformer output: {id_month:41,month:May,quarter:AMJ,year:2013}
[41:vertex] DEBUG Transformer input: {id_month:41,month:May,quarter:AMJ,year:2013}
[41:vertex] DEBUG Transformer output: v(Month)[#45:20]
[41:edge] DEBUG Transformer input: v(Month)[#45:20]
[41:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 40 vertices (1 vertices/sec) Total time: 

60074ms [0 warnings, 0 errors]
[41:edge] DEBUG created new edge=e[#53:20][#45:20-BelongsTo->#42:1]
[41:edge] DEBUG Transformer output: v(Month)[#45:20]
[42:csv] DEBUG Transformer input: 42,June,AMJ,2013
[42:csv] DEBUG parsing=42,June,AMJ,2013
[42:csv] DEBUG document={id_month:42,month:June,quarter:AMJ,year:2013}
[42:csv] DEBUG Transformer output: {id_month:42,month:June,quarter:AMJ,year:2013}
[42:vertex] DEBUG Transformer input: {id_month:42,month:June,quarter:AMJ,year:2013}
[42:vertex] DEBUG Transformer output: v(Month)[#46:20]
[42:edge] DEBUG Transformer input: v(Month)[#46:20]
[42:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 41 vertices (1 vertices/sec) Total time: 

61074ms [0 warnings, 0 errors]
[42:edge] DEBUG created new edge=e[#54:20][#46:20-BelongsTo->#42:1]
[42:edge] DEBUG Transformer output: v(Month)[#46:20]
[43:csv] DEBUG Transformer input: 43,July,JAS,2013
[43:csv] DEBUG parsing=43,July,JAS,2013
[43:csv] DEBUG document={id_month:43,month:July,quarter:JAS,year:2013}
[43:csv] DEBUG Transformer output: {id_month:43,month:July,quarter:JAS,year:2013}
[43:vertex] DEBUG Transformer input: {id_month:43,month:July,quarter:JAS,year:2013}
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 42 vertices (0 vertices/sec) Total time: 

62076ms [0 warnings, 0 errors]
[43:vertex] DEBUG Transformer output: v(Month)[#45:21]
[43:edge] DEBUG Transformer input: v(Month)[#45:21]
[43:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 42 vertices (0 vertices/sec) Total time: 

63081ms [0 warnings, 0 errors]
[43:edge] DEBUG created new edge=e[#53:21][#45:21-BelongsTo->#42:1]
[43:edge] DEBUG Transformer output: v(Month)[#45:21]
[44:csv] DEBUG Transformer input: 44,August,JAS,2013
[44:csv] DEBUG parsing=44,August,JAS,2013
[44:csv] DEBUG document={id_month:44,month:August,quarter:JAS,year:2013}
[44:csv] DEBUG Transformer output: {id_month:44,month:August,quarter:JAS,year:2013}
[44:vertex] DEBUG Transformer input: {id_month:44,month:August,quarter:JAS,year:2013}
[44:vertex] DEBUG Transformer output: v(Month)[#46:21]
[44:edge] DEBUG Transformer input: v(Month)[#46:21]
[44:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 43 vertices (1 vertices/sec) Total time: 

64081ms [0 warnings, 0 errors]
[44:edge] DEBUG created new edge=e[#54:21][#46:21-BelongsTo->#42:1]
[44:edge] DEBUG Transformer output: v(Month)[#46:21]
[45:csv] DEBUG Transformer input: 45,September,JAS,2013
[45:csv] DEBUG parsing=45,September,JAS,2013
[45:csv] DEBUG document={id_month:45,month:September,quarter:JAS,year:2013}
[45:csv] DEBUG Transformer output: {id_month:45,month:September,quarter:JAS,year:2013}
[45:vertex] DEBUG Transformer input: {id_month:45,month:September,quarter:JAS,year:2013}
[45:vertex] DEBUG Transformer output: v(Month)[#45:22]
[45:edge] DEBUG Transformer input: v(Month)[#45:22]
[45:edge] DEBUG joinCurrentValue=2013, lookupResult=#42:1
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 44 vertices (1 vertices/sec) Total time: 

65081ms [0 warnings, 0 errors]
ETL process has problem: java.util.concurrent.TimeoutException
END ETL PROCESSOR
+ extracted 97 rows (0 rows/sec) - 97 rows -> loaded 44 vertices (0 vertices/sec) Total time: 

65225ms [0 warnings, 0 errors]
Error in Pipeline execution: com.orientechnologies.orient.core.exception.ODatabaseException: Error 

during saving of record with rid #45:22
    DB name="testdb"
C:\orientdb-community-2.2.5\bin>

重现问题的步骤

  1. 使用ETL(无错误加载)**
  2. 导入year.csv的JSON
  3. 使用ETL导入month.csv的JSON(加载一些顶点和边缘,然后在大约64000ms后抛出错误)

1 个答案:

答案 0 :(得分:0)

我试过

"config": {
        "log":"debug",
        "maxRetries":10,
        "parallel":true
    },

它有效。