Question

我是OrientDb的新手，但我和ETL脚本有问题，我只是想弄清楚为什么它不起作用。这是场景：

CSV文件：

PersonName,PersonAge,CarMake,CarAge
John,23,Nissan,3
Peter,45,Mini,25
Paul,23,Morgan,15
Jim,24,Mini,24

ETL json如下：

{
    "config": {"log": "INFO",
    "fileDirectory": "/Users/johnbrogan/Downloads/orientdb-community-2.2.20/import/",
    "fileName": "SimpleTest.csv",
    "dbDirectory": "plocal:/Users/johnbrogan/Downloads/orientdb-community-2.2.20/databases/",
    "dbName": "simpleTest",
    "parallel": false
  },
  "begin": [
    {"let": {"name": "$filePath", "expression": "$fileDirectory.append($fileName)"}},
    {"let": {"name": "$dbPath", "expression": "$dbDirectory.append($dbName)"}}
  ],
  "source": { "file": { "path": "$filePath", "lock": true }
  },
  "extractor": { 
      "csv": {}
  },
  "transformers": [{ 

    "vertex": { "class": "Person", "skipDuplicates": true},
    "edge": {
        "unresolvedLinkAction": "CREATE",
        "class": "hasCar",
        "joinFieldName": "CarMake",
        "lookup": "Car.CarMake"  
    }  
  }],
  "loader": {
    "orientdb": {
       "dbURL": "$dbPath",
       "dbType": "graph",
       "batchCommit": 1000,
       "dbAutoCreate": true,
       "dbAutoDropIfExists": true,
       "standardElementConstraints": false,
       "classes": [
         {"name": "Person", "extends": "V"}, 
         {"name": "Car", "extends": "V"},
         {"name": "hasCar", "extends": "E"}
       ], 
       "indexes": [
         {"class":"Person", "fields":["PersonName:string"], "type":"UNIQUE" },
         {"class":"Car", "fields":["CarMake:string"], "type":"UNIQUE" }
       ]
    }
  }
}

脚本运行时没有任何警告且没有错误。创建类和索引以及4 x Person记录，但不创建Car记录和hasCar边缘。

终端上的输出是：

OrientDB etl v.2.2.20 (build 76ab59e72943d0ba196188ed100c882be4315139) https://www.orientdb.com
[file] INFO Load from file /Users/johnbrogan/Downloads/orientdb-community-2.2.20/import/SimpleTest.csv
[orientdb] INFO Dropping existent database 'plocal:/Users/johnbrogan/Downloads/orientdb-community-2.2.2
0/databases/simpleTest'...
BEGIN ETL PROCESSOR
[file] INFO Reading from file /Users/johnbrogan/Downloads/orientdb-community-2.2.20/import/SimpleTest.c
sv with encoding UTF-8
Started execution with 1 worker threads
[orientdb] DEBUG - OrientDBLoader: created vertex class 'Person' extends 'V'
[orientdb] DEBUG orientdb: found 0 vertices in class 'null'
[orientdb] DEBUG - OrientDBLoader: created vertex class 'Car' extends 'V'
[orientdb] DEBUG orientdb: found 0 vertices in class 'null'
[orientdb] DEBUG - OrientDBLoader: created edge class 'hasCar' extends 'E'
[orientdb] DEBUG orientdb: found 0 vertices in class 'null'
[orientdb] DEBUG orientdb: found metadata field 'null'
[orientdb] DEBUG - OrientDBLoader: created property 'Person.PersonName' of type: string
[orientdb] DEBUG - OrientDocumentLoader: created index 'Person.PersonName' type 'UNIQUE' against Class
'Person', fields [PersonName:string]
[orientdb] DEBUG orientdb: found metadata field 'null'
[orientdb] DEBUG - OrientDBLoader: created property 'Car.CarMake' of type: string
[orientdb] DEBUG - OrientDocumentLoader: created index 'Car.CarMake' type 'UNIQUE' against Class 'Car',
 fields [CarMake:string]
Start extracting
[csv] DEBUG document={CarMake:Nissan,PersonName:John,CarAge:3,PersonAge:23}
[1:vertex] DEBUG Transformer input: {CarMake:Nissan,PersonName:John,CarAge:3,PersonAge:23}
[csv] DEBUG document={CarMake:Mini,PersonName:Peter,CarAge:25,PersonAge:45}
[csv] DEBUG document={CarMake:Morgan,PersonName:Paul,CarAge:15,PersonAge:23}
[csv] DEBUG document={CarMake:Mini,PersonName:Jim,CarAge:24,PersonAge:24}
Extraction completed
[1:vertex] DEBUG Transformer output: v(Person)[#25:0]
[2:vertex] DEBUG Transformer input: {CarMake:Mini,PersonName:Peter,CarAge:25,PersonAge:45}
[2:vertex] DEBUG Transformer output: v(Person)[#26:0]
[3:vertex] DEBUG Transformer input: {CarMake:Morgan,PersonName:Paul,CarAge:15,PersonAge:23}
[3:vertex] DEBUG Transformer output: v(Person)[#27:0]
[4:vertex] DEBUG Transformer input: {CarMake:Mini,PersonName:Jim,CarAge:24,PersonAge:24}
[4:vertex] DEBUG Transformer output: v(Person)[#28:0]
[orientdb] INFO committing
Pipeline worker done without errors: true
END ETL PROCESSOR
+ extracted 4 rows (0 rows/sec) - 4 rows -> loaded 4 vertices (0 vertices/sec) Total time: 256ms [0 war
nings, 0 errors]

看起来边缘变换没有运行，有什么想法？

Answer 1

为了任何其他有类似问题的Orientdb新手的好处回答我自己的问题。这只是一个不够{}的情况。每个变换器都需要在它们自己的{}集合中，如下所示

{
    "config": {"log": "INFO",
    "fileDirectory": "/Users/johnbrogan/Downloads/orientdb-community-2.2.20/import/",
    "fileName": "InventoryResults-05-17-2017-08h27m56s_top25.csv",
    "dbDirectory": "plocal:/Users/johnbrogan/Downloads/orientdb-community-2.2.20/databases/",
    "dbName": "msamlin-test001"
  },
  "begin": [
    {"let": {"name": "$filePath", "expression": "$fileDirectory.append($fileName)"}},
    {"let": {"name": "$dbPath", "expression": "$dbDirectory.append($dbName)"}}
  ],
  "source": { "file": { "path": "$filePath", "lock": true }
  },
  "extractor": { 
      "csv": {}
  },
  "transformers": [

    {"vertex": { "class": "Asset", "skipDuplicates": true}},
    {"edge": {
        "unresolvedLinkAction": "CREATE",
        "class": "hasStatus",
        "joinFieldName": "WMIStatus",
        "lookup": "Status.name",
        "targetVertexFields": {
            "name": "${input.WMIStatus}"
        }
    }}
    ],
  "loader": {
    "orientdb": {
       "dbURL": "$dbPath",
       "dbType": "graph",
       "batchCommit": 1000,
       "dbAutoCreate": true,
       "dbAutoDropIfExists": true,
       "classes": [
         {"name": "Asset", "extends": "V"}, 
         {"name": "Status", "extends": "V"},
         {"name": "hasStatus", "extends": "E"}
       ], 
       "indexes": [
         {"class":"Asset", "fields":["ComputerName:string"], "type":"UNIQUE" },
         {"class":"Status", "fields":["name:string"], "type":"UNIQUE" }
       ]
    }
  }
}

ETL脚本不创建边

1 个答案: