如何将CSV数据上传到数据存储区

时间:2014-04-02 20:15:00

标签: java macos google-app-engine csv google-cloud-datastore

我正在尝试使用csv文件中的数据填充数据存储区实体,但没有成功。

这是我的CSV文件places.csv:

name,placeId,location,key,address
A store at City1 Shopping Center,store101,"47,-122",1,"Some address of the store in City 1"
A big store at Some Mall,store102,"47,-122",2,"Some address of the store in City 2"

bulkloader.yaml:

python_preamble:
- import: base64
- import: re
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.ext.db
- import: google.appengine.api.datastore
- import: google.appengine.api.users

transformers:

- kind: Place
  connector: csv
  connector_options:
  property_map:
    - property: __key__
      external_name: key
      export_transform: transform.key_id_or_name_as_string

    - property: address
      external_name: address
      # Type: String Stats: 6 properties of this type in this kind.

    - property: location
      external_name: location
      # Type: GeoPt Stats: 6 properties of this type in this kind.
      import_transform: google.appengine.api.datastore_types.GeoPt

    - property: name
      external_name: name
      # Type: String Stats: 6 properties of this type in this kind.

    - property: placeId
      external_name: placeId
      # Type: String Stats: 6 properties of this type in this kind

upload_data.sh:

#!/bin/sh
../Eclipse/plugins/com.google.appengine.eclipse.sdkbundle_1.9.1/appengine-java-sdk-1.9.1/bin/appcfg.sh upload_data --config_file bulkloader.yaml --url=http://localhost:8888/remote_api --filename places.csv --kind=Place -e nobody@nowhere.com

我创建了文件夹gae并放置了upload_data.shbulkloader.yamlplaces.csv

运行sudo ./upload_data.sh后,收到消息:

sudo: ./upload_data.sh: command not found

运行sudo sh upload_data.sh后,我收到以下错误:

    Bad argument: Expected an action: [update, request_logs, rollback, update_indexes, update_cron, update_dispatch, update_dos, update_queues, cron_info, vacuum_indexes, help, download_app, version, set_default_version, resource_limits_info, start_module_version, stop_module_version, backends list, backends rollback, backends update, backends start, backends stop, backends delete, backends configure, backends, list_versions, delete_version, debug]
usage: AppCfg [options] <action> [<app-dir>] [<argument>]

Action must be one of:
  help: Print help for a specific action.
  download_app: Download a previously uploaded app version.
  request_logs: Write request logs in Apache common log format.
  rollback: Rollback an in-progress update.
  start_module_version: Start the specified module version.
  stop_module_version: Stop the specified module version.
  update: Create or update an app version.
  update_indexes: Update application indexes.
  update_cron: Update application cron jobs.
  update_queues: Update application task queue definitions.
  update_dispatch: Update the application dispatch configuration.
  update_dos: Update application DoS protection configuration.
  version: Prints version information.
  set_default_version: Set the default serving version.
  cron_info: Displays times for the next several runs of each cron job.
  resource_limits_info: Display resource limits.
  vacuum_indexes: Delete unused indexes from application.
  backends list: List the currently configured backends.
  backends update: Update the specified backend or all backends.
  backends rollback: Roll back a previously in-progress update.
  backends start: Start the specified backend.
  backends stop: Stop the specified backend.
  backends delete: Delete the specified backend.
  backends configure: Configure the specified backend.
  list_versions: List the currently uploaded versions.
  delete_version: Delete the specified version.
Use 'help <action>' for a detailed description.

options:
  -s SERVER, --server=SERVER
                        The server to connect to.
  -e EMAIL, --email=EMAIL
                        The username to use. Will prompt if omitted.
  -H HOST, --host=HOST  Overrides the Host header sent with all RPCs.
  -p PROXYHOST[:PORT], --proxy=PROXYHOST[:PORT]
                        Proxies requests through the given proxy server.
                        If --proxy_https is also set, only HTTP will be
                        proxied here, otherwise both HTTP and HTTPS will.
  --proxy_https=PROXYHOST[:PORT]
                        Proxies HTTPS requests through the given proxy server.
  --no_cookies          Do not save/load access credentials to/from disk.
  --sdk_root=root       Overrides where the SDK is located.
  --passin              Always read the login password from stdin.
  -A APP_ID, --application=APP_ID
                        Override application id from appengine-web.xml or app.yaml
  -M MODULE, --module=MODULE
                        Override module from appengine-web.xml or app.yaml
  -V VERSION, --version=VERSION
                        Override (major) version from appengine-web.xml or app.yaml
  --oauth2              Use OAuth2 instead of password auth.
  --enable_jar_splitting
                        Split large jar files (> 10M) into smaller fragments.
  --jar_splitting_excludes=SUFFIXES
                        When --enable-jar-splitting is set, files that match
                        the list of comma separated SUFFIXES will be excluded
                        from all jars.
  --disable_jar_jsps
                        Do not jar the classes generated from JSPs.
  --enable_jar_classes
                        Jar the WEB-INF/classes content.
  --delete_jsps
                        Delete the JSP source files after compilation.
  --retain_upload_dir
                        Do not delete temporary (staging) directory used in
                        uploading.
  --compile_encoding
                        The character encoding to use when compiling JSPs.
  -n NUM_DAYS, --num_days=NUM_DAYS
                        Number of days worth of log data to get. The cut-off
                        point is midnight UTC. Use 0 to get all available
                        logs. Default is 1.
  --severity=SEVERITY   Severity of app-level log messages to get. The range
                        is 0 (DEBUG) through 4 (CRITICAL). If omitted, only
                        request logs are returned.
  -a, --append          Append to existing file.
  -n NUM_RUNS, --num_runs=NUM_RUNS
                        Number of scheduled execution times to compute
  -f, --force           Force deletion of indexes without being prompted.

如何将数据上传到数据存储区?谢谢。

2 个答案:

答案 0 :(得分:0)

我认为你使用的是appcfg.sh而不是appcfg.py。参见:

https://developers.google.com/appengine/docs/python/tools/uploadingdata

此外,您的输出清楚地显示了您收到Bad Argument错误的原因 - appcfg.sh列出的操作参数不包括&#34; update_data&#34;,但这是您的脚本作为操作传递的内容。

答案 1 :(得分:0)

我正在做这件事并且没有立即实现直觉的飞跃:

下载python SDK,它将为您提供appcfg.py工具。只需在upload_data.sh脚本中调用该文件即可。

appcfg.sh程序没有upload_data操作。我觉得很奇怪。