Question

我需要在ruby上运行一个ruby程序，它将数据从Postgres DB导出到CSV文件。该程序将每天运行并处理数百万行数据，因此我将该命令放在SQL中。

query = "copy (select plays.* from plays inner join games on games.id=plays.game_id where games.league=3 and games.date >= '1995-01-01' and games.date <= '2017-07-23') to '/home/stuff9000/sports/public/processed_data/ncaaf_plays.csv' with csv header"

connection = ActiveRecord::Base.connection
connection.execute(query)

这会产生以下错误

ERROR:  must be superuser to COPY to or from a file (ActiveRecord::StatementInvalid)
HINT:  Anyone can COPY to stdout or from stdin. psql's \copy command also works for anyone.

所以我将查询命令更改为

query = "\\copy (select plays.* from plays inner join games on games.id=plays.game_id where games.league=3 and games.date >= '1995-01-01' and games.date <= '2017-07-23') to '/home/stuff9000/sports/public/processed_data/ncaaf_plays.csv' with csv header"

我得到了

syntax error at or near "\" (ActiveRecord::StatementInvalid)
LINE 1: \copy (select plays.* from plays inner join games on games.i...
        ^

那么我该如何让它发挥作用呢？

Answer 1

我无法弄清楚如何使用copy命令执行此操作。我将copy命令调整为指向STDOUT而不是写入文件。然后让ruby处理STDOUT来归档。结果一目了然。

query = "COPY (select plays.* from plays inner join games on games.id=plays.game_id where games.league=3 and games.date >= '1995-01-01' and games.date <= '2017-07-23') to STDOUT with csv header;"
destination = "/"

csv = []

raw_connection.copy_data(copy_data) do
  while row = raw_connection.get_copy_data
    csv.push(row)
  end
end

sql_data_as_string = csv.join("")

File.open(destination, "w") {|file| file.write(sql_data_as_string.force_encoding("UTF-8"))

Ruby Rails - 从Postgres复制到CSV

1 个答案: