Question

我正在使用Terraform创建一个使用GCP cloudsql实例作为hivemetastore的dataproc集群，terrafrm项目创建了该集群及其所有先决条件（网络，服务帐户，cloudsql实例和用户等）。
提供了cloud-sql-proxy.sh来协助解决此问题，但是在创建集群时cloud-sql-proxy.sh失败并出现错误：

nc：连接到本地主机端口3306（tcp）失败：连接被拒绝

我把头撞在墙上，试图找出原因，但无法深入了解它，所以希望这里有人可以提供帮助。

我在https://github.com/jamiekt/democratising-dataproc托管了terraform项目。重现问题非常容易，请按照下列步骤操作：

Install terraform（如果您还没有）
如果尚未安装gcloud
创建一个新的GCP项目
为新项目启用Cloud Dataproc API
gcloud auth application-default login #creates a file containing credentials that terraform will use
git clone git@github.com:jamiekt/democratising-dataproc.git && cd democratising-dataproc
export GCP_PROJECT=name-of-project-you-just-created
make init
make apply

这应该成功启动网络，子网，cloudsql实例，几个存储桶（其中一个包含cloud-sql-proxy.sh），一个服务帐户，一个防火墙，然后在尝试创建dataproc集群时失败

如果有人可以看看并告诉我为什么失败了，我将非常感激。

Answer 1

似乎您没有在cloud-sql-proxy.sh模板（cloud-sql-proxy.sh.tmpl）中使用最新版本的diff脚本。

您可能想尝试使用Dataproc Cloud SQL I/O and Hive Metastore初始化操作中的最新脚本版本来更新模板。

Answer 2

There were a number of problems here that have now been solved:

service_account_scopes needed specifying and hive:hive.metastore.warehouse.dir property needed setting
Terraform removes the default root user from MySQL so it has to be recreated. I was missing (amongst other things) the host attribute (host = '%')
various other things

The state of the repo at the time of posting this message will work as intended (i.e. create, using Terraform, a dataproc cluster that uses a shared hive metastore).

Thank you @igor-dvorzhak for your responses, your link to the article on configuring Hive Metastore to use Cloud SQL put me on the right track..

Answer 3

根据here

的教程，我遇到了同样的问题

我看不到您提供的解决方案是什么。似乎蜂巢试图通过本地主机访问数据库我不明白为什么这是dataproc的日志：

Synchronizing state of mysql.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install disable mysql
Removed /etc/systemd/system/multi-user.target.wants/mysql.service.
2020-10-07 14:05:49 URL:https://dl.google.com/cloudsql/cloud_sql_proxy.linux.amd64 [16361206/16361206] -> "cloud_sql_proxy.linux.amd64" [1]
Created symlink /etc/systemd/system/multi-user.target.wants/cloud-sql-proxy.service → /usr/lib/systemd/system/cloud-sql-proxy.service.
About to run 'nc -zv localhost 3306' with retries...
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 1 failed! Sleeping 1.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 2 failed! Sleeping 1.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 3 failed! Sleeping 2.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 4 failed! Sleeping 3.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 5 failed! Sleeping 5.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 6 failed! Sleeping 8.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 7 failed! Sleeping 13.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 8 failed! Sleeping 21.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 9 failed! Sleeping 34.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 10 failed! Sleeping 55.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 11 failed! Sleeping 89.
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused
'nc -zv localhost 3306' attempt 12 failed! Sleeping 144.
Final attempt of 'nc -zv localhost 3306'...
nc: connect to localhost port 3306 (tcp) failed: Connection refused
nc: connect to localhost port 3306 (tcp) failed: Connection refused

GCP Dataproc的cloud-sql-proxy失败，nc：连接到本地主机端口3306（tcp）失败：连接被拒绝

3 个答案: