Question

我正在尝试更新/etc/presto/conf/catalog/hive.properties中的hive连接器属性。我添加了以下内容：

hive.s3.max-client-retries=50
hive.s3.max-error-retries=50
hive.s3.max-connections=500
hive.s3.connect-timeout=5m
hive.s3.socket-timeout=5m

但由于某些原因，这些都不被接受。 presto服务器重新启动失败，并显示以下错误：

2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   UNUSED PROPERTIES
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.connect-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-client-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-connections
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.max-error-retries
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   hive.s3.socket-timeout
2018-04-16T19:34:26.369Z    WARN    main    Bootstrap   
2018-04-16T19:34:26.587Z    ERROR   main    com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:

1) Configuration property 'hive.s3.connect-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

2) Configuration property 'hive.s3.max-client-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

3) Configuration property 'hive.s3.max-connections' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

4) Configuration property 'hive.s3.max-error-retries' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

5) Configuration property 'hive.s3.socket-timeout' was not used
  at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:234)

5 errors

Answer 1

当hive.s3-file-system-type设置为PRESTO时，这些属性适用。当它是例如。 EMRFS，这些配置属性不适用，设置它们会导致您观察到的错误。

Answer 2

正如Piotr所说，EMR将默认的FileSystem实现从emr-5.12.0开始从PrestoS3FileSystem更改为EmrFS。 PrestoS3FileSystem属性将不可用，但您可以像配置任何其他应用程序一样配置EmrFS。有关详细信息，请参阅https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-presto-considerations.html上的“EMRFS和PrestoS3FileSystem配置”。

在EMR中为Presto设置配置单元配置属性

2 个答案: