无法使用Spark-Submit连接Hive mysql Metastore

时间:2019-06-01 18:54:48

标签: mysql hive spark-submit

当我尝试连接Hive Mysql metastore时,由于连接被拒绝而引发错误。下面给出了pom.xml,hive-site.xml和代码中的详细信息

    SELECT DISTINCT 
    `media`.`id`,`media`.`description`, 
    `keywords`.`id` AS `keywords.id`, 
    `keywords->media_keywords`.`id` AS `keywords.media_keywords.id`, 
    `keywords->media_keywords`.`media_id` AS `keywords.media_keywords.media_id`, 
    `keywords->media_keywords`.`keyword_id` AS `keywords.media_keywords.keyword_id` 

    FROM database.media 

    LEFT OUTER JOIN 
    ( 
        `media_keywords` AS `keywords->media_keywords` 

        INNER JOIN `keywords` AS `keywords` 
        ON `keywords`.`id` = `keywords->media_keywords`.`keyword_id`
    ) 

    ON `media`.`id` = `keywords->media_keywords`.`media_id` 

    WHERE 
    ( 
            (`media`.`description` LIKE '%dog%' ) 

            OR `keywords`.`id` IN (100) 
    )

    AND NOT 
    ( 
            (`media`.`description` LIKE '%cat%' ) 

            OR `keywords`.`id` IN (400,500)
    )

    GROUP BY `media`.`id` ;

1 个答案:

答案 0 :(得分:0)

可能有两个原因:

  1. 您的配置单元metastore似乎已被锁定,spark无法访问。您需要从元存储中删除锁定文件。

    rm metastore_db/*.lck
    
  2. 此外,您还需要检查Metastore服务器是否已启动并正在运行。尝试以下代码:

    ps -ef | grep metastore
    

如果没有,您可以尝试查看日志:/var/log/hive/hivemetastore.log

希望这会有所帮助:)