Question

CREATE TABLE `cluster_diagnostic_report`(
  `run_id` string COMMENT 'format: <hostname>_<datetime> - to uniquely identify the a particular execution instance of Cluster Diag job',
  `execution_hostname` string COMMENT 'Machine Name from where Test Case Executed',
  `module` string COMMENT 'Test Case Module',
  `expected_result` string COMMENT 'Test Case Module expected Result',
  `actual_result` string COMMENT 'Test Case Module actual Result',
  `validation_result` string COMMENT 'Test Case Module validation Result',
  `start_time` string COMMENT 'Test Case Module Start Time',
  `end_time` string COMMENT 'Test Case Module Elapsed Time',
  `elapsed_time` string COMMENT 'from deserializer',
  `total_time_seconds` int COMMENT 'total elapsed time for this step')
PARTITIONED BY (
  `cluster_name` string,
  `rptg_dt` string)
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'

上面的内容

仅需要获取分区列的名称和类型。对于上面的示例，我想获得以下详细信息：

col_name = cluster_name, type = string
rptg_dt= cluster_name, type = string

我尝试过的东西在下面给出，但没有返回：

partitionResult = re.match(r"PARTITIONED\s\w+\s\((\n){2}",line)
if partitionResult == None:
    pass
else:
    print(partitionResult.group(1),sep='\t')

有人可以建议做什么吗？

Answer 1

这是一个使用\G（从开始或上一个匹配继续）来匹配任意数量的群集列/类型的解决方案：

Online Test（需要在PCRE中运行）

示例代码（需要Python的替代regex软件包）

import regex as re

regex = r"(?|PARTITIONED\s+BY\s+\(\s+`(\w+)`\s+(\w+),?|\G\s*`(\w+)`\s+(\w+),?)\K"

test_str = ("CREATE TABLE `cluster_diagnostic_report`(\n"
    "  `run_id` string COMMENT 'format: <hostname>_<datetime> - to uniquely identify the a particular execution instance of Cluster Diag job',\n"
    "  `execution_hostname` string COMMENT 'Machine Name from where Test Case Executed',\n"
    "  `module` string COMMENT 'Test Case Module',\n"
    "  `expected_result` string COMMENT 'Test Case Module expected Result',\n"
    "  `actual_result` string COMMENT 'Test Case Module actual Result',\n"
    "  `validation_result` string COMMENT 'Test Case Module validation Result',\n"
    "  `start_time` string COMMENT 'Test Case Module Start Time',\n"
    "  `end_time` string COMMENT 'Test Case Module Elapsed Time',\n"
    "  `elapsed_time` string COMMENT 'from deserializer',\n"
    "  `total_time_seconds` int COMMENT 'total elapsed time for this step')\n"
    "PARTITIONED BY (\n"
    "  `cluster_name` string,\n"
    "   `cluster_name2` string,`rptg_dt` string,\n"
    "`cluster_name2` string,)\n"
    "ROW FORMAT SERDE\n"
    "  'org.apache.hadoop.hive.ql.io.orc.OrcSerde'")

matches = re.finditer(regex, test_str, re.MULTILINE)

for matchNum, match in enumerate(matches):    
    for groupNum in range(0, len(match.groups())):
        groupNum = groupNum + 1

        print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))

输出：

Group 1 found at 789-801: cluster_name
Group 2 found at 803-809: string
Group 1 found at 813-826: cluster_name2
Group 2 found at 828-834: string
Group 1 found at 836-843: rptg_dt
Group 2 found at 845-851: string
Group 1 found at 854-867: cluster_name2
Group 2 found at 869-875: string

Answer 2

善用正面的眼光，望着(Example)的后方：

pat = re.compile(r'(?<=PARTITIONED BY \(\n)(.*)(?=\))', re.S)
# Look behind for "PARTITIONED BY (" and look ahead to ")"
# use the re.S flag to match across lines with .*

results = pat.search(text).group()

# '  `cluster_name` string,\n  `rptg_dt` string'

# ... do what you need with the strings.

[i.lstrip() for i in results.split('\n')]
# ['`cluster_name` string,', 
#  '`rptg_dt` string']

python使用正则表达式基于前一行读取后n行

2 个答案: