我想替换一个文件(例如IP地址)中的多个字符串,并希望对其进行标记,以便所有重复出现都被标记为相同的名称。
例如,如果这是我的文件:
2018-09-13 19:00:00,317 INFO -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,317 INFO -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,891 INFO -filters.BasicAuthFilter: Client IP:192.168.100.98
2018-09-13 19:00:01,891 INFO -filters.BasicAuthFilter: Validating token ...
2018-09-13 19:00:01,892 INFO -authentication.Tokenization: Token:192.168.100.98:20180913_183401is present in map
2018-09-13 19:00:01,892 INFO -configure.ConfigStatusCollector: status.
2018-09-13 19:00:01,909 INFO -filters.BasicAuthFilter: Client IP:192.168.100.98
2018-09-13 19:00:01,909 INFO -filters.BasicAuthFilter: Validating token ...
2018-09-13 19:00:01,910 INFO -authentication.Tokenization: Token:192.168.100.98:20180913_183401is present in map
2018-09-13 19:00:01,910 INFO -restadapter.ConfigStatusService: configuration status.
2018-09-13 19:00:01,910 INFO -configure.Collector: Getting configuration status.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Processing the ssh command execution results standard output.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Processing the ssh command execution standard error.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Remote command using SSH execution status: Host : [10.2.251.129] User : [root] Password : [***********] Command : [shell ntpdate -u 132.132.0.88] STATUS : [0]
2018-09-13 19:00:02,318 INFO -util.SSHUtil: STDOUT : [Shell access is granted to root
14 Sep 01:00:01 ntpdate[16063]: adjust time server 132.132.0.88 offset 0.353427 sec
]
2018-09-13 19:00:02,318 INFO -util.SSHUtil: STDERR : []
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Successfully executed remote command using SSH.
2018-09-13 19:00:02,318 INFO Successfully executed the command on VCenter :10.2.251.129
它应该变成:
2018-09-13 19:00:00,317 INFO -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,317 INFO -util.SSHUtil: Waiting for channel close
2018-09-13 19:00:01,891 INFO -filters.BasicAuthFilter: Client IP:IP_1
2018-09-13 19:00:01,891 INFO -filters.BasicAuthFilter: Validating token ...
2018-09-13 19:00:01,892 INFO -authentication.Tokenization: Token:IP_1:20180913_183401is present in map
2018-09-13 19:00:01,892 INFO -configure.ConfigStatusCollector: status.
2018-09-13 19:00:01,909 INFO -filters.BasicAuthFilter: Client IP:IP_1
2018-09-13 19:00:01,909 INFO -filters.BasicAuthFilter: Validating token ...
2018-09-13 19:00:01,910 INFO -authentication.Tokenization: Token:IP_1:20180913_183401is present in map
2018-09-13 19:00:01,910 INFO -restadapter.ConfigStatusService: configuration status.
2018-09-13 19:00:01,910 INFO -configure.Collector: Getting configuration status.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Processing the ssh command execution results standard output.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Processing the ssh command execution standard error.
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Remote command using SSH execution status: Host : [IP_2] User : [root] Password : [***********] Command : [shell ntpdate -u IP_3] STATUS : [0]
2018-09-13 19:00:02,318 INFO -util.SSHUtil: STDOUT : [Shell access is granted to root
14 Sep 01:00:01 ntpdate[16063]: adjust time server IP_3 offset 0.353427 sec]
2018-09-13 19:00:02,318 INFO -util.SSHUtil: STDERR : []
2018-09-13 19:00:02,318 INFO -util.SSHUtil: Successfully executedremote command using SSH.
2018-09-13 19:00:02,318 INFO Successfully executed the command on VCenter :IP_2
以下脚本实际上完成了我想要的,但是随后它是特定于文件的:
import typing, re
def change_ips(ips:typing.List[str]) -> typing.Generator[str, None, None]:
val = {}
count = 1
for i in ips:
if i not in val:
yield f'IP_{count}'
val[i] = count
count += 1
else:
yield f'IP_{val[i]}'
with open(r'server.log') as f:
content = f.read()
with open(r'logfile2.txt', 'w') as f1:
f1.write(re.sub('\d+\.\d+\.\d+\.\d+', '{}', content).format(*change_ips(re.findall('\d+\.\d+\.\d+\.\d+', content))))
这可以工作,但随后它是特定于文件的,并且不能与其他日志文件一起工作,我想以某种方式使其健壮,使得任何一行中都有IP地址的文件都无法工作,特定的日志文件。
不起作用的示例:
2018-09-15 15:58:20,083 INFO [Timer-0]-util.SSHUtil: STDERR : []
2018-09-15 15:58:20,083 INFO [Timer-0]-util.SSHUtil: Successfully executed remote command using SSH.
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
---------------------------------------------------------------------
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
Validate [33mKBDash2121 Node[0m installation BEGIN:
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
Show KBDash2121 system configuration: [33m1.1.2.371[0m
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
*****************************************************************
2018-09-15 15:58:20,090 INFO [Timer-0]-util.SSHUtil: Connecting to host [10.60.9.44] using provided credentials.
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"cis_url" : "https://localhost:441/cis/v1.1",
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"app_name" : "KBDash2121",
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"node_name" : "idpa-1-dps",
2018-09-15 15:59:40,093 ERROR [Timer-0]-dashboard.DPSDashboard: Unable to validate ssh credential.Host 10.60.9.44 is not reachable.
2018-09-15 15:59:40,093 ERROR [Timer-0]-dashboard.DPSDashboard: loadDataNodeStatus --> unable to find data node process statuscom.common.exception.ApplianceException: Host 10.60.9.44 is not reachable.
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"system_index_name" : "system",
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"worker_id" : "aWRwYS0xLWRwc3wwMDo1MDo1Njo5RDoyRDo4RSA=",
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"work_base_folder": "/mnt/KBDash2121_work",
2018-09-15 15:58:20,083 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"service_work_folder" : "tmp/dpworker",
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"web_download_folder" : "tmp/dpweb",
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"admin_api_url" : "https://localhost:448/admin_api/v1",
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
"search_api_url" : "https://localhost:449/search_api/v1",
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
*****************************************************************
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mDirectory: /usr/local/KBDash2121 has been created [0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mFile: /usr/local/KBDash2121/etc/system.conf has been created [0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mService: dpworker is on[0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mService: nginx is on[0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mProccess: WorkerService is running[0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mProccess: nginx is running[0m
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[33mchecking admin api url:https://localhost:448......
2018-09-15 15:58:20,084 INFO [Timer-0]-dashboard.KBDash: getProcessSummary --> processing output line
[32mOk: {"status":200,"name":"myspace","version":"1.1.2.371","cis":"online","tagline":"none"}[0m
2018-09-15 15:59:40,106 INFO [Timer-0]-util.SSHUtil: Connecting to host [10.60.9.59] using provided credentials.
2018-09-15 15:59:40,209 INFO [Timer-0]-util.SSHUtil: Connected to host [10.60.9.59] using provided credentials.
答案 0 :(得分:1)
您可以保留一个唯一IP地址数组,并将其在数组中的索引用作替换值。
在下面的代码中,\1
中的replace_func
是指正则表达式中的第一个匹配项。我们在数组中查找(如有必要,可以添加),对其进行正确的格式化,然后将其返回以用作下面re.sub
的替换值。
类似这样的东西:
import fileinput
import re
ips = []
def replace_func(match):
ip = match.expand(r'\1')
if ip not in ips:
ips.append(ip)
return 'IP_%s' % ips.index(ip)
with fileinput.input('server.log', inplace=True, backup='.bak') as file:
for line in file:
print(re.sub(r'(\d+\.\d+\.\d+\.\d+)', replace_func, line), end='')