将日志数据转换为具有所需格式的csv文件

时间:2018-07-04 05:33:23

标签: windows batch-file


我的日志数据为:

Name:Mark
City:London
Country:UK

Name:Ben
City:Paris
Country:France

Name:Tom
City:Athens
Country:Greece

我需要以以下格式进行CSV输出:

Name   City      Country
Mark   London    UK
Ben    Paris     France
Tom    Athens    Greece

为此创建的批处理很简单,可以转换为CSV。 如下:

@echo off

cd /d %~dp0
set infilenm=abc.log
set outfilenm=abc.csv
set beforestr=
set afterstr=, 

type nul >%outfilenm%

setlocal enabledelayedexpansion

for /f "tokens=1,2,3 delims=" %%A in (%infilenm%) do (   
    set line=%%A      
    echo !line:%beforestr%=%afterstr%!>>%outfilenm%
)
endlocal

exit /b

由于我是批处理脚本的新手,因此任何人都可以帮我这个忙!

4 个答案:

答案 0 :(得分:4)

您的脚本逻辑错误; for /F会连续读取一行,因此您必须先收集三行的数据,然后再写入一行输出。

以下是如何完成任务的示例,而不是使用input redirection (<)set /P来读取日志文件,而不是使用for /F

@echo off
setlocal EnableDelayedExpansion
for /F %%C in ('^< "abc.log" find /C /V ""') do set /A "COUNT=(%%C+1)/2"
set "FIRST=#"
< "abc.log" > "abc.csv" (
    for /L %%I in (1,1,%COUNT%) do (
        set "LINE1=" & set /P LINE1=""
        if defined LINE1 (
            set "LINE2=" & set /P LINE2=""
            set "LINE3=" & set /P LINE3=""
            if defined FIRST (
                echo Name,City,Country
                set "FIRST="
            )
            echo(!LINE1:*:=!,!LINE2:*:=!,!LINE3:*:=!
        )
    )
)
endlocal

这取决于您的日志文件显示的格式,因此它不会验证留给冒号的字符串。


这是基于上述方法的一种更灵活的方法,但是它通过字段值的名称来收集字段值,这些字段值保存在预定义的可配置列表(常数_LIST)中。一个或多个空行完成返回的行。如果在日志文件的当前处理的块中找不到某个字段名称,则其返回的CSV字段为空。这是代码:

@echo off
setlocal EnableExtensions EnableDelayedExpansion

rem // Define constants here:
set "_INPUT=abc.log"  & rem // (log file to process)
set "_OUTPUT=abc.csv" & rem // (CSV file to return)
set "_LIST=Name,City,Country" & rem /* (comma-separated list of field names, which must
                                rem     not contain any of the following characters:
                                rem     `:`, `,`, `*`, `?`, `<`, `>`, `!`, `"`, `=`) */
set "_SEPARATOR=,"    & rem /* (separator character to be used; the default is `,`;
                        rem     the following separator characters are forbidden:
                        rem     `!`, `^`, `&`, `(`, `)`, `<`, `>`, `|`) */
set "_QUOTED=#"       & rem // (if not empty, defines to quote the returned items)
set "_HEADER=#"       & rem // (if not empty, defines to write a header row)

set "_SEPARATOR=!_SEPARATOR!," & set "_SEPARATOR=!_SEPARATOR:~,1!"
if not defined _QUOTED (set "QUOTE=") else set "QUOTE="^" & rem/^"
for /F "delims==" %%D in ('2^> nul set $ARRAY[') do set "%%D="
for /F %%C in ('^< "abc.log" find /C /V ""') do set /A "COUNT=%%C+1"
< "abc.log" > "abc.csv" (
    set "FLAG=" & if defined _HEADER if defined _LIST (
        echo(%QUOTE%!_LIST:,=%QUOTE%%_SEPARATOR%%QUOTE%!%QUOTE%
    ) else echo(%QUOTE%%QUOTE%
    for /L %%I in (1,1,%COUNT%) do (
        set "LINE=" & set /P LINE=""
        if defined LINE (
            for /F "delims=: eol=:" %%J in ("!LINE!") do set "$ARRAY[%%J]=!LINE:*:=!"
            set "FLAG=#"
        ) else (
            if defined FLAG if defined _LIST (
                set "COLL=" & for %%J in ("!_LIST:,=","!") do (
                    set "COLL=!COLL!%_SEPARATOR%%QUOTE%!$ARRAY[%%~J]!%QUOTE%"
                    set "$ARRAY[%%~J]="
                )
                echo(!COLL:~1!
            ) else echo(%QUOTE%%QUOTE%
            set "FLAG="
        )
    )
)
endlocal
exit /B

此脚本以某种数组$ARRAY[]的形式收集列表项,其索引为字段名称,因此将字符串保留到日志文件块中每一行的(第一)冒号,以及其元素值是(第一个)冒号右边的字符串,可能看起来像这样(相对于示例日志数据的第一个块):

$ARRAY[Name]=Mark
$ARRAY[City]=London
$ARRAY[Country]=UK

答案 1 :(得分:1)

@echo off
setlocal

set "output=abc.csv"
2> "%output%" echo.

set "line=Name,City,Country"
call :write

for /f "tokens=1,* delims=:" %%A in (abc.log) do call :append %%A %%B
exit /b

:append
setlocal
set  "key=%~1"
set  "value=%~2"
endlocal & (
    if /i "%key%" == "Name" set "line=%value%"
    if /i "%key%" == "City" set "line=%line%,%value%"
    if /i "%key%" == "Country" set "line=%line%,%value%"& call :write
)
exit /b

:write
setlocal
for /f "tokens=1-3 delims=," %%A in ("%line%") do (
    set "a=%%~A          "
    set "b=%%~B          "
    set "c=%%~C          "
)
>> "%output%" echo %a:~,10% %b:~,10% %c:~,10%
set "line="
exit /b

首先将标头设置为 名为line的变量,并调用标签:write进行格式化 并写入csv输出文件。

for循环使用标记:1,*分隔每一行 在:之前获得第一个令牌,而第二个令牌作为 :之后的余数。它称为标签:append 根据第一个标记连接行。如果 令牌等于Country,然后调用标签:write 格式化行并将其写入csv输出文件。

答案 2 :(得分:1)

您的问题在几点上不清楚,因此我们只能猜测...

tagName[name^='Ui_u']

使用此 logData.txt

datalist = list()

datalist[[1]] <- formtype
datalist[[2]] <- cl
datalist[[3]] <- date


data.out = as.data.frame(do.call(cbind,datalist))

这是输出:

@echo off
setlocal EnableDelayedExpansion

rem Put here the width of the output columns
set "width=10"

set "spaces="
for /L %%i in (1,1,%width%) do set "spaces= !spaces!"
set "head=" & "out=" & set "output="
for /F "tokens=1-3 delims=:" %%a in ('findstr /N "^" logData.txt') do (
   if "%%b" neq "" (
      if not defined output (
         set "col=%%b%spaces%"
         set "head=!head!!col:~0,%width%!"
         set "out=!out!^!%%b:~0,%width%^!"
      )
      set "%%b=%%c%spaces%"
   ) else (
      if not defined output (
         echo !head!
         set "output=!out!"
      )
      for /F %%o in ("!output!") do echo %%o
      for %%a in (!head!) do set "%%a=%spaces%"
   )
)

该程序要求第一组数据包括 all 列,最后一组数据后跟一个空行...

答案 3 :(得分:1)

PowerShell解决方案,它不关心地址属性的数量。
它唯一需要的常量是用空行分隔地址和
property:value

之间的冒号

如果需要,可以从批处理中调用它(更多关于主题)

  • 它使用正则表达式将其拆分为多个部分 (地址),
    将每个部分分成几行,然后将每一部分 排成属性和价值。
  • 它将带有值的属性插入每个新地址
  • 在结果表中缺少属性的调整由PowerShell automagically完成
  • 显示为具有由Format-Table自动检测到的列宽的表

## Q:\Test\2018\07\04\SO_51166380.ps1
$InputFile = '.\abc.log'
$OutputFile= '.\abc.csv'

$Sections = ((Get-Content $InputFile -Raw) -split "`r?`n *`r?`n" -ne '')

$Csv = ForEach($Section in $Sections){
    $Address = New-Object PSCustomObject
    ForEach($PropVal in ($Section -Split "`r?`n" -ne '')){
        $Prop,$Val = $PropVal.Split(':',2)
        Add-Member -InputObject $Address `
                   -NotePropertyName $Prop `
                   -NotePropertyValue $Val
    }
    $Address
}
$Csv | Format-Table -Auto
$Csv | Export-Csv $OutputFile -NoTypeInformation

使用修改后的abc.log输出示例

> type abc.log
Name:Mark
City:London
Country:UK
LastName:Anonymus

Name:Ben
Country:France

Name:Tom
City:Athens

Name:Antonio
City:Mexico
Country:Mexico

> .\SO_51166380.ps1

Name    City   Country LastName
----    ----   ------- --------
Mark    London UK      Anonymus
Ben            France
Tom     Athens
Antonio Mexico Mexico

> type .\abc.csv
"Name","City","Country","LastName"
"Mark","London","UK","Anonymus"
"Ben",,"France",
"Tom","Athens",,
"Antonio","Mexico","Mexico",