Question

我需要提取从S3中提取的AWS cloudtrail日志。这些文件包含单个json负载，其中包含单个cloudtrail事件。但是，splunk无法识别单个事件并且无法正确拆分。它只是一个大事件而已

每个文件都包含以下格式：

  "Records": [
    {
      "apiVersion": "2012-06-01",
      "awsRegion": "us-west-1",
      "eventID": "c-c245-2c4-32v6-vfff",
      "eventName": "DescribeLoadBalancers",
      "eventSource": "elasticloadbalancing.amazonaws.com",
      "eventTime": "2019-11-30T18:15:33Z",
      "eventType": "AwsApiCall",
      "eventVersion": "1.05",
      "recipientAccountId": "redacted",
      "requestID": "2xc454xc-2345-234cv5-2345",
      "requestParameters": null,
      "responseElements": null,
      "sourceIPAddress": "1.1.1.1",
      "userAgent": "aws-sdk-ruby3/3.75.0 jruby/2.3.3 java aws-sdk-elasticloadbalancing/1.19.0 cloudhealth",
      "userIdentity": {
        "accessKeyId": "redacted",
        "accountId": "redacted",
        "arn": "arn:aws:sts::redacted:assumed-role/team/AssumeRoleSession",
        "principalId": "redacted:AssumeRoleSession",
        "sessionContext": {
          "attributes": {
            "creationDate": "2019-11-30T17:45:06Z",
            "mfaAuthenticated": "false"
          },
          "sessionIssuer": {
            "accountId": "redacted",
            "arn": "arn:aws:iam::redacted:team/company",
            "principalId": "redacted",
            "type": "Role",
            "userName": "redacted"
          },
          "webIdFederationData": {}
        },
        "type": "AssumedRole"
      }
    },{
      "apiVersion": "2012-06-01",
      "awsRegion": "us-west-1",
      "eventID": "c-c245-2c4-32v6-vfff",
      "eventName": "DescribeLoadBalancers",
      "eventSource": "elasticloadbalancing.amazonaws.com",
      "eventTime": "2019-11-30T18:16:33Z",
      "eventType": "AwsApiCall",
      "eventVersion": "1.05",
      "recipientAccountId": "redacted",
      "requestID": "2xc454xc-2345-234cv5-2345",
      "requestParameters": null,
      "responseElements": null,
      "sourceIPAddress": "1.1.1.1",
      "userAgent": "aws-sdk-ruby3/3.75.0 jruby/2.3.3 java aws-sdk-elasticloadbalancing/1.19.0 cloudhealth",
      "userIdentity": {
        "accessKeyId": "redacted",
        "accountId": "redacted",
        "arn": "arn:aws:sts::redacted:assumed-role/team/AssumeRoleSession",
        "principalId": "redacted:AssumeRoleSession",
        "sessionContext": {
          "attributes": {
            "creationDate": "2019-11-30T17:45:06Z",
            "mfaAuthenticated": "false"
          },
          "sessionIssuer": {
            "accountId": "redacted",
            "arn": "arn:aws:iam::redacted:role/team",
            "principalId": "redacted",
            "type": "Role",
            "userName": "redacted"
          },
          "webIdFederationData": {}
        },
        "type": "AssumedRole"
      }
    }
  ]
}

我的道具看起来像这样

[cloudtrail]
KV_MODE = json

Answer 1

进一步的谷歌搜索和反复试验导致该道具配置似乎正确地打破了事件

[cloudtrail]
KV_MODE = json
SHOULD_LINEMERGE=false
LINE_BREAKER=((?<=}),(?={)|[\r\n]+)
SEDCMD-remove_prefix=s/{"Records":\[//g
SEDCMD-remove_suffix=s/\]}//g

适用于AWS Cloudtrail JSON日志的Splunk道具配置

1 个答案: