删除与模式匹配的文本块

时间:2018-01-23 02:49:26

标签: shell awk sed text-processing

我有一个难题,我想从一个有特殊字的文本中删除一些块,这里是一个例子,我想要的是删除匹配backup_all和匹配行前的一行,以及匹配行后的三行< / p>

# Time: 2018-01-23T03:41:41.454104+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3168695
# Query_time: 0.129250  Lock_time: 0.000062 Rows_sent: 3535  Rows_examined: 3535
SET timestamp=1516650101;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `legend_gather_customer_note_effect`;
# Time: 2018-01-23T03:41:41.527587+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3168695
# Query_time: 0.066378  Lock_time: 0.000059 Rows_sent: 193  Rows_examined: 193
SET timestamp=1516650101;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `legend_gather_performance_config`;
# Time: 2018-01-23T03:41:41.558254+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3168695
# Query_time: 0.025533  Lock_time: 0.000058 Rows_sent: 296  Rows_examined: 296
SET timestamp=1516650101;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `legend_gift`;
# Time: 2018-01-23T03:42:09.536559+08:00
# User@Host: zabbix_agent[zabbix_agent] @  [127.0.0.1]  Id: 3169056
# Query_time: 0.000304  Lock_time: 0.000162 Rows_sent: 1  Rows_examined: 1
SET timestamp=1516650129;
SELECT SUM(trx_rows_locked) AS rows_locked, SUM(trx_rows_modified) AS rows_modified, SUM(trx_lock_memory_bytes) AS lock_memory FROM information_schema.INNODB_TRX;

删除后,文本成为以下输出,如何使用sed来实现这个目标?

# Time: 2018-01-23T03:42:09.536559+08:00
# User@Host: zabbix_agent[zabbix_agent] @  [127.0.0.1]  Id: 3169056
# Query_time: 0.000304  Lock_time: 0.000162 Rows_sent: 1  Rows_examined: 1
SET timestamp=1516650129;
SELECT SUM(trx_rows_locked) AS rows_locked, SUM(trx_rows_modified) AS rows_modified, SUM(trx_lock_memory_bytes) AS lock_memory FROM information_schema.INNODB_TRX;

更新

这个文本来自mysql慢日志,我想从用户'backup_all'中删除所有sql信息,如你所见,用户'backup_all'的这个文本产品肯定是常规的,但是其他用户的其他文本产品可能不一样,他们会

# Time: 2018-01-23T03:41:24.490723+08:00
# User@Host: zabbix_agent[zabbix_agent] @  [127.0.0.1]  Id: 3169038
# Query_time: 0.000669  Lock_time: 0.000334 Rows_sent: 1  Rows_examined: 27
SET timestamp=1516650084;
select count(*) Slownum from information_schema.processlist where COMMAND = 'Query' and info not like '%information_schema.processlist%' and TIME > 0;
# Time: 2018-01-23T03:41:40.628284+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3168695
# Query_time: 78.333179  Lock_time: 0.000073 Rows_sent: 13269064  Rows_examined: 13269064
SET timestamp=1516650100;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `legend_final_inventory`;
# Time: 2018-01-23T03:41:40.925596+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3168695
# Query_time: 0.175956  Lock_time: 0.000065 Rows_sent: 5101  Rows_examined: 5101
SET timestamp=1516650100;
SELECT /*!40001 SQL_NO_CACHE */ * FROM `legend_finance_account`;

# Time: 2018-01-23T04:29:26.903048+08:00
# User@Host: yun_rw_legend[yun_rw_legend] @  [10.162.86.162]  Id: 3167670
# Query_time: 0.000150  Lock_time: 0.000053 Rows_sent: 6  Rows_examined: 32
SET timestamp=1516652966;
select id, value from legend_precheck_value where value_type = 8 and is_deleted = 'N';
# Time: 2018-01-23T04:29:31.825823+08:00
# User@Host: yun_rw_legend[yun_rw_legend] @  [10.162.86.162]  Id: 3167670
# Query_time: 0.000826  Lock_time: 0.000146 Rows_sent: 0  Rows_examined: 947
SET timestamp=1516652971;
select

         id as id,
        is_deleted as isDeleted,
        gmt_create as gmtCreate,
        creator as creator,
        gmt_modified as gmtModified,
        modifier as modifier,

            shop_id as shopId,
            customer_id as customerId,
            account_id as accountId,
            customer_car_id as customerCarId

        from legend_customer_car_rel

         WHERE is_deleted = 'N'




                   and shop_id = 3374




                    and customer_car_id = 1307177;
# Time: 2018-01-23T04:30:01.149529+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3170398
# Query_time: 0.003047  Lock_time: 0.000169 Rows_sent: 0  Rows_examined: 385
SET timestamp=1516653001;
SELECT LOGFILE_GROUP_NAME, FILE_NAME, TOTAL_EXTENTS, INITIAL_SIZE, ENGINE, EXTRA FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'UNDO LOG' AND FILE_NAME IS NOT NULL AND LOGFILE_GROUP_NAME IS NOT NULL GROUP BY LOGFILE_GROUP_NAME, FILE_NAME, ENGINE, TOTAL_EXTENTS, INITIAL_SIZE, EXTRA ORDER BY LOGFILE_GROUP_NAME;
# Time: 2018-01-23T04:30:01.151783+08:00
# User@Host: backup_all[backup_all] @  [127.0.0.1]  Id: 3170398
# Query_time: 0.002082  Lock_time: 0.000119 Rows_sent: 0  Rows_examined: 385
SET timestamp=1516653001;
SELECT DISTINCT TABLESPACE_NAME, FILE_NAME, LOGFILE_GROUP_NAME, EXTENT_SIZE, INITIAL_SIZE, ENGINE FROM INFORMATION_SCHEMA.FILES WHERE FILE_TYPE = 'DATAFILE' ORDER BY TABLESPACE_NAME, LOGFILE_GROUP_NAME;

2 个答案:

答案 0 :(得分:1)

似乎有效的强制性awk解决方案:

awk '
    $2!~/backup_all/ {
        printf "# Time%s", $0
    }' RS='# Time' ORS='' FS='
' example_in

它将输入拆分为每个# Time的记录和\n的字段,以将每行作为字段。如果第二行(字段)中没有backup_allprint

第一个例子的输出:

# Time: 2018-01-23T03:42:09.536559+08:00
# User@Host: zabbix_agent[zabbix_agent] @  [127.0.0.1]  Id: 3169056
# Query_time: 0.000304  Lock_time: 0.000162 Rows_sent: 1  Rows_examined: 1
SET timestamp=1516650129;
SELECT SUM(trx_rows_locked) AS rows_locked, SUM(trx_rows_modified) AS rows_modified, SUM(trx_lock_memory_bytes) AS lock_memory FROM information_schema.INNODB_TRX

和第二个:

# Time: 2018-01-23T04:29:26.903048+08:00
# User@Host: yun_rw_legend[yun_rw_legend] @  [10.162.86.162]  Id: 3167670
# Query_time: 0.000150  Lock_time: 0.000053 Rows_sent: 6  Rows_examined: 32
SET timestamp=1516652966;
select id, value from legend_precheck_value where value_type = 8 and is_deleted = N;
# Time: 2018-01-23T04:29:31.825823+08:00
# User@Host: yun_rw_legend[yun_rw_legend] @  [10.162.86.162]  Id: 3167670
# Query_time: 0.000826  Lock_time: 0.000146 Rows_sent: 0  Rows_examined: 947
SET timestamp=1516652971;
select

         id as id,
        is_deleted as isDeleted,
        gmt_create as gmtCreate,
        creator as creator,
        gmt_modified as gmtModified,
        modifier as modifier,

            shop_id as shopId,
            customer_id as customerId,
            account_id as accountId,
            customer_car_id as customerCarId

        from legend_customer_car_rel

         WHERE is_deleted = N




                   and shop_id = 3374




                    and customer_car_id = 1307177;

答案 1 :(得分:0)

这可能适合你(GNU sed):

sed -n '/^#/{:a;N;/^SELECT/M!ba;/backup_all/!p}' file

由于这是对行的缩减/过滤,因此通过设置-n选项可以实现类似于grep的性质。收集以评论行(#)开头的行,并以以SELECT开头的行结束。如果这些行不包含单词backup_all,则打印它们。

N.B。 regexp上的M标志调用多行模式,其中^$可以在行的开头/结尾匹配。另一种正则表达式是/\nSELECT/