从.sql文件中“删除”一行行

时间:2014-07-10 16:24:08

标签: python mysql

我正在编写一个脚本,它将从SQL转储中删除一行代码(基本上,清理我们不想恢复的任何表)。我以为我有它工作,但是当我尝试将生成的文件恢复到我的数据库时,我意识到当脚本重写到文件时,它会丢失比它应该更多的行,并且恢复失败。这是我对此的微不足道的尝试:

#!/usr/bin/python

to_keep = []
to_remove = []

f = open("backuptest.sql","r")
lines = f.readlines()

f.close()

### Function to remove lines associated with a table block
def remove_lines(table_name):
    for i in range(0, len(lines)):
        line = lines[i]
        if "structure" in line and table_name in line:
            for j in range(i, len(lines)):
                to_remove.append(lines[j])
                if "UNLOCK TABLES;" in lines[j]:
                    break
        if line not in to_remove:
            to_keep.append(line)
            f.write(line)

print "Finding lines"

f = open("backuptest.sql", "w")
remove_lines("advanced_searches")
f.close()

f = open("backuptest.sql", "w")
remove_lines("test_table2")
f.close()

f = open("backuptest.sql", "w")
remove_lines("test_table3")
f.close()

以下是sql文件的一个块的样子,供参考:

-- Table structure for table `advanced_searches`
--

DROP TABLE IF EXISTS `advanced_searches`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `advanced_searches` (
  `batch_size` int(11) NOT NULL DEFAULT '0'
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Dumping data for table `advanced_searches`
--

LOCK TABLES `advanced_searches` WRITE;
/*!40000 ALTER TABLE `advanced_searches` DISABLE KEYS */;
/*!40000 ALTER TABLE `advanced_searches` ENABLE KEYS */;
UNLOCK TABLES;

我通过匹配这样一个块中的第一行来启动脚本(包括单词“structure”和表名),然后我试图通过“UNLOCK TABLES”从该行迭代。发生的事情是它正在删除“UNLOCK TABLES”的所有实例;在脚本中,甚至从我不想删除的块,它也从每个块中删除以下行“)ENGINE = MyISAM DEFAULT CHARSET = utf8 COLLATE = utf8_unicode_ci;”(通过该行的其余部分的右括号)

感谢您的帮助。我会继续关注我错过的东西。

2 个答案:

答案 0 :(得分:1)

尝试sqlparse

$ pip install sqlparse

教程快速测试:

>>> import sqlparse
>>> sql = "select * from foo; select * from bar;"
>>> res = sqlparse.split(sql)
>>> res
[u'select * from foo;', u'select * from bar;']

应用于您的SQL脚本:

>>> sql = """-- Table structure for table `advanced_searches`
... --
... 
... DROP TABLE IF EXISTS `advanced_searches`;
... /*!40101 SET @saved_cs_client     = @@character_set_client */;
... /*!40101 SET character_set_client = utf8 */;
... CREATE TABLE `advanced_searches` (
...   `batch_size` int(11) NOT NULL DEFAULT '0'
... ) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
... /*!40101 SET character_set_client = @saved_cs_client */;
... 
... --
... -- Dumping data for table `advanced_searches`
... --
... 
... LOCK TABLES `advanced_searches` WRITE;
... /*!40000 ALTER TABLE `advanced_searches` DISABLE KEYS */;
... /*!40000 ALTER TABLE `advanced_searches` ENABLE KEYS */;
... UNLOCK TABLES;"""
...
>>> res = sqlparse.split(sql)
>>> res
[u'-- Table structure for table `advanced_searches`\n--\n\nDROP TABLE IF EXISTS `advanced_searches`;',
 u'/*!40101 SET @saved_cs_client     = @@character_set_client */;',
 u'/*!40101 SET character_set_client = utf8 */;',
 u"CREATE TABLE `advanced_searches` (\n  `batch_size` int(11) NOT NULL DEFAULT '0'\n) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;",
 u'/*!40101 SET character_set_client = @saved_cs_client */;',
 u'--\n-- Dumping data for table `advanced_searches`\n--\n\nLOCK TABLES `advanced_searches` WRITE;',
 u'/*!40000 ALTER TABLE `advanced_searches` DISABLE KEYS */;',
 u'/*!40000 ALTER TABLE `advanced_searches` ENABLE KEYS */;',
 u'UNLOCK TABLES;']

这提供了正确解析的脚本,最后要做的是逐个过滤它,只选择那些似乎需要的脚本。我留给你了。

答案 1 :(得分:0)

最好不要使用全局变量代替函数参数 - 使用参数可以使函数与其环境之间的“耦合”更加清晰。由于您要从单个文件中删除多个表,因此最好先从读取所有行开始,然后将输出行作为新结果传回。

def remove_lines(lines, table_name):
   output_lines = []
   for line in lines:
       if line.startswith("-- structure for table") and table_name in line:
           while "UNLOCK TABLE" not in line:
               line = lines.next()
       else:
           output_lines.append(line)
   return output_lines

应该使用一组行和一个table_name调用此函数,并返回一个行列表,必要时可以进一步处理这些行。最后一组可以用来覆盖原始文件(你确定你不想保留备份吗?),如下所示:

with open("backuptest.sql","r") as f:
    lines = f.readlines()
for table_name in ("advanced_searches", "test_table2", "test_table1"):
    lines = removelines(lines, table_name)
with open("backuptest.sql", "w") as f
    f.writelines(lines)