使用OLEDB（VFPOLEDB）在DBF文件上运行查询太慢

时间：2015-10-17 21:29:36

标签： c# visual-foxpro

我正在开发一个界面来显示WinForm应用程序中DBF文件的数据。 我开始使用OdbcConnection。尽管它有效，但由于Visual FoxPro驱动程序的某些限制（不支持子查询），我开启了使用OLEDB（VFPOLEDB）。现在我能够运行复杂的查询，但是出现了必须解决的新困难。问题是这些查询太慢了。比预期慢100倍。

以下是演示代码。 有一个DBF表'PROD'。索引字段PRICE_N用于查询的Where子句中。该表位于运行应用程序的同一台PC上。正如您所看到的，通过ODBC（Microsoft Visual FoxPro驱动程序）和OLEDB（VFPOLEDB）运行查询所花费的时间变化很大。

            TimeSpan timeSpanODBC;
        DateTime timeODBC = DateTime.Now;

        OdbcConnection odbcConnection = new OdbcConnection(@"Driver={Microsoft Visual FoxPro Driver};SourceType=DBF;SourceDB=C:\Users\Vakshul\Documents\dbfs;Exclusive=No;Collate=Machine;NULL=NO;DELETED=NO;BACKGROUNDFETCH=NO;");
        odbcConnection.Open();
        OdbcCommand odbcCommand = new OdbcCommand("SELECT utk_ved FROM prod WHERE (price_n='641857')", odbcConnection);
        odbcCommand.ExecuteScalar();
        timeSpanODBC = DateTime.Now - timeODBC;
        double timeOdbcEqual = timeSpanODBC.TotalMilliseconds;
        System.Console.WriteLine("Time spent via ODBC(milliseconds) using '=' to compare - {0}", timeOdbcEqual.ToString());


        timeODBC = DateTime.Now;

        odbcCommand = new OdbcCommand("SELECT utk_ved FROM prod WHERE (price_n like'641857')", odbcConnection);
        odbcCommand.ExecuteScalar();
        timeSpanODBC = DateTime.Now - timeODBC;
        double timeOdbcLike = timeSpanODBC.TotalMilliseconds;
        System.Console.WriteLine("Time spent via ODBC(milliseconds) using 'Like' to compare - {0}", timeOdbcLike.ToString());

        TimeSpan timeSpanOLEDB;
        DateTime timeOLEDB = DateTime.Now;

        OleDbConnection oleDbCon = new OleDbConnection(@"Provider=VFPOLEDB.1;Data Source=C:\Users\Vakshul\Documents\dbfs;Collating Sequence=MACHINE;Mode=Read");
        oleDbCon.Open();
        OleDbCommand oleDbcommand = new OleDbCommand("SELECT utk_ved FROM prod WHERE (price_n = '641857')", oleDbCon);
        oleDbcommand.ExecuteScalar();
        timeSpanOLEDB = DateTime.Now - timeOLEDB;
        double timeOLEDBEqual = timeSpanOLEDB.TotalMilliseconds;
        System.Console.WriteLine("Time spent via OLEDB(milliseconds) using '=' to compare - {0}", timeOLEDBEqual.ToString());

        timeOLEDB = DateTime.Now;

        oleDbcommand = new OleDbCommand("SELECT utk_ved FROM prod WHERE (price_n like '641857')", oleDbCon);
        oleDbcommand.ExecuteScalar();
        timeSpanOLEDB = DateTime.Now - timeOLEDB;
        double timeOLEDLike = timeSpanOLEDB.TotalMilliseconds;
        System.Console.WriteLine("Time spent via OLEDB(milliseconds) using 'Like' to compare - {0}", timeOLEDLike.ToString());

        System.Console.WriteLine("ODBC is faster than OLEDB {0} times using '=' to compare", Math.Round(timeOLEDBEqual / timeOdbcEqual, 0));
        System.Console.WriteLine("ODBC is faster than OLEDB {0} times using 'Like' to compare", Math.Round(timeOLEDBEqual / timeOdbcEqual, 0));

首次运行后的控制台：

Time spent via ODBC(milliseconds) using '=' to compare - 5,0006
Time spent via ODBC(milliseconds) using 'Like' to compare - 3,5005
Time spent via OLEDB(milliseconds) using '=' to compare - 1630,207
Time spent via OLEDB(milliseconds) using 'Like' to compare - 1755,2228
ODBC is faster than OLEDB 326 times using '=' to compare
ODBC is faster than OLEDB 326 times using 'Like' to compare

Console, after the second run:
Time spent via ODBC(milliseconds) using '=' to compare - 4,5006
Time spent via ODBC(milliseconds) using 'Like' to compare - 4,5005
Time spent via OLEDB(milliseconds) using '=' to compare - 1526,1938
Time spent via OLEDB(milliseconds) using 'Like' to compare - 1595,2026
ODBC is faster than OLEDB 339 times using '=' to compare
ODBC is faster than OLEDB 339 times using 'Like' to compare

Console, after the third run:
Time spent via ODBC(milliseconds) using '=' to compare - 4,0005
Time spent via ODBC(milliseconds) using 'Like' to compare - 3,0004
Time spent via OLEDB(milliseconds) using '=' to compare - 1449,184
Time spent via OLEDB(milliseconds) using 'Like' to compare - 1451,1843
ODBC is faster than OLEDB 362 times using '=' to compare
ODBC is faster than OLEDB 362 times using 'Like' to compare

Console, after the fourth run:
Time spent via ODBC(milliseconds) using '=' to compare - 3,5004
Time spent via ODBC(milliseconds) using 'Like' to compare - 4,5006
Time spent via OLEDB(milliseconds) using '=' to compare - 1475,6874
Time spent via OLEDB(milliseconds) using 'Like' to compare - 1621,2059
ODBC is faster than OLEDB 422 times using '=' to compare
ODBC is faster than OLEDB 422 times using 'Like' to compare

在此示例中，索引字段PRICE_N包含在查询的Where子句中。 我还测试了相同的查询，包括Where子句中的非索引字段而不是索引字段。结果是相同的~1400-1600毫秒。 我的印象是，在OLEDB（VFPOLEDB）的情况下，不使用索引。 我对结果不满意，我需要使用索引。

如果有人有任何建议，我将非常感激。

4 个答案:

答案 0 :(得分：0)

只是想知道......你的“Price_n”列是NUMERIC还是STRING列。如果是数字，那么我想知道VFP OleDb是否试图将所有数字转换为测试的STRING等价物，而不是将引用的字符串转换为Price_n数据类型的数字等价物，因为我希望自然索引基于

如果是这样，请尝试测试并将所有WHERE子句分别更改为

WHERE price_n = 641857

WHERE price_n赞641857

但是如果基于数字的列，那么Like真的不适用，因为数字是一个数字，而不像像

那样的部分字符串匹配 
"1" LIKE "1"
"1" LIKE "10"
"1" LIKE "19302"... etc where they all start with a same value

答案 1 :(得分：0)

@Sergiy， 你正在做的事情有很大的不同。对于VFP6之后的版本（即2.5或2.6是包含ODBC驱动程序的最后一个包），ODBC驱动程序不存在。 IOW ODBC仅支持VFP6引擎。 OTOH VFPOLEDB支持VFP9引擎（以及所有那些添加了SQL功能的铃声和口哨声）。

在这些引擎之间，存在一个问题，使得查询在文本字段上变慢： 如果OS代码页与表的代码页不同，则对表达式为字符类型的索引进行搜索。然后它不使用索引但是进行表扫描。这个＆＃34; bug＆＃34;在VFP9初始释放后浮出水面并且没有更正AFAIK。

根据= vs like，类似于隐式ANSI，所以表现得像＆lt; g>使用==运算符（完全匹配）。使用=，如果ANSI为OFF，则部分匹配为真。

PS：VFPOLEDB即使在修改了代码页之后，也会稍微慢一点但可以忽略不计。

以下是我的代码时序，在一个包含1,000,000行的测试表中：

Time spent via ODBC(milliseconds) using '=' to compare - 41.0023
Time spent via ODBC(milliseconds) using 'Like' to compare - 0
Time spent via OLEDB(milliseconds) using '=' to compare - 68.0038
Time spent via OLEDB(milliseconds) using 'Like' to compare - 2.0002
ODBC is faster than OLEDB 2 times using '=' to compare
ODBC is faster than OLEDB 2 times using 'Like' to compare

这是第二次运行后的时间：

Time spent via ODBC(milliseconds) using '=' to compare - 1
Time spent via ODBC(milliseconds) using 'Like' to compare - 1.0001
Time spent via OLEDB(milliseconds) using '=' to compare - 3.0001
Time spent via OLEDB(milliseconds) using 'Like' to compare - 0
ODBC is faster than OLEDB 3 times using '=' to compare
ODBC is faster than OLEDB 3 times using 'Like' to compare

答案 2 :(得分：0)

@Cetin Basoz，

  

在这些引擎之间，存在使查询变慢的问题   文本字段：如果操作系统代码页与表的代码页不同   正在对表达式具有特征的索引进行搜索   类型。然后它不使用索引，而是进行表扫描。这个“虫子”   在VFP9初始释放后浮出水面并且没有更正AFAIK。

你遇到了问题的关键！

我不知道那种特殊性。我决定检查一下是否真的如此。如果你的假设是对的，低速的原因是DBF文件的代码页和我的操作系统是不同的。为了测试我安装了Visual Fox Pro 9（我以前从未处理过它）并将所有数据传输到新表。然后我在Table Designer中打开了表，并在PRICE_N字段上创建了一个常规索引。因此，新表和我的操作系统的代码页变得相同。

然后我再次运行测试。结果发生了巨大变化。

第一次运行后：

Time spent via ODBC(milliseconds) using '=' to compare - 12,5016
Time spent via ODBC(milliseconds) using 'Like' to compare - 3,5005
Time spent via OLEDB(milliseconds) using '=' to compare - 20,0025
Time spent via OLEDB(milliseconds) using 'Like' to compare - 3,0004
ODBC is faster than OLEDB 2 times using '=' to compare
ODBC is faster than OLEDB 2 times using 'Like' to compare

第二次运行后：

Time spent via ODBC(milliseconds) using '=' to compare - 3,0004
Time spent via ODBC(milliseconds) using 'Like' to compare - 2,5003
Time spent via OLEDB(milliseconds) using '=' to compare - 11,0014
Time spent via OLEDB(milliseconds) using 'Like' to compare - 3,5005
ODBC is faster than OLEDB 4 times using '=' to compare
ODBC is faster than OLEDB 4 times using 'Like' to compare

谢谢，Cetin Basoz。评论很棒：）

即使我不允许更改生产DBF文件的代码页，但至少现在当我知道发生了什么时，我可以放松一下。

答案 3 :(得分：0)

@Sergiy， 我可能有一个解决方案：

string sqlEq = "SELECT utk_ved FROM prod WHERE Price_N = '641857'";
string sqlLike = "SELECT utk_ved FROM prod WHERE Price_N like '641857'";

TimeSpan timeSpanODBC;
DateTime timeODBC = DateTime.Now;

OdbcConnection odbcConnection = new OdbcConnection(@"Driver={Microsoft Visual FoxPro Driver};SourceType=DBF;SourceDB=C:\Users\Vakshul\Documents\dbfs;Exclusive=No;Collate=Machine;NULL=NO;DELETED=NO;BACKGROUNDFETCH=NO;");
odbcConnection.Open();
OdbcCommand odbcCommand = new OdbcCommand(sqlEq, odbcConnection);
odbcCommand.ExecuteScalar();
timeSpanODBC = DateTime.Now - timeODBC;
double timeOdbcEqual = timeSpanODBC.TotalMilliseconds;
System.Console.WriteLine("Time spent via ODBC(milliseconds) using '=' to compare - {0}", timeOdbcEqual.ToString());


timeODBC = DateTime.Now;

odbcCommand = new OdbcCommand(sqlLike, odbcConnection);
odbcCommand.ExecuteScalar();
timeSpanODBC = DateTime.Now - timeODBC;
double timeOdbcLike = timeSpanODBC.TotalMilliseconds;
System.Console.WriteLine("Time spent via ODBC(milliseconds) using 'Like' to compare - {0}", timeOdbcLike.ToString());

TimeSpan timeSpanOLEDB;
DateTime timeOLEDB = DateTime.Now;

OleDbConnection oleDbCon = new OleDbConnection(@"Provider=VFPOLEDB.1;Data Source=C:\Users\Vakshul\Documents\dbfs;Collating Sequence=MACHINE;Mode=Read");
oleDbCon.Open();
new OleDbCommand("set enginebehavior 80", oleDbCon).ExecuteNonQuery();
OleDbCommand oleDbcommand = new OleDbCommand(sqlEq, oleDbCon);
oleDbcommand.ExecuteScalar();
timeSpanOLEDB = DateTime.Now - timeOLEDB;
double timeOLEDBEqual = timeSpanOLEDB.TotalMilliseconds;
System.Console.WriteLine("Time spent via OLEDB(milliseconds) using '=' to compare - {0}", timeOLEDBEqual.ToString());

timeOLEDB = DateTime.Now;

oleDbcommand = new OleDbCommand(sqlLike, oleDbCon);

oleDbcommand.ExecuteScalar();
timeSpanOLEDB = DateTime.Now - timeOLEDB;
double timeOLEDLike = timeSpanOLEDB.TotalMilliseconds;
System.Console.WriteLine("Time spent via OLEDB(milliseconds) using 'Like' to compare - {0}", timeOLEDLike.ToString());

请注意此行 在同一连接上 ：

new OleDbCommand("set enginebehavior 80", oleDbCon).ExecuteNonQuery();

这不会影响您的结果，但会得到您想要的结果。在我的头顶，它可能影响的唯一地方是“分组依据”查询。认为你不会以旧的越野车VFP方式通过查询来编写小组，你应该是安全的。

我的时间没有“Set EngineBehavior 80”：

Time spent via ODBC(milliseconds) using '=' to compare - 4.0002
Time spent via ODBC(milliseconds) using 'Like' to compare - 1.0001
Time spent via OLEDB(milliseconds) using '=' to compare - 352.0201
Time spent via OLEDB(milliseconds) using 'Like' to compare - 659.0377

并使用“Set EngineBehavior 80”：

Time spent via ODBC(milliseconds) using '=' to compare - 3.0001
Time spent via ODBC(milliseconds) using 'Like' to compare - 2.0002
Time spent via OLEDB(milliseconds) using '=' to compare - 15.0008
Time spent via OLEDB(milliseconds) using 'Like' to compare - 3.0002
相关问题
最新问题