where子句中的最佳搜索字符串

时间:2018-11-29 10:26:13

标签: sql-server tsql sql-server-2008-r2 soundex patindex

想在WHERE子句中使用PATINDEX和SOUNDEX来搜索字符串。

我有一张带有一些示例数据的下表,用于使用PATINDEXSOUNDEX搜索给定的字符串。

create table tbl_pat_soundex
(
    col_str varchar(max)
);

insert into tbl_pat_soundex values('Smith A Steve');
insert into tbl_pat_soundex values('Steve A Smyth');
insert into tbl_pat_soundex values('A Smeeth Stive');
insert into tbl_pat_soundex values('Steve Smith A');
insert into tbl_pat_soundex values('Smit Steve A');

注意:我在表中有100 Millions条记录可供搜索。

要搜索的字符串:-'史密斯史蒂夫'

SELECT col_str
FROM tbl_pat_soundex
WHERE PATINDEX('%Smith%',col_str) >= 1 AND PATINDEX('%A%',col_str) >= 1 AND PATINDEX('%Steve%',col_str) >= 1

获取输出:

col_str
--------------
Smith A Steve
Steve Smith A

预期输出:

col_str         
----------------
Smith A Steve   
Steve A Smyth   
A Smeeth Stive  
Steve Smith A   
Smit Steve A    

尝试:

1

SELECT col_str
FROM tbl_pat_soundex
WHERE PATINDEX('%Smith%',col_str) >= 1 AND 
      PATINDEX('%A%',col_str) >= 1 AND 
      PATINDEX('%Steve%',col_str) >= 1

2

SELECT col_str
FROM tbl_pat_soundex
WHERE PATINDEX('%'+SOUNDEX('Smith')+'%',SOUNDEX(col_str)) >= 1 AND 
      PATINDEX('%'+SOUNDEX('A')+'%',SOUNDEX(col_str)) >= 1 AND 
      PATINDEX('%'+SOUNDEX('Steve')+'%',SOUNDEX(col_str)) >= 1

3

SELECT col_str
FROM tbl_pat_soundex    
WHERE DIFFERENCE('Smith',col_str) = 4 AND 
      DIFFERENCE('A',col_str) =4 AND 
      DIFFERENCE('Steve',col_str) = 4

4

--Following was taking huge time(was kept running more than 20 minutes) to execute.
SELECT DISTINCT col_str
FROM tbl_pat_soundex [a]
CROSS APPLY SplitString([a].[col_str], ' ') [b]
WHERE DIFFERENCE([b].Item,'Smith') >= 1 AND 
      DIFFERENCE([b].Item,'A') >= 1 AND 
      DIFFERENCE([b].Item,'Steve') >= 1

2 个答案:

答案 0 :(得分:2)

如此多的行,我唯一能给您的提示是:更改设计。每个名称部分都应放在单独的列中...

以下方法会起作用,但我保证它会很慢...

-设置测试数据库

USE master;
GO
CREATE DATABASE shnugo;
GO
USE shnugo;
GO

-在您的表格中,我添加了一个ID列

create table tbl_pat_soundex
(
    ID INT IDENTITY --needed to distinguish rows
   ,col_str varchar(max)
);
GO

-一个函数,它将以空格分隔的字符串作为由/分隔的不同soundex值的字母排序列表返回:“史密斯史蒂夫”以/A000/S310/S530/的形式返回

CREATE FUNCTION dbo.ComputeSoundex(@str VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
BEGIN
    DECLARE @tmpXML XML=CAST('<x>' + REPLACE((SELECT @str AS [*] FOR XML PATH('')),' ','</x><x>') + '</x>' AS XML);
    RETURN (SELECT DISTINCT '/' + SOUNDEX(x.value('text()[1]','varchar(max)')) AS [se]
            FROM @tmpXML.nodes('/x[text()]') A(x)
            ORDER BY se
            FOR XML PATH(''),TYPE).value('.','nvarchar(max)') + '/';
END
GO

-添加一列以永久存储计算出的soundex链

ALTER TABLE tbl_pat_soundex ADD SortedSoundExPattern VARCHAR(MAX);
GO

-我们需要一个触发器来在任何插入或更新中维护计算出的soundex链

CREATE TRIGGER RefreshComputeSoundex ON tbl_pat_soundex
FOR INSERT,UPDATE
AS
BEGIN
    UPDATE s SET SortedSoundExPattern=dbo.ComputeSoundex(i.col_str)
    FROM tbl_pat_soundex s
    INNER JOIN inserted i ON s.ID=i.ID;
END
GO

-测试数据

insert into tbl_pat_soundex(col_str) values
 ('Smith A Steve')
,('Steve A Smyth')
,('A Smeeth Stive')
,('Steve Smith A')
,('Smit Steve A')
,('Smit Steve') --no A
,('Smit A') --no Steve
,('Smit Smith Robert Peter A') --add noise
,('Shnugo'); --something else entirely

-检查中间结果

SELECT * 
FROM tbl_pat_soundex

/*
+----+---------------------------+-----------------------+
| ID | col_str                   | SortedSoundExPattern  |
+----+---------------------------+-----------------------+
| 1  | Smith A Steve             | /A000/S310/S530/      |
+----+---------------------------+-----------------------+
| 2  | Steve A Smyth             | /A000/S310/S530/      |
+----+---------------------------+-----------------------+
| 3  | A Smeeth Stive            | /A000/S310/S530/      |
+----+---------------------------+-----------------------+
| 4  | Steve Smith A             | /A000/S310/S530/      |
+----+---------------------------+-----------------------+
| 5  | Smit Steve A              | /A000/S310/S530/      |
+----+---------------------------+-----------------------+
| 6  | Smit Steve                | /S310/S530/           |
+----+---------------------------+-----------------------+
| 7  | Smit A                    | /A000/S530/           |
+----+---------------------------+-----------------------+
| 8  | Smit Smith Robert Peter A | /A000/P360/R163/S530/ |
+----+---------------------------+-----------------------+
| 9  | Shnugo                    | /S520/                |
+----+---------------------------+-----------------------+
*/

-现在我们可以开始搜索:

DECLARE @StringToSearch VARCHAR(MAX)=' A Steve';

WITH SplittedSearchString AS
(
    SELECT soundexCode.value('text()[1]','nvarchar(max)') AS SoundExCode
    FROM (SELECT CAST('<x>' + REPLACE(dbo.ComputeSoundex(@StringToSearch),'/','</x><x>') + '</x>' AS XML)) A(x)
    CROSS APPLY x.nodes('/x[text()]') B(soundexCode)
)
SELECT a.ID,col_str
FROM tbl_pat_soundex a
INNER JOIN SplittedSearchString s On SortedSoundExPattern LIKE '%/' +  s.SoundExCode + '/%'
GROUP BY ID,col_str
HAVING COUNT(ID)=(SELECT COUNT(*) FROM SplittedSearchString)
ORDER BY ID 
GO

-清理

USE master;
GO
DROP DATABASE shnugo;

简短说明

它是这样工作的:

  • cte将使用相同的功能来返回输入片段的声音链。
  • 然后该查询将通过INNER JOIN测试LIKE进行此操作-这将会是sloooooow ...
  • 最后的检查是,命中数是否与片段数相同。

最后一个提示:如果您要搜索完全匹配的内容,但又想包含不同的文字,则可以直接比较两个字符串。您甚至可以在新列SortedSoundExPattern上放置索引。由于创建方式的不同,“史蒂文·史密斯”,“史蒂文·史密斯”甚至以“史密斯·史蒂文·史密斯”之类的不同顺序都将产生完全相同的图案。

答案 1 :(得分:0)

在我看来,您应该尝试使用动态SQL。

例如,您有一个表:

create table tbl_pat_soundex
(
    id int,
    col_str varchar(max)
)

并且您具有以下聚集索引或任何其他索引(具有超过1亿行的表应具有某些索引):

CREATE NONCLUSTERED INDEX myIndex ON dbo.tbl_pat_soundex(id) INCLUDE (col_str)*/

因此,请尝试根据您的逻辑创建以下动态SQL查询并执行它。希望的结果应如下所示:

DECLARE @statement NVARCHAR(4000)
SET @statement = N'
SELECT col_str
FROM tbl_pat_soundex
WHERE col_str like '%Smith%' AND id > 0
UNION ALL
SELECT col_str
FROM tbl_pat_soundex
WHERE col_str like '%Steve%' AND id > 0
UNION ALL
SELECT col_str
FROM tbl_pat_soundex
WHERE 
    PATINDEX('%Smith%',col_str) >= 1 AND PATINDEX('%A%',col_str) >= 1 AND 
    PATINDEX('%Steve%',col_str) >= 1
    AND id > 0'

基本上,我们要做的是创建单个搜索查询,该查询将具有索引搜索,然后合并所有结果。

当我们使用谓词id > 0时,此查询将具有索引查找(假设所有ID均大于0或您可以编写自己的负数):

SELECT col_str
FROM tbl_pat_soundex
WHERE col_str like '%Smith%' AND id > 0