Sql 2005 Table to Table合并了一些数据转换

时间:2009-07-01 16:51:56

标签: .net sql-server sql-server-2005

因此,我所处理的网站正在迁移到新的数据Feed。当前数据馈送从FTP站点馈送,并以两个denomlized文件呈现给我。然后将其分成两个表到MS SQL 2005服务器,然后该站点用于运行所有搜索(例如,房地产站点)。

问题是,新的数据源被规范化,不只有一个文件,而是12.所有这些文件都已合并到视图中以模仿旧的遗留表。它的速度很慢,因为每行必须将其数据从“a,d,e”转移到“Range,Refrigerator,Dishwasher”。每行必须完成最多17次这些更改。所以我的想法是进行一次转换,并将它们作为静态值而不是通过视图保存到表中。这可以工作但是将数据放入空白表需要40多分钟。只需1小时45分钟即可更新所需的行,并插入新行。这里的主键是一个char(8)用于一切。我也有每行的更新日期。在加快速度方面我需要注意什么。我需要对服务器,文件组,索引,表等进行哪些更改?

还有一些其他信息,每行大约7K,服务器运行SQL 2005 Workgroup Edition,我有一个开发和生产框,所以我可以轻松测试任何东西。任何帮助都会得到满足,我的目标是让它降低到10到20分钟,而不是47分钟。

编辑, 我被要求发布视图,这里是合并在一起的文件之一。有问题发布另一个,似乎它在下一个Union之后将它们全部合并在一起。


        select LN,null as [PropertyType],ST,CTGF,TYP,UD,null as [LAG],null as [LO],null as [OFFICE_PHONE],null as [AGENT_FORM_NAME],
                    null as [OFFICE_FORM_NAME],LP,HSN,CP,STR,null as [UN],CIT,ZP,ADI,AR,null as [AREA_Name],SUB,CNY,RMS,BR,BTH,MBB,CARS,dbo.listLookupToString(gar,'a,Attached,b,Detached,c,1 Car Garage,d,1.5 Car Garage,e,2 Car Garage,f,2.5 Car Garage,g,3 Car Garage,h,3+ Car Garage,m,Garage Door Opener(s) (Auto),n,Transmitter(s),o,Carport,p,Heated,q,Leased,r,Owned,s,Underground,u,None,v,Deeded Sold Separately,w,On-Site,x,Off-Site,z,Tandem') as GAR,PKN,
                    PAR,FP,BLT,LSZ,ACR,WF,BMT,MOD,TAX,TXY,ASM,MBS,dbo.listLookupToString(MBF,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as MBF,B2S,dbo.listLookupToString(B2F,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as B2F,B3S,dbo.listLookupToString(B3F,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as B3F,B4S,dbo.listLookupToString(B4F,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as B4F,LRS,dbo.listLookupToString(LRF,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as LRF,DRS,dbo.listLookupToString(DRF,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as DRF,KTS,dbo.listLookupToString(KTF,'c,Carpet,h,Hardwood,p,Parquet,v,Vinyl,t,Ceramic Tile,o,Other') as KTF,FRS,FRF,REMARKS_INTERNET,REMARKS,
                    null as [RE1],null as [RE2],null as [RE3],null as [RE4],null as [RE5],null as [RE6],null as [RE7],null as [RE8],null as [RE9],
                    null as [RE10],A1N,A1S,A1F,A2N,A2S,A2F,A3N,A3S,A3F,A4N,A4S,A4F,A5N,A5S,A5F,GS,GSD,JH,JHD,HS,HSD,OT,OSD,FEA,dbo.listLookupToString(OTR,'a,1st Floor Bedroom,b,Darkroom,c,Den/Office/Study,d,Enclosed Balcony,e,Exercise Room,f,Family Room,g,Gallery/Foyer,h,Great Room,i,In-Law Arrangement,j,Kitchen Second,k,Library,l,Loft,m,Maids Room,n,Porch - Enclosed,o,Porch - Screened,p,Recreation,q,Sitting Room,r,Sunroom/Florida Room,t,Utility/Laundry - 1st Floor,u,Utility/Laundry - 2nd Floor,v,Workshop') as OTR,STY,TPE,
                    null as [TPC],null as [TMU],null as [TMF],DIR,null as [DR1],null as [DR2],null as [DR3],null as [UFL],null as [FLN],ASF,
                    null as [PTA],null as [CAA],EXP,null as [UFE],null as [TNU],null as [UF1],null as [RM1],null as [BR1],null as [BT1],
                    null as [UF2],null as [RM2],null as [BR2],null as [BT2],null as [UF3],null as [RM3],null as [BR3],null as [BT3],null as [UF4],
                    null as [RM4],null as [BR4],null as [BT4],null as [GRI],null as [NOI],null as [AP1],null as [AP2],null as [AP3],null as [AP4],
                    null as [AZN],null as [SLN],null as [ASQ],null as [TLA],null as [RU],null as [LT],null as [AML],null as [BIM],null as [CUU],
                    null as [FMT],null as [LND],null as [LOCAT],null as [PTU],null as [GSI],null as [TAE],null as [GSA],null as [TO],
                    null as [UNT],null as [SUBTPE],null as [RP],null as [MIN],null as [MAX],null as [BSQ],null as [BAG],null as [BUT],
                    null as [APT],null as [OFC],null as [STO],null as [DID],null as [NDK],null as [CEMXF],null as [CEMXI],null as [CEMIF],
                    null as [CEMII],null as [HT],null as [GD],null as [PARK],null as [DBL],null as [MK],PHOTOCOUNT as [NBR_PHOTOS],
                    null as [SUPP_PHOTOS],SP,CLOSED_DATE
from rets_de where st<>'CLSD'

1 个答案:

答案 0 :(得分:1)

您可以发布将所有这些数据汇总在一起的当前视图吗?

听起来好像改变你的表实际上将主键声明为集群并使用一些写得好的JOIN语句可以帮助你的查询运行得更快,而不需要太多的工作。此外,每当在SQL Server中运行选择时,除非您特别需要锁定您正在查看的表,否则请确保运行Select with NOLOCK,因为如果您正在加载数据并从中选择,则可以提高速度一组表同时。例如:

SELECT X, Y, Z
FROM Table AS t WITH(NOLOCK)

在查看你的查询之后,看起来你可能正在放慢速度,因为你在整个地方调用ListLookupToString函数,必须为每一行调用它。更好,更快的方法是使用规范化表和跨表的JOIN。我已经进行了一次查找,并展示了如何在下面进行转换。

当前查找:

dbo.listLookupToString(gar,'a,Attached,b,Detached,c,1 Car Garage,d,1.5 Car Garage,e,2 Car Garage,f,2.5 Car Garage,g,3 Car Garage,h,3+ Car Garage,m,Garage Door Opener(s) (Auto),n,Transmitter(s),o,Carport,p,Heated,q,Leased,r,Owned,s,Underground,u,None,v,Deeded Sold Separately,w,On-Site,x,Off-Site,z,Tandem') as GAR

创建查找表

CREATE TABLE GarageLookup (GarageID VARCHAR(4), GarageTypeName VARCHAR(64))
INSERT INTO GarageLookup (GarageID, GarageTypeName) VALUES('a', 'Attached')
INSERT INTO GarageLookup (GarageID, GarageTypeName) VALUES('b', 'Detached')
/* Insert all rows into GarageLookup that are in the Lookup function currently */

使用SELECT中的查找表而不是查找函数

SELECT /*Everything ommitted to be brief*/
        gl.GarageTypeName
FROM rets_de AS r WITH(NOLOCK)
JOIN GarageLookup AS gl WITH(NOLOCK) ON r.gar = gl.GarageID