Question

我有一个我正在尝试运行的SSIS包，但它总是因为缓冲区到很多数据而失败。这是我的第一次运行。然后我想我应该只从今天的日期和30天抓取我的数据然后插入。我的问题是，我将如何获取最近30天的日期，然后在接下来的30天再次执行此操作，直到我将所有数据插入到我的数据仓库中？

我的查询看起来像这样

SELECT db_name() dbname,  TicketType, TicketNo source_bk, UniqueID, ItemNo, CASE WHEN  VehicleID  = '' THEN '-1' ELSE VehicleID END VehicleID 
, CASE WHEN  TicketID  = '' THEN '-1' ELSE TicketID END TicketID,
 case when p.purchaseOrder = '' then 'unknown' else p.PurchaseOrder end as PurchaseOrder, TicketDate, TicketTime, S1.LocationID
, S1.CustomerID, S1.OrderID, OrderItem, ProductID, MixID, S1.TaxCodeID,  S1.CarrierID,  Description, DeliveryAddress1
,  Gross, Tare, Net, Qty, Unit, FreightQty, FreightPayQty, S1.Price, S1.FreightRate, S1.FreightAmount,  S1.FreightPay
, FreightPayAmount, TodayLoads, TodayQty, OrderLoads, OrderQty,  AltTicketQty, AltTicketQtyEdited, TodayAmount
, 'Posted' as [Source] FROM tkhist1 S1  WITH (NOLOCK)
join [dbo].[Slordnam] p 
on s1.customerID = p.CustomerID
where s1.TicketDate >= CURRENT_TIMESTAMP -30
UNION
SELECT db_name() dbname,  TicketType, TicketNo source_bk, UniqueID, ItemNo, CASE WHEN  VehicleID  = '' THEN '-1' ELSE VehicleID END VehicleID 
, CASE WHEN  TicketID  = '' THEN '-1' ELSE TicketID END TicketID,
 case when p.purchaseOrder = '' then 'unknown' else p.PurchaseOrder end as PurchaseOrder, TicketDate, TicketTime, S1.LocationID
, S1.CustomerID, S1.OrderID, OrderItem, ProductID, MixID, S1.TaxCodeID,  S1.CarrierID,  Description, DeliveryAddress1
,  Gross, Tare, Net, Qty, Unit, FreightQty, FreightPayQty, S1.Price, S1.FreightRate, S1.FreightAmount,  S1.FreightPay
, FreightPayAmount, TodayLoads, TodayQty, OrderLoads, OrderQty,  AltTicketQty, AltTicketQtyEdited, TodayAmount
, 'Posted' as [Source] FROM Tkbatch S1  WITH (NOLOCK)
join [dbo].[Slordnam] p 
on s1.customerID = p.CustomerID
where s1.TicketDate >= CURRENT_TIMESTAMP -30
UNION
SELECT db_name() dbname,  TicketType, TicketNo source_bk, UniqueID, ItemNo, CASE WHEN  VehicleID  = '' THEN '-1' ELSE VehicleID END VehicleID 
, CASE WHEN  TicketID  = '' THEN '-1' ELSE TicketID END TicketID,
 case when p.purchaseOrder = '' then 'unknown' else p.PurchaseOrder end as PurchaseOrder, TicketDate, TicketTime, S1.LocationID
, S1.CustomerID, S1.OrderID, OrderItem, ProductID, MixID, S1.TaxCodeID,  S1.CarrierID,  Description, DeliveryAddress1
,  Gross, Tare, Net, Qty, Unit, FreightQty, FreightPayQty, S1.Price, S1.FreightRate, S1.FreightAmount,  S1.FreightPay
, FreightPayAmount, TodayLoads, TodayQty, OrderLoads, OrderQty,  AltTicketQty, AltTicketQtyEdited, TodayAmount
, 'Posted' as [Source] FROM Tkscale S1  WITH (NOLOCK)
join [dbo].[Slordnam] p 
on s1.customerID = p.CustomerID
where s1.TicketDate >= CURRENT_TIMESTAMP -30

插入完成后我会希望它在下次运行时再次执行此操作但是从此结束后的30天内完成。所以我已经拥有的是因为这些数据来自许多数据库，我为每个循环都有一个数据库并为每个数据库执行此操作。我希望它从现在起30天后再进行第一次运行的过程，因为我无法让它完全运行。然后我会做CDC只做新数据

Answer 1

要考虑的两个答案......

一个是，你是什么意思＆＃34;对于大量数据＆＃34; - SSIS对您希望通过数据流传输的数据量没有限制。您是说服务器无法处理查询，或者您试图在SSIS中执行查找以及内存不足或其他内容？最重要的是，听起来你可能会以不同的方式处理这个问题，因为应该没有这样的限制。

其中两个是，如果你真的想一次处理多组行，请使你的查询动态化并进行For循环保存，为每个循环增加30天的日期范围。您可以通过多种方式创建动态SQL，这些方法取决于您使用的连接类型以及您喜欢的方法，因为它们都有优点/缺点。与任何关系连接一起使用的最灵活和一致的方法是创建一个字符串变量并根据vbscript设置它的值。

我希望有所帮助。

由于多行数据，SSIS包在首次运行时失败

1 个答案: