我创建了一个新的SSIS包(SQL 2017),该包会读取“导出请求”表,并根据计划将数据导出到外部存储。例如,用户可以向该表添加新行,如“选择*从表1中col1 = 2的表中选择*”,其计划为2018年6月25日下午3:30。
我的程序包(使用SQL代理运行)一直循环搜索该表中的新任务,并按计划将查询结果当时导出到文件夹中。
有一项新功能要求将这些导出计划为定期进行。例如,客户可能希望从2018年6月25日下午3:30开始每天进行出口。每年,每月,每天,每3个月,...的时间表可能会变得很复杂。
在SSIS中实现重复执行的最佳方法是什么?我可以为Cron使用5个字符的字段,但不知道如何使用其Cron时间表查询新任务。 有什么帮助吗?
答案 0 :(得分:1)
确认已拥有的内容:您有一个TASK表,其中包含任务,以及某种日期标记,用于指示何时需要运行该表。然后,您的作业每x分钟运行一次,并检查是否有任何要拾取/运行的任务。如果是这样,它将运行它们。然后,我猜您在设置的表中有一个标记,说在循环中运行它后完成了吗?如果您一次要运行多个任务,那么一次只能运行一次?
然后您想要添加的内容是:我做了一些非常相似的事情。我为每个任务创建了一个查找表(在我的过程中,每个任务都有一个名称,并且我使用该名称来引用该查找表,我将其称为计划查找)。
在该表中,我放置了报表的运行时间和频率(如此2列),以及时间和频率。一个示例是7:00和工作日。因此,对于此报告,它仅在每个工作日(m-F)晚上7点运行。
然后,当我的作业运行时,它将像上面一样运行该任务,但是还有另一步,即将该任务标记为完成,然后在任务表中插入一个新任务(任务详细信息是相同的),但我将查看上述的计划查找表,以找出作业应再次运行的下一个日期/时间,并将其用作任务表中的下一个运行日期/时间。
下面是我在流程中使用的SP,用于获取下一个计划的日期/时间并更新现有的计划/日期,然后创建新的计划。
注意:我的日程安排中有一些您可能不需要的高级选项,我有很多评论可以解释为什么/为什么要做什么。我正在调用一些我在某些地方创建的函数,但是我认为您不需要这些函数,而是可以找出要做什么而不是我的函数,但是如果您有任何疑问,请告诉我:
这就是我所使用的,因此它正在使用我的表结构/ etc,但是您可以很容易地将其转换为您的表结构/
。--Purpose
----------------------------------------------------------------------------------
-- calculates the next time to run/schedule the job
----------------------------------------------------------------------------------
-- NOTE: TO USE you have to insert the first value manually in queue table
----- possibile scenerios
-- if we want to schedule every x hours, or x days
-- run every month only
-- run weekdays only
-- run on certain days of month only
-- TO ADD MORE COMPLEX or different types of schedules:
-- special - different times for different days of week
-- ex - so have dayofweek:2:00,dayofweek:3:00 (and we parse out the day of week and number splitting out the strings)
-- hourly - to do more then once a day??
-- WHEN @ScheduleLookupType = 'hourly' THEN DATEADD(DAY, 1, @CurrentScheduleDate) -- FIX FIX FIX
-- EXEC dbo.JobsDynamicRescheduleFindNextTimeToScheduleJob @ReportName = 'TestReport1'
----------------------------------------------------------------------------------
ALTER PROCEDURE [dbo].[JobsDynamicRescheduleFindNextTimeToScheduleJob]
@ReportName VARCHAR(50)
AS
SET NOCOUNT ON
BEGIN TRY
-- left here for testing outside of SP
-- this will be passed from SP
--DECLARE @ReportName AS VARCHAR(50)
--SET @ReportName = 'TESTREport'
-- this sets the first day of the week to Monday (we need it set to a value to do calcluations for weekdays, I set it to 1 for monday and 7 for sunday)
-- this is due to server settings could have somethign else so forcing it here
SET DATEFIRST 1
DECLARE @CurrentScheduleDate AS DATE -- find the current date for the job that just ran
DECLARE @CurrentScheduleDayNumberOfWeek AS SMALLINT -- this pulls the number of the day of week 1=monday, 2=tuesdday
DECLARE @ScheduleLookupType AS VARCHAR(20) -- this is the type of schedule to do calculations on
DECLARE @TimeOfDayToScheduleJob AS VARCHAR(20) -- look this up, its the time to schedule the job
DECLARE @SpecialScheduleValue AS VARCHAR(8000) -- this is special value to lookup (only needed if non standard one)
DECLARE @NewScheduleDateONLY AS DATETIME -- to hold just the date of the schedule before combinng with time
DECLARE @NewScheduleDateTime AS DATETIME -- to hold the new schedule date and time, actual value to insert into queue
-- pull the current schedule date/time from the queue table
SELECT @CurrentScheduleDate = NextRunDateTime
FROM dbo.GenericReportingQueue (NOLOCK)
WHERE IsGenerated IS NULL
AND ReportName = @ReportName
-- to override for testing
--SET @CurrentScheduleDate = '5/20/2016'
SET @CurrentScheduleDayNumberOfWeek = DATEPART(WEEKDAY, @CurrentScheduleDate)
-- pull these values from lookup table
SELECT @ScheduleLookupType = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_ScheduleLookupType'),
@TimeOfDayToScheduleJob = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_TimeOfDayToScheduleJob'),
@SpecialScheduleValue = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_SpecialScheduleValue')
/*
-- reset for testing
SET @ScheduleLookupType = 'specialdays' -- weekly, weekdays, monthly, specialdays
SET @TimeOfDayToScheduleJob = '8:00'
SET @SpecialScheduleValue = '5,6'
*/
-- calculations to get the date to schedule the job next time based off logic
SELECT @NewScheduleDateONLY = CASE
WHEN @ScheduleLookupType = 'daily' THEN DATEADD(DAY, 1, @CurrentScheduleDate)
WHEN @ScheduleLookupType = 'weekly' THEN DATEADD(DAY, 7, @CurrentScheduleDate)
WHEN @ScheduleLookupType = 'monthly' THEN DATEADD(MONTH, 1, @CurrentScheduleDate)
WHEN @ScheduleLookupType = 'yearly' THEN DATEADD(YEAR, 1, @CurrentScheduleDate)
-- only run on weekdays and skip weekends
WHEN @ScheduleLookupType = 'weekdays' THEN
CASE
WHEN @CurrentScheduleDayNumberOfWeek IN (1, 2, 3, 4) THEN DATEADD(DAY, 1, @CurrentScheduleDate)
WHEN @CurrentScheduleDayNumberOfWeek = 5 THEN DATEADD(DAY, 3, @CurrentScheduleDate)
END -- end case for day of week
-- only run on weekends and skip weekdays
WHEN @ScheduleLookupType = 'weekends' THEN
CASE
WHEN @CurrentScheduleDayNumberOfWeek = 6 THEN DATEADD(DAY, 1, @CurrentScheduleDate)
WHEN @CurrentScheduleDayNumberOfWeek = 7 THEN DATEADD(DAY, 6, @CurrentScheduleDate)
END -- end case for weekends only
WHEN @ScheduleLookupType = 'specialdays' THEN
-- for this we need to determine the current day, and the next day we want to run on, then add that many days
-- if next day is not till the following week we just find the first day in the list
-- Take taht number and do dateadd to it
DATEADD(DAY,
-- this does the select to determine what number to add based off current day and next day list
(SELECT ISNULL(
-- if this one I want to take today value and subtract from next value found
-- then add that number to todays date to give me the next schedule date
(SELECT TOP 1 StringValue - @CurrentScheduleDayNumberOfWeek
FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')
WHERE StringValue > @CurrentScheduleDayNumberOfWeek
ORDER BY StringValue)
,
-- if none found above I need to go to the next weeks first value
-- I need to take 7 - todays number (to get the rest of the week) then add the next number for the next week to it
(SELECT TOP 1 (7 - @CurrentScheduleDayNumberOfWeek) + StringValue
FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')
ORDER BY StringValue)
)-- end is null
) -- end select
, @CurrentScheduleDate) -- end dateadd for speical days
END -- outer case
SET @NewScheduleDateTime = @NewScheduleDateONLY + ' ' + @TimeOfDayToScheduleJob
-- for testing
--SELECT @ScheduleLookupType AS ReportLookupType, @TimeOfDayToScheduleJob AS TimeOfDayToSchedule, @SpecialScheduleValue AS SpecialValuesForCalc, @NewScheduleDateTime AS NewDateTimeToRun,
--@CurrentScheduleDate AS CurrentDateSchedule, @CurrentScheduleDayNumberOfWeek AS CurrentNumberDayOfWeek, @NewScheduleDateONLY AS NewScheduleDateOnly
-- &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
-- now update and insert the new schedule date/time into the table
-- update existing record
UPDATE dbo.GenericReportingQueue
SET IsGenerated = 1,
DateReportRun = GETDATE(),
LastUpdateDate = GETDATE()
WHERE ISGenerated IS NULL
AND ReportName = @ReportName
-- insert new record with new date
INSERT INTO dbo.GenericReportingQueue (
ReportName, NextRunDateTime, CreatorID, CreateDate
)
SELECT @ReportName, @NewScheduleDateTime, 1, GETDATE()
END TRY
BEGIN CATCH
RETURN
END CATCH