使用SSIS的定期任务

时间:2018-06-20 22:22:57

标签: sql ssis cron

我创建了一个新的SSIS包(SQL 2017),该包会读取“导出请求”表,并根据计划将数据导出到外部存储。例如,用户可以向该表添加新行,如“选择*从表1中col1 = 2的表中选择*”,其计划为2018年6月25日下午3:30。

我的程序包(使用SQL代理运行)一直循环搜索该表中的新任务,并按计划将查询结果当时导出到文件夹中。

有一项新功能要求将这些导出计划为定期进行。例如,客户可能希望从2018年6月25日下午3:30开始每天进行出口。每年,每月,每天,每3个月,...的时间表可能会变得很复杂。

在SSIS中实现重复执行的最佳方法是什么?我可以为Cron使用5个字符的字段,但不知道如何使用其Cron时间表查询新任务。 有什么帮助吗?

1 个答案:

答案 0 :(得分:1)

确认已拥有的内容:您有一个TASK表,其中包含任务,以及某种日期标记,用于指示何时需要运行该表。然后,您的作业每x分钟运行一次,并检查是否有任何要拾取/运行的任务。如果是这样,它将运行它们。然后,我猜您在设置的表中有一个标记,说在循环中运行它后完成了吗?如果您一次要运行多个任务,那么一次只能运行一次?

然后您想要添加的内容是:我做了一些非常相似的事情。我为每个任务创建了一个查找表(在我的过程中,每个任务都有一个名称,并且我使用该名称来引用该查找表,我将其称为计划查找)。

在该表中,我放置了报表的运行时间和频率(如此2列),以及时间和频率。一个示例是7:00和工作日。因此,对于此报告,它仅在每个工作日(m-F)晚上7点运行。

然后,当我的作业运行时,它将像上面一样运行该任务,但是还有另一步,即将该任务标记为完成,然后在任务表中插入一个新任务(任务详细信息是相同的),但我将查看上述的计划查找表,以找出作业应再次运行的下一个日期/时间,并将其用作任务表中的下一个运行日期/时间。

下面是我在流程中使用的SP,用于获取下一个计划的日期/时间并更新现有的计划/日期,然后创建新的计划。

注意:我的日程安排中有一些您可能不需要的高级选项,我有很多评论可以解释为什么/为什么要做什么。我正在调用一些我在某些地方创建的函数,但是我认为您不需要这些函数,而是可以找出要做什么而不是我的函数,但是如果您有任何疑问,请告诉我:

这就是我所使用的,因此它正在使用我的表结构/ etc,但是您可以很容易地将其转换为您的表结构/

--Purpose
----------------------------------------------------------------------------------
-- calculates the next time to run/schedule the job
----------------------------------------------------------------------------------
    -- NOTE:  TO USE you have to insert the first value manually in queue table

    ----- possibile scenerios
    -- if we want to schedule every x hours, or x days
    -- run every month only
    -- run weekdays only
    -- run on certain days of month only


    -- TO ADD MORE COMPLEX or different types of schedules:
    --  special - different times for different days of week
    --          ex - so have dayofweek:2:00,dayofweek:3:00  (and we parse out the day of week and number splitting out the strings)

    --  hourly - to do more then once a day??
    --          WHEN @ScheduleLookupType = 'hourly' THEN DATEADD(DAY, 1, @CurrentScheduleDate) -- FIX FIX FIX


--  EXEC dbo.JobsDynamicRescheduleFindNextTimeToScheduleJob @ReportName = 'TestReport1'

----------------------------------------------------------------------------------


ALTER PROCEDURE [dbo].[JobsDynamicRescheduleFindNextTimeToScheduleJob]
    @ReportName VARCHAR(50)
AS

SET NOCOUNT ON

BEGIN TRY
            -- left here for testing outside of SP
            -- this will be passed from SP
            --DECLARE @ReportName AS VARCHAR(50)    
            --SET @ReportName = 'TESTREport'


            -- this sets the first day of the week to Monday (we need it set to a value to do calcluations for weekdays, I set it to 1 for monday and 7 for sunday)
            --  this is due to server settings could have somethign else so forcing it here
            SET DATEFIRST 1

            DECLARE @CurrentScheduleDate AS DATE                    -- find the current date for the job that just ran
            DECLARE @CurrentScheduleDayNumberOfWeek AS SMALLINT     -- this pulls the number of the day of week 1=monday, 2=tuesdday
            DECLARE @ScheduleLookupType AS VARCHAR(20)              -- this is the type of schedule to do calculations on
            DECLARE @TimeOfDayToScheduleJob AS VARCHAR(20)          -- look this up, its the time to schedule the job
            DECLARE @SpecialScheduleValue AS VARCHAR(8000)          -- this is special value to lookup (only needed if non standard one)
            DECLARE @NewScheduleDateONLY AS DATETIME                -- to hold just the date of the schedule before combinng with time
            DECLARE @NewScheduleDateTime AS DATETIME                -- to hold the new schedule date and time, actual value to insert into queue


            -- pull the current schedule date/time from the queue table
            SELECT @CurrentScheduleDate = NextRunDateTime
            FROM dbo.GenericReportingQueue (NOLOCK)
            WHERE IsGenerated IS NULL
            AND ReportName = @ReportName


            -- to override for testing
            --SET @CurrentScheduleDate = '5/20/2016'


            SET @CurrentScheduleDayNumberOfWeek = DATEPART(WEEKDAY, @CurrentScheduleDate)

            -- pull these values from lookup table
            SELECT @ScheduleLookupType = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_ScheduleLookupType'), 
            @TimeOfDayToScheduleJob = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_TimeOfDayToScheduleJob'),
            @SpecialScheduleValue = dbo.fn_GetValueLookupTableValue(@ReportName, 'RescheduleJobDynamic_SpecialScheduleValue')


            /*
            -- reset for testing
            SET @ScheduleLookupType = 'specialdays' -- weekly, weekdays, monthly, specialdays
            SET @TimeOfDayToScheduleJob = '8:00'
            SET @SpecialScheduleValue = '5,6'
            */

            -- calculations to get the date to schedule the job next time based off logic
            SELECT @NewScheduleDateONLY = CASE  
                WHEN @ScheduleLookupType = 'daily' THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'weekly' THEN DATEADD(DAY, 7, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'monthly' THEN DATEADD(MONTH, 1, @CurrentScheduleDate)
                WHEN @ScheduleLookupType = 'yearly' THEN DATEADD(YEAR, 1, @CurrentScheduleDate)

                -- only run on weekdays and skip weekends
                WHEN @ScheduleLookupType = 'weekdays' THEN 
                    CASE 
                        WHEN @CurrentScheduleDayNumberOfWeek IN (1, 2, 3, 4) THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                        WHEN @CurrentScheduleDayNumberOfWeek = 5 THEN DATEADD(DAY, 3, @CurrentScheduleDate)
                    END -- end case for day of week
                -- only run on weekends and skip weekdays
                WHEN @ScheduleLookupType = 'weekends' THEN 
                    CASE
                        WHEN @CurrentScheduleDayNumberOfWeek = 6 THEN DATEADD(DAY, 1, @CurrentScheduleDate)
                        WHEN @CurrentScheduleDayNumberOfWeek = 7 THEN DATEADD(DAY, 6, @CurrentScheduleDate)
                    END -- end case for weekends only

                WHEN @ScheduleLookupType = 'specialdays' THEN
                    -- for this we need to determine the current day, and the next day we want to run on, then add that many days
                    --      if next day is not till the following week we just find the first day in the list
                    --      Take taht number and do dateadd to it
                    DATEADD(DAY, 
                            -- this does the select to determine what number to add based off current day and next day list
                            (SELECT ISNULL(
                                -- if this one I want to take today value and subtract from next value found
                                --      then add that number to todays date to give me the next schedule date
                                (SELECT TOP 1 StringValue - @CurrentScheduleDayNumberOfWeek
                                FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')
                                WHERE StringValue > @CurrentScheduleDayNumberOfWeek
                                ORDER BY StringValue)
                                ,
                                -- if none found above I need to go to the next weeks first value
                                --      I need to take 7 - todays number (to get the rest of the week) then add the next number for the next week to it
                                (SELECT TOP 1 (7 - @CurrentScheduleDayNumberOfWeek) + StringValue 
                                FROM dbo.fn_ParseText2Table(@SpecialScheduleValue, ',')                 
                                ORDER BY StringValue)
                                )-- end is null
                            ) -- end select 
                        , @CurrentScheduleDate) -- end dateadd for speical days             
            END -- outer case


            SET @NewScheduleDateTime = @NewScheduleDateONLY + ' ' + @TimeOfDayToScheduleJob

            -- for testing
            --SELECT @ScheduleLookupType AS ReportLookupType, @TimeOfDayToScheduleJob AS TimeOfDayToSchedule, @SpecialScheduleValue AS SpecialValuesForCalc, @NewScheduleDateTime AS NewDateTimeToRun, 
            --@CurrentScheduleDate AS CurrentDateSchedule, @CurrentScheduleDayNumberOfWeek AS CurrentNumberDayOfWeek, @NewScheduleDateONLY AS NewScheduleDateOnly


            -- &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
            --      now update and insert the new schedule date/time into the table

            -- update existing record
            UPDATE dbo.GenericReportingQueue
            SET IsGenerated = 1,
            DateReportRun = GETDATE(),
            LastUpdateDate = GETDATE()
            WHERE ISGenerated IS NULL
            AND ReportName = @ReportName


            -- insert new record with new date
            INSERT INTO dbo.GenericReportingQueue (
                ReportName, NextRunDateTime, CreatorID, CreateDate
            )
            SELECT @ReportName, @NewScheduleDateTime, 1, GETDATE()



END TRY

BEGIN CATCH

   RETURN

END CATCH