长时间运行的Azure WebJob获取重复消息

时间:2016-08-30 16:08:07

标签: c# azure azure-webjobs

我有一个长期运行的WebJob,它将上传的文本文件导入数据库。整个过程大约需要5到10分钟。作业由消息触发(即,它不是预定的WebJob)。

在调试时查看日志消息我看到在作业的大约一半时,另一个实例似乎已启动(即,我可以看到初始化日志消息与已经运行的实例的日志消息交错)。

第二个实例失败,但原始实例继续运行并最终成功完成任务。

我是否需要手动将触发消息出列?或者有没有办法重置所涉及的超时等待更长时间?

这是Functions类:

public class Functions
{
private static readonly ImporterFactory _importFactory = new ImporterFactory();

public static void ProcessUploadedFile( [QueueTrigger( BlobStorage.UploadQueue )] UploadMessage msg, [Blob("uploads/{BlobPath}", FileAccess.Read)] Stream blobStream, TextWriter log )
{
    StringWriter swLog = new StringWriter();

    LoggerConfiguration logConfig = new LoggerConfiguration()
        .WriteTo.TextWriter( swLog );

    var config = new JobHostConfiguration();
    if( config.IsDevelopment ) logConfig = logConfig.WriteTo.TextWriter( log );

    ILogger logger = logConfig.CreateLogger();

    logger.Information( $"{msg.SourceFile} uploaded for processing" );
    logger.Information( $"File type is {msg.UploadType.ToString()}, primary ID is {msg.PrimaryID}" );

    if( blobStream == null )
    {
        logger.Information( $"blobStream is undefined; is file available?" );
        return;
    }

    // if we got a zip file, make sure it has only one entry
    string ext = Path.GetExtension( msg.SourceFile ) ?? "";
    Stream src = null;

    if( ext.ToLower() == ".zip" )
    {
        logger.Information( $"{msg.SourceFile} is a zip archive, checking to see if it contains only one file" );

        ZipArchive archive = new ZipArchive( blobStream, ZipArchiveMode.Read );

        switch( archive.Entries.Count )
        {
            case 0:
                logger.Fatal( "Uploaded zip file is empty" );
                break;

            case 1:
                src = archive.Entries[ 0 ].Open();
                break;

            default:
                logger.Fatal( "Uploaded zip file has more than 1 entry" );
                break;
        }
    }
    else src = blobStream;

    if( src == null ) return;

    TextReader tr = new StreamReader( src );

    switch( msg.UploadType )
    {
        case UploadType.CountyVoter:
            ITextFileImporter importer = _importFactory.GetCountyVoterImporter( msg.PrimaryID, tr, logger );
            if( importer == null )
            {
                logger.Fatal( $"Couldn't find a TextFileImporter Type supporting CountyID = {msg.PrimaryID}" );
                return;
            }

            bool okay = importer.Import();

            if( okay )
            {
                logger.Information( $"import succeeded" );
            }
            else logger.Fatal( "import failed" );

            break;

        default:
            logger.Fatal( $"Unknown import file type ({msg.UploadType.ToString()})" );
            break;
    }
}
}

0 个答案:

没有答案