Question

我们使用Azure服务总线SDK 3.1.7的SendBatch（）方法将数据发送到Azure Event Hub。以下是代码段：

foreach (var packet in transformedPackets)
        {
            EventData eventData = new EventData(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(packet, settings)));;
            try
            {
                eventData.Properties.Add(AppConstants.TenantDeploymentUniqueIdKey, tenantDeploymentUniqueId);
                eventData.Properties.Add(AppConstants.DataTypeKey, DataTypeKey);
                byteCount += eventData.SerializedSizeInBytes;                   
                if (byteCount > MaxBatchSize)
                {
                    sender.SendBatch(transformedMessages);
                    transformedMessages.Clear();
                    byteCount = eventData.SerializedSizeInBytes;
                }
                transformedMessages.Add(eventData);
            }
            catch (System.Exception)
            {
                eventData.Dispose();
                throw;
            }

        }

在将事件数据添加到批处理（最大允许限制：256 KB）之前检查SerializedSizeInBytes属性后的事件，我们得到以下异常：

收到的消息（delivery-id：0，大小：262279字节）超过了链接上当前允许的限制（262144字节）。在Microsoft.ServiceBus.Common.ExceptionDispatcher.Throw（异常异常）在Microsoft.ServiceBus.Common.AsyncResult.End [TAsyncResult]（IAsyncResult结果）在Microsoft.ServiceBus.Common.AsyncResult 1.End(IAsyncResult asyncResult) at Microsoft.ServiceBus.Messaging.MessageSender.RetrySenderEventDataAsyncResult.End(IAsyncResult r) at Microsoft.ServiceBus.Messaging.MessageSender.EndSendEventData(IAsyncResult result) at Microsoft.ServiceBus.Messaging.EventHubSender.SendBatch(IEnumerable 1 eventDataList）

Answer 1

当事件被转换为单个$('body').on('change', '#id', function() { // Action goes here. });时，一些附加信息将添加到批处理标题中。

确保所有AmqpMessage的总大小低于250k。

您可以在此处找到此问题的详细信息：How to use client-side event batching functionality while Sending to Microsoft Azure EventHubs

Answer 2

我也在Event Hub上面临同样的问题。但处理这些功能的聪明方法是利用递归调用。在这里发布适合我的示例代码。

方法1：处理正常SendBatchAsync操作

public async Task SendBatchToEHAsync(IEnumerable<string> inputContent)  // Taking List of String to process (50000 Lines Approx)
{
    try
    {
        var batch = new List<EventData>();
        foreach (var item in inputContent) // Add it to Batch
            {
                var bind = new EventData(Encoding.UTF8.GetBytes(item));
                batch.Add(bind);
            }
            try
            {
                await eventHubClient.SendBatchAsync(batch);

            }
            catch (MessageSizeExceededException)
            {
                await ReprocessQuotaExceededBatch(batch);
            }
    }
    catch (Exception ex)
    {

        throw;
    }
}

方法2：处理和纠正递归函数以处理＆＃34; MessageSizeExceededException＆＃34;

 public async Task ReprocessQuotaExceededBatch(List<EventData> batch)
{
    try
    {
        var noOfCalls = 2;
       var noOfRecordsPerSplitBatch = (int)Math.Ceiling((double)batch.Count / noOfCalls);
        var counter = 0;
        while (counter < noOfCalls)
        {
            var insertSubList = GetSubListForEventDataBatch(batch, counter);
            try
            {
                await eventHubClient.SendBatchAsync(insertSubList);

            }
            catch (MessageSizeExceededException)
            {
                await ReprocessQuotaExceededBatch(insertSubList);
                // Perform SPlit processing in case of this exception occures
            }
            counter++;
        }
    }
    catch (Exception ex)
    {

        throw;
    }
}

方法3：为物品分割的辅助方法

 private static List<EventData> GetSubListForEventDataBatch(List<EventData> list, int count)
{

    var sList = list.Skip(count * noOfRecordsPerSplitBatch).Take(noOfRecordsPerSplitBatch);
    List<EventData> subList = new List<EventData>();
    subList.AddRange(sList);
    return subList;
}

希望这能为他人解决问题。如果有的话，请写下您对此代码的改进。

超过Azure服务总线SDK的SendBatch（）方法上的链接上的当前允许的限制（262144字节）

2 个答案: