我很难为C＃中实现的解决方案提供最佳设计或模式。
问题 -
我的应用程序将异步调用外部WebService来检索响应。外部Web服务每个请求只能使用5000个地址。但是我希望我的方法是，如果我的用户上传了3000个地址我想将它们分成1000个批次并传递给外部WebService并得到响应但是如果用户上传了15,000个我希望将它们拆分成包5000并传递给外部Web服务。
我要调用的外部Web服务上的方法的伪类型代码如下：
tokenId = GetInfoForAddress(Addresses[] addresses)
progStatus = GetProgStatus(tokenId);
if (progrStatus == "Complete")
{
addressResponse = GetResults(tokenId);
}
else
{
//handle other statuses
}
所以基本上webservice返回我传入的一些地址的令牌ID，然后我可以根据令牌ID获取服务的状态，然后如果状态完成，我可以得到结果。
但是我不确定处理这个案例的最佳方法是我需要首先考虑一下地址，然后说如果少于5000分成1000个包，如果超过5000分成5000个包看起来喜欢。我需要在拆分之前生成一个唯一的GUID，以便我可以将结果打包到原始的3000或15000，例如当结果返回时。
有没有人做过类似的事情，或许可以提供一个简单的伪代码或使用过程/模式的序列图？
答案 0 :(得分：1)
“最佳”模式是将所有这些管道内部保留在内部，并在发送1000或10000个地址时获得以API形式公开的唯一调用方法和行为。
这在AMQP或JMS等消息传递应用中很常见。
我编写了一个嵌入式AMQP brocker的代码，其中transfert可以根据客户端/浏览器接受的大小分成chunck。 整个请求存储在未结算的映射中（使用唯一键）并保持不稳定状态，直到所有部分都已成功处理。
希望这会有所帮助。
private async Task<uint> InternalTransfertAsyncWithGaranty(IAMQPEnvelope p_envelope, CancellationToken p_token)
{
// ask session the delivery id
uint l_deliveryId = Session.MessageIdentificationProvider.NewIdentifier(this.Handle);
// add to unsettled map
_unsettled.Add(l_deliveryId, p_envelope);
try
{
// transfert
await InternalSplitTransfertAsync(p_envelope.Key, l_deliveryId, p_envelope.Payload, true, p_token);
}
catch
{
// if something goes wrong, such cancellation or I/O
IAMQPEnvelope l_env;
if (_unsettled.Remove(l_deliveryId, out l_env))
{
}
// lets Exception exit
throw;
}
return l_deliveryId;
}
private async Task InternalSplitTransfertAsync(string p_deliveryTag, uint p_deliveryId, ArraySegment<byte> p_payload, bool p_settled, CancellationToken p_token)
{
IAMQPConnection l_conn = Session.Connection as IAMQPConnection;
uint l_blockSize = l_conn.Capabilities.MaxFrameSize;
int l_initialsize = p_payload.Count;
if (l_initialsize < l_blockSize)
{
// just send one transfert
await InternalSingleTransfertAsync(p_deliveryTag, p_deliveryId, false, p_payload, p_settled, p_token);
}
else
{
// cut transfert onto pieces. Message MUST NOT be interleaved. The order of delivery lets message beeing reconstrucetd.
int l_offset = p_payload.Offset;
int l_count = (int)l_blockSize;
int l_remain = l_initialsize;
bool l_more = true;
int l_size = 0;
do
{
l_size = l_remain <= l_count ? l_remain : l_count;
l_more = l_offset + l_size < l_initialsize;
await InternalSingleTransfertAsync(null, p_deliveryId, l_more, new ArraySegment<byte>(p_payload.Array, l_offset, l_size), p_settled, p_token);
l_offset += l_size;
l_remain -= l_size;
} while (l_more);
}
}
private async Task InternalSingleTransfertAsync(string p_deliveryTag, uint p_deliveryId, bool p_more, ArraySegment<byte> p_payload, bool p_settled, CancellationToken p_token)
{
AMQPTransfert l_transfert = InternalBuildTransfert(p_deliveryTag, p_settled, p_deliveryId, p_more);
AMQPFrame l_frame = InternalBuildFrame(l_transfert, p_payload);
await Session.SendAsync(l_frame, p_token);
}
答案 1 :(得分：0)
我认为在请求者中我会有这样的方法：
public List<string> GetResponses(params string[] addresses)
{
var result = new List<string>();
var skipCount = 0;
var takeCount = addresses.Length < 5000 ? 1000 : 5000;
string[] request;
while ((request = addresses.Skip(skipCount).Take(takeCount).ToArray()).Length > 0)
{
skipCount += takeCount;
result.Add(service.Request(request));
}
return result;
}
在控制器中，我想我会使用BackgroundWorker来检查响应的状态。因此，可能在控制器的ctor中我会初始化private字段：
private BackgroundWorker _worker = new BackgroundWorker();
public Controller()
{
_worker.WorkerReportsProgress = true;
_worker.DoWork += DoWork;
_worker.ProgressChanged += ProgressChanged;
_worker.RunWorkerCompleted += RunWorkerCompleted;
}
private void ControllerMethod()
{
var tokens = requester.GetResponses(addresses);
_worker.RunWorkerAsync(tokens);
}
private void DoWork(object sender, DoWorkEventArgs e)
{
var tokens = e.Argument as List<string>;
if (tokens == null) { return; }
// declare the result variable here
// I don't know what it is; but you'll concatenate
// the results below where I show the response
var completedTokens = new List<string>();
while (completedTokens.Count < tokens.Count)
{
foreach (var tokenId in tokens)
{
var status = GetProgStatus(tokenId);
if (status == "Complete")
{
completedTokens.Add(tokenId);
// maybe call
// worker.ReportProgress(completedTokens.Count, "some status text");
var response = GetResults(tokenId);
// if response is a list or array, just use LINQ to
// add the response to an overall result that is declared
// outside of the while loop
}
else
{
// check other statuses here and maybe call
// worker.ReportProgress(completedTokens.Count, "some status text");
}
}
// sleep for some duration and check again
Thread.Sleep(1000);
}
e.Result = result;
}
private void ProgressChanged(object sender, ProgressChangedEventArgs e)
{
// here you can use that real percentage complete value to set
// a progress bar; or maybe SignalR to send a progress if you're
// inside a web application
// this.progressBar1.Value = e.ProgressPercentage;
// and now here you can even grab that textual progress
var progress = e.UserState as string;
}
private void RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
// here you can now get at that result
var result = e.Result as {SomeType};
}
通过completedTokens.Count向ReportProgress发送，您获得了真正的进步。如果有3个令牌返回，1个完成，那么你真的完成了33％。
答案 2 :(得分：0)
可能类似以下内容：
public class GetInfoForAddressAggregate
{
private AddressBatch[] _batches;
public GetInfoForAddressAggregate(Address[] addresses)
{
if (addresses.Length >= 5000)
{
_batches = SplitIntoBatches(addresses, 5000);
}
else
{
_batches = SplitIntoBatches(addresses, 1000);
}
}
private AddressBatch[] SplitIntoBatches(Address[] addresses, int size)
{
// stubbed method
return null;
}
public void Run()
{
foreach (var batch in _batches)
{
batch.Run();
}
}
public bool IsComplete
{
get { return _batches.All(x => x.IsComplete); }
}
private class AddressBatch
{
private Address[] _addresses;
private string _token;
public string Token
{
get { return _token; }
}
public AddressBatch(Address[] addresses)
{
_addresses = addresses;
}
public void Run()
{
_token = GetInfoForAddress(_addresses);
}
private string GetProgStatus(string token)
{
// stubbed method
return "Complete";
}
private string GetInfoForAddress(Address[] addresses)
{
// stubbed method
return "token";
}
public bool IsComplete
{
get { return _token != null && GetProgStatus(_token) == "Complete"; }
}
}
我认为您的目标是简化和隐藏批量拆分的细节，以及此聚合的功能。调用者不必了解批量拆分规则，并且可以像使用它一样使用它。