为批处理任务实现通用进度跟踪器

时间:2015-07-23 14:34:27

标签: java background-process generic-programming

我目前正在开发一个项目,其中包含许多后台执行任务。创建每个任务,然后将其发送到管理所有任务执行过程的并发服务。每个任务在执行期间都存储在数据库表中。

我的困境是每个任务执行非常具体的功能,并且通常大多数委托给系统的另一部分中的服务,工作的主要部分完成,然后任务返回。

目前我有一个非常简单的系统实现跟踪任务的进度,它运行良好,但是每个执行的任务都需要添加大量额外的代码来适应它所委托的服务的功能。

因此,作为一个例子,我的任务将有一个方法:

@Override
public void execute() {
    service.calculateAverage();
}

然后相应地在服务中:

public float calculateAverage() {
    float total = 0.0f; 
    for (i = 0; i < 20; i++) {
        total += i;
    }
    return total / 20;
}

跟踪这一过程相当简单,我只是在超过某个迭代阈值后更新我在数据库中的任务。然而,要证明这一点是一项相当重要的任务,因为执行的每个任务都可能完全委托给另一个服务。这意味着在每个服务中,我需要添加特定于该服务的实现的代码。

我做了一些搜索,我似乎无法找到任何有助于创建跟踪每项任务进度的通用系统的好模式。任何指针甚至只是寻找或阅读的地方都会很好。

1 个答案:

答案 0 :(得分:2)

如果您使用自己的// Target endpoint. Use the default Barix UDP 'high priority' // port. IPEndPoint target = new IPEndPoint( IPAddress.Parse("192.168.1.100"), 3030 ); // Create reader... NAudio.Wave.Mp3FileReader reader = new Mp3FileReader("hello.mp3"); // Build a simple udp-socket for sending. Socket sender = new Socket( AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp ); // Now for some 'constants.' double ticksperms = (double)Stopwatch.Frequency; ticksperms /= 1000.0; // We manage 'buffering' by just accumulating a linked list of // mp3 frame times. The maximum time is defined by our buffer // time. For this example, use a 2-second 'buffer'. // 'framebufferticks' tracks the total time in the buffer. double framebuffermaxticks = ticksperms * 2000.0; LinkedList<double> framebuffer = new LinkedList<double>(); double framebufferticks = 0.0f; // We'll track the total mp3 time in ticks. We'll also need a // stopwatch for the internal timing. double expectedticks = 0.0; Stopwatch sw = new Stopwatch(); long startticks = Stopwatch.GetTimestamp(); // Now we just read frames until a null is returned. int totalbytes = 0; Mp3Frame frame; while( (frame = reader.ReadNextFrame()) != null ) { // Make sure the frame buffer is valid. If not, we'll // quit sending. byte [] rawdata = frame.RawData; if( rawdata == null ) break; // Send the complete frame. totalbytes += rawdata.Length; sender.SendTo(rawdata, target); // Timing is next. Get the current total time and calculate // this frame. We'll also need to calculate the delta for // later. double expectedms = reader.CurrentTime.TotalMilliseconds; double newexpectedticks = expectedms * ticksperms; double deltaticks = newexpectedticks - expectedticks; expectedticks = newexpectedticks; // Add the frame time to the list (and adjust the total // frame buffer time). If we haven't exceeded our buffer // time, then just go get the next packet. framebuffer.AddLast(deltaticks); framebufferticks += deltaticks; if( framebufferticks < framebuffermaxticks ) continue; // Pop one delay out of the queue and adjust values. double framedelayticks = framebuffer.First.Value; framebuffer.RemoveFirst(); framebufferticks -= framedelayticks; // Now we just wait.... double lastelapsed = 0.0f; sw.Reset(); sw.Start(); while( lastelapsed < framedelayticks ) { // We do short burst delays with Socket.Poll() because it // supports a much higher timing precision than // Thread.Sleep(). sender.Poll(100, SelectMode.SelectError); lastelapsed = (double)sw.ElapsedTicks; } // We most likely waited more ticks than we expected. Timewise, // this isn't a lot. But it could cause accumulate into a large // 'drift' if this is a very large file. We lower the total // buffer/queue tick total by the overage. if( lastelapsed > framedelayticks ) { framebufferticks -= (lastelapsed - framedelayticks); } } // Done sending the file. Now we'll just do one final wait to let // our 'buffer' empty. The total time is our frame buffer ticks // plus any latency. double elapsed = 0.0f; sw.Reset(); sw.Start(); while( elapsed < framebufferticks ) { sender.Poll(1000, SelectMode.SelectError); elapsed = (double)sw.ElapsedTicks; } // Dump some final timing information: double diffticks = (double)(Stopwatch.GetTimestamp() - startticks); double diffms = diffticks / ticksperms; Console.WriteLine("Sent {0} byte(s) in {1}ms (filetime={2}ms)", totalbytes, diffms, reader.CurrentTime.TotalMilliseconds); ,而不是让它创建循环。

Iterator

然后,您可以创建一个class SumService { private float calculateSum(Iterable<Integer> i) { float total = 0.0f; for (Integer x : i) { total += x; } return total; } } 来跟踪进度并将其报告给进度跟踪器。

Iterable

这管理责任分离。该服务仅为/** * State of progress - returns a double result between 0 and 1. * * Will be called repeatedly by the progress tracker. */ interface Progress { public double getProgress(); } /** * The progress tracker. */ static class ProgressTracker { // All processes are registered. static void registerProgressor(Progress p) { // Add it to mmy list of people to watch. } } /** * An Iterable that reports its progress. */ class ProgressingIterable<T> implements Iterable<T>, Progress { // The iterable we are hosting. final Iterable<T> it; // How far we are to go. final int steps; // Where we're at now. volatile int at = 0; public ProgressingIterable(Iterable<T> it, int steps) { this.it = it; this.steps = steps; } @Override public Iterator<T> iterator() { return new Iterator<T>() { // Grab an Iterator from the Iterable. Iterator<T> i = it.iterator(); @Override public boolean hasNext() { // Delegate. return i.hasNext(); } @Override public T next() { // Keep track of the steps. at++; return i.next(); } }; } @Override public double getProgress() { // How are we doing? return (double) at / (double) steps; } } /** * A range (from http://stackoverflow.com/a/6828887/823393). * * @param begin inclusive * @param end exclusive * @return list of integers from begin to end */ public static List<Integer> range(final int begin, final int end) { return new AbstractList<Integer>() { @Override public Integer get(int index) { return begin + index; } @Override public int size() { return end - begin; } }; } /** * A process. */ class Process { ProgressingIterable<Integer> progress = new ProgressingIterable<>(range(0, 20), 20); public void execute() { // Register the Progress ProgressTracker.registerProgressor(progress); // Make the service use my progress object. service.calculateSum(progress); } } // The service it uses. SumService service = new SumService(); ,而对于进度跟踪器,它会在被询问时提供当前进度。

我将此称为Janus模式,因为您有一个对象可以完成两件事。它允许您将两个进程绑定在一个对象中。

我选择了最简单的进度指标 - Iterabledouble之间的0。我相信你能做得更好。