Question

所以我目前正在尝试提高应用程序的下载速度，为此我试图实现分段下载。

代码就像魅力一样，但在5个测试用例中，2次生成的文件无法打开或看起来很奇怪。

import java.nio.ByteBuffer
import java.nio.channels.ClosedChannelException
import java.nio.channels.FileChannel

//https://www.fareway.com/stores/ia/cresco/112-south-elm-street/ad/weekly/download

class Playground {

final int SEGMENT_SIZE = 8192 * 8
final int THREAD_COUNT = 8
final int UPDATE_INTERVAL = 2

boolean running = false
int target_size

static void main(String[] args){
    new Playground()
}

Playground(){
    running = true
    long time_started_execution = System.currentTimeMillis()
    URL testurl = new URL("https://www.fareway.com/stores/ia/cresco/112-south-elm-street/ad/weekly/download")
    target_size = testurl.openConnection().getContentLength()

    URL url = new URL("https://www.fareway.com/stores/ia/cresco/112-south-elm-street/ad/weekly/download")
    File outFile = new File("testfile.pdf")
    if(outFile.exists()) outFile.delete()

    RandomAccessFile raf = new RandomAccessFile(outFile, "rw")
    FileChannel channel = raf.getChannel()

    def thread_list = []
    THREAD_COUNT.times { int n ->
        def t = Thread.start {
            int pos = n * SEGMENT_SIZE
            while(downloadSegment(url, pos, channel)){
                pos+=THREAD_COUNT*SEGMENT_SIZE
            }
        }
        t.setName("Download Thread T$n")
        thread_list << t
    }

    def monitor_thread = Thread.start{
        while(running){
            try{
                int before = channel.size()
                sleep(UPDATE_INTERVAL * 1000)
                double speed = (channel.size() - before) / UPDATE_INTERVAL
                print("\rDownloading with ${speed / 1024} kb/s. (${channel.size() / 1024} / ${(target_size / 1024) as int} kb)")
            }catch (ClosedChannelException cce){
                running = false
            }
        }
    }

    thread_list.each{
        it.join()
    }
    running = false
    println("\n(${channel.size()} / ${(target_size)} bytes)")
    raf.close()
    println("Download finished in ${(System.currentTimeMillis() - time_started_execution) / 1000} s!")
}

boolean downloadSegment(URL url, int position, FileChannel channel){
    if(position > target_size){
        return false
    }
    int endPos = position+SEGMENT_SIZE-1
    if(endPos > target_size){
        endPos = target_size
    }
    HttpURLConnection conn = url.openConnection() as HttpURLConnection
    conn.setRequestProperty("Range", "bytes=$position-$endPos")
    //println("${Thread.currentThread().getName()}: bytes=$position-${position+SEGMENT_SIZE-1}")
    conn.connect()
    int current_segment_size = conn.getContentLength()
    if(conn.getResponseCode() != 200 && conn.getResponseCode() != 206){
        //println("\n${Thread.currentThread().getName()}: No more data!")
        return false
    }

    byte[] buffer = conn.getInputStream().getBytes()
    ByteBuffer bf = ByteBuffer.wrap(buffer)

    //Set Channel Position to write at the correct place.
    channel.position(position)

    while(bf.hasRemaining()){
        channel.write(bf)
        channel.force(false)
    }
    bf.clear()

    return current_segment_size >= SEGMENT_SIZE
}
}

那么有没有人知道我可以尝试避免损坏的文件？我确实尝试降低Thread_Count和Segment_Size，如果它们更低，文件不太可能被破坏，但它也将下载速度从大约1.5 Mb / s降低到400 kb / s ...而且我想得到这里的最高速度。

提前感谢您的帮助。

P.S。：我知道缺少try-catch或其他东西。但这只是一个Groovy Playground，可以在安全的环境中测试这个功能。

分段下载文件，有时已损坏？

0 个答案: