多线程无法正常工作,无法将文件上传到S3中

时间:2018-11-28 09:15:14

标签: java multithreading amazon-s3

我已经问了很多问题,但是仍然无法调整我上传到S3的文件。 同样,我也不想把每个细节都放在一个问题中,而把阅读范围扩大到其他问题。

我已经用多线程编写了我的应用程序,以便将文件上传到3中。 我的多线程在某些地方无法正常工作,为什么我要说的原因是,当我打印上载状态时,我的输出低于输出

> Transfer: Uploading etc
>   - State: Waiting
>   - Progress: 0 Transfer: Uploading etc
>   - State: Waiting
>   - Progress: 0 Transfer: Uploading etc
>   - State: Waiting
>   - Progress: 0 Transfer: Uploading etc
>   - State: Waiting
>   - Progress: 0 Transfer: Uploading etc
>   - State: InProgress
>   - Progress: 0

我可以看到只有一个正在进行状态,其余所有线程都处于等待状态。 这是否表明我的线程无法正常工作。

我怀疑我是在TransferManager中创建Runnable并且正在传递构造函数。

这是我的代码。

这是我的执行人

public class UploadExecutor {
    private static String fileLocation = "C:\\Users\\u6034690\\Desktop\\TWOFILE\\Samplefiles";
    // private static String fileLocation="D:\\TRFAudits_Moved\\";
    private static final String _logFileName = "s3FileUploader.log";
    private static Logger _logger = Logger.getLogger(UploadExecutor.class);

    @SuppressWarnings("unchecked")
    public static void main(String[] args) {
        _logger.info("----------Stating application's  main method----------------- ");
        AWSCredentials credential = new ProfileCredentialsProvider("TRFAuditability-Prod-ServiceUser").getCredentials();
        final ClientConfiguration config = new ClientConfiguration();
        AmazonS3Client s3Client = (AmazonS3Client) AmazonS3ClientBuilder.standard().withRegion("us-east-1")
                .withCredentials(new AWSStaticCredentialsProvider(credential)).withForceGlobalBucketAccessEnabled(true)
                .build();
        s3Client.getClientConfiguration().setMaxConnections(100);
        TransferManager tm = new TransferManager(s3Client);
        // while (true) {
        FileProcessThreads fp = new FileProcessThreads();
        List<File> records = fp.process(fileLocation);
        while (records.size() <= 0) {
            try {
                _logger.info("No records found willl wait for 10 Seconds");
                TimeUnit.SECONDS.sleep(10);
                records = fp.process(fileLocation);
            } catch (InterruptedException e) {
                _logger.error("InterruptedException: " + e.toString());
            }
        }
        _logger.info("Total no of Audit files = " + records.size());
        ExecutorService es = Executors.newFixedThreadPool(6);
        int recordsInEachThread = (int) (records.size() / 6);
        _logger.info("No of records in each thread = " + recordsInEachThread);
        UploadObject my1 = new UploadObject(records.subList(0, recordsInEachThread), tm);
        UploadObject my2 = new UploadObject(records.subList(recordsInEachThread, recordsInEachThread * 2), tm);
        UploadObject my3 = new UploadObject(records.subList(recordsInEachThread * 2, recordsInEachThread * 3), tm);
        UploadObject my4 = new UploadObject(records.subList(recordsInEachThread * 3, recordsInEachThread * 4), tm);
        UploadObject my5 = new UploadObject(records.subList(recordsInEachThread * 4, recordsInEachThread * 5), tm);
        UploadObject my6 = new UploadObject(records.subList(recordsInEachThread * 5, records.size()), tm);
        es.execute(my1);
        es.execute(my2);
        es.execute(my3);
        es.execute(my4);
        es.execute(my5);
        es.execute(my6);
        es.shutdown();
    }
}

0 个答案:

没有答案