在从S3存储桶读取文件时无法模拟AWSClient,S3Object

时间:2018-01-22 09:01:43

标签: java amazon-s3 aws-sdk s3-bucket

我正在读取S3存储桶中的文件,如下所示:

class VersionServiceImpl implements VersionService {
VersionDto versiondto = new VersionDto();
    try {
        BasicAWSCredentials awsCreds = new BasicAWSCredentials(versionConfig.getAccessKeyId(), versionConfig.getSecretAccessKey());
        AmazonS3 s3Client=new AmazonS3Client(awsCreds);
        S3Object s3object = s3Client.getObject(new GetObjectRequest(versionConfig.getBucketKey(), versionConfig.getFileNameKey()));
        BufferedReader reader = new BufferedReader(new InputStreamReader(s3object.getObjectContent()));
        while (true) {
            String line = reader.readLine();
            //System.out.println(" "+line);
            if (line == null) 
                break;

            String[] fields=line.split("=");
            switch (fields[0]) {
                case "MAJOR":
                    versiondto.setMajor(fields[1] != null ? fields[1] : "0");
                    break;
                case "MINOR":
                    versiondto.setMinor(fields[1] != null ? fields[1] : "0");
                    break;
                case "HOTFIXPATCH":
                    versiondto.setHotFixPatch(fields[1] != null ? fields[1] : "0");
                    break;
                default:
                    LOGGER.info("INVALID/EXTRA FIELD IN VERSION.TXT FILE", fields[0]);
            }
        }
        s3object.close();
    } catch (AmazonServiceException ase) {
        LOGGER.error("Caught an AmazonServiceException from GET requests", ase);
    } catch (AmazonClientException ace) {
        LOGGER.error("Caught an AmazonClientException", ace);
    } catch (IOException ioe) {
        LOGGER.error("IOE Exception reading file from S3", ioe);
    }
    return versiondto;
}

我正在尝试模拟AWS类并针对此方法运行jUnit测试,但我得到的例外情况很少。我尝试过mockito.spy,模拟aws类。但在测试读文件方法时仍然会发现异常 以下是JUnit测试:

public class VersionServiceTest {

@Mock
S3ObjectInputStream s3ObjectInputStream;

@InjectMocks
private VersionServiceImpl roleService;

@Mock
private VersionConfig versionConfig;

@Mock
private  S3Object s3Object ;

@Mock
BasicAWSCredentials awsCreds;

@Mock
private AmazonS3 s3Client;

@Before
public void init(){
    MockitoAnnotations.initMocks(this);
    when(versionConfig.getAccessKeyId()).thenReturn("AKIAJ6HT2QIXO452VW4A");
    when(versionConfig.getBucketKey()).thenReturn("version-test-ops/version");
    when(versionConfig.getFileNameKey()).thenReturn("version.txt");
    when(versionConfig.getSecretAccessKey()).thenReturn("6f+qJ/i0tQprEftE+pwa0CmnjTtdzoOYnuG0DGbN");
    awsCreds= new BasicAWSCredentials(versionConfig.getAccessKeyId(), versionConfig.getSecretAccessKey());
}

@Test
public void TestGetVersion() throws IOException {

    String bucket = "version-test-ops/version";
    String keyName = "version.txt";
    s3Object.setBucketName(bucket);
    s3Object.setKey(keyName);
    when(any(BasicAWSCredentials.class)).thenReturn(awsCreds);
    when(any(AmazonS3Client.class)).thenReturn((AmazonS3Client) s3Client);
    when(s3Client.getObject(any(GetObjectRequest.class))).thenReturn(s3Object);
    when(s3Object.getObjectContent()).thenReturn(s3ObjectInputStream);

    BufferedReader reader = Mockito.mock(BufferedReader.class);
    Mockito.when(reader.readLine()).thenReturn("ENV=DEV3", "MAJOR=0", "MINOR=0", "CRSPNG_VERSION_HOTFIXPATCH=384",
            "DATEOFDEPLOY=15-12-2017", "SPRINT=30");
    VersionDto dto= roleService.getVersionDetails();


}

}

以下是例外:

java.lang.LinkageError: loader constraint violation: when resolving overridden method "com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(ILjava/net/Socket;Lorg/apache/http/HttpHost;Ljava/net/InetSocketAddress;Ljava/net/InetSocketAddress;Lorg/apache/http/protocol/HttpContext;)Ljava/net/Socket;" the class loader (instance of org/powermock/core/classloader/MockClassLoader) of the current class, com/amazonaws/http/conn/ssl/SdkTLSSocketFactory, and its superclass loader (instance of sun/misc/Launcher$AppClassLoader), have different Class objects for the type org/apache/http/protocol/HttpContext used in the signature
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.getPreferredSocketFactory(ApacheConnectionManagerFactory.java:87)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:65)
at com.amazonaws.http.apache.client.impl.ApacheConnectionManagerFactory.create(ApacheConnectionManagerFactory.java:58)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:50)
at com.amazonaws.http.apache.client.impl.ApacheHttpClientFactory.create(ApacheHttpClientFactory.java:38)
at com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:260)
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:160)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:519)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:499)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:481)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:453)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:435)
at com.VersionServiceImpl.getVersionDetails(VersionServiceImpl.java:38)

2 个答案:

答案 0 :(得分:0)

模拟S3的最简单方法是不直接使用S3库。我使用Java 7中引入的FileSystem api,它专门设计为允许您编写一次代码并将其用于任何文件系统。对于单元测试,我使用了默认的FileSystem FileSystem.getDefault(),它只是您的本地FileSystem。为了生产,我创建了一个与S3对话的S3FileSystem类的实例。 Upplication在这里提供了一个非常好的S3文件系统api实现https://github.com/Upplication/Amazon-S3-FileSystem-NIO2

这种方法的优点是你不必模拟任何东西来测试你的代码,这使得测试和调试变得非常容易。

答案 1 :(得分:0)

我不确定为什么会收到您看到的错误,但是似乎由于没有注入所需类的模拟而是尝试直接实例化它们而使您变得复杂。我建议您按照以下方式注入AmazonS3

class VersionServiceImpl implements VersionService {
    private AmazonS3 s3Client;

    public VersionServiceImpl(AmazonS3 s3Client) {
        this.s3Client = s3Client;
    }

    public VersionDto getVersion() {
        VersionDto versiondto = new VersionDto();
        try {
            S3Object s3object = s3Client
                    .getObject(new GetObjectRequest(versionConfig.getBucketKey(), versionConfig.getFileNameKey()));
            BufferedReader reader = new BufferedReader(new InputStreamReader(s3object.getObjectContent()));
            while (true) {
                String line = reader.readLine();
                // System.out.println(" "+line);
                if (line == null)
                    break;

                String[] fields = line.split("=");
                switch (fields[0]) {
                case "MAJOR":
                    versiondto.setMajor(fields[1] != null ? fields[1] : "0");
                    break;
                case "MINOR":
                    versiondto.setMinor(fields[1] != null ? fields[1] : "0");
                    break;
                case "HOTFIXPATCH":
                    versiondto.setHotFixPatch(fields[1] != null ? fields[1] : "0");
                    break;
                default:
                    LOGGER.info("INVALID/EXTRA FIELD IN VERSION.TXT FILE", fields[0]);
                }
            }
            s3object.close();
        } catch (AmazonServiceException ase) {
            LOGGER.error("Caught an AmazonServiceException from GET requests", ase);
        } catch (AmazonClientException ace) {
            LOGGER.error("Caught an AmazonClientException", ace);
        } catch (IOException ioe) {
            LOGGER.error("IOE Exception reading file from S3", ioe);
        }
        return versiondto;
    }
}

从那里开始,您的测试仅需关心通过AmazonS3模拟。同样,仅当您在其中注入该读取器时,测试中BufferedReader的模拟声明才起作用。但是,模拟InputStream可能是您的最佳选择。对于该流模拟,您可以在测试中采用这种方式:

    String expectedContents = String.join("\n", new string[] {
        "ENV=DEV3", "MAJOR=0", "MINOR=0", 
        "CRSPNG_VERSION_HOTFIXPATCH=384", "DATEOFDEPLOY=15-12-2017", "SPRINT=30"
        });
    InputStream testInputStream = new StringInputStream(expectedContents);
    when(s3ObjectInputStream.read(any(byte[].class))).thenAnswer(invocation -> {
        return testInputStream.read(invocation.getArgument(0));
    });

我有一个更广泛的示例here 您可以在there

上找到更多帮助。