Question

我们有一个应用程序，该应用程序在五分钟内将数百行附加到Azure Gen2 Datalake中的文件。下面的代码模拟了该应用程序。我们注意到，有时有0x00行写入了文件而不是文本。没有引发任何错误。您能否建议我们如何诊断原因。

public class AdlAppender {

    private static final String datalake = "<<datalake-name>>";
    private static final String url = "https://<<datalake-name>>.dfs.core.windows.net";
    private static final String key = "<<access-key>>";
    private static final String container = "<<container-name>>";
    private static final String fileName = "<<file-path>>";

    final DataLakeFileClient fileClient;

    public AdlAppender() {
        final StorageSharedKeyCredential credential = new StorageSharedKeyCredential(datalake, key);

        final DataLakeServiceClient dataLakeServiceClient = new DataLakeServiceClientBuilder()
            .endpoint(url)
            .credential(credential)
            .buildClient();

        final DataLakeFileSystemClient fileSystemClient = dataLakeServiceClient.getFileSystemClient(container);

        fileClient = fileSystemClient.createFile(fileName);
    }

    public long append(final String content, final long offset) {
        final byte[] bytes = content.getBytes(StandardCharsets.UTF_8);
        final int length = bytes.length;
        final ByteArrayInputStream inputStream = new ByteArrayInputStream(bytes);

        fileClient.append(inputStream, offset, length);

        final long updatedOffset = offset + length;
        fileClient.flush(updatedOffset);

        return updatedOffset;
    }
}

public class AdlAppenderTest {

    @Test
    public void testAppendFile() throws InterruptedException {
        final AdlAppender adlAppender = new AdlAppender();
        final String content = "The quick brown fox jumps over the lazy dog\n";
        final int size = content.length();
        long offset = 0;

        final int numLines = 200;
        final int duration = 5;
        for (int i = 1; i <= numLines; i++) {
            offset = adlAppender.append(content, offset);
            assert offset == size * i;

            Thread.sleep(duration * 60 * 1000 / numLines);
        }
    }
}

示例：

$ xxd -c 44 foo.file
00000000: 5468 6520 7175 6963 6b20 6272 6f77 6e20 666f 7820 6a75 6d70 7320 6f76 6572 2074 6865 206c 617a 7920 646f 670a  The quick brown fox jumps over the lazy dog.
0000002c: 5468 6520 7175 6963 6b20 6272 6f77 6e20 666f 7820 6a75 6d70 7320 6f76 6572 2074 6865 206c 617a 7920 646f 670a  The quick brown fox jumps over the lazy dog.
00000058: 5468 6520 7175 6963 6b20 6272 6f77 6e20 666f 7820 6a75 6d70 7320 6f76 6572 2074 6865 206c 617a 7920 646f 670a  The quick brown fox jumps over the lazy dog.
00000084: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000  ............................................
000000b0: 5468 6520 7175 6963 6b20 6272 6f77 6e20 666f 7820 6a75 6d70 7320 6f76 6572 2074 6865 206c 617a 7920 646f 670a  The quick brown fox jumps over the lazy dog.
000000dc: 5468 6520 7175 6963 6b20 6272 6f77 6e20 666f 7820 6a75 6d70 7320 6f76 6572 2074 6865 206c 617a 7920 646f 670a  The quick brown fox jumps over the lazy dog.

Azure Datalake Gen2 Java SDK FileClient有时会附加`0x00`s行

0 个答案: