使用MTKView显示JPEG图像

时间:2015-11-09 08:40:50

标签: ios objective-c iphone metal

有没有办法通过MTKView和MTLBuffer(以及iPhone 6+内)显示JPEG图像。我已经通过以下方式尝试了这一点(这只是测试):

- (id<MTLBuffer>)testBuffer
{
    if (!_testBuffer)
    {
// 
        NSString *path = [[NSBundle mainBundle] pathForResource:@"testImage" ofType:@"jpg"];
        NSData *imageData = [NSData dataWithContentsOfFile:path];
        _testBuffer = [self.device newBufferWithBytes:imageData.bytes length:imageData.length options:MTLResourceCPUCacheModeWriteCombined];
    }

    return _testBuffer;
}

- (void)drawInMTKView:(MTKView *)view
{
    MTLRenderPassDescriptor* renderPassDescriptor = view.currentRenderPassDescriptor;
    id <MTLCommandBuffer> commandBuffer = [_commandQueue commandBuffer];
    id <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
    [renderEncoder drawPrimitives:MTLPrimitiveTypePoint indirectBuffer:self.testBuffer indirectBufferOffset:0];
    [renderEncoder endEncoding];
    [commandBuffer presentDrawable:view.currentDrawable];
    [commandBuffer commit];
}

我在这个方法调用中遇到错误:

[renderEncoder drawPrimitives:MTLPrimitiveTypePoint indirectBuffer:self.testBuffer indirectBufferOffset:0];
  

/BuildRoot/Library/Caches/com.apple.xbs/Sources/Metal/Metal-54.31/ToolsLayers/Debug/MTLDebugRenderCommandEncoder.mm:2123:   断言失败` - [MTLDebugRenderCommandEncoder   drawPrimitives:indirectBuffer:indirectBufferOffset:]仅受支持   在MTLFeatureSet_iOS_GPUFamily3_v1及更高版本&#39;

这是因为此API仅由iPhone 6s(+)支持。

我认为我做的事情完全错了。也许我需要在其他方向思考。有人可以帮助我还是指向正确的方向? 感谢。

2 个答案:

答案 0 :(得分:2)

由于工作繁忙,我忘了写解决方案。 @ Denn的回答看起来是正确的,所以我会接受它。 我的解决方案如下(在objective-c中)。我已经简化了苹果示例(以及它们的着色器文件)并制作了test project。 我已经创建了名为MetalView的MTKView子类。它有'配置'#39;方法,准备MetalView进行绘图:设置MTKView的属性并创建pipelineState对象以及顶点缓冲区。

- (void)configure
{
    self.device = MTLCreateSystemDefaultDevice();
    self.colorPixelFormat = MTLPixelFormatBGRA8Unorm;
    self.framebufferOnly = YES;
    self.sampleCount = 1;

    // create command queue for usage during drawing
    self.commandQueue = [self.device newCommandQueue];

    // load shaders functions from texturedQuad.metal file. These functions needed for configuration of MTLRenderPipelineDescriptor
    id <MTLLibrary> shaderLibrary = [self.device newDefaultLibrary];
    id <MTLFunction> fragmentProgram = [shaderLibrary newFunctionWithName:@"texturedQuadFragment"];
    id <MTLFunction> vertexProgram = [shaderLibrary newFunctionWithName:@"texturedQuadVertex"];

    //  create a pipeline state
    MTLRenderPipelineDescriptor *pQuadPipelineStateDescriptor = [MTLRenderPipelineDescriptor new];
    pQuadPipelineStateDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;
    pQuadPipelineStateDescriptor.sampleCount      = self.sampleCount;
    pQuadPipelineStateDescriptor.vertexFunction  = vertexProgram;
    pQuadPipelineStateDescriptor.fragmentFunction = fragmentProgram;
    NSError *pipErr = nil;
    self.pipelineState = [self.device newRenderPipelineStateWithDescriptor:pQuadPipelineStateDescriptor
                                                                     error:&pipErr];

    if (pipErr) NSLog(@"newRenderPipelineStateWithDescriptor err: %@", pipErr);

    // buffers for MTLRenderCommandEncoder in drawRect: method
    self.vertexBuffer = [self.device newBufferWithBytes:kQuadVertices
                                                 length:kSzQuadVertices
                                                options:MTLResourceOptionCPUCacheModeDefault];

    self.texCoordBuffer = [self.device newBufferWithBytes:kQuadTexCoords
                                                   length:kSzQuadTexCoords
                                                  options:MTLResourceOptionCPUCacheModeDefault];

    self.paused = YES;
    self.enableSetNeedsDisplay = NO;
}

这些东西将由drawRect:中的renderEncoder用于实际绘图:

- (void)drawRect:(CGRect)rect
{
    id <MTLCommandBuffer> commandBuffer = [self.commandQueue commandBuffer];
    MTLRenderPassDescriptor *renderPassDescriptor = self.currentRenderPassDescriptor;

    id <MTLRenderCommandEncoder>  renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor];
    // Encode into a renderer
    [renderEncoder setRenderPipelineState:self.pipelineState];

    [renderEncoder setVertexBuffer:self.vertexBuffer
                            offset:0
                           atIndex:0];

    [renderEncoder setVertexBuffer:self.texCoordBuffer
                            offset:0
                           atIndex:1];

    [renderEncoder setFragmentTexture:self.texture
                              atIndex:0];

    // tell the render context we want to draw our primitives. We will draw triangles that's
    // why we need kQuadVertices and kQuadTexCoords (arrays of points)
    [renderEncoder drawPrimitives:MTLPrimitiveTypeTriangle
                      vertexStart:0
                      vertexCount:6
                    instanceCount:1];

    [renderEncoder endEncoding];
    [commandBuffer presentDrawable:self.currentDrawable];
    [commandBuffer commit];
}

着色器文件代码(texturedQuad.metal):

struct VertexInOut
{
    float4 m_Position [[position]];
    float2 m_TexCoord [[user(texturecoord)]];
};

vertex VertexInOut texturedQuadVertex(constant float4         *pPosition   [[ buffer(0) ]],
                                      constant packed_float2  *pTexCoords  [[ buffer(1) ]],
                                      constant float4x4       *pMVP        [[ buffer(2) ]],
                                      uint                     vid         [[ vertex_id ]])
{
    VertexInOut outVertices;

    outVertices.m_Position = pPosition[vid];
    outVertices.m_TexCoord = pTexCoords[vid];

    return outVertices;
}

fragment half4 texturedQuadFragment(VertexInOut     inFrag    [[ stage_in ]],
                                    texture2d<half>  tex2D     [[ texture(0) ]])
{
    constexpr sampler quad_sampler;
    half4 color = tex2D.sample(quad_sampler, inFrag.m_TexCoord);

    return color;
}

MetalView类用作ViewController故事板中视图出口的类。 ViewController的viewDidLoad方法检索指定图像的对象,并将此对象设置为MetalView。

- (void)viewDidLoad
{
    [super viewDidLoad];

    self.metalView = (MetalView *)self.view;
    NSString *path = [[NSBundle mainBundle] pathForResource:@"testImage"
                                                     ofType:@"jpg"];
    MTKTextureLoader *loader = [[MTKTextureLoader alloc] initWithDevice:self.metalView.device];
    NSError *err = nil;
    self.metalView.texture = [loader newTextureWithContentsOfURL:[NSURL fileURLWithPath:path] options:nil error:&err];
    if (err) NSLog(@"newTextureWithContentsOfURL err: %@", [err localizedDescription]);
}

最后,我们需要为MetalView调用draw方法

- (void)viewDidLayoutSubviews
{
    [self.metalView draw];
}

示例代码为here

答案 1 :(得分:1)

没有简单的方法可以使用MTLBuffer通过MTKView显示图像数据。在这种情况下,您必须为右输出纹理准备正确的像素显示,用于在绘图层中由MTKView显示。最简单的方法是使用MTKTextureLoader,或从UIImage / CGImage实例将数据加载到MTLTexture对象。

我做过类似的实验,在iOS设备上使用MTL Layer和MTLTexture(而不是MTLBuffer)呈现图像,我已经决定了显示图像的最佳方式(如果这只需要:显示图像)它使用passthrough没有渲染的内核函数。

在Swift2中,这看起来如下:

<?php
$datetime1 = date_create('2015-11-16 10:01:13');
$datetime2 = date_create('2015-05-06 09:47:16');
$interval = date_diff($datetime1, $datetime2);
echo $interval->format('%R%a days');
?>

在金属底纹文件中:

...

let textureLoader = MTKTextureLoader(device: self.device!)
if let image = UIImage(named: file){
   imageTexture = try! textureLoader.newTextureWithCGImage(image.CGImage!, options: nil)
   threadGroups = MTLSizeMake(
                (imageTexture.width+threadGroupCount.width)/threadGroupCount.width,
                (imageTexture.height+threadGroupCount.height)/threadGroupCount.height, 1)
    }
} 
...  

let function:MTLFunction! = library.newFunctionWithName("kernel_passthrough")
...
pipeline = try! self.device.newComputePipelineStateWithFunction(function)
...
let commandBuffer = commandQueue.commandBuffer()
let encoder = commandBuffer.computeCommandEncoder()
encoder.setComputePipelineState(pipeline)
encoder.setTexture(actualImageTexture, atIndex: 0)
encoder.setTexture(metalView.currentDrawable!.texture, atIndex: 1)
encoder.setBuffer(self.saturationUniform, offset: 0, atIndex: 0)
encoder.dispatchThreadgroups(threadGroups!, threadsPerThreadgroup: threadGroupCount)
encoder.endEncoding()
commandBuffer.presentDrawable(metalView.currentDrawable!)
commandBuffer.commit()
...

完整示例来源:ImageMetalling-00