Lets have a look at Apple's Metal Framework
•
2 likes•1,275 views
Frederik Vogel from LINE Fukuoka will give a presentation on Apple's Metal framework. The presentation will provide a history of Metal and graphics processing, demonstrate Metal concepts like the render pipeline, and show code examples. Attendees will learn about rendering 3D graphics, compute capabilities, and possibilities for using Metal in areas like machine learning and finance.
Report
Share
Lets have a look at Apple's Metal Framework
- 3. Frederik Vogel LINE Fukuoka 株式会社 iOS Engineer Freddy
- 4. Why this topic? • I am curious about Metal • What can I do with Metal? • Is is difficult to use? • Can i integrated it in my Project? • I want to use Swift!! • Not many resources • Hope to meet other people with same interest ? Motivation
- 5. Let's have a look at Apple's Metal Framework • History • Metal concepts • Render pipeline • Compute pipeline • Shader • Demo • Other possibilities Agenda
- 6. History
- 13. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded History
- 14. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL -> OpenGL History
- 15. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL History
- 16. John Carmack, id Software, now Oculus VR
- 19. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular History
- 20. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular • 2003 OpenGL ES was released, subset of OpenGL History
- 21. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular • 2003 OpenGL ES was released, subset of OpenGL • 2007 iPhone released, supported OpenGL ES History
- 22. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular • 2003 OpenGL ES was released, subset of OpenGL • 2007 iPhone released, supported OpenGL ES • 2010 iPhone supports OpenGL ES 2.0 + Shader History
- 23. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular • 2003 OpenGL ES was released, subset of OpenGL • 2007 iPhone released, supported OpenGL ES • 2010 iPhone supports OpenGL ES 2.0 + Shader • 2014 Metal was released History
- 24. OpenGL Where does it come from? • 1981 Silicon Graphics (SGI) was founded • 1992 opened and renamed IRIS GL API -> OpenGL • 1996 Quake used OpenGL • 3D Graphic cards became popular • 2003 OpenGL ES was released, subset of OpenGL • 2007 iPhone released, supported OpenGL ES • 2010 iPhone supports OpenGL ES 2.0 + Shader • 2014 Metal was released • 2017 Metal 2 was released History
- 25. What is Metal? • Provides access to the GPU • Low-level access • Low-overhead access • Accelerated 3D graphics rendering • Data-parallel computation • Full control of GPU computation power Metal 🤘
- 26. Why not OpenGL ES? • For graphics programming • Not for general-purpose programming • Need to copy memory between CPU and GPU • Very old, has its beginning in the 80’s • Does not fit for Apple’s plan OpenGL ES
- 27. Why Metal? • For graphics programming • For general-purpose programming • CPU and GPU are on the same chip • No Need to copy memory between CPU and GPU • Optimized for the chip • Developer has full control of buffer • Queue work in preferred order • Can delay buffer execution • More control over the performance Metal ?
- 28. Render Pipeline
- 29. Render pipeline
- 30. Vertex Function • Runs once on every vertex • Calculates position for every vertex • Assemble position and other data
- 31. Vertex Shader struct RasterizerData { float4 position [[position]]; float4 color; }; vertex RasterizerData myVertexFunction( uint vid [[ vertex_id ]], constant packed_float4* position [[ buffer(0) ]], constant packed_float4* color [[ buffer(1) ]]) { RasterizerData outVertex; outVertex.position = position[vid]; outVertex.color = color[vid]; return outVertex; };
- 32. Assemble vertices to primitive • Groups vertices into primitive types • Primitive Types: point, line, lineStrip, triangle, triangleStrip • Needed for rasterization
- 33. Rasterization • Rasterize primitive types • Interpolates Position • Interpolates other Data
- 34. Fragment Function • Runs for every pixel on the primitive • Calculates color for each pixel
- 35. Fragment Function struct RasterizerData { float4 position [[position]]; float4 color; }; fragment half4 myFragmentFunction(RasterizerData inFrag [[stage_in]]) { return half4(inFrag.color); };
- 36. Render pipeline
- 37. What belongs to Metal? • Metal framework • MetalKit framework • Metal Performance Shaders framework (MPS) • Metal shading language (MSL) • Metal standard library Metal World
- 38. Let’s build our render pipeline!
- 39. Concept
- 40. MTLDevice • Represent the actual GPU • Create command Queue • Creates pipeline • Creates Data Buffer • Access to custom function library What is it for?
- 41. MTLRenderPipelineState • Represents rendering pipeline • Access to custom function library • Creates compute or render pipeline What is it for?
- 42. MTLCommandQueue • Created by the MetalDevice • Creates CommandBuffers • Queues and schedules CommandBuffers What is it for?
- 43. MTLCommandBuffer • Created every frame • Creates CommandEncoders • RenderCommandEncoder • ParallelRenderCommandEncoder • ComputeCommandEncoder • BlitCommandEncoder • Encodes to the specific GPU hardware What is it for?
- 44. MTLCommandEncoder • Encode commands to the specific GPU hardware What is it for?
- 45. 🤯 A lot of information!
- 46. Setup
- 47. Every frame
- 48. Demo Let’s see the code!
- 49. MetalKit MetalKit framework? • Reduces the boilerplate code • No need to setup RenderPassDescriptor • No need to setup MetalLayer • No need to setup Displaylink • Provides MTKView • Provides MTKTextureLoader • Model Handling Metal UIKit + Metal = MetalKit
- 50. Demo Let’s see the code again!
- 51. Compute Pipeline Compute Pipeline • Only one stage • shader is marked with kernel keyword • kernel reads and write directly from/to resource • Must specify “thread” size • Fine control performance with threadGroups Metal
- 52. Demo More code!
- 53. Resume Resume • A lot of information • But not too difficult • An own world • Compute Pipeline very powerful • Particles • Apple says its good for AR, VR, Cryptography, Machine learning, Physics and Finance !
- 56. Bitcoin Trading Bot!! 😃
- 57. Further Sources ・developer.apple.com/documentation/metal Apple Documentation ・flexmonkey.blogspot.co.uk ・github.com/FlexMonkey/ParticleLab FlexMonkey Creative Coding in Swift ・by Janie Clayton Metal Programming Guide: Tutorial and Reference via Swift