SlideShare a Scribd company logo

Rest3d BOF presentation at SigGraph 2013

•Download as PPTX, PDF•
•
Remi Arnaud
Remi Arnaud

This document recaps the development of rest3d from 2011-2013, including prototyping a REST API and COLLADA to JSON converter to simplify accessing 3D content on the web. It describes the open sourcing of rest3d and a COLLADA to glTF converter. The rest3d pipeline is explained, involving content creation, processing, and delivery to a web viewer via REST. Key components include a database, HTTP server, and HTML client. Examples demonstrate accessing models via the REST API and compressing 3D meshes.

Related slideshows

Art and Science of Web Sites Performance: A Front-end Approach
Art and Science of Web Sites Performance: A Front-end ApproachArt and Science of Web Sites Performance: A Front-end Approach
Art and Science of Web Sites Performance: A Front-end Approach
 
Javascript Update May 2013
Javascript Update May 2013Javascript Update May 2013
Javascript Update May 2013
 
Thrombus Training Dec. 2013
Thrombus Training Dec. 2013Thrombus Training Dec. 2013
Thrombus Training Dec. 2013
 
1 of 37
rest3d SigGraph 2013 remi.arnaud@amd.com remi@acm.org
Recap 2011 • 2011 – forming the idea and brainstorming – GDC: small group discussed the idea for a rest based API to simplify usage of 3D content in the web – http://www.slideshare.net/remi_arnaud/6-10presentation – http://www.slideshare.net/remi_arnaud/collada-webgl – SigGraph BOF gathering • Resources: – http://rest3d.org – Mailing list: (join us!)https://groups.google.com/forum/#!forum/3d-rest
Recap 2012 • 2012 – prototyping rest3d, COLLADA2json – http://www.slideshare.net/remi_arnaud/rest-3- dupdatewebglmeetup11072012 – http://www.scribd.com/doc/102183295/Siggraph 2012-Pamphlet – http://www.youtube.com/watch?v=Dja5achkgZc – SigGraph BOF • Use case discussion • Group said – lets focus on WebGL viewer APP
Status for SigGraph 2013 BOF • COLLADA2JSON became Khronos glTF – http://gltf.gl/ – https://github.com/KhronosGroup/glTF – Fabrice Robinet, Patrick Cozzi, Tony Parisi, Rémi Arnaud • rest3d prototype recently open-sourced – https://github.com/amd/rest3d – Rémi Arnaud, Khaled Mamou
Where in the world is rest3d
Content creation 3Dapp3D model Images Animations Physics Audio Script …
Content ‘Pipeline’ Packaging Optimization Physics Textures, Shaders Animations Modeler Object files Animation files + skins/bones Shader files + psd Physics files Binary files package file Multiple indexes, Quads Clips, bézier, complex Transformation graph Vertex / Fragment shaders Convex Mesh, colliders Single index, triangles DXTs, simplified scene Compressed folder
3D loader? Packaging Optimization A Physics A Textures, Shaders A Animations A Modeler A My game Polygons multiple indexes image format mesh optimization, split complex transform hierarchy no shader programs no collision volumes verbose text (xml) parsing
COLLADA Refinery (2007) https://collada.org/mediawiki/index.php/COL LADA_Refinery COLLADA Refinery 2.0.3 conditioners Axisconverter Compress Transforms Stripper Image conversion Axis transform Coherencytest ** Use this !! Copyrighter De-indexer Triangulate Vertex cache optimization Optimizer Packager
Integrated pipeline – Unity Editor Packaging IOS, Android, PC, Web (plugin), Flash Optimization Built-in Physics PhysX Textures, Shaders Shader Factory Animations Mecanim Modeler Max Maya Blender Modo Cinema4D XSI Fbx or simple COLLADA (e.g. no physics, morphing, shaders..) Unity Engine C o n t e n t p i p e l i n e
cloud (rest3d) pipelineModeler Max Maya Blender Databank Editor Desktop Or web app Assets Database Queries Upload/download Processing Conversion (glTF) Compression Image conversion WebApp Turbulenz Three.js Cesium …WebGL… Desktop Browser Cloud REST3D REST3D REST3D
Use case: viewer • Assets are stored in the cloud • App query / search for assets • App ask for ‘compiled’ assets for specific target • Need geometry, Shaders, textures, animations, lights, cameras… • Compress data for bandwidth use and potentially HW decode • Cache!! – client and server cache are very important
Rest3d BOF presentation at SigGraph 2013
glTF as a command line tool Modeler Max Maya Blender … Exporter COLLADA converter COLLADA2GLTF WebGL Turbulenz Three.js ... Converter flags (or profile) to control conversion for specific target (e.g. shader generation, animation, transparency control, image format)
Rest3d BOF presentation at SigGraph 2013
Three main components XML Database • BaseX server • (sockets or http interface) • Xquery http server • Node.js + rest modules • Javascript Html client • Html/Javascript • WebGL viewer • UI framework
Hacking it together
Local Scene • Single asset only – Viewer simple API only allows for 1 model view – click on model in scene send load even in viewer – Viewer loads json scene description… [GET] /rest3d/assets/Dog/models/model.json – .. then pages [partial http requests] data • Issues – Partial content non cacheable by database – Node.js server caching implemented – not ideal – Client side does not cache 206
Database assets browser • Browse asset database – Tree UI – Hierarchical JSON from XML query – Typed assets [model, image, other] – Converted assets [collada2json, glsl shaders] – Should implement lazy tree evaluation – Show asset info in right viewport, direct URL access [GET] /rest3d/assets • Drag/drop into scene – Local operation .. • Considering shared scene through websockets
Rest3d BOF presentation at SigGraph 2013
Rest3d BOF presentation at SigGraph 2013
Sketchup warehouse proxy • Browse – Dynamic tree MV UI – Node.js html scraping with cheerio and request – Warehouse collection browsing issue: leads to unavailable content (protected or not converted) – QooxDoo complex view system and dynamic update issues [GET] /rest3d/warehouse/ • Search – Simple text box UI – [enter] / search button linked – Warehouse search can filter specific available format – Search return is a list, not a tree… [GET] /rest3d/warehouse/search/search_string
Sketchup warehouse proxy • Fetch & Convert (most complex API so far) – Drag/drop UI (from warehouse tree to asset tree) – http get zip file to rest3d server – Node.js multi-file zip archive to file system decompress – Datase file upload / store – asset.xml management through xquery – COLLADA2json server side conversion • Could not get nodejs module to work in Ubuntu cloud • Need to spawn async process ! [PUT] /rest3d/assets/dog [body]http://sketchup.google.com/3dwarehouse/download?mid:20093 34a95ee8f4c4038ed23c800b042
Rewrite – and open source • ~4000 lines of new JavaScript code brand new AMD client code – glTF loader – COLLADA loader – WebGL rendering – Simpler UI framework (based on jquery, jquery-UI, jquery- layout) • ~450,000 lines removed ! (qooxdoo, Khronos glTF viewer sample) https://github.com/amd/rest3d
Rest3d BOF presentation at SigGraph 2013
Rest3d BOF presentation at SigGraph 2013
samples http://rest3d-remi.rhcloud.com/index.html Tested in Chrome, Firefox, Safari
 (new) free and paying service from RedHat – 3 small ‘gears’ for free – Could not fit database and node server in a small gear – Deploy directly from github • Provided the github project contains the .openshift configuration files – Scalability • ‘cartridge’ can spread over multiple ‘gears’
Online converter http://rest3d-remi.rhcloud.com/upload/index.html
Open 3D Graphics Compression (Open3DGC) • Efficient cross platform implementation of patent free MPEG graphics compression tools – 3D Meshes with multiple attributes per vertex • Positions, normals and texture coordinates • Float attributes (e.g., skinning animation weights) • Integer attributes (e.g., bone ID, material ID) – Bone based animations (coming soon) • Time varying transforms (e.g., rotation, scale, translation) • C/C++ Implementation under MIT License https://github.com/amd/rest3d/tree/master/server/o3dgc
Open3DGC Mesh Compression • Triangle Fan-based Compression [Mammou’09] [Mammou’09] K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm” Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
Open3DGC Mesh Compression • Compression Efficiency  160 models from different sources (e.g., CAD, 3D scans, modeling tools)  Various topologies (i.e., open/closed, manifold/non-manifold, arbitrary genus)  WebGL-Loader and Open3DGC: 14 bits for positions et 10 bits for normals/texture coord.  OpenCTM: default parameters (not fair) Open3DGC is 7.3 times more efficient than Gzip and 1.6-1.8 times more efficient than WebGL Loader and OpenCTM
Open3DGC Mesh Compression • Speed  Non-optimized C++ implementation, single thread (rooms for parallelization)  Intel i7-2600 CPU @ 3.4 GHz, Win7 x64, RAM 10 GB
Open3DGC Mesh Decompression • Decoder
Demo & deep dive • Running rest3d local – Debugging • Running on server • Database vs. node.js vs. cache – Socket vs. http • API details
Let’s talk • Roadmap ? – More work on the viewer / UI – More work on the server – Where is the spec ? • Contribute ? – Join the email list https://groups.google.com/forum/#!forum/3d-rest • See also: – COLLADA/glTF BOF !
Thank you !!

More Related Content

Rest3d BOF presentation at SigGraph 2013

  • 2. Recap 2011 • 2011 – forming the idea and brainstorming – GDC: small group discussed the idea for a rest based API to simplify usage of 3D content in the web – http://www.slideshare.net/remi_arnaud/6-10presentation – http://www.slideshare.net/remi_arnaud/collada-webgl – SigGraph BOF gathering • Resources: – http://rest3d.org – Mailing list: (join us!)https://groups.google.com/forum/#!forum/3d-rest
  • 3. Recap 2012 • 2012 – prototyping rest3d, COLLADA2json – http://www.slideshare.net/remi_arnaud/rest-3- dupdatewebglmeetup11072012 – http://www.scribd.com/doc/102183295/Siggraph 2012-Pamphlet – http://www.youtube.com/watch?v=Dja5achkgZc – SigGraph BOF • Use case discussion • Group said – lets focus on WebGL viewer APP
  • 4. Status for SigGraph 2013 BOF • COLLADA2JSON became Khronos glTF – http://gltf.gl/ – https://github.com/KhronosGroup/glTF – Fabrice Robinet, Patrick Cozzi, Tony Parisi, Rémi Arnaud • rest3d prototype recently open-sourced – https://github.com/amd/rest3d – Rémi Arnaud, Khaled Mamou
  • 5. Where in the world is rest3d
  • 7. Content ‘Pipeline’ Packaging Optimization Physics Textures, Shaders Animations Modeler Object files Animation files + skins/bones Shader files + psd Physics files Binary files package file Multiple indexes, Quads Clips, bézier, complex Transformation graph Vertex / Fragment shaders Convex Mesh, colliders Single index, triangles DXTs, simplified scene Compressed folder
  • 8. 3D loader? Packaging Optimization A Physics A Textures, Shaders A Animations A Modeler A My game Polygons multiple indexes image format mesh optimization, split complex transform hierarchy no shader programs no collision volumes verbose text (xml) parsing
  • 9. COLLADA Refinery (2007) https://collada.org/mediawiki/index.php/COL LADA_Refinery COLLADA Refinery 2.0.3 conditioners Axisconverter Compress Transforms Stripper Image conversion Axis transform Coherencytest ** Use this !! Copyrighter De-indexer Triangulate Vertex cache optimization Optimizer Packager
  • 10. Integrated pipeline – Unity Editor Packaging IOS, Android, PC, Web (plugin), Flash Optimization Built-in Physics PhysX Textures, Shaders Shader Factory Animations Mecanim Modeler Max Maya Blender Modo Cinema4D XSI Fbx or simple COLLADA (e.g. no physics, morphing, shaders..) Unity Engine C o n t e n t p i p e l i n e
  • 11. cloud (rest3d) pipelineModeler Max Maya Blender Databank Editor Desktop Or web app Assets Database Queries Upload/download Processing Conversion (glTF) Compression Image conversion WebApp Turbulenz Three.js Cesium …WebGL… Desktop Browser Cloud REST3D REST3D REST3D
  • 12. Use case: viewer • Assets are stored in the cloud • App query / search for assets • App ask for ‘compiled’ assets for specific target • Need geometry, Shaders, textures, animations, lights, cameras�� • Compress data for bandwidth use and potentially HW decode • Cache!! – client and server cache are very important
  • 14. glTF as a command line tool Modeler Max Maya Blender … Exporter COLLADA converter COLLADA2GLTF WebGL Turbulenz Three.js ... Converter flags (or profile) to control conversion for specific target (e.g. shader generation, animation, transparency control, image format)
  • 16. Three main components XML Database • BaseX server • (sockets or http interface) • Xquery http server • Node.js + rest modules • Javascript Html client • Html/Javascript • WebGL viewer • UI framework
  • 18. Local Scene • Single asset only – Viewer simple API only allows for 1 model view – click on model in scene send load even in viewer – Viewer loads json scene description… [GET] /rest3d/assets/Dog/models/model.json – .. then pages [partial http requests] data • Issues – Partial content non cacheable by database – Node.js server caching implemented – not ideal – Client side does not cache 206
  • 19. Database assets browser • Browse asset database – Tree UI – Hierarchical JSON from XML query – Typed assets [model, image, other] – Converted assets [collada2json, glsl shaders] – Should implement lazy tree evaluation – Show asset info in right viewport, direct URL access [GET] /rest3d/assets • Drag/drop into scene – Local operation .. • Considering shared scene through websockets
  • 22. Sketchup warehouse proxy • Browse – Dynamic tree MV UI – Node.js html scraping with cheerio and request – Warehouse collection browsing issue: leads to unavailable content (protected or not converted) – QooxDoo complex view system and dynamic update issues [GET] /rest3d/warehouse/ • Search – Simple text box UI – [enter] / search button linked – Warehouse search can filter specific available format – Search return is a list, not a tree… [GET] /rest3d/warehouse/search/search_string
  • 23. Sketchup warehouse proxy • Fetch & Convert (most complex API so far) – Drag/drop UI (from warehouse tree to asset tree) – http get zip file to rest3d server – Node.js multi-file zip archive to file system decompress – Datase file upload / store – asset.xml management through xquery – COLLADA2json server side conversion • Could not get nodejs module to work in Ubuntu cloud • Need to spawn async process ! [PUT] /rest3d/assets/dog [body]http://sketchup.google.com/3dwarehouse/download?mid:20093 34a95ee8f4c4038ed23c800b042
  • 24. Rewrite – and open source • ~4000 lines of new JavaScript code brand new AMD client code – glTF loader – COLLADA loader – WebGL rendering – Simpler UI framework (based on jquery, jquery-UI, jquery- layout) • ~450,000 lines removed ! (qooxdoo, Khronos glTF viewer sample) https://github.com/amd/rest3d
  • 28.  (new) free and paying service from RedHat – 3 small ‘gears’ for free – Could not fit database and node server in a small gear – Deploy directly from github • Provided the github project contains the .openshift configuration files – Scalability • ‘cartridge’ can spread over multiple ‘gears’
  • 30. Open 3D Graphics Compression (Open3DGC) • Efficient cross platform implementation of patent free MPEG graphics compression tools – 3D Meshes with multiple attributes per vertex • Positions, normals and texture coordinates • Float attributes (e.g., skinning animation weights) • Integer attributes (e.g., bone ID, material ID) – Bone based animations (coming soon) • Time varying transforms (e.g., rotation, scale, translation) • C/C++ Implementation under MIT License https://github.com/amd/rest3d/tree/master/server/o3dgc
  • 31. Open3DGC Mesh Compression • Triangle Fan-based Compression [Mammou’09] [Mammou’09] K. Mammou, T. Zaharia, F. Prêteux, “TFAN: A low complexity 3D mesh compression algorithm” Computer Animation and Virtual Worlds, Vol. 20(2-3), pp. 343–354, 2009
  • 32. Open3DGC Mesh Compression • Compression Efficiency  160 models from different sources (e.g., CAD, 3D scans, modeling tools)  Various topologies (i.e., open/closed, manifold/non-manifold, arbitrary genus)  WebGL-Loader and Open3DGC: 14 bits for positions et 10 bits for normals/texture coord.  OpenCTM: default parameters (not fair) Open3DGC is 7.3 times more efficient than Gzip and 1.6-1.8 times more efficient than WebGL Loader and OpenCTM
  • 33. Open3DGC Mesh Compression • Speed  Non-optimized C++ implementation, single thread (rooms for parallelization)  Intel i7-2600 CPU @ 3.4 GHz, Win7 x64, RAM 10 GB
  • 35. Demo & deep dive • Running rest3d local – Debugging • Running on server • Database vs. node.js vs. cache – Socket vs. http • API details
  • 36. Let’s talk • Roadmap ? – More work on the viewer / UI – More work on the server – Where is the spec ? • Contribute ? – Join the email list https://groups.google.com/forum/#!forum/3d-rest • See also: – COLLADA/glTF BOF !

Editor's Notes

  1. All this content need to get in the game, somehow
  2. Typical content toolchain – it is a pipeline. Each stage has its own tools its own format.
  3. It’s really hard….
  4. 6 years later, we are still doing the same things