Surfaces behave like smart groups which mask and warp content placed inside of it to match the real world scene. Surfaces simplify the content creation process for complex real-world scenes through an entirely 2D workflow.
In this article:
Overview
Content creation in Lightform Creator is powered by the Smart Scan image and user-defined Surfaces.
In Lightform Creator, content such as images, videos, and effects, live within a Surface. The Surface behaves like a smart group which masks and warps the content placed inside of it to match the real world scene.
This allows users to quickly augment or add digital content to pixels that have been aligned and perspectively corrected without needing to use 3D tools or create a 3D model of the scene beforehand.
Creating Masks & Surfaces
In most content creation software, content and objects are often created in arbitrary positions based on the users preference and masks are optional. In projected augmented reality, the position and orientation of content is predetermined by the scene being augmented.
In Lightform Creator, you create surfaces on top of the Scan Image and then apply content to the surface. Surfaces are created using any of the masking tools - the Pen, Shape, Magic Wand, Quick Select, or Brush tool. By creating a surface, you are in effect, masking out a part of the real world that you’d like to augment.
Fit Accuracy: This slider adjusts the accuracy between the mask and the generated curve. Higher values mean more vertices in your surface and a closer fit.
Once a mask is completed, a Surface is automatically created so that content can be seamlessly added to it. Each content element inside the Surface has its own layer but shares the same mask of the parent Surface.
Modifying Structures
The structure tool allows users to modify the perspective of content and warp it onto non-planar objects to approximate 3D surfaces using 2D tools.
Modifying Perspective
Structures can be corner pinned for flat surfaces.
Changing the perspective of content is useful for when your projector is not pointing directly at a flat surface, but you want your content to look as if it is flat/undistorted when playing back. A common example is projecting text onto a menu; using the corners of the Structure, you can quickly adjust your text to fit correctly inside the menu. By pinning the Structure's corners to the corresponding corners of the menu, the perspective of the text is shifted to match the perspective of the sign.
This is also known as “corner pinning”, or the adjustment of planar perspective of a rectangular piece of content. Corner pinning uses 4 points to construct a new 3D perspective of the content.
Initially, the content is aligned/parallel to the image plane. By modifying the corners, the perspective of the content is changed so that the vanishing points for this content are now defined by vertical and horizontal line segments. There is also a 3D interpretation of this, namely that by modifying the corners, a 3D rotation can be solved for and applied to this content to make the content appear as if it is being viewed from this new perspective.
To corner pin, select a surface and click the Structure icon in the Toolbar or press S. A green bounding box with corners should appear around the selected surface. Drag a corner of the Structure and release to pin it in place. Corner pinning edits can be reset by clicking the Fit Structure to Masks button in the Properties Panel under Structure.
Mesh Warping
Structures can be mesh warped for modeling of non-flat surfaces.
Warping content (the second function of the Structure) is extremely useful when projecting onto a 3D, non-flat surface.
Imagine you are trying to project a logo onto a cylindrical mug. Because the mug’s surface is curved, the projected logo may look slightly distorted at the edges. You can use the Structure control points to pull the logo a bit closer to the center of the mug, making it look more like a decal on the surface rather than a projection. This is also known as “mesh warping”, or the adjustment/warping of subregions of the content.
While corner pinning is a “global” modification to the content, mesh warping is a “local” adjustment that only affects subregions of the content. Moving points of the mesh warp grid implies a displacement that smoothly falls off to further regions of the content (specifically, 2D cubic spline interpolation is used to determine the amount of displacement and the appearance of the newly warped content).
To mesh warp, select a surface and click the Structure icon in the Toolbar or press S. A green bounding box with corners should appear around the selected surface. Double click anywhere within the Structure to edit its mesh. You can double click again to return to corner pinning mode. Specify the number of columns and rows in the Properties Panel under Structure. Click and drag the control points to warp the mesh. You can select multiple points by Shift+Selecting control points. Mesh warping edits can be reset by clicking the Reset Structure Points button in the Properties Panel under Structure.
Reusing Surfaces
When one Surface is created, it is reusable on every Slide in the project. Unique content can be added to the same Surface on every individual Slide.
If you refine the mask or structure of the main Surface, it applies globally to every slide. We designed slides to work in this way to make adjusting the content in your scene much more efficient.