iOS

Prerequisites:

• XCode >= 11.0

• Cocoapods >= 1.9.1 (can be installed via brew)

Overview

The app demonstrates simple anchoring of content, in this case - colored boxes. To test the anchoring is working, tap on the screen in some locations to leave boxes there and restart the app. The boxes shall appear where you left them.

Build instructions

• Download the sample app from

• Unzip it to <Path/To/SampleApp>

• Cd to <Path/To/SampleApp>

• Run: pod install

• In Finder double click to open the workspace in XCode:

  <Path/To/SampleApp>/SampleApp.xcworkspace

• While in xcode, go to Signing&Capabilities and change Team to your certified team

• In ContentView.swift put your API key as a string:

  LibResight.RealityKitPlugin.instance.delegate = self
  LibResight.RealityKitPlugin.instance.initialize(<put-sdk-key>, namespace: "ns1", view: arView)

• Connect your device, select it in the Device List and hit “Run”

Code Explained

The SampleApp is basically an XCode AR template app (SwiftUI as its UX, RealityKit as its 3D engine) with the minimal additions to activate ReSight's powerful engine.

The main file here is ContentView.swift. We’ll go through the special API calls which enable the Spatial Anchoring (the ‘glue’).

ReSight’s SDK contains RealityKit plugin which consists of 2 high-level components: anchors and entities. Anchors can be considered as placeholders for content. Once added to the scene, they maintain their location and will appear in any subsequent scene which shares the same namespace (more about namespaces will be explained later).

Entities represent rendered content. They consist of 3D position and rendering properties (mesh, texture, dimensions etc.). Once an entity is added as a child to an anchor in the scene, its 3D position during its life cycle is determined by 3 factors:

• The anchor position in the world.

• The entity position as set by the user of the app.

• The entity position as set by some remote user.

All Entity’s position handling is done by ReSight SDK, hence removing this burden from the developer.

As for the rendering properties of an entity, due to their highly customizable characteristic, ReSight SDK needs the programmer to tell it how to serialize the entity for synchronization over the network and deserialize once updates are received from the network. For that purpose, the developer shall provide its implementation for those 2 functions, namely:

func entity(encode entity: Entity) -> Data
func entity(decode data: Data) -> Entity

These 2 methods are part of a larger protocol named ResightEntityDelegate but only those are required in order to start using the plugin.

In our SampleApp, ARViewContainer is implementing ResightEntityDelegate’s encode and decode functions by taking advantage of Swift’s String builtin serialization methods. Of course this is only for demonstration purposes and one can use any complex serialization for his/her content.

    func entity(encode entity: Entity) -> Data {
        return entity.name.data(using: .utf8)!
    }

    func entity(decode data: Data) -> Entity {
        guard let name = String(data: data, encoding: .utf8), name == "box" else {
             return AnchorEntity()
        }

        // this is our object, return its instance
         let boxAnchor = try! Experience.loadBox()
        let steelBox = boxAnchor.steelBox!
        return steelBox
    }

Next, we initialize the plugin by setting its delegate and calling the initialization function:

LibResight.RealityKitPlugin.instance.delegate = self
LibResight.RealityKitPlugin.instance.initialize(<put-sdk-key>, namespace: "ns1", view: arView)

Once inititalize is called, Resight library cycles through several states:

enum RSEngineState {
    Uninitialized = -1, // default state, pre "initialize" call
    Init = 0, // library is initialized and ready for mapping
    Mapping, // Mapping and localization services are active
    Stopping, // "stop" has been called on LibResight.instance
    Stopped
};

We can take advantage of LibResight's "onState" callback to hook our app's initialization procedure on the Init state:

LibResight.instance.onState += { (status) in
            switch status {
            case Init:
              ...
            }
}

Within "Init" state we instantiate ResightAnchor at ARKit’s origin (0,0,0) to act as our placeholder for content in this session and we add it to the scene:

let rsAnchor = LibResight.ResightAnchor(world: matrix_identity_float4x4)
arView.scene.addAnchor(rsAnchor)

In order to place content, we take advantage of ReSight Plugin’s TapGesture publisher to create a new box at the current user’s location:

ARViewContainer.cancellable = LibResight.RealityKitPlugin.instance.onTapGesture.sink() { (gesture) in
         let point = gesture.location(in: gesture.view)
         let ray = arView.ray(through: point)

         let transform = Transform(scale: [1, 1, 1], rotation: simd_quaternion(matrix_identity_float4x4), translation: ray!.origin + ray!.direction * 0.2)
         ...
         ...
         let steelBox = CustomBox(color: UIColor(red: r, green:g, blue: b, alpha: 1.0))
         steelBox.setTransformMatrix(transform.matrix, relativeTo: nil)
         ...
}

Finally, we add the entity as a child to the anchor we created earlier. This will add the box to the scene and render it to the screen.

rsAnchor.addChild(steelBox, preservingWorldTransform: true)

That’s it! That's all it takes to create an awesome spatial app in less than 60 lines of code.

We can’t wait to see what you’ll make with our SDK.