Kotlin Cardboard

This is a demo Kotlin Multiplatform Mobile project that implements the Google Cardboard VR platform. It is implemented primarily in Kotlin and exposes everything so that many applications would not have to worry about any native code but just write shared Kotlin code. The demo given is the same as the "Hello Cardboard" demo included in the official Cardboard SDK, but written in Kotlin and uses Compose for the UI.

Getting Started

Development has only been tested on macOS due to the need for Xcode to build the iOS app. Before being able to compile you first must initialize the submodules (it uses the official Cardboard SDK as a submodule). You can do this by running the following command in the root of the project:

git submodule update --init --recursive

Then you to compile the Cardboard SDK iOS static library (the Android ones are automatically built by Gradle). You can do this by running the following command in the root of the project:

cd iosApp
./setup.sh

Project Structure

Standard Kotlin Multiplatform project structure is used in this project but with several additions due to the native bridges required to access the Cardboard SDK which is written in C/C++.

• cardboard contains the official Cardboard SDK code as a submodule. It's contents are not meant to be modified, but rather used as a dependency in your project.
• proto is a symlink to cardboard/proto which contains the Cardboard protocol buffers definitions. Currently required because of how gradle is set up, but could likely be fixed to not require this symlink at all.
• composeApp is for almost all code that used for the Compose Multiplatform application. It contains several subfolders:
  • commonMain is for code and resources that is common/shared for both Android and iOS, it includes:
    • kotlin which contains the shared Kotlin code for the application
    • resources/assets is for shared assets that will be used in both Android and iOS, currently this includes the 3D models and textures used in the demo.
  • androidMain is for code that is unique to the Android platform, it includes:
    • cpp and CMakeLists.txt which are used to compile the native JNI bridge to the Cardboard SDK C/C++ code for Android.
    • AndroidManifest.xml which is the Android manifest for the application which includes several required permissions and features for the Cardboard SDK to work properly.
    • kotlin which contains the Android-specific Kotlin code.
  • iosMain is for code that is unique to the iOS platform
  • nativeInterop contains the description of the native interop libraries that are used to bridge the Cardboard SDK C/C++ code to Kotlin in iOS. It is not used for Android (which uses JNI instead).
• iosApp contains the iOS application entry point and XCode project, for the most part this will not need to be modified (expect for including the Cardboard SDK static library and other dependent frameworks).

For code/settings added to the default Gradle files and similar, search for **ADDED**.

Code Structure

The demo app is launched from composeApp/commonMain/kotlin/org/example/cardboard_demo/App.kt. The other code in org.example.cardboard_demo is used to implement the demo apps, all written in pure shared Kotlin code (along with some shader code).

The cardboard library includes the code required to easily get a Cardboard view up and running:

• cardboard.sdk is the Kotlin wrapper around the official Cardboard SDK C/C++ with a few convenience functions to make it easier/faster to use.
• cardboard.util are utility classes/functions for Matrices, Vectors, playing media, and loading assets from the shared resources.
• cardboard.CardboardRenderer is the main class that handles rendering the Cardboard view and managing the Cardboard lifecycle. It must be subclassed to implement the rendering logic for a specific application. It requires a:
  • NativeRenderView which is a platform-specific wrapper around a native view that can be used to render the Cardboard view. It also implements the hooks for lifecycle events such as onInit, onResume, onPause, and onDestroy. On Android this wraps a GLSurfaceView and on iOS it wraps a MTKView.
  • NativeRenderer which is a platform-specific implementation of the base rendering logic. It exposes several functions for loading meshes, textures, and rendering them. On Android, the default it uses GLES and on iOS it uses Metal. In both cases it uses a single offscreen texture to render the combined left and right eyes. Other native renderers can be implemented if more complex rendering is required.
  • Shaders which is a collection of shader strings and variables for the different platforms. These are not built into the NativeRenderer for flexibility but certain rules must be followed to be compatible with the default NativeRenderers (e.g. there is a position and UV attribute, a model view matrix uniform, and a texture sampler uniform).
  • Cardboard rendering parameters, see the CardboardRendererParams class.

TODO

• Rendering issue on iOS where some parts are "inverted", missing, or similar?
• Force screen to be full-screen landscape, full brightness, and prevent screen from sleeping whenever the Cardboard view is visible (Android mostly working, iOS not yet)
• Test with QR code scanner to initialize the Cardboard SDK parameters
• Do not require the proto symlink by fixing the Gradle setup
• Do not require manual running of iosApp/setup.sh, instead use Gradle to do run it automatically
• Use a slightly improved cross-platform asset system