Developing Mobile Augmented Reality with Unity and Vuforia

Fred Leighton, University of Wisconsin-Whitewater, USA

Abstract

This is an introduction to developing mobile augmented reality applications for Android and iOS using Unity software, including the development process in Unity with the Vuforia Engine for 2D image and 3D object recognition and use cases.

Keywords: Augmented Reality, AR, Mobile Devices, Development, Unity, Vuforia

Introduction

In this How-To Session, a development process for authoring mobile applications with Unity and Vuforia Engine for Android and iOS applications is shown. Vuforia allows for the use of visual elements and 3D object information for AR markers. With 2D and 3D markers, Unity allows for the development of further interactive content and user interface design.

Intended Audience

The session is intended for designers and developers with experience and/or interest in Mobile AR and Unity as a software development tool. Demonstrations in the session include Unity and Vuforia Engine. The Resources section includes links to online support files including demonstrations.

Session Goals

This session includes information on a development process for using Unity and Vuforia Engine, with examples, demonstrations, and reference material.

Development

Software Tools

Unity

Unity is software that allows for authoring of digital content in 2D or 3D scenes for animations, cinematics, games, simulations and other interactive applications. Along with authoring tools, it includes the C# scripting language and allows for publishing to multiple operating systems and platforms for AR, VR, mobile, desktop, console, TV and web. The software is free to use for development. There are licensing fees for commercial applications.

The Vuforia Engine

Vuforia Engine allows for the building of augmented reality applications in Android, Apple iOS, and Universal Windows Platform (UWP) applications for mobile devices and AR glasses. The Vuforia Engine package for Unity is free to download and install for development. Unity can be used to design augmented reality content by incorporating the Vuforia Engine that adds capabilities that allow for 2D image and 3D object recognition in the real world. The Vuforia Engine Developer Portal is fee to join and provides documentation along with tools used in the development process.

Working with the Vuforia Engine with Unity requires creating a license key and setting up an online database for augmented reality targets. Both are free for development, allowing for one thousand marker recognitions in a month. Applications with a higher number of marker recognitions include fees. Once the database is created, images and datasets are uploaded and managed through the online Target Manager database web page.

 

Development Examples

Example One: Image as AR Marker with User Interface

In this example, a 2D image is used as the augmented reality marker. When recognized by the Vuforia Engine as part of the Unity application, the image will trigger the presentation of a contextual 2D image, registered to the marker. A user interface is added to the scene using authoring tools in Unity.

Unity app with Vuforia 2D image recognition for AR marker

Figure 1: Vuforia AR Marker from 2D image.

Steps

Install the latest version of Unity with the Unity Hub.

Unity Hub Projects Interface

Figure 2: Launching Unity Projects from the Unity Hub.

 

Register as a developer with Vuforia and create a license key. (Vuforia Engine Developer Portal)

Vuforia Engine Developer Portal

Figure 3: Vuforia Engine Developer Portal

Create an augmented reality marker database as a developer with Vuforia.

Create a new 3D project in Unity.

Access the Player Settings, button on the lower left in the Build Settings, from the File menu.

Scroll down to the XR Settings in the Player Settings, and select Vuforia Augmented Reality Supported.

Unity XR Settings in Player Settings panel

Figure 4: XR Settings in Player Settings panel.

In Unity, from the Window menu, select Vuforia Configuration.

In the Vuforia Configuration panel, copy the license key from the Vuforia developer site and paste it into the App License Key space in the panel.

License Manager on Vuforia Engine Developer Portal

Figure 5: Vuforia License Key in License Manager on Vuforia Developer Site.

Unity - License Key in Vuforia Configuration panel

Figure 6: Vuforia App License Key in Unity configuration panel.

Create a new scene in Unity.

Upload the image file to be used as a visual marker to the AR marker database on the Vuforia developer website. When uploading, identify the target type as “image.”

In Unity, in the Game Object menu, select Vuforia Engine/AR Camera, this will place the AR Camera in the scene menu, and add it to the list of items in the scene hierarchy.

From the same Game Object/Vuforia Engine menu, select Image Target. Unity will prompt to download Vuforia related assets. It may be necessary to add the Image Target again after this is completed.

Open the Vuforia Configuration panel, there will be a database listed from the Vuforia sample library. Add the database with the image to be used by selecting the Add Database button.

Target Manager in Vuforia Engine Developer Portal

Figure 7: Selecting Items for a database in the Vuforia Target Manager

Select the database created on the Vuforia developer site, and select individual files to be downloaded from the online database list. When the selected target(s) are chosen, select the Download Database button on the upper right. Select the Unity Editor development platform, and download.

Downloading Vuforia Target Database from Vuforia Developer Portal for use in Unity

Figure 8: Downloading Vuforia Target Database for Unity.

Open the [database name].unitypackage file once it is downloaded.

In Unity, the Import Unity Package window will automatically open, select Import to add the database items. The database will be added and will now be listed in the Vuforia Configuration panel.

Importing the Vuforia Target Database in Unity

Figure 9: Importing the Vuforia Target Database into Unity.

From the Scene menu, select the Image Target. In the Inspector panel, in the Image Target Behavior, select Type: Predefined, Database: [Vuforia DB name] and Image Target: name of image file to be used as the AR target.

From the Scene menu, select the Image Target, CTRL-Click to open a fly out menu to select Game Object / 3D Object / Quad. This will place a flat 3D plane in the scene that will be a child of the Image Target.

Add the image file to replace the Image Target when recognized, to Assets / Editor / Vuforia / ImageTargetTextures / [Project Name] folder.

Create a material for the Quad, with the ImageTarget parent, using the intended replacement image. Add the image to the Albedo map for the material.

Information for a single Target from the Vuforia Target Database

Figure 10: Vuforia Target information.

Place the Quad in 3D space in relation to the Image Target at the origin, for the intended placement in viewing space.

Preview by pressing the play button above the #Scene window and test the AR marker with the computer’s webcam.

Adding a User Interface

Inserting a second camera into the scene with the Vuforia AR Camera provides the addition of 2D user interface elements, for example, buttons, information, and other scripted interactions.

 

Development in Unity - Testing an AR app with a Vuforia Image Target

Figure 11: Unity testing Vuforia using web cam.

Publishing

To publish the application, open the Build Settings from the File menu.

Select the Scenes to include in the Build and choose the Platform and options, then select the Build button and save the application to a location on the computer.

Build Settings Panel in Unity for exporting application

Figure 12: Unity Build settings.

For Android applications, an .apk file will be generated.

Using Android File Transfer, or other similar Android tools, the .apk file is installed on the target mobile device for use. Files should be placed in a location where they can be accessed, for example, com.sec.android.app.myfiles or other similar location that will allow for user access.

For iOS applications, Unity will publish to Xcode where code signing and the final steps can be completed to publish for iOS devices.

 

Development Example Two

Object as AR Marker

In this example, a 3D object is used as the augmented reality marker. When recognized by the Vuforia Engine as part of the Unity application, the image will trigger the presentation of a contextual media, registered to the marker.

Unity app with Vuforia 3D object recognition for AR marker

Figure 13: Unity application with Vuforia Engine object as AR marker.

Steps

Download and install the Vuforia Object Scanner. This application is for Android devices only.

To scan an object to be used as an AR marker with the Vuforia Engine, download and print the Object Scanning Target. This is included in the Vuforia Object Scanner download package. Set up the scanning environment as shown in the Vuforia Object Scanner Guide in the developer online library.

Collecting Data for a 3D object with the Vuforia Object Scanner app

Figure 14: Vuforia Object Scanner app.

Using the Vuforia Object Scanner, scan an object within the size limits allowed by the Object Scanning Target environment. Save the object file.

In the Object Scanner application, there is an option to Test the scan with the object. If the object is recognized, a green rectangle 3D model will appear at the origin point from the original scan orientation.

To upload the scan data, share the .od data file to a location that allows for uploading it to the online Vuforia AR Target database. When uploading, choose “object” for the type of marker.

Follow the same steps as in the first example for selecting AR markers in the database for use in the Unity project and any related scenes, and importing this database into Unity for use with the Vuforia Engine.

From the same Game Object / Vuforia Engine menu, select 3D Scan.

From the Scene menu, select the Object Target. In the Inspector panel, in the Object Target Behavior, select Database: [Vuforia DB name] and Object Target: name of object scan file to be used as the AR target.

Create Unity assets as children of the Object Target and place them in the scene in relation to the Object Target.

For example, a Cylinder with a child Particle System.

Development in Unity - Creating Media for use with a Vuforia Object Target

Figure 15: Unity Particles with Object target.

Resources

Unity.com

Unity with Vuforia

Vuforia Developer Portal

Vuforia Object Scanner User Guide

You Tube demo, adding user interface elements in Unity: https://youtu.be/lhHliig-W-g

You Tube demo, adding button and dynamic text to user interface in Unity: https://www.youtube.com/watch?v=6rr7RkIs54U

Android File Transfer

Xcode

https://www.artsdigital.com/mw20

Vincent van Gogh, 1889, Cypresses (https://commons.wikimedia.org/wiki/File:1889_Vincent_van_Gogh_Zypressen_anagoria.JPGhttps://commons.wikimedia.org/wiki/File:Van_Gogh_-_Zypressen1.jpeg)


Cite as:
Leighton, Fred. "Developing Mobile Augmented Reality with Unity and Vuforia." MW20: MW 2020. Published January 15, 2020. Consulted .
https://mw20.museweb.net/paper/developing-mobile-augmented-reality-with-unity-and-vuforia/