Developing Mobile Augmented Reality with Unity and Vuforia
Fred Leighton, University of Wisconsin-Whitewater, USA
AbstractThis is an introduction to developing mobile augmented reality applications for Android and iOS using Unity software, including the development process in Unity with the Vuforia Engine for 2D image and 3D object recognition and use cases.
Keywords: Augmented Reality, AR, Mobile Devices, Development, Unity, Vuforia
In this How-To Session, a development process for authoring mobile applications with Unity and Vuforia Engine for Android and iOS applications is shown. Vuforia allows for the use of visual elements and 3D object information for AR markers. With 2D and 3D markers, Unity allows for the development of further interactive content and user interface design.
The session is intended for designers and developers with experience and/or interest in Mobile AR and Unity as a software development tool. Demonstrations in the session include Unity and Vuforia Engine. The Resources section includes links to online support files including demonstrations.
This session includes information on a development process for using Unity and Vuforia Engine, with examples, demonstrations, and reference material.
Unity is software that allows for authoring of digital content in 2D or 3D scenes for animations, cinematics, games, simulations and other interactive applications. Along with authoring tools, it includes the C# scripting language and allows for publishing to multiple operating systems and platforms for AR, VR, mobile, desktop, console, TV and web. The software is free to use for development. There are licensing fees for commercial applications.
The Vuforia Engine
Vuforia Engine allows for the building of augmented reality applications in Android, Apple iOS, and Universal Windows Platform (UWP) applications for mobile devices and AR glasses. The Vuforia Engine package for Unity is free to download and install for development. Unity can be used to design augmented reality content by incorporating the Vuforia Engine that adds capabilities that allow for 2D image and 3D object recognition in the real world. The Vuforia Engine Developer Portal is fee to join and provides documentation along with tools used in the development process.
Working with the Vuforia Engine with Unity requires creating a license key and setting up an online database for augmented reality targets. Both are free for development, allowing for one thousand marker recognitions in a month. Applications with a higher number of marker recognitions include fees. Once the database is created, images and datasets are uploaded and managed through the online Target Manager database web page.
Example One: Image as AR Marker with User Interface
In this example, a 2D image is used as the augmented reality marker. When recognized by the Vuforia Engine as part of the Unity application, the image will trigger the presentation of a contextual 2D image, registered to the marker. A user interface is added to the scene using authoring tools in Unity.
Figure 1: Vuforia AR Marker from 2D image.
Install the latest version of Unity with the Unity Hub.
Figure 2: Launching Unity Projects from the Unity Hub.
Register as a developer with Vuforia and create a license key. (Vuforia Engine Developer Portal)
Figure 3: Vuforia Engine Developer Portal
Create an augmented reality marker database as a developer with Vuforia.
Create a new 3D project in Unity.
Access the Player Settings, button on the lower left in the Build Settings, from the File menu.
Scroll down to the XR Settings in the Player Settings, and select Vuforia Augmented Reality Supported.
Figure 4: XR Settings in Player Settings panel.
In Unity, from the Window menu, select Vuforia Configuration.
In the Vuforia Configuration panel, copy the license key from the Vuforia developer site and paste it into the App License Key space in the panel.
Figure 5: Vuforia License Key in License Manager on Vuforia Developer Site.
Figure 6: Vuforia App License Key in Unity configuration panel.
Create a new scene in Unity.
Upload the image file to be used as a visual marker to the AR marker database on the Vuforia developer website. When uploading, identify the target type as “image.”
In Unity, in the Game Object menu, select Vuforia Engine/AR Camera, this will place the AR Camera in the scene menu, and add it to the list of items in the scene hierarchy.
From the same Game Object/Vuforia Engine menu, select Image Target. Unity will prompt to download Vuforia related assets. It may be necessary to add the Image Target again after this is completed.
Open the Vuforia Configuration panel, there will be a database listed from the Vuforia sample library. Add the database with the image to be used by selecting the Add Database button.
Figure 7: Selecting Items for a database in the Vuforia Target Manager
Select the database created on the Vuforia developer site, and select individual files to be downloaded from the online database list. When the selected target(s) are chosen, select the Download Database button on the upper right. Select the Unity Editor development platform, and download.
Figure 8: Downloading Vuforia Target Database for Unity.
Open the [database name].unitypackage file once it is downloaded.
In Unity, the Import Unity Package window will automatically open, select Import to add the database items. The database will be added and will now be listed in the Vuforia Configuration panel.
Figure 9: Importing the Vuforia Target Database into Unity.
From the Scene menu, select the Image Target. In the Inspector panel, in the Image Target Behavior, select Type: Predefined, Database: [Vuforia DB name] and Image Target: name of image file to be used as the AR target.
From the Scene menu, select the Image Target, CTRL-Click to open a fly out menu to select Game Object / 3D Object / Quad. This will place a flat 3D plane in the scene that will be a child of the Image Target.
Add the image file to replace the Image Target when recognized, to Assets / Editor / Vuforia / ImageTargetTextures / [Project Name] folder.
Create a material for the Quad, with the ImageTarget parent, using the intended replacement image. Add the image to the Albedo map for the material.
Figure 10: Vuforia Target information.
Place the Quad in 3D space in relation to the Image Target at the origin, for the intended placement in viewing space.
Preview by pressing the play button above the #Scene window and test the AR marker with the computer’s webcam.
Adding a User Interface
Inserting a second camera into the scene with the Vuforia AR Camera provides the addition of 2D user interface elements, for example, buttons, information, and other scripted interactions.
Figure 11: Unity testing Vuforia using web cam.
To publish the application, open the Build Settings from the File menu.
Select the Scenes to include in the Build and choose the Platform and options, then select the Build button and save the application to a location on the computer.
Figure 12: Unity Build settings.
For Android applications, an .apk file will be generated.
Using Android File Transfer, or other similar Android tools, the .apk file is installed on the target mobile device for use. Files should be placed in a location where they can be accessed, for example, com.sec.android.app.myfiles or other similar location that will allow for user access.
For iOS applications, Unity will publish to Xcode where code signing and the final steps can be completed to publish for iOS devices.
Development Example Two
Object as AR Marker
In this example, a 3D object is used as the augmented reality marker. When recognized by the Vuforia Engine as part of the Unity application, the image will trigger the presentation of a contextual media, registered to the marker.
Figure 13: Unity application with Vuforia Engine object as AR marker.
Download and install the Vuforia Object Scanner. This application is for Android devices only.
To scan an object to be used as an AR marker with the Vuforia Engine, download and print the Object Scanning Target. This is included in the Vuforia Object Scanner download package. Set up the scanning environment as shown in the Vuforia Object Scanner Guide in the developer online library.
Figure 14: Vuforia Object Scanner app.
Using the Vuforia Object Scanner, scan an object within the size limits allowed by the Object Scanning Target environment. Save the object file.
In the Object Scanner application, there is an option to Test the scan with the object. If the object is recognized, a green rectangle 3D model will appear at the origin point from the original scan orientation.
To upload the scan data, share the .od data file to a location that allows for uploading it to the online Vuforia AR Target database. When uploading, choose “object” for the type of marker.
Follow the same steps as in the first example for selecting AR markers in the database for use in the Unity project and any related scenes, and importing this database into Unity for use with the Vuforia Engine.
From the same Game Object / Vuforia Engine menu, select 3D Scan.
From the Scene menu, select the Object Target. In the Inspector panel, in the Object Target Behavior, select Database: [Vuforia DB name] and Object Target: name of object scan file to be used as the AR target.
Create Unity assets as children of the Object Target and place them in the scene in relation to the Object Target.
For example, a Cylinder with a child Particle System.
Figure 15: Unity Particles with Object target.
You Tube demo, adding user interface elements in Unity: https://youtu.be/lhHliig-W-g
You Tube demo, adding button and dynamic text to user interface in Unity: https://www.youtube.com/watch?v=6rr7RkIs54U
Vincent van Gogh, 1889, Cypresses (https://commons.wikimedia.org/wiki/File:1889_Vincent_van_Gogh_Zypressen_anagoria.JPG, https://commons.wikimedia.org/wiki/File:Van_Gogh_-_Zypressen1.jpeg)
Leighton, Fred. "Developing Mobile Augmented Reality with Unity and Vuforia." MW20: MW 2020. Published January 15, 2020. Consulted .