Unity + Vuforia: Building Marker-Based AR
A walkthrough of how a Unity application recognises a printed image and overlays 3D content on it in real time — the core concepts, then the full build pipeline, with the reasoning behind each step. Built as a live reference for walking an audience through a practical AR demonstration.
The AR Technology Tree
Every node below is a real alternative at that layer of the stack. Click any node to trace its path from "Augmented Reality" and read what it is on the right. Nodes highlighted in green are the ones this build actually uses — they converge into the final stack at the bottom.
Core Concepts
What is Augmented Reality (AR)?
Augmented Reality overlays computer-generated content — 3D models, video, text, sound — onto a live view of the real world, usually through a phone or headset camera. Unlike Virtual Reality, which replaces what you see entirely, AR adds to it: the real environment stays visible, with digital elements blended in on top and anchored so they appear to belong in the scene.
There are two broad approaches, and the distinction matters for what you're about to demonstrate:
- Marker-based AR — content appears when the camera recognises a specific pre-registered image or object (a "marker"). This is what Vuforia's Image Targets do, and what today's demo is built on.
- Markerless AR — content is placed using the device's own sensors (camera, GPS, accelerometer, depth sensing) to understand surfaces and location, without needing a pre-registered image. This is how ARKit/ARCore "place an object on this table" experiences work.
Why marker-based, for this demo: it's deterministic and easy for an audience to follow — show the printed image, the content appears, move the image, the content follows. There's a clear, visible cause and effect, which is harder to narrate live with markerless/surface-detection AR.
What is a Marker / Image Target?
A marker is a real-world image or object that an AR application is trained to recognise. In Vuforia, this is called an Image Target. Once the camera detects the target in the physical world, the engine tracks its position and orientation continuously — even as the camera or the target moves — and uses that position as the "anchor" for digital content.
Under the hood, Vuforia converts the marker image to grayscale and extracts feature points — sharp, high-contrast corners and edges — then stores them as a searchable set of coordinates. At runtime, it looks for that same pattern of feature points in the live camera feed. The more distinct, asymmetric detail an image has, the more reliable the match.
This is why Vuforia's Target Manager gives every uploaded marker a star rating from 1 to 5 before you ever open Unity — it's scoring how trackable the image actually is.
Why this matters live: if your printed marker isn't tracking well on stage, it's almost always a feature-point problem — too plain, too symmetric, too low-contrast, or too small/blurry when printed. Worth checking the star rating beforehand, not debugging it live.
What is an SDK?
A Software Development Kit (SDK) is a ready-made toolbox for building on a specific platform — a bundle of libraries, APIs, sample code, and documentation, so developers don't have to solve the same low-level problems from scratch. Vuforia is an SDK for computer-vision-based AR: it handles image recognition, camera access, and real-time tracking, so a developer only has to decide what content to show and where.
Why it matters: without an SDK like Vuforia, recognising and tracking an image through a camera feed in real time would mean writing your own computer-vision pipeline — feature detection, matching, pose estimation — before you could even begin placing 3D content. The SDK is what turns a multi-week computer-vision research problem into a feature you can set up inside Unity in an afternoon.
What is a Game Engine?
A game engine is a software framework that provides the core systems a real-time application needs — rendering, physics, audio, input, and scene management — so developers build the experience itself rather than that underlying infrastructure. Unity is one of the most widely used engines for this, supporting 2D, 3D, AR, and VR projects across many platforms from a single codebase.
In an AR pipeline, the game engine is where everything comes together: it runs the SDK's tracking data, renders the 3D content, and handles how the user interacts with it. Vuforia alone only tells you where the marker is — Unity is what actually draws the 3D object on top of it and lets you script what happens next.
What is Vuforia, specifically?
Vuforia Engine is an augmented reality SDK that plugs directly into Unity. It provides the computer-vision pipeline — detecting and tracking Image Targets, Model Targets, and other marker types through the device camera — and exposes that tracking data to Unity's scene graph, so a 3D object can simply be "childed" under a tracked target and will follow it automatically, with no manual position-tracking code required.
Developed by PTC. Account, license keys, and documentation live on the Vuforia Developer Portal; the Unity integration itself is distributed as a free package on the Unity Asset Store.
The Full Build Flow
Create a Vuforia Developer Account
Go to the Vuforia Developer Portal and register for a free account. This account is what every license key and target database you create is tied to.
Generate a Vuforia License Key
In the portal, go to Develop → License Manager → Get Basic (free tier), give the license a name, and create it. Click into the license you just created and copy the key shown there.
Why: Vuforia won't initialise in Unity without a license key tied to your account — it's what authorises the app to use Vuforia's recognition and tracking features.
Create the Unity Project
In Unity Hub, click New Project, pick the Universal 3D template, name it, and click Create — a clean starting point with the camera, lighting, and render pipeline already in place.
Import Vuforia Engine
On the Vuforia Engine Asset Store page, click Add to My Assets (sign in if asked). Back in Unity, open Window → Package Manager, switch the dropdown at the top to Packages: My Assets, select Vuforia Engine AR, click Download, then Import, and confirm with Import in the dialog that appears.
Add the License Key to Unity
In Unity, open Window → Vuforia Configuration. In the Inspector panel that opens, paste your key from step 2 into the App License Key field. It saves automatically — there's no separate save button.
Replace the Main Camera with an AR Camera
In the Hierarchy panel, select Main Camera and delete it. Then go to GameObject → Vuforia Engine → AR Camera to add the replacement. This camera streams the live device feed into the scene and feeds it through Vuforia's tracking.
Why: a normal Unity camera only renders the 3D scene — it has no idea what the physical camera is seeing. The AR Camera is what bridges the two: real camera feed in, tracked marker positions out.
Design the Marker
With Unity now AR-ready, switch to designing the marker itself: sketch or draw something with rich, asymmetric detail. Avoid plain shapes, symmetry, or large flat areas of a single colour.
Why: this connects directly to the feature-point tracking covered above — detail, contrast, and asymmetry are literally what Vuforia extracts and matches against. A plain or symmetric drawing will track poorly no matter how well everything else is set up.
Photograph the Marker
Take a clear photo of the finished drawing. Don't edit the colours or apply any filter — just crop it to the drawing's edges — then transfer the image to your laptop.
Why: Vuforia extracts features from this exact image. Colour edits or filters change what gets analysed versus what the camera will actually see live, which can quietly hurt tracking.
Create a Target Database
In the Developer Portal, go to Develop → Target Manager, click Add Database, give it a name, choose Device as the database type (not Cloud), and click Create.
Upload the Marker Image
Click into your new database, then click Add Target. Set Type to Single Image, choose your cropped photo as the file, enter a Width (in scene units) and a Name, then click Add.
Check the Star Rating
Back on the database page, the star rating (1–5) appears next to your uploaded target's thumbnail. Only proceed at 4 or 5 stars. Below that, go back to step 7 and redesign the marker with more contrast and detail.
Why: this rating is a direct preview of how reliably the marker will track live — catching a weak marker here is far better than discovering it isn't tracking in front of an audience.
Download the Database
Tick the checkbox next to your target, click Download Database (All), choose Unity Editor as the format, then Download and save the .unitypackage file.
Import the Target Database into Unity
Back in the Unity project from step 6, double-click the downloaded .unitypackage (or go to Assets → Import Package → Custom Package and select it), then click Import in the dialog — this brings your specific marker's tracking data in.
Add an Image Target
Go to GameObject → Vuforia Engine → Image Target. Select it in the Hierarchy, then in its Inspector (the Image Target Behaviour component), set the Database dropdown to the database you just imported and the Image Target dropdown to your specific marker.
Why: this is the link between "an image Vuforia can recognise" and "a point in the Unity scene" — it's the anchor everything else attaches to.
Add Your 3D Content
Drag your own 3D model — or the one provided — from the Project window into the Hierarchy, dropping it directly onto the ImageTarget object so it becomes a child of it.
Why: Unity's parent-child transform system means the content's position updates automatically whenever the Image Target's tracked position updates — no manual code needed to make it "follow" the marker.
Scale and Position the Model
Select the model and use its Inspector's Transform → Scale fields (or press R for the Scale tool in the Scene view) so the content reads clearly against the marker — not so small it's hard to see, not so large it overwhelms the frame or clips oddly. Keep proportions close to how the object would actually look at that size.
Test with the Webcam
Click the Play button at the top of the Unity Editor, allow camera access if prompted, and hold the printed marker up to your laptop's webcam. Confirm the model appears, stays anchored, and tracks correctly as you move the marker.
Build the UI
Go to GameObject → UI → Canvas (this also auto-creates an EventSystem), then add buttons, labels, or on-screen instructions as children via GameObject → UI → Button / Text, etc.
Add Particle Effects
Go to GameObject → Effects → Particle System for visual flourishes — sparks, glow, dust, and similar — as additional children of the Image Target or model.
Set Up Animation
Drag a rigged, animated model (FBX) into the Project window. Select it, open the Rig tab in the Inspector, set Animation Type to Humanoid or Generic, and click Apply. Right-click in the Project window → Create → Animator Controller, then open Window → Animation → Animator to drag in clips and connect transitions between them. Finally, select your model's Animator component and assign this controller to its Controller field.
Add Unity Events
On a UI Button's Inspector, scroll to the On Click () list, click +, drag in the target GameObject, then pick a function from the dropdown — for example, a particle system's SetActive to toggle visibility, or a custom script's public method to trigger an animation.
Write a Script
In the Project window, right-click → Create → C# Script, name it, and double-click to open it in your code editor. After writing the logic — for example, a "billboard" script that makes a text label continuously face the camera, or a script that keeps the AR model rotating — drag the script onto the target GameObject in the Hierarchy (or use Add Component in its Inspector) to attach it.
Open Build Settings
Go to File → Build Settings, select Android (or iOS) in the Platform list, and click Switch Platform.
Configure Player Settings
In the same Build Settings window, click Player Settings. Under Other Settings, set the Package Name and Minimum API Level; under Icon, set the app icon; under Resolution and Presentation, set the Default Orientation.
Manage Scenes in the Build
In the Build Settings window's Scenes In Build list, click Add Open Scenes to add your AR scene, then select and remove (uncheck or delete) any unused or demo scenes already listed.
Build
Click Build (or Build And Run) in the Build Settings window, choose an output folder, and Unity will compile the final APK (or IPA).
Why Unity + Vuforia (and not something else)
Unity is the dominant engine for AR/VR work — an estimated 60% of AR/VR content is built on it — largely because of its mobile-first architecture, built-in AR Foundation/ARCore/ARKit support, and a much gentler learning curve (C#) than Unreal's C++ toolchain. For a live demo meant to run reliably on a phone in front of an audience, that mobile optimisation and setup speed matters more than Unreal's graphical ceiling.
Vuforia remains one of the most established marker-tracking SDKs for exactly this use case: it's been doing image-target recognition since well before ARKit/ARCore existed, and it works consistently across both Android and iOS from one Unity project — useful when you don't know in advance which device the audience will be watching it on.