When it Rains: An AR storybook

Padmasree M M

Introduction

Augmented Reality (AR) is taking over in education. It allows for a completely new and easy method for learning. For years, bedtime stories have been every kid’s favorite memory. Interactive books for children have tons of developmental benefits. It helps with critical thinking, developmental skills, also foster love of reading and support early literacy. With traditional books, interactions were purely based on imagination and the child could only visualize the story vicariously. Children find reading traditionally dull and boring. Kids these days spend so much time on screen. With AR, storytelling becomes fun and exciting. It makes even a simple story very much interesting with animations, sound and other alive visuals.

The project

What am I doing?

This is an Interactive children’s storybook for ages 5-6 years. The stories come alive in front of eyes in Augmented Reality. The Illustrations in each page in the book act as a marker. It comes alive when scanned through the app. It can be used with ipad, tablets and smartphones. With the AR, the storybook gets a new dimension, where children can experience animation and sound along with the comfort of portability.

Narrative

About story

It is a simple story about two friends coming back from their school on a rainy day. It’s about things they see and the little moments. Story depicts the love, kindness and friendship for each other. Story is narrated through conversation between the kids.

Character Design

There are two main characters in the story. Sophie and Ammu are best friends. They care for each other. Both are mischievous and playful.

Storyboard

Visual Development and final illustrations

Research and Inspirations

As part of research, I looked at current AR projects, Interactive storytelling including comics and storybooks. I also looked at books with physical interactivity.

Explorations

Slide AR

Slide AR is an augmented reality app where artists can upload layer by layer illustrations and view it in AR. But the disadvantage of the app is that you cannot export it.

Adobe Aero

Adobe aero is another augmented reality app where you can upload the psd and view it as AR. It has a lot of options to add interactivity also without any coding knowledge. But the problem with the app is that you can only send through links to view this. And the export formats won’t support unity or any other software. So you can’t make it into an apk file.

Unity

Augmented reality (AR) can be defined as the overlaying of digital content (images, video, text, sound) onto locations and it is typically experienced by looking through the camera of an electronic device such as a smartphone, tablets, etc.

Using sprites

The illustration is kept as a marker for the content. Each layer is imported to unity as sprites and sprite animations. It is kept above the marker in the order of appearance and height. Sprite masking is also done to view it inside panels.

In order to view it hold the illustration in front of the webcam. The layers and animation will appear above the illustration.

Using video playback

Instead of uploading layer by layer, a video playback is kept by adding a video component for the marker. On viewing the marker illustration, an animated video will be playing.