When it Rains: An AR storybook

Padmasree M M

Introduction

Augmented Reality (AR) is revolutionizing education, making learning fun and engaging. AR books transform storytelling with animations, sound, and vivid visuals, fostering imagination and critical skills. Unlike traditional books, AR brings stories to life, making reading exciting and nurturing a love for learning in the digital age.

The project

What am I doing?

This is an Interactive children’s storybook for ages 5-6 years. The stories come alive in front of eyes in Augmented Reality. The Illustrations in each page in the book act as a marker. It comes alive when scanned through the app. It can be used with ipad, tablets and smartphones. With the AR, the storybook gets a new dimension, where children can experience animation and sound along with the comfort of portability.

Narrative

About story

It is a simple story about two friends coming back from their school on a rainy day. It’s about things they see and the little moments. Story depicts the love, kindness and friendship for each other. Story is narrated through conversation between the kids.

Character Design

There are two main characters in the story. Sophie and Ammu are best friends. They care for each other. Both are mischievous and playful.

Storyboard

Story

Simple story from childhood About two friends coming back from the school on a rainy day Things they see and little moments About their love, kindness and friendship for each other Narrated through the conversation between each other

1.Ammu and Sophie are on their way back home from school enjoying the scenic scenery in the background of kerala. Passing by a flower bloomed meadow.

Girl1(Sophie) : Look at those flowers, I can make a beautiful garland with those.
Girl 2(Ammu): Oh no...Don’t pluck them they will love us if we let them live. Sophie: They are alive? Can they see us? Ammu: Yeah, they will die if we pluck them. They have emotions too..

2. Weather is changing to slightly dark cloudy- about to start rain and windy. A frog croaks next to them- it’s about to rain.

Sophie smiles at her best friend and leaves the flower unplucked.

3. Both children are sharing one umbrella, leaving them wet on both sides anyway.

Sophie: I really really love this fragrance of wet mud. And look at those greenery around us! Wow! Ammu: Yeah true! How good is that every year our school reopens on ‘Edavapathy rains’. And we get to enjoy this rain so much.

4. They decide to drop the umbrella and move forward without it and actually enjoying it!

5. Both about to start a race from the hilly top to bottom.
Sophie: Who wins will buy the other ‘Mittayi’ from Dasettan’s shop, cross your heart. Ammu: Yippie, let’s start the race!

6. Local fisherman catching fish. Children are both watching in wonder
Ammu: I will also catch ‘Piloppi’ Sophie: Uncle will you give us one ‘Piloppi’ so that will take it home and put it in the tank?

Uncle(Local fisherman) laughs Children Laughs

7. Both children are about to turn in two different directions, to their homes.
Sophie: If Mummy finds me wet, she will surely scold me. Ammu: Don’t tell her. As soon as you reach home, change your clothes and wipe yourself well. Sophie: Yes, you too and take this ‘Piloppi’ with you.

8. A distant discussion of friends
Ammu: Don’t forget to bring ‘Morukoottan’ tomorrow for lunch Sophie: Yes, will tell Mummy.

9. A traditional home interior, where the kid is half dried with the towel.
Sophie/Ammu: Achheeee!!(Sneeze)

Visual Development and final illustrations

Research and Inspirations

As part of research, I looked at current AR projects, Interactive storytelling including comics and storybooks. I also looked at books with physical interactivity.

Explorations

Slide AR

Slide AR is an augmented reality app where artists can upload layer by layer illustrations and view it in AR. But the disadvantage of the app is that you cannot export it.

Adobe Aero

Adobe aero is another augmented reality app where you can upload the psd and view it as AR. It has a lot of options to add interactivity also without any coding knowledge. But the problem with the app is that you can only send through links to view this. And the export formats won’t support unity or any other software. So you can’t make it into an apk file.

Unity

Augmented reality (AR) can be defined as the overlaying of digital content (images, video, text, sound) onto locations and it is typically experienced by looking through the camera of an electronic device such as a smartphone, tablets, etc.

Using sprites

The illustration is kept as a marker for the content. Each layer is imported to unity as sprites and sprite animations. It is kept above the marker in the order of appearance and height. Sprite masking is also done to view it inside panels.

In order to view it hold the illustration in front of the webcam. The layers and animation will appear above the illustration.

Using video playback

Instead of uploading layer by layer, a video playback is kept by adding a video component for the marker. On viewing the marker illustration, an animated video will be playing.