ECHO Demo Video

Standard

Check out this preview of our smart mirror, ECHO.                

To learn more, join us for the debut of our latest demo on April 10th at Ryerson University’s Rogers Communication Centre.

Follow @RyersonRC4@TAsupercourse, @RTARyerson, and @Hosinux  to stay posted on all of the latest updates for our upcoming showcase.

Advertisements

Statement of Feasibility

Standard

Below is a breakdown of the components required to design our prototype. The construction of our prototype will be built as follows:

Mirror (see fig.1):                                                                              The mirror surface has been built by adhering rear projection film to a 36”-48” cut of Plexiglas, which is mounted upright using 2 spider-feet stands.

Augmented Display:                                                                 Option A: A projector (situated at the base of the mirror) will cast the augmented display onto a small mirror placed behind the Plexiglass, which will ricochet onto the larger mirror surface (see fig.1).

Option B: A projector (situated at the base of the mirror) will cast the augmented display directly onto the mirror surface

Gesture-based Interaction/Receptive Display Screen: Gesture-based Interactions will be detected through a combination of Microsoft Kinect (to detect gestures in motion) and a second camera (which will enable functions such as facial detection, photo capture and magnification zooming)

Backend:                                                                                           The prototype will be hooked up to a desktop monitor placed locally near the display.

smartMirror                                       (Fig. 1)

Use Case Profiles/Scenarios

Standard

Our prototype will be designed for domestic use, which we will demonstrate via the following use-case profiles:  (1) Mother (2) Father (3) Child.  Each profile will be customized to suit the routines pertinent to each individual user (see expanded use-case profiles below for details).

User Profile #1 – Mother:

Profile Components:

Widgets: Time, current weather, traffic, calendar notifications

Notification Display: The notification display will disseminate updates (extrapolated from a user’s widgets) to an end-user to notify a user of any changes that need to be made to routines and/or daily plans due to environmental parameters (i.e. weather/traffic delays which conflict with a user’s event calendar)

Receptive Display Screen: Upon opening the receptive display screen [using a single right to left swiping motion over the illuminated display screen bar] a photo-recall system will become activated. The system will enable the end-user to capture daily photos to monitor signs of aging, weight loss or weight gain, health ailments, etc.  All photos will be stored in an archive – and can be played-back via time-lapse upon the user’s request.  Further, photos can be sent to an end-user’s mobile device or emailed to their respective health care provider.

Use-Case Scenario:

Jane wakes up minutes before her alarm and gets out of bed, eager to start her day. She walks into her bathroom and begins her morning routine.  Upon standing before bathroom sink, ECHO’s facial recognition activates her personalized profile. The time, current weather & traffic updates, and calendar notifications begin to appear on the left-hand side, illuminating the mirror’s surface.  Additionally, a notification display centre appears on the left-side panel, which reads “Good morning Jane, there are no current weather or traffic delays.  Estimated arrival to your development meeting at 9am is on schedule.” Jane swipes the notification clear by motioning her hand from left to right in-line with the notification display.

 As Jane washes her face, she notices faint areas of hyper-pigmentation on her skin.   Concerned that her skin imperfection might be a result of sun damage, she decides to begin a photo-log to monitor changes in her facial complexion.

Using gesture recognition, she opens the receptive display screen by hovering her hands in front of the illuminated bar located on the left-side panel, and moving it towards the forefront of the mirror [using a single swipe motion].  She then selects the photo-recall system by pointing at the photo-capture icon on the display screen.  The photo is captured and stored into an archive, which she plans to monitor over the next few weeks.  Jane swipes the display screen closed (by moving her hand over the illuminated bar from left to right). While doing so, she decides that if in a few weeks time, the time-lapse playback shows a significant increase in her facial complexion, she will sync the archive to her mobile or desktop device to send to her dermatologist.

Jane then begins to brush her teeth. The time on the mirror reads 7:15 – 30 minutes before she usually expects to leave for work.   Approximately 5 minutes later, Jane receives an update that appears on the notification display centre.  It reads, “Gardiner Expressway traffic is moving 10 minutes slower than normal. Leave the house at 7:40 to arrive on time for your development meeting at 9am”.  Jane reads the notification, and swipes it clear by motioning her hand from left to right.  She then heads to her kitchen to make a quick cup of coffee before heading out to make it on time for her morning meeting.

Use-Case Profile #2 – Father:

Profile Components:

Widgets: time, current weather, traffic, and calendar notifications

Notification Display: The notification display will disseminate updates (extrapolated from a user’s widgets) to an end-user to notify a user of any changes that need to be made to routines and/or daily plans due to environmental parameters (i.e. weather/traffic delays which conflict with a user’s event calendar)

Receptive Display Screen: Upon gesture recognition [opening the reactive screen open via right to left hand swiping motions], the end-user will be prompted with a medium-sized on-screen overlay that will function to increase the mirror’s magnification.

Use-Case Scenario: 

John wakes up at 7:40am to a text-message from his wife, explaining that she had left the house 10 minutes early to make it on time for her meeting.  She asks him to make sure their 7-year-old son, Chris, brushes his teeth before John brings him to his dentist appointment.   John, who has a dentist appointment in the morning as well, decides to get himself up early so he can get ready before waking his son, Chris.  He walks into the bathroom and begins to brush his teeth. Through facial recognition, his personalized profile becomes activated.  His customized widgets begin to fade-in on-screen.

His notification display centre (on the left-hand panel) reads:

2 New Calendar Reminders:

                      “Dentist appointment for John at 9am                        “Dentist appointment for Chris at 10am”

He swipes the notifications clear by motioning his hands from left to right.

He leans into the mirror and tilts his head slightly to the right, thinking, I could definitely use a shave today. He grabs his razor off to his left [using his left-hand] and swipes the receptive display screen open by motioning his right hand from right to left.  The receptive display screen appears, suggesting the mirror magnification tool.  John points at the display to activate the mirror magnification. As he lathers shaving cream on his cheeks, he decides to zoom-in a little bit closer; he holds his thumb and index finger in front of the display and spreads them apart, signalling the mirror to zoom-in.  John continues to shave and then rinses his sink clean. He leaves the bathroom to go and get his son ready for their dentist appointment.

User Profile #3-Child:

Widgets: time and weather

Notification Display: The notification display will disseminate updates to the end-user.

Receptive Display Screen:  The end-user will activate the receptive screen to appear by swiping the display open by motioning his hands from right to left.  In doing so, an augmented reality game will appear. Upon completing the game, the display can be closed by swiping the illuminated panel from left to right.

Use-Case Scenario:

Chris wakes up to his father, John, asking Chris to start getting ready for their dentist appointments.  John asks Chris to make sure he brushes and flosses his teeth before they leave the house.  Chris walks into the bathroom and looks in the mirror, combing out his bed-head with his fingertips.   By looking into the mirror, Chris’s profile has become activated.  An animated clock appears alongside the weather. Chris notices rain in the forecast and calls out for his dad- asking him to get Chris’s rain boots out from the closet.  Chris proceeds to get ready, and squeezes out a dab of toothpaste onto his toothbrush.  As he begins to brush his teeth, he motions his hands over the illuminated receptive display panel from right to left, triggering an augmented game to appear on the receptive display screen. Plaque monsters begin to multiply around the on-screen reflection on Chris’s mouth, and a countdown appears off to the right.  Eager to vanquish the growth of the plaque monsters, Chris brushes his teeth thoroughly, and as his countdown decreases, the plaque monsters begin to dissipate.  Once the countdown comes to a close, Chris banishes the last remaining plaque monster. Chris rinses off his mouth and his toothbrush.  Feeling a sense of victory, he swipes the receptive screen closed and sports a proud smile in front of the mirror.  After feeling the satisfaction of a job well done, he finishes up by flossing his teeth.  Chris closes the bathroom light and calls for his dad to let him know that he is ready to go to the dentist. 

Echo is now live on WordPress!

Standard

Stay tuned for all of our latest updates/progress on the development of our project,

logo_001

Extant Smart Mirrors on the market enable users to view email, social media, weather, play music, and send information to other smart objects relative to their environment [1][2][3][4]. While these products offer a multitude of widgets to users, the volume of content displayed on-screen, combined with touch-screen/remote control navigation fundamentally transforms the utility of the mirror into a computer monitor instead.  It is imperative that ECHO addresses this concern, by enhancing the functionality of a mirror as opposed to transforming the object from its intended use.

Our system will adhere to the abovementioned industry gaps as follows:

 ECHO is an interactive mirror equipped with an augmented display of interoperable widgets, as well as a context-aware panel (receptive display screen) that adapts and responds to a user’s actions.

 Echo is designed to improve the efficiency of your day-to-day life by adapting to your personal routines. Through learning about your personal behavior, ECHO is able to provide customized and contextually relevant information to a user.

 Each personalized profile is activated utilizing facial recognition.  Initiating a user’s individual profile will prompt a series of customized widgets that provide live updates and information pertinent to the end-user’s day-to-day schedule.  From weather and traffic patterns to calendar notification reminders, ECHO widgets serve to help users organize their schedule accordingly, and resultantly, optimize their time management.

With ECHO, there is no need to search or navigate for information on a mobile app or desktop computer- our widgets are interoperable; that is, they communicate with one another so users won’t have to browse through each idiosyncratic widgets, as all relevant and timely updates will be displayed in real-time on the notification display, which can be found on the left-side panel of the mirror.

 Through gesture recognition and motion detection, ECHO’s receptive display screen will provide end-users with augmented tools that will offer extensive functionality to their present task at hand.  Once a particular action has been detected, a vertical panel [attached to the notification display] will become illuminated to notify an end-user that the receptive display screen [pertinent to their current task, see use-cases for further details] is actively available.  The end-user is then able to call up the screen to the centre of the mirror by motioning their hands over the luminescent bar from right to left.  Once a user has finished using the receptive display screen, they can dismiss the augmented tool by swiping their hand over the illuminated bar from left to right.

 In designing ECHO, we recognized the importance to maintain the utility of the mirror, which is why ECHO is configured to respond to a user’s preferences and actions, thereby allowing a user to sustain their primary focus on themselves. Through this trajectory, ECHO will improve the efficiency of end-users real life actions by understanding and enhancing a user’s interests and needs.