How Beauty Care Companies Can Combine Augmented Reality with Chatbots for Customized Online Experience

Anastasia Morozova
Chief Operating Officer

Augmented reality (AR) is a perfect match for the beauty industry, for this is where customers tend to try the products on before buying them.

Each face is unique, so it is essential to be sure that this particular shade goes well with the person’s color type.

Augmented reality suits this purpose to the maximum, allowing customers to put virtual makeup on their face.

sephora augmented reality

Related article: How Much Does an Augmented Reality App Development Cost?

Many brands have already been experimenting with technologies such as artificial intelligence and face tracking – they filter the customer’s requirements by asking some questions and then suggest products which the customer can immediately “put on”.

However, the latest news from the beauty sector shows that some brands have started cooperating with software developers to bring another component into the picture – chatbots.

The use of artificial intelligence, which is the fundamental technology under all chatbots, and augmented reality results in practical solutions combining a personalized approach to each customer and the option of a virtual try-on of the selected products.

The shopping experience customers get from interacting with AR-based chatbots is utterly stunning.

Imagine a beauty consultant attending you in a cosmetics store asking you questions, finding out your preferences, making personal makeup recommendations and finally suggesting the products which will be “just right” for you. The consultant is holding a mirror for you while you are putting on the makeup and admiring your new look. Oh, and you do not even have to leave home for that because this beautiful experience is now available to you via beauty brand chatbots.

augmented reality makeup app

What technology is needed for AR-enhanced chatbots

Beauty brand chatbots rest on top of a brilliant technology solution which is a product of advanced software development.

If we look under the hood, we will see that the chatbot includes the following know-hows:

–        Artificial intelligence, allowing the bot to recognize the customer’s answers and respond accordingly. Advanced chatbots can hold quite realistic conversations because they have been “trained” with hundreds of phrases during their development. Besides, chatbots can learn from their conversations with customers by enhancing their knowledge base. AI is the component creating that “personalized” feeling we experience when talking to a chatbot;

–        Face tracking, the technology of recognizing the face features in an uploaded image and locating their place in the picture;

–        Augmented reality, working closely with face tracking to apply virtual elements to the real image, for example, putting a virtual lipstick on the user’s lips in a photo. We have already talked about augmented reality face recognition and facial tracking apps in our other article;

–        E-commerce, the platform supporting the shopping component of the chatbot functionality.

E-commerce software provides the ability to browse the inventory, pull up product prices, display images from the product catalogs, and many other commerce-related functions.

This is a very brief summary of the powerful technology at the core of a beauty chatbot.

Some beauty brands have already rolled out their solutions combining augmented reality, artificial intelligence, face tracking, and e-commerce to make their customers’ e-shopping as close to that “pampered” sensation of a visit to a beauty boutique as possible.

Let’s look at a couple of them to see what these industry forerunners have to show.

Sephora Virtual Artist – see how a new lipstick shade fits you

If you are thinking of getting a new lipstick and do not mind trying on an unusual shade to create an entirely new look, start a chat with Sephora Virtual Artist.

The bot can help you find a lipstick shade from a Sephora collection, select a lipstick matching the color of your dress or your general color type or surprise you by suggesting a random product from the Sephora lipstick line.

In all cases, there is the “Try it on” option, and this is when the magic begins. The chatbot asks you to upload a front-facing photo, and in a second you see what you look like with the lipstick applied. You can go on with makeup for hours trying various shades and getting new, unexpected looks.

Of course, for all products, there is the “Buy now” option, so if you come across a lipstick which makes you look truly stunning, Sephora can make it real. The purchase is completed on Sephora’s website, though, as the payment mechanism has not yet been implemented in its chatbot.

There is also an attractive additional feature – for all looks created with Sephora lipsticks, the bot offers the “Download image” option. Think of all the fabulous avatars you can make!

augmented reality makeup

Estée Lauder Lip Artist – get professional makeup advice without leaving home

In 2017, Estée Lauder released a Facebook Messenger bot providing a complete shopping experience for customers. You can search for products, browse collections, try lipsticks on and purchase the ones you like – all within the same Facebook Messenger environment.

However, Estée Lauder Lip Artist does everything you expect from a beauty consultant – suggests the shades based on your color type, asks questions about your lipstick preferences and the occasions on which you are planning to wear it, and makes surprise recommendations. Nudes, reds and purples, matte and shimmer – the Lip Artist bot has everything.

And, of course, you can try it on by uploading your photo or snapping a selfie with your device camera. You will immediately see the result and, should you like it, just click “Shop now” to purchase the selected lipstick.

makeup augmented reality

However, the Estée Lauder chatbot has not yet been integrated with a payment system, so for purchase, you are redirected to the Estée Lauder website where you can complete your shopping.

At the same time, messaging platforms, Facebook, in particular, are working on implementing payment engines right within the chat functionality, so we may expect the beauty chatbots to stay with you right until the checkout.


Browsing the beauty brands, we come across a number of other beauty recommendation bots which are still a couple of steps away from providing a complete augmented reality shopping experience.

They include advanced AI and face tracking feature managing to create a sensation of highly personalized shopping supplying recommendations and suggesting products based on your appearance or self-description. The integrated e-commerce software supports product catalogs, price-lists, and product image display within the chatbot content.

Madison Reed – get a personalized hair coloring recommendation

Madison Reed, a company, producing hair colors for at-home treatment, talks to you via its Facebook Messenger chatbot, Madi. With Madi, you can discuss your current hair condition and get a choice of products matching your requirements.

Madi can identify your hair color from an uploaded photo or a selfie and, after a couple of questions, Madi can suggest several hair coloring products which may suit you.

Madi’s chatting style with smileys and compliments is somewhat appealing, as the chatbot seems to create a relaxed and comfortable atmosphere similar to talking to a beauty consultant.

At the moment, Madi is not yet able to apply hair color to your face. However, that seems to be just around the corner.

For browsing the full collection and purchasing the hair products, you will follow the links leading to the Madison Reed website.

beauty augmented reality

Color Me Happy Cosmetics – your makeup assistant and more

The chatbot of Color Me Happy Cosmetics, a Korean brand, not only helps you purchase beauty products but also assists in performing skin tone analysis to find the best lipstick for you, tells lipstick-related jokes and trivia and generally keeps a laid-back chat to make you feel easy and relaxed. It has a rather interesting approach to determining the skin tone based on your jewelry preferences and the way your veins look under your skin.

The Color Me Happy Cosmetics chatbot may not be in the AR league yet.

However, it creates an incredibly personalized shopping experience with product recommendations based on the analysis of your answers to quite a number of questions.

You will get a choice of lipstick options matching your skin tone together with photos of models wearing the same shades and a link to visit the online store if you decide to buy.

Additionally, you can direct the virtual assistant by asking to suggest something different, for example, “less sheer” or “more tan”.

augmented reality beauty

In an attempt to win more customers by creating fantastic shopping experiences, e-commerce absorbs the most advanced technologies becoming an astounding fusion of industry innovations.

The integration with artificial intelligence and augmented reality makes online shopping platforms stand out from the competitors giving them additional options to reach out to their customers.

The world is getting busier with every passing day, and the customers, among other things, value the time that online shopping saves for them.

The solutions combining the convenience of shopping without leaving home and the exquisite feeling of personalized attendance seems to be just the thing for today’s hectic schedules.

The state-of-the-art technology allows tailor-made service’s in an environment that is most suitable for customers enabling personalized shopping at a time and place which the customers find convenient.

In case you need a professional consultation regarding augmented reality development, feel free to contact us anytime!

Learn also: How to Build an On-Demand Service for Tattoo and Beauty Salons Using Augmented Reality

Get our fresh tech stories delivered to your inbox!