Category: apps

Web Augmented Reality (AR) Hands-On

Web AR is young, state-of-art technology. In this paper I will discuss entry to provide Augmented Reality (AR) experience to the users. Recently, the Web-based AR implementation method has also received focused attention from the W3C group, and the Web XR Editor’s draft was released in March 2018. This technology looks very promising unfortunately is not widely supported yet.

Options

Currently there are 4 major open-source libraries to be used to build AR experience. First one is supported by most of the current web browsers, but is quite fragile. The another two are more robust but are not widely supported yet.

AR.js

Don’t Worry It’s Less Than 10 Lines of HTML

This is title from aframe.io and you know, it is very rare that cheap decisions will make great impact. You are able to create AR app with 10 lines of code. 10 lines of this perverse html markup, that works, but to be honest – that’s all. 

From this point all that will follow is blood, sweat, and tears. This AR.js is total garbage that is not usable in standart project because it doesn’t even allow you to install and use it as npm module. You have to <script> it into you web app. More over even easy adjustments as is changing the marker are almost impossible because there is no documentation and what worked in previous version doesn’t work in current version. Here are few issues that document stat of this repo very well:

three.ar

Three.ar.js is library from google-ar team. This library helps to create AR experience with helper classes and utilities on top of the three.js 3D library, which interfaces with the WebVR Extension for AR exposed by WebARonARKit and WebARonARCore.

To play wioth three.ar aside from Chrome Canary, you’ll also need:

See three.ar.js API documentation for details.

aframe-xr

System & components by Mozilla team to build WebXR experiences with A-frame. You can check the examples here URL with all the examples. Again you need supported browser.

aframe-ar

This library add basic A-Frame support for browser-based AR, supporting the new three.ar.js library and WebARonARKit/Core browsers, as well as WebXR Viewer. By simply adding the ar component to your A-Frame scene declaration, aframe-ar will, when using a supported browser, take over the scene camera using information from ARKit / ARCore. You can read more here.

Proof of Concept

Let’s get our hands dirty. When you want to create an app that your friends and family can play, you have basically one option and that’s the AR.js library. This is what we were able to create in few hours. Just point your smartphone to the QR code to open the webpage then point it again to the marker.

Web browser-based augmented reality (AR)

Web browser-based augmented reality (AR)

You should go through similar process on any platform as is below.

AR delivery experience

AR delivery experience

 

Where To Get 3D Models

GlTF, OBJ and COLLADA are already supported on A-Frame and have good documentation available on the A-Frame docs, but if you can multiple options, I would go with gLTFA lot of great models can be downloaded from these two pages:

  • https://poly.google.com/search/koala
  • https://sketchfab.com/feed

To enable animations you have to include aframe-extras library.

Summary

After playing with WebAR for a while, it’s obvious that it’s not yet mature. However, even today, it is possible to build decent AR experience.

Notes

I have found also another libraries as is jeelizAR, THREEAR which seems to have more robust javascript api. Will give it a try.

References

  1. https://aframe.io/
  2. https://ieeexplore.ieee.org/document/8643424
  3. https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e
  4. https://developers.google.com/web/updates/2018/06/ar-for-the-web
  5. https://github.com/JamesMilnerUK/THREEAR
Learn More

Is It a Koala? Artificial Intelligence in Javascript and TensorFlow.js

AI in your Browser

Tensorflow.js brings machine learning and its possibilities to JavaScript. It is an open source library built to create, train, and run machine learning models in the browser and Node.js

Training and building complex models can take a considerable amount of resources and time. Some models require massive amounts of data to provide acceptable accuracy. And, if computationally intensive, may require hours or days of training to complete. All that taken into consideration, you may not find the browser to be the ideal environment for building such models.

A more appealing use case is importing and running existing models. Depending on the problem you are trying to solve, there might be a model already trained with a data set and for a specific purpose which you can use and import in your code.

Proof of Concept

For example, let’s say you want to build a web app to predict if an image is a picture of a koala. A popular image classification model is called MobileNet and is available as a pre-trained model with Tensorflow.js.

Using that, you can create the app in a few minutes with just a few lines of code.
I’m not joking – this is it:

For presentational purposes I have created a simple app in ReactJS that does exactly what I have described above – predicts if an image is a picture of a koala. 

You can find the whole source code here

As you can see, it is really easy to build an app like this in your browser. TensorFlow.js does everything for you, you just work with the data you get back.

Summary

Implementing Tensorflow models in your web app is a really easy task that requires no knowledge of neural networks or AI as a whole. You can implement it with just under 20 lines of code.

Learn More