Category Archives: Workshop

Browser based AR and VR

OK, so “WebAR and WebVR” may have been a snappier title but also technically incorrect. “WebVR” is an actual standard still under development and “WebAR” doesn’t exist as such (oh look, I spoke too soon! It looks like it may be a pending standard, after all).

The terms VR¹ and AR², along with MR³, RR⁴, ER⁵, XR⁶ and god knows what else can be used to mean a surprising variety of things (check Wikipedia for more info).
¹ Virtual Reality
² Augmented Reality
³ Mixed Reality
⁴ Real Reality (WTF?)
⁵ Extended Reality
⁶ X Reality (I don’t think they’re even trying anymore)

Milgram’s Continuum of Mixed Reality

Google’s version of Milgram’s Continuum

These are supposedly convenient labels for different things on a spectrum, but I think they largely serve to muddy the water and are little more than an exercise in marketing bullocks. To this end, I prefer to just use the terms AR and VR.

Definition of terms
I claim no authority, but I’ll be defining AR and VR as follows (hopefully to provide clarity for the length of this article only):
AR – adding digital content to the existing environment
VR – (near) total immersion in an artificial environment

A “quick” note about hardware
We’ll be browser based, so there’s no bulky, expensive or high-end equipment (although we might burn out your GPU or drain your battery). I’m using my Nexus 4 (ancient my modern standards). A tablet can be used for AR, but might be a bit awkward for VR 🙂 .

While researching this article, I used a SketchFab cardboard and a pair of Go4D glasses. Neither was ideal, the cardboard provides an easy touch button input, while the Go4D has adjustable lenses. Both were needed for a proper experience, so I was sadly disappointed.

Researching for this article is the first time I’ve used used VR for any length of time on a mobile, and it’s both literally headache inducing and arm tiring. I’ve found dedicated systems/headsets far more comfortable to use (although they do make my head hot).
I’m not sure what caused the headaches, possibly poor resolution or poor placement causing eye-strain, adjusting the inter-ocular distance seemed to ease the problem.
It’s evident that for prolonged use some kind of strap is required along with some kind of cushion for the nose.

APIs

Jerome Etiene’s AR.js
argon.js
Aruco

VR Demos

https://airtable.com/embed/shr2Lc7pmlJis02R4/tblZbV2S0W0T5DDth?viewControls=on

AR Demos

Hatsune Miku Dancing in Augmented Reality
WebAR Playground: AR in a Few Clicks
(more info)

Browser Based VR

Requirements
Phone
Headset
Orientation/Positioning/Tracking
Input method

Implementation

VR is based on old-fashioned stereoscopic viewing (and I do mean old-fashioned). The basic requirement is to provide the viewer with two slightly offset stereoscopic images giving the impression of depth. True “VR” is achieved by adding positional awareness — as the viewer moves the images change to reflect the viewer’s new point of view.

Stereoscopy is OLD!

Without positional tracking the VR experience is severely limited, only allowing the user to look around, resulting in an experience akin to 360° video. Teleporting is a practical alternative to moving around as this is already a widely used method of moving around in a virtual environment.

While the WebVR standard makes VR easier to implement, it’s not widely supported (at the time of writing only Edge, Firefox and Chrome for Android have limited support). Polyfills and various other hacks are used to achieve the necessary functionality. Ultimately, happily, all the required legwork has already been done by many brave, hearty souls, leaving us to simply find the API we like the most.

A number of headsets are available, most notably the cardboard. The cardboard become uncomfortable after a short time.
Recent iterations have moved away from the neodymium magnets, which was the most expensive part of the cardboard and could interfere with the magnetometer/compass functionality of the phone.

The “big three” Three.js, Babylon.js and PlayCanvas make it relatively easy to generate VR content

Polyfill
WebVR
Early browser VR

Browser Based AR

Requirements
Positioning (markers or visual positioning)
Camera
Image processing
Input

AR has significantly more requirements as images additionally interact with the local visual environment (to some extent).

Editors

ARKit – Apple only and can’t find actual demos only movies (Apple restricted?)

A-Frame

Code it yourself or use on-line editor

APIs and Code

VR

A-Frame
Three.js
Babylon.js
PlayCanvas

AR

A-Frame

1
2
3
4
<script src="vendor/aframe/build/aframe.min.js"></script>
 
<script src='../build/aframe-ar.js'></script>
<script>ARjs.Context.baseURL = '../../three.js/'</script>
1
2
3
4
5
6
7
8
9
10
11
12
<body style='margin : 0px; overflow: hidden;'>
  <a-scene embedded arjs='trackingMethod: best;'>
    <a-anchor hit-testing-enabled='true'>
      <a-box position='0 0.5 0' material='opacity: 0.5; side:double; color:red;'>
        <a-torus-knot radius='0.26' radius-tubular='0.05'>
          <a-animation attribute="rotation" to="360 0 0" dur="5000" easing='linear' repeat="indefinite"></a-animation>
        </a-torus-knot>
      </a-box>
    </a-anchor>
    <a-camera-static/>
    </a-scene>
</body>

Three.js

1
2
3
<script src='vendor/three.js/build/three.min.js'></script>
<script src="../build/ar.js"></script>
<script>THREEx.ArToolkitContext.baseURL = '../'</script>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
var renderer = new THREE.WebGLRenderer({
  antialias: true,
  alpha: true
});
 
renderer.setClearColor(new THREE.Color('lightgrey'), 0)
renderer.setSize( 640, 480 );
renderer.domElement.style.position = 'absolute'
renderer.domElement.style.top = '0px'
renderer.domElement.style.left = '0px'
document.body.appendChild( renderer.domElement );
 
var onRenderFcts= [];
 
var scene = new THREE.Scene();
 
var camera = new THREE.Camera();
scene.add(camera);
 
var arToolkitSource = new THREEx.ArToolkitSource({ sourceType : 'webcam' });
 
arToolkitSource.init(function onReady() {
  onResize()
});
 
window.addEventListener('resize', function() {
  onResize()
});
 
function onResize() {
  arToolkitSource.onResize()
    arToolkitSource.copySizeTo(renderer.domElement)
    if( arToolkitContext.arController !== null ) {
    arToolkitSource.copySizeTo(arToolkitContext.arController.canvas)
  }
}
 
var arToolkitContext = new THREEx.ArToolkitContext({
  cameraParametersUrl: THREEx.ArToolkitContext.baseURL + '../data/data/camera_para.dat',
  detectionMode: 'mono',
});
 
arToolkitContext.init(function onCompleted() {
    camera.projectionMatrix.copy( arToolkitContext.getProjectionMatrix() );
});
 
onRenderFcts.push(function() {
  if( arToolkitSource.ready === false )  return
 
  arToolkitContext.update( arToolkitSource.domElement )
      scene.visible = camera.visible
  });
 
  var markerControls = new THREEx.ArMarkerControls(arToolkitContext, camera, {
    type : 'pattern',
    patternUrl : THREEx.ArToolkitContext.baseURL + '../data/data/patt.hiro',
    changeMatrixMode: 'cameraTransformMatrix'
});
 
scene.visible = false
 
var geometry = new THREE.CubeGeometry(1,1,1);
var material = new THREE.MeshNormalMaterial({
  transparent : true,
  opacity: 0.5,
  side: THREE.DoubleSide
}); 
 
var mesh = new THREE.Mesh( geometry, material );
mesh.position.y = geometry.parameters.height/2
scene.add( mesh );
 
var geometry = new THREE.TorusKnotGeometry(0.3,0.1,64,16);
var material = new THREE.MeshNormalMaterial(); 
var mesh = new THREE.Mesh( geometry, material );
mesh.position.y = 0.5
scene.add( mesh );
onRenderFcts.push(function(delta) {
  mesh.rotation.x += Math.PI*delta
});
 
onRenderFcts.push(function() {
  renderer.render( scene, camera );
});
 
var lastTimeMsec= null
requestAnimationFrame(function animate(nowMsec) {
  requestAnimationFrame( animate );
  lastTimeMsec = lastTimeMsec || nowMsec-1000/60
  var deltaMsec = Math.min(200, nowMsec - lastTimeMsec)
  lastTimeMsec = nowMsec
  onRenderFcts.forEach(function(onRenderFct) {
    onRenderFct(deltaMsec/1000, nowMsec/1000)
  });
});

Babylon.js

1
2
3
4
5
6
7
<script src="vendor/babylon.max.js"></script> 
<script src="../build/babylon-ar.js"></script>
<script>ARjs.Context.baseURL = '../../three.js/'</script> 
 
<div id="canvasZone">
  <canvas id="renderCanvas"></canvas>
</div>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
var canvas = document.getElementById("renderCanvas");
var engine = new BABYLON.Engine(canvas, true);
 
function createScene(){
  var scene = new BABYLON.Scene(engine);
  scene.clearColor = new BABYLON.Color4(0, 0, 0, 0);
 
  var camera = new BABYLON.ArcRotateCamera("Camera", 1.0, 1.0, 12, BABYLON.Vector3.Zero(), scene);
 
  var light = new BABYLON.HemisphericLight("hemi", new BABYLON.Vector3(0, 1, 0), scene);
 
  light.groundColor = new BABYLON.Color3(0.5, 0, 0.5);
 
  var box = BABYLON.Mesh.CreateBox("mesh", 1, scene);
  box.position.y = 0.5;
  box.showBoundingBox = true;
 
  var material = new BABYLON.StandardMaterial("std", scene);
  material.diffuseColor = new BABYLON.Color3(0.5, 0, 0.5);
 
  box.material = material;
 
  return scene;
}
 
var scene = createScene();
scene.clearColor = new BABYLON.Color4(0, 0, 0, 0);
 
ARjs.Babylon.init(engine, scene, scene.cameras[0])
 
engine.runRenderLoop(function () {
  scene.render();
});
 
window.addEventListener("resize", function () {
  engine.resize();
});

Workshop 16 – Light and shadow made easy

2nd June 2015

Workshop 16 – Light and shadow made easy… with Three.js

Work files
All work files as a zip

Worksheets
Worksheets in pdf format

Basic scene
A basic scene rendered as wireframe

Coloured material
Each object gas its own basic coloured material. Still no lighting.

Coloured material with per vertex shading

Coloured material with per vertex shading and added shadow

Coloured material with per pixel shading and added shadow

Point light

Spotlight

Workshop 15 – Light and shade

21st April 2015

Worksheets
Worksheets in pdf format

Work files and worksheet
All work files and worksheet pdf as zip

Basic scene
No lighting applied, each object within the scene is displayed with its colour.
Each object looks like a silhouette with no visible detail.

00lightnone
Scene with ambient light (and emissive material)
Ambient lighting is now applied to the scene.
Ambient lighting is applied evenly to an object. So, each object still looks like a silhouette with no visible detail. However the lightness of the scene can be controlled.
But, note that the “light” is not affected by the ambient light as it only has an emissive material.

01lightambient
Directional light
A directional light is added to the scene.

02lightdirectional
Directional light – fragment shader
Again directional light is added to the scene.
However, the lighting calculations are done in the fragment shader, making the lighting more accurate.

03lightdirectionalfrag
Directional light with materials (frag)
Support for all materials (ambient, emissive, diffuse and specular) is added to the scene.
Setting these values appropriately simulates different materials such as gold or copper.

04lightdirectionalmaterialfrag
Scene with point light and materials (frag)
A point light is added to the scene.
A point light has a position as well as a direction so as it approaches an object it has a greater effect.

05lightpointmaterial

Workshop 11 – WebGL Models

WebGL Models
Thursday, 30th October – Skills Matter, London

How to make (or find), load and display them. And a bit of debugging, too.

Prerequisites
The workshop will be about 50:50 presentation and hands on.
• A laptop with a browser that supports WebGL (http://get.webgl.org/) and your favourite editor.
• Some knowledge of HTML and programming preferably Javascript (or similar).
• Some features will require a server, although there are workarounds. (Brackets.io and Visual Studio provide one, or you can use node.js or whatever you’re comfortable with.)

Register
If you’re planning on attending, please RSVP on the MeetUp page (http://www.meetup.com/WebGL-Workshop-London/events/210678942/) and register on the Skills Matter site (https://www.skillsmatter.com/meetups/6576-webgl-models).

Workshop 9 – Back to Basics: Part 2

Back to Basics: Part 2 (WebGL, Three.js and Babylon.js)
Our ninth Workshop will take place on Thursday, 28th August at SkillsMatter, 116-120 Goswell Road, London
This month we finish the rotating dice in raw WebGL, then do the same in THREE.js and Babylon.js for comparison.

Prerequisites
• A laptop with a browser that supports WebGL (http://get.webgl.org/) and an editor that can handle HTML and JavaScript.
• Some knowledge of HTML and programming, preferably JavaScript (or similar).
• Some features may require a server (e.g. image loading), although there are workarounds. (Brackets.io and Visual Studio provide one, or you can use node.js or whatever you’re comfortable with.)

Register
If you’re planning on attending please RSVP on our MeetUp site and register on the SkillsMatter site (SkillsMatter.com).

Workshops

Workshop 01 - Introduction
Workshop 02 - Programming dice
Workshop 03 - Light and Shadow
Workshop 04 - A look at three.js
Workshop 05 - A bit more Three.js
Workshop 06 - Writing Shaders and Effects for Three.js
Workshop 07 – Data Visualisation
Workshop 08 - Back To Basics
Workshop 09 - Back to Basics: Part 2 (WebGL, Three.js and Babylon.js)
Workshop 10 - Fonts and models
Workshop 11 - Models
Workshop 11 - Models
Workshop 12 - Christmas Comes to Babylon Three
Workshop 14 - Introduction to WebGL
Workshop 15 - Light and shade
Workshop 16 - Light and shadow made easy
Workshop 18 – Practical Learnings of WebGL for Make Benefit Glorious Internet of Web
Workshop 22 - GLSL
Workshop 23 - Workflow, Clara.io & Babylon.js
Workshop 24 - WebGL and Animation
Workshop 25 - Into The Second Dimension
Workshop 26 - Babylon, Games 'n' Stuff
Workshop 28 - Intro to WebGL 2.0
Workshop 31 - AR, VR & WebGL Academy
Workshop 32 - WebGL Academy & Pixel Spirit