Torus: wrapping the world around a donut

Try it: https://lab.amcc.io/tilt-torus

I'm pretty amazed at what is possible in the browser, on mobile expecially. With a bit of wrangling and checking for permission you have access to the tilt sensors, compass sensors and more. Of course on most phones we also have access to 2 cameras, combining all of these things with some WEBGL is a lot of fun.

I had an idea that wrapping the video feed from a camera around a torus would be interesting, but I didn't expect the amazing effect it produces. Combining this with tilt and compass sensors (or just dragging and orbiting the form) does something quite special, with a relatively simple process.

Getting everything to play was a little bit of effort, you must allow for a click action to enable tilt on an iOS mobile device (after iOS13), most other devices just work (if they have the sensors and wont break if they don't).

p5js makes everything pretty simple - its got good integration in a number of areas, allowing for using WEBGL with simple forms, it helps you easily make graphics with the video feed to use as a texture and also helps add/remove CSS classes for the UI. However I make buttons and help text in the html to help with accessibility, p5js can make buttons and text too, but its no way near as acessible doing it that way.

If you want to try yourself you can find the code on Github and the p5 Editor at the links below:

https://github.com/amcc/tilt-torus

https://editor.p5js.org/amcc/sketches/b9evuS3Qm

Exploring the code

In the links above you can explore the code. The easiest way to have a look is to go to the p5 editor on this link: https://editor.p5js.org/amcc/sketches/b9evuS3Qm. The editor is fantastic, you can easily tinker in an environment that allows for sensor access (as its secure). Feel free to duplicate this and play.

Getting access to the sensors

Android just lets you access the sensors right now, but since iOS13 you need to request permission on Apple devices, and this must be on a 'click' event (taps trigger a click!). I've simply added this to the body of the webpage. I end up setting various variables to help with the UX on an iOS device, but you actually get data as soon as you click.

The thing you need permission for is DeviceMotionEvent which you do with a DeviceMotionEvent.requestPermission(), and if you get it you can trigger other things - which as i mention above i'm doing. By the way if you want the orientation there's also a DeviceOrientationEvent and a ton of other similar things.

I find there is fantastic documentation for this on Mozilla's docs: https://developer.mozilla.org/en-US/

Once i've got this data i'm using p5js to animate a WEBGL torus, which is really easy with this library. I just do it like this:
rotateZ(radians(rotationZ));
rotateX(radians(rotationX));
rotateY(-radians(rotationY));

(i'm flipping the Y rotation as it feels better that way!)

Wrapping the video around the torus...

This wasn't so straightforward, video doesn't behave that well on an iPhone from standard tutorials and examples provided. Plus using a video feed as a texture doesn't work the way you'd think it should.

Getting the rear camera video feed

To get the video working i'm using p5js's createCapture function. You would normally pass it VIDEO, but this doesn't work so well on a mobile in my experience, plus it uses the wrong camera for my purposes. However you can pass it an object and include a lot of different variables, plus a callback function.

To get the rear camera you can determine the facingMode, again check out the Mozilla docs to see all the options as there are a few. You can even set frameRate for the video and other stuff here.

Using a camera feed as a texture

This was not obvious. There's plenty of examples of getting a video feed then displaying it on each frame as an image. the p5js website has many examples of this https://p5js.org/examples/dom-video-capture.html.

But this image can't be used as a texture in WEBGL mode, which is a shame. However I found a great tutorial by Daniel Shiffman showing how to use createGraphics as a texture: https://youtu.be/3tTZlTq4Cxs. So what I did was create a new renderer object with createGraphics like so:

vidGraphics = createGraphics(width, height)

Then I got my video feed (as a variable video) and did this in the draw function:

vidGraphics.image(video, 0, 0, width, height);

Now i can use that as a texture like so:

texture(vidGraphics);
torus(x, y, z);

Making it run smooth on a slow/mobile browser

Most of the tutorials on video on the p5js website capture the video then draw it frame by frame to an image in the draw loop. While this gives you loads of control and lots of image manipulation options it is slow. Using createCapture already creates a html <video> element, so why not use that. With a bit of CSS and some z-index stuff you can simply put the canvas ontop of the video and get the same effect, but like anything thats an html element, or CSS its way way faster.

The caveat here is that if you need to really line things up its harder to do. You've got to get your CSS perfect, and the variety of browsers, device rotations etc means that its tricky. If you want absolute position control and you're not going fullscreen like I am then perhaps don't worry. With this particular website fullscreen video on a high density display means your canvas is absolutely huge.

One thing I don't see that much on p5js examples is using a transparent canvas and utilising the page behind, but if you want to mix them up simply use clear() at the start of the draw loop rather than a background. I find this technique allows you to get away with pretty interesting things at fullscreen sizes.