Tag Archives: technical

Shaped Bokeh in Cycles

Posted on by
3

It all started with the release of Cycles – a new GPU based path-tracing rendering engine for Blender. For people unfamiliar with the concept, a rendering engine is a software that turns a computer generated 3D scene into a final picture. A rendering engine has to calculates lights, shadows, surface materials and all the intricate details that make up a believable virtual world. Most rendering engines use all kind of tricks and shortcuts to simulate surfaces and lightings, whereas Cycles rendering is purely based on the physics of light, which means it is very easy to get photo-realistic renderings like this:

However, unlike a real camera, a virtual camera in the 3D world doesn’t use a compound set of lenses and film to capture the image. It relies on a set of equations to do the work. For example, to change the zoom of our virtual camera is a simple process of changing some variables in the perspective calculation equation.

How does this all tie in to photography? Well for a real camera, if you place a small opening in front of a camera lens, it will block off some light entering the lens, and we call this opening the “the aperture”.

Bokeh, as you probably know, refers to the out-of-focus highlights in a photograph as a result of shallow depth-of-field composition. With me so far? Turns out, the shape of the aperture dictates the shape of the bokeh. A typical bokeh of a wide open lens looks like little round disks because the lens aperture is round:

A “stopped-down” lens might produce bokeh that looks like hexagons(because the aperture is made up of distinct blades).

Here comes the coolest part, if you placed a custom shaped opening in front of the lens,

the bokeh will take the shape of that opening.

Back to computer graphics. I was curious to see if the Cycles is as robust as it is advertised. After all, a ‘physical-based’ rendering engine should be able to simulate light interactions just as they behave in the real world. So when I set up a simple scene and placed a heart shaped opening in front of my virtual camera, I expected to see the out-of-focus area being rendered into little heart shapes.

Neat eh?

What’s even more amazing is the bokeh shape is distorted near the frame edge, as if suffering from some kind of spherical abberation. Exactly how my 50mm F1.4 behaves when used wide-open.

This little experienment was nothing more than a test to see what can be accomplished with today’s cutting edge rendering technology, and bring together my two passions – computer graphics and photography.

This article was written by me, and first published at http://www.diyphotography.net/can-you-make-computerized-shaped-bokeh

Jaw-dropping online 3D Map

Posted on by
4

“Any sufficiently advanced technology is indistinguishable from magic.”

 

Paul Debevec published his seminal work on image based modelling in 1997, just over 14 years ago. By taking multiple images of an object from different angles, he was able to reconstruct the 3D geometry of the bell tower at  UC Berkeley.  Programmatically generating 3D models from 2D images!  It was so revolutionary at the time that the sfx team behind the Matrix movie worked with Paul to use this exact technique in the film.

Pretty impressive eh?  Now imagine going this for an entire campus, or an entire city, or the entire world.  Without human intervention. At 10cm resolution.

This is what C3 Technology and Nokia came up with, and it runs inside a web browser.

A better writer might be able to describe to you the magnitude of what you are witnessing, but I am truly at a loss at conveying the significance of this to the future of mapping.  Click on the image to see the demo for yourself.  You’ll need to install a plugin, but it’s well worth the trouble.  Someone less technically might not fully appreciate the complexity of what’s going on behind the scene, and as someone who only knows enough to be dangerous, I am still not convinced technology like this even exists today.

Wow.

Some more shader experiments

Posted on by
Reply

After seeing Ian’s latest update on his awesome particles system, I am shifting the focus of my own project a bit, moving away from the goal of making an entire particles engine in the shader to something more attainable: use GLSL shaders for what they are designed for: making visual effects.  Below is attempt at making some abstract art using pure code.  The base shape is a simple plane.

You might notice that some of the shader code looks strange, that’s because I am trying to keep things simple by forcing myself to use the latest version of OpenGL Shading Language available on my computer hardware, so for a Geforce 8, that’s OpenGL 3.3 | GLSL 3.3.

#version 120
#vertex shader

in vec4 vertex;
uniform float timer;

void main() {
vec4 v = vertex;
v.z = sin(timer+v.y+v.x)*0.5+v.z;
v.x = sin(timer*10.0+v.y+v.y)*0.2+v.x;
gl_Position = gl_ModelViewProjectionMatrix * v;
}

#==========================

#version 330
#geometry shader

layout(triangles) in;
layout(triangle_strip) out;
out vec2 texCoord;

#define radius 0.01
#define layer 1

void main() {
//for(int i = 0; i < gl_in.length(); i++) { // avoid duplicate draw

for (int j=0; j<layer; j++){

vec4 p = gl_in[0].gl_Position;

texCoord = vec2(1.0,1.0);
gl_Position = vec4(p.r+radius, p.g+radius+j*0.05, p.b, p.a);
EmitVertex();

texCoord = vec2(0.0,1.0);
gl_Position = vec4(p.r-radius, p.g+radius+j*0.05, p.b, p.a);
EmitVertex();

texCoord = vec2(1.0,0.0);
gl_Position = vec4(p.r+radius, p.g-radius+j*0.05, p.b, p.a);
EmitVertex();

texCoord = vec2(0.0,0.0);
gl_Position = vec4(p.r-radius, p.g-radius+j*0.05, p.b, p.a);
EmitVertex();

EndPrimitive();
}
//}
}

#========================================

#version 330
#fragment shader
in vec2 texCoord;
out vec4 outColor;

uniform sampler2D col;
uniform float emit, alpha;

void main(void) {

// load color texture
vec4 color;
color = texture2D(col, texCoord);

// apply material panel values
color.rgb *= emit;
color.a *= alpha;

outColor = color;
}

 

The first section, the vertex shader, is executed for each vertex of the geometry, it is here that I apply the deformation to form the base shape of the particle systems.  The resulting coordinates is projected from modelspace onto viewspace.  This is important, because it saves the next step from becoming overly complex!  The geometry that follows is able to do per-primitive operations on the vertex, here I take each vertex from the previous stage(which is already projected into viewspace), and ‘expand’ that into a billboard by creating a quad for each point.  The quad naturally faces the camera, since everything is already in viewspace.  The final stage of the pipeline is the fragment shader, which applies a texture onto the quad.

One blatant oversight of this approach is that it does not take into account the viewport aspect ratio, thus the generated particles will only look square on a 1:1 canvas.  None unity aspect-ratio will stretch the billboard.

As you can see, the code is very short, but being GLSL, the program is executed on the graphic hardware, which is highly tuned for this type of computation.  Freeing the CPU to do other things.

24GB of RAM

Posted on by
Reply

My lucky day!  Naturally I tried to do some heavy-weight visual effects on this awesome machine.  Sadly, Blender’s fluid engine is not really reliable at this setting, the OpenMP version failed time after time (using a x64 verion on Win7), and the single-threaded version simply takes an hour to return a single frame.  Since I only had access to this computer for a day, nothing was made in the end.

The Blender SFX engine (Fluid, Smoke, Softbody) is still lagging the industry by quite a bit in terms of speed and functionality, especially considering all the recent research into GPU-powered physics calculation.  It would be interesting to see how Project Mango (which is the Blender Institute’s next production, allegedly SFX-heavy) addresses these problems.

Enable HD video recording on iPhone 3GS

Posted on by
266

Summary:

  • Last updated December 2010 for iOS4.1, Click here for the newest release for iOS 4.2.
  • I added ‘Super’ high quality mode in rev4, which uses the camera sensor in still image mode  to capture video, so it will only record at 15fps, but the quality is amazing as evident in the above picture.
  • In ‘Super’ high quality mode (15fps), the camera sometimes makes a ‘clicking’ noise when focusing and then the video recording lags for half a second.  It helps if you close all other background apps.
  • You are recording at 1080×800, which is ~4:3 aspect ratio, just like the original video.
  • The video is a lot higher in resolution and bitrate than the original, thus the files are bigger, and it will take more power to encode.  Keep an eye on the disk space and battery life!  Also, disable background apps might improve performance.
  • if you email the video or send it to youtube using iOS, it will reencode it to a lower quality.  Use Pixelpipe from the appstore to upload in original quality, or download it to your computer to view the video in full resolution.
  • This hack is for iOS 4.1, I have not tested this with older versions of iOS.  But supposedly it works on 4.2 as well.
  • Someone made a cydia App called 3GS HD Enabler under the modmyi repo for the less technical inclined.

One quick look at the ARM Cortex processor that the 3GS uses [PDF Spec], it’s apparent that the chip is capable of handling 720p video encoding.  So all we need to do is by-pass the artificial limitation Apple has imposed on the hardware.  Here is a way to get the phone to record video (and playback on the iPhone) at 1080×800 @ 30fps at up to 15Mbps.  (up from the original 640×480 @ 3Mbps)  (I am using the odd resolution of 1080×800 in order to keep the video at the right aspect ratio)

The quick method:
Jailbreak your iDevice, install Cydia, and search for the app ’3GS HD Enabler’.  Enjoy.

The manual method:
Jailbreak your iDevice and install SSH
Download the pre-edited file HDVideo3GS.zip.
Decide if you want HD or SuperHD, see comparison of different video mode.
Copy the 3 .plist files to /System/Library/PrivateFrameworks/Celestial.framework/N88/ (Backup your original!)
Respring your iPhone 3GS and enjoy HD video

Details, for the tinkerers:

  • Open /System/Library/PrivateFrameworks/Celestial.framework/N88/AVCapture.plist with a Plist editor
  • Under the AVCaptureMode_AudioVideoRecording key, set the width and height for the capture key to 1080×800, respectively.  This value controls the dimension of the video it is encoded at.
  • Set the width and height for the preview key to 440×320.  This value controls the size of the on-screen preview.  It does not have any effect on the final video.
  • Set the width and height for the sensor key to 1920×1080.  This value controls the active region of the sensor, so basically, we want to use the entire sensor.  This source will be automatically downscaled to the proper resolution at encoding time.  Setting it to the full size (2048×1536 seems to turn on a special ‘photo mode’, which is slower than video mode, but the quality is much better)
  • Increase the encoding average bitrate to 20Mbps (20000000bps), this records a much larger video file (~ 2MB for every second of video)
  • Increase the encoding bitrate cap to 40Mbps.  Also, lower the minimum quantizer from 19 to 15 or something, just for the heck of it.
  • Save and close this file.
  • To make the phone playback the recorded HD video. Simply edit CameraRollValidator.plist and MediaValidator.plist, they can be found in the same place as AVCapture.plist. (If you don’t do this, the file can only be played back on your computer)
  • Replace all instances of 768 with 1920
  • And replace  <key>MaxLevel</key><integer>30</integer> with  <key>MaxLevel</key><integer>31</integer>
  • Save these two files and respring your iDevice

Results: