However, unlike a real camera, a virtual camera in the 3D world doesn’t use a compound set of lenses and film to capture the image. It relies on a set of equations to do the work. For example, to change the zoom of our virtual camera is a simple process of changing some variables in the perspective calculation equation.

How does this all tie in to photography? Well for a real camera, if you place a small opening in front of a camera lens, it will block off some light entering the lens, and we call this opening the “the aperture”.

Bokeh, as you probably know, refers to the out-of-focus highlights in a photograph as a result of shallow depth-of-field composition. With me so far? Turns out, the shape of the aperture dictates the shape of the bokeh. A typical bokeh of a wide open lens looks like little round disks because the lens aperture is round:

A “stopped-down” lens might produce bokeh that looks like hexagons(because the aperture is made up of distinct blades).

Here comes the coolest part, if you placed a custom shaped opening in front of the lens,

the bokeh will take the shape of that opening.

Back to computer graphics. I was curious to see if the Cycles is as robust as it is advertised. After all, a ‘physical-based’ rendering engine should be able to simulate light interactions just as they behave in the real world. So when I set up a simple scene and placed a heart shaped opening in front of my virtual camera, I expected to see the out-of-focus area being rendered into little heart shapes.

Neat eh?

What’s even more amazing is the bokeh shape is distorted near the frame edge, as if suffering from some kind of spherical abberation. Exactly how my 50mm F1.4 behaves when used wide-open.

This little experienment was nothing more than a test to see what can be accomplished with today’s cutting edge rendering technology, and bring together my two passions – computer graphics and photography.

This article was written by me, and first published at http://www.diyphotography.net/can-you-make-computerized-shaped-bokeh

Paul Debevec published his seminal work on image based modelling in 1997, just over 14 years ago. By taking multiple images of an object from different angles, he was able to reconstruct the 3D geometry of the bell tower at  UC Berkeley.  Programmatically generating 3D models from 2D images!  It was so revolutionary at the time that the sfx team behind the Matrix movie worked with Paul to use this exact technique in the film.

Pretty impressive eh?  Now imagine going this for an entire campus, or an entire city, or the entire world.  Without human intervention. At 10cm resolution.

This is what C3 Technology and Nokia came up with, and it runs inside a web browser.

A better writer might be able to describe to you the magnitude of what you are witnessing, but I am truly at a loss at conveying the significance of this to the future of mapping.  Click on the image to see the demo for yourself.  You’ll need to install a plugin, but it’s well worth the trouble.  Someone less technically might not fully appreciate the complexity of what’s going on behind the scene, and as someone who only knows enough to be dangerous, I am still not convinced technology like this even exists today.

Wow.

You might notice that some of the shader code looks strange, that’s because I am trying to keep things simple by forcing myself to use the latest version of OpenGL Shading Language available on my computer hardware, so for a Geforce 8, that’s OpenGL 3.3 | GLSL 3.3.

#version 120
#vertex shader

in vec4 vertex;
uniform float timer;

void main() {
vec4 v = vertex;
v.z = sin(timer+v.y+v.x)*0.5+v.z;
v.x = sin(timer*10.0+v.y+v.y)*0.2+v.x;
gl_Position = gl_ModelViewProjectionMatrix * v;
}

#==========================

#version 330
#geometry shader

layout(triangles) in;
layout(triangle_strip) out;
out vec2 texCoord;

#define radius 0.01
#define layer 1

void main() {
//for(int i = 0; i < gl_in.length(); i++) { // avoid duplicate draw

The first section, the vertex shader, is executed for each vertex of the geometry, it is here that I apply the deformation to form the base shape of the particle systems.  The resulting coordinates is projected from modelspace onto viewspace.  This is important, because it saves the next step from becoming overly complex!  The geometry that follows is able to do per-primitive operations on the vertex, here I take each vertex from the previous stage(which is already projected into viewspace), and ‘expand’ that into a billboard by creating a quad for each point.  The quad naturally faces the camera, since everything is already in viewspace.  The final stage of the pipeline is the fragment shader, which applies a texture onto the quad.

One blatant oversight of this approach is that it does not take into account the viewport aspect ratio, thus the generated particles will only look square on a 1:1 canvas.  None unity aspect-ratio will stretch the billboard.

As you can see, the code is very short, but being GLSL, the program is executed on the graphic hardware, which is highly tuned for this type of computation.  Freeing the CPU to do other things.

This is a test with 1million fully textured particles running in Blender’s using geometry shaders and vertex shaders to do most of the heavy-lifting.

My lucky day!  Naturally I tried to do some heavy-weight visual effects on this awesome machine.  Sadly, Blender’s fluid engine is not really reliable at this setting, the OpenMP version failed time after time (using a x64 verion on Win7), and the single-threaded version simply takes an hour to return a single frame.  Since I only had access to this computer for a day, nothing was made in the end.

The Blender SFX engine (Fluid, Smoke, Softbody) is still lagging the industry by quite a bit in terms of speed and functionality, especially considering all the recent research into GPU-powered physics calculation.  It would be interesting to see how Project Mango (which is the Blender Institute’s next production, allegedly SFX-heavy) addresses these problems.

Summary:

• Last updated December 2010 for iOS4.1, Click here for the newest release for iOS 4.2.
• I added ‘Super’ high quality mode in rev4, which uses the camera sensor in still image mode  to capture video, so it will only record at 15fps, but the quality is amazing as evident in the above picture.
• In ‘Super’ high quality mode (15fps), the camera sometimes makes a ‘clicking’ noise when focusing and then the video recording lags for half a second.  It helps if you close all other background apps.
• You are recording at 1080×800, which is ~4:3 aspect ratio, just like the original video.
• The video is a lot higher in resolution and bitrate than the original, thus the files are bigger, and it will take more power to encode.  Keep an eye on the disk space and battery life!  Also, disable background apps might improve performance.
• if you email the video or send it to youtube using iOS, it will reencode it to a lower quality.  Use Pixelpipe from the appstore to upload in original quality, or download it to your computer to view the video in full resolution.
• This hack is for iOS 4.1, I have not tested this with older versions of iOS.  But supposedly it works on 4.2 as well.
• Someone made a cydia App called 3GS HD Enabler under the modmyi repo for the less technical inclined.

One quick look at the ARM Cortex processor that the 3GS uses [PDF Spec], it’s apparent that the chip is capable of handling 720p video encoding.  So all we need to do is by-pass the artificial limitation Apple has imposed on the hardware.  Here is a way to get the phone to record video (and playback on the iPhone) at 1080×800 @ 30fps at up to 15Mbps.  (up from the original 640×480 @ 3Mbps)  (I am using the odd resolution of 1080×800 in order to keep the video at the right aspect ratio)

The quick method:
Jailbreak your iDevice, install Cydia, and search for the app ’3GS HD Enabler’.  Enjoy.

The manual method:
Jailbreak your iDevice and install SSH
Download the pre-edited file HDVideo3GS.zip.
Decide if you want HD or SuperHD, see comparison of different video mode.
Copy the 3 .plist files to /System/Library/PrivateFrameworks/Celestial.framework/N88/ (Backup your original!)
Respring your iPhone 3GS and enjoy HD video

Details, for the tinkerers:

• Open /System/Library/PrivateFrameworks/Celestial.framework/N88/AVCapture.plist with a Plist editor
• Under the AVCaptureMode_AudioVideoRecording key, set the width and height for the capture key to 1080×800, respectively.  This value controls the dimension of the video it is encoded at.
• Set the width and height for the preview key to 440×320.  This value controls the size of the on-screen preview.  It does not have any effect on the final video.
• Set the width and height for the sensor key to 1920×1080.  This value controls the active region of the sensor, so basically, we want to use the entire sensor.  This source will be automatically downscaled to the proper resolution at encoding time.  Setting it to the full size (2048×1536 seems to turn on a special ‘photo mode’, which is slower than video mode, but the quality is much better)
• Increase the encoding average bitrate to 20Mbps (20000000bps), this records a much larger video file (~ 2MB for every second of video)
• Increase the encoding bitrate cap to 40Mbps.  Also, lower the minimum quantizer from 19 to 15 or something, just for the heck of it.
• Save and close this file.
• To make the phone playback the recorded HD video. Simply edit CameraRollValidator.plist and MediaValidator.plist, they can be found in the same place as AVCapture.plist. (If you don’t do this, the file can only be played back on your computer)
• Replace all instances of 768 with 1920
• And replace  <key>MaxLevel</key><integer>30</integer> with  <key>MaxLevel</key><integer>31</integer>
• Save these two files and respring your iDevice

Results:

Isn’t Relief Mapping great?

Apart from all games that will inevitably take advantage of this technique, another application I can think of that will reap huge benefit from this is the loading and displaying of large-scale terrain visualization at virtually no CPU overhead.  A 4096×4096 DEM texture can be easily displayed at 60fps.  Whereas if the same data were to be converted to real geometry, most computers will die trying to render those 33million triangles.

The effect is achieved by a rather complex fragment shader (pixel shader) that effectively does ray-casting on each pixel to determine the location and visibility of each pixel.  The shader requires at least 2 textures.  A diffuse color texture; and a normal-height texture that contains the surface normal stored in the RGB channel and the height value stored in the Alpha channel.  Supplied with a miminum of these two textures, the shader is able to reconstruct the 3D surface like you saw.  (I also used a specular texture to control the specularity of the lakes, but this is optional)

Download the demo (4MB)

Lesson 1: Know the limit of your camera, and try to work around it.

Small cameras are known for their low dynamic range, so to get the most out of this sunset, I took two images at different exposure and merged them together with Pro HDR, right on the phone.  This increased the dynamic range dramatically, making it equal to that of a dSLR.  Then a bit of contrast and saturation boost completed the look.

Lesson 2: Post Processing

This picture came out very bland at first, but a few tweaks in Lightroom 3made it marginally passable.  There are enough apps in the Apple Appstore to edit your photo into oblivion and back.  Some of my favorite apps are Pro HDR, TiltShiftGen and Best Camera. Of course, for the most control, you’d have to use a desktop-based tool like Lightroom or Aperture.

Lesson 3: Color

What the tiny camera lacks in light-gathering ability, it makes up by upping the vibrancy and contrast.  iPhone photos are usually a lot more vibrant and contrasty than what you get out of a high-end dSLR.  Use this to your advantage to capture some eye popping pictures.

Lesson 4: Bokeh

You CAN throw the background out of focus even on a tiny sensor like the iPhone camera, but only if you are shooting in macro.  Just be patient with the focus, it can take a while to get the razor sharp image that you wanted.

That’s all for now, I am still discovering the capability of the iPhone camera.  Let me know if you have any tips regarding digital photography.

Core 2 Duo SU7300 Processor:
It has a 1.3Ghz (up to 1.7Ghz with TurboBoost technology), ultra-low-voltage processor manufactured on 45nm technology.  Which really just means the processor can do a LOT of work while barely sipping on your precious battery.  Honestly, it’s unbelievable how fast this processor is while using less than 10W of power.  [Compare with a 13 MacBook Pro: slightly slower, but uses 1/3rd the power)

Nvidia G210M 512MB and Intel 4500 Graphics:
This .93 inch thin laptop has TWO graphics cards.  The Nvidia G210 is probably the fastest graphics card available on a 13 inch laptop.  It's build on 40nm technology, and uses a max of 14W of power, which is still impressive for a graphics card of this caliber.  The Intel is slower, but uses even less power.  You can toggle between the 2 to trade off performance for battery life. [Compare with a 13" MacBook Pro:  The Asus is twice as fast, while using the same amount of juice]

4GB DDR3 RAM:
Also, the laptop can support a max of 8GB of memory, you know… in case you need that much. [Same as Macbooks.]

13inch LED-lit Screen:
LED also means it’s uses less power than regular backlights. [same as Macbooks]

12 hour battery life:
…or so Asus claims, you can probably expect 10 hours of real world usage, and maybe 4-5 hours of gaming/heavy 3D work. [much longer than the MacBook Pros]

Design:
Not quite on par with Apple’s drool inducing one-piece aluminum finish, but it’s one of the nicer laptops I’ve seen (once you get rid of all the stickers).  The design is obviously Macbook inspired.  And it’s really light and thin.

Price:
$800 USD.  I know you can get an even faster 15″ laptop for $900, but the battery life and portability also suffers as you up the screen size.  13″ with this amount of computing power is perfect.  It also comes with all the standard bells and whistles: webcam, bluetooth, 802.11n, and a half terabyte harddrive.

The bottom line:
if you are looking for a portable powerhouse, take a closer look at this laptop.  If you want raw power and don’t care about battery life or portability, skip this.

Update: I’ve had this laptop for almost a year now and am very happy about it.  It’s light enough to be carried everywhere, and with 8 hours of up time on the integrated video card (can be stretched to 10 if you are stingy on screen brighness or wifi), I don’t even need to carry the AC adapter unless I plan an overnight trip.  My only complain is that the screen has an absolutely horrific viewing angle, color shifts are common when viewed off-axis; and black is a muddy gray at best.  But once you plug this thing into an IPS display via the build in VGA or HDMI cable, the Nvidia G210M chip is beefy enough to play StarCraft II at medium settings.

1. Get Blender 2.5 Alpha, also go for the 64bit version if you have 3GB or more of RAM.  2.5 is simply a lot faster and refined than Blender 2.49.  The tools and interface is also much cleaner and more intuitive.  Also, a lot of work has gone into optimizing the sculpting feature in Blender 2.5.

2. Start with a base mesh, then apply the multires modifier.  Do not use a default cube and rely on the multires modifier to do *all* the sub-division.  It’s always better to start with a base mesh with a few thousand polygons, and use multires with a sub-division level of 2-5.

3. Turn off “Double Sided” in the Object Data panel.  This will significantly speed up the redraw.

4. Delete UV texture data and Vertex Color data *might* help speed things up, but I don’t really know for sure.

5. Turn on VBO in the Preference menu.  This will further speed up drawing speed.  (Thanks Gustav!) Okay apparently it doesn’t according to another commenter, since sculpt mode already uses VBO by default.