iphone - Gaussian filter with OpenGL Shaders

Question

I am trying to learn Shaders to implement something in my iPhone app. So far I have understood easy examples like making a color image to gray scale, thresholding, etc. Most of the examples involve simple operations in where processing input image pixel I(x,y) results in a simple modification of the colors of the same pixel

But, how about Convolutions?. For example, the easiest example would the Gaussian filter,

in where output image pixel O(x,y) depends not only on I(x,y) but also on surrounding 8 pixels.

O(x,y) = (I(x,y)+ surrounding 8 pixels values)/9;

Normally, this cannot be done with one single image buffer or input pixels will change as the filter is performed. How can I do this with shaders? Also, should I handle the borders myself? or there is a built-it function or something that check invalid pixel access like I(-1,-1) ?

Thanks in advance

PS: I will be generous(read:give a lot of points) ;)

Answer 1

A highly optimized shader-based approach for performing a nine-hit Gaussian blur was presented by Daniel Rákos. His process uses the underlying interpolation provided by texture filtering in hardware to perform a nine-hit filter using only five texture reads per pass. This is also split into separate horizontal and vertical passes to further reduce the number of texture reads required.

I rolled an implementation of this, tuned for OpenGL ES and the iOS GPUs, into my image processing framework (under the GPUImageFastBlurFilter class). In my tests, it can perform a single blur pass of a 640x480 frame in 2.0 ms on an iPhone 4, which is pretty fast.

I used the following vertex shader:

 attribute vec4 position;
 attribute vec2 inputTextureCoordinate;

 uniform mediump float texelWidthOffset; 
 uniform mediump float texelHeightOffset; 

 varying mediump vec2 centerTextureCoordinate;
 varying mediump vec2 oneStepLeftTextureCoordinate;
 varying mediump vec2 twoStepsLeftTextureCoordinate;
 varying mediump vec2 oneStepRightTextureCoordinate;
 varying mediump vec2 twoStepsRightTextureCoordinate;

 void main()
 {
     gl_Position = position;

     vec2 firstOffset = vec2(1.3846153846 * texelWidthOffset, 1.3846153846 * texelHeightOffset);
     vec2 secondOffset = vec2(3.2307692308 * texelWidthOffset, 3.2307692308 * texelHeightOffset);

     centerTextureCoordinate = inputTextureCoordinate;
     oneStepLeftTextureCoordinate = inputTextureCoordinate - firstOffset;
     twoStepsLeftTextureCoordinate = inputTextureCoordinate - secondOffset;
     oneStepRightTextureCoordinate = inputTextureCoordinate + firstOffset;
     twoStepsRightTextureCoordinate = inputTextureCoordinate + secondOffset;
 }

and the following fragment shader:

 precision highp float;

 uniform sampler2D inputImageTexture;

 varying mediump vec2 centerTextureCoordinate;
 varying mediump vec2 oneStepLeftTextureCoordinate;
 varying mediump vec2 twoStepsLeftTextureCoordinate;
 varying mediump vec2 oneStepRightTextureCoordinate;
 varying mediump vec2 twoStepsRightTextureCoordinate;

// const float weight[3] = float[]( 0.2270270270, 0.3162162162, 0.0702702703 );

 void main()
 {
     lowp vec3 fragmentColor = texture2D(inputImageTexture, centerTextureCoordinate).rgb * 0.2270270270;
     fragmentColor += texture2D(inputImageTexture, oneStepLeftTextureCoordinate).rgb * 0.3162162162;
     fragmentColor += texture2D(inputImageTexture, oneStepRightTextureCoordinate).rgb * 0.3162162162;
     fragmentColor += texture2D(inputImageTexture, twoStepsLeftTextureCoordinate).rgb * 0.0702702703;
     fragmentColor += texture2D(inputImageTexture, twoStepsRightTextureCoordinate).rgb * 0.0702702703;

     gl_FragColor = vec4(fragmentColor, 1.0);
 }

to perform this. The two passes can be achieved by sending a 0 value for the texelWidthOffset (for the vertical pass), and then feeding that result into a run where you give a 0 value for the texelHeightOffset (for the horizontal pass).

I also have some more advanced examples of convolutions in the above-linked framework, including Sobel edge detection.