iphone - How do I perform a fast pixellation filter on an image?

Question

I have a little problem with my pixellation image processing algorithm.

I load the image from the beginning into an array of type unsigned char* After that, when needed, I modify this data and have to update the image. This updating takes too long. This is how I am doing it:

CGDataProviderRef dataProvider = CGProviderCrateWithData(.....);
CGImageRef cgImage = CGImageCreate(....);
[imageView setImage:[UIImage imageWithCGImage:cgImage]]];

Everything is working but it's very slow to process a large image. I tried running this on a background thread, but that didn't help.

So basically, this takes too long. Does anyone have any idea how to improve it?

Answer 1

As others have suggested, you'll want to offload this work from the CPU to the GPU in order to have any kind of decent processing performance on these mobile devices.

To that end, I've created an open source framework for iOS called GPUImage that makes it relatively simple to do this kind of accelerated image processing. It does require OpenGL ES 2.0 support, but every iOS device sold for the last couple of years has this (stats show something like 97% of all iOS devices in the field do).

As part of that framework, one of the initial filters I've bundled is a pixellation one. The SimpleVideoFilter sample application shows how to use this, with a slider that controls the pixel width in the processed image:

This filter is the result of a fragment shader with the following GLSL code:

 varying highp vec2 textureCoordinate;
 uniform sampler2D inputImageTexture;
 uniform highp fractionalWidthOfPixel;

 void main()
 {
    highp vec2 sampleDivisor = vec2(fractionalWidthOfPixel);

    highp vec2 samplePos = textureCoordinate - mod(textureCoordinate, sampleDivisor);
    gl_FragColor = texture2D(inputImageTexture, samplePos );
 }

In my benchmarks, GPU-based filters like this perform 6-24X faster than equivalent CPU-bound processing routines for images and video on iOS. The above-linked framework should be reasonably easy to incorporate in an application, and the source code is freely available for you to customize however you see fit.