G’MIC 1.7.1: When the flowers are blooming, image filters abound!

Disclaimer: This article is a duplicate of this post, originally published on the Pixls.us website, by the same authors.

Then we shall all burn together by Philipp Haegi.

 A new version 1.7.1Spring 2016” of G’MIC (GREYC’s Magic for Image Computing),
the open-source framework for image processing, has been released recently (26 April 2016). This is a great opportunity to summarize some of the latest advances and features over the last 5 months.

G’MIC: A brief overview

G’MIC is an open-source project started in August 2008. It has been developed in the IMAGE team of the GREYC laboratory from the CNRS (one of the major French public research institutes). This team is made up of researchers and teachers specializing in the algorithms and mathematics of image processing. G’MIC is released under the free software licence CeCILL (GPL-compatible) for various platforms (Linux, Mac and Windows). It provides a set of various user interfaces for the manipulation of generic image data, that is images or image sequences of multispectral data being 2D or 3D, and with high-bit precision (up to 32bits floats per channel). Of course, it manages “classical” color images as well.


Logo and (new) mascot of the G’MIC project, the open-source framework for image processing.

Note that the project just got a redesign of its mascot Gmicky, drawn by David Revoy, a French illustrator well-known to free graphics lovers for being responsible for the great libre webcomics Pepper&CarottG’MIC is probably best known for it’s GIMP plug-in, first released in 2009. Today, this popular GIMP extension proposes more than 460 customizable filters and effects to apply on your images.


Overview of the G’MIC plug-in for GIMP.

But G’MIC is not a plug-in for GIMP only. It also offers a command-line interface, that can be used in addition with the CLI tools from ImageMagick or GraphicsMagick (this is undoubtly the most powerful and flexible interface of the framework). G’MIC also has a web service G’MIC Online to apply effects on your images directly from a web browser. Other G’MIC-based interfaces also exist (ZArt, a plug-in for Krita, filters for Photoflow…). All these interfaces are based on the generic C++ libraries CImg and libgmic which are portable, thread-safe and multi-threaded (through the use of OpenMP). Today, G’MIC has more than 900 functions to process images, all being fully configurable, for a library of only approximately 150 kloc of source code. It’s features cover a wide spectrum of the image processing field, with algorithms for geometric and color manipulations, image filtering (denoising/sharpening with spectral, variational or patch-based approaches…), motion estimation and registration, drawing of graphic primitives (up to 3d vector objects), edge detection, object segmentation, artistic rendering, etc. This is a versatile tool, useful to visualize and explore complex image data, as well as elaborate custom image processing pipelines (see these slides to get more information about the motivations and goals of the G’MIC project).

A selection of some new filters and effects

Here we look at the descriptions of some of the most significant filters recently added. We illustrate their usage from the G’MIC plug-in for GIMP. All of these filters are of course available from other interfaces as well (in particular within the CLI tool gmic).

Painterly rendering of photographs

The filter Artistic / Brushify tries to transform an image into a painting. Here, the idea is to simulate the process of painting with brushes on a white canvas. One provides a template image and the algorithm first analyzes the image geometry (local contrasts and orientations of the contours), then attempt to reproduce the image with a single brush that will be locally rotated and scaled accordingly to the contour geometry. By simulating enough of brushstrokes, one gets a “painted” version of the template image, which is more or less close to the original one, depending on the brush shape, its size, the number of allowed orientations, etc. All these settings being customizable by the user as parameters of the algorithm: This filter allows thus to render a wide variety of painting effects.


Overview of the filter “Brushify” in the G’MIC plug-in GIMP. The brush that will be used by the algorithmis visible on the top left.

The animation below illustrates the diversity of results one can get with this filter, applied on the same input picture of a lion. Various brush shapes and geometries have been supplied to the algorithm. Brushify is computationally expensive so its implementation is parallelized (each core gives several brushstrokes simultaneously).


A few examples of renderings obtained with “Brushify” from the same template image, but with different brushes and parameters.

Note that it’s particularly fun to invoke this filter from the command line interface (using the option -brushify available in gmic) to process a sequence of video frames (see this example of “ brushified “ video):

Reconstructing missing data from sparse samples

G’MIC gets a new algorithm to reconstruct missing data in images. This is a classical problem in image processing, often named “Image Inpainting“, and G’MIC already had a lot of useful filters to solve this problem. Here, the newly added interpolation method assumes only a sparse set of image data is known, for instance a few scattered pixels over the image (instead of continuous chuncks of image data). The analysis and the reconstruction of the global image geometry is then particularly tough.

The new option -solidify in G’MIC allows the reconstruction of dense image data from such a sparse sampling, based on a multi-scale diffusion PDE’s-based technique. The figure below illustrates the ability of the algorithm with an example of image reconstruction. We start from an input image of a waterdrop, and we keep only 2.7% of the image data (a very little amount of data!). The algorithm is able to reconstruct a whole image that looks like the input, even if all the small details have not been fully reconstructed (of course!). The more samples we have, the finer details we can recover.


Reconstruction of an image from a sparse sampling.

As this reconstruction technique is quite generic, several new G’MIC filters takes advantage of it:

  • Filter Repair / Solidify applies the algorithm in a direct manner, by reconstructing transparent areas from the interpolation of opaque regions. The animation below shows how this filter can be used to create an artistic blur on the image borders.

Overview of the “Solidify” filter, in the G’MIC plug-in for GIMP.

From an artistic point of view, there are many possibilities offered by this filters. For instance, it becomes really easy to generate color gradients with complex shapes, as shown with the two examples below (also in this video that details the whole process).


Using the “Solidify” filter of G’MIC to easily create color gradients with complex shapes (input images on the left, filter results on the right).

  • Filter Artistic / Smooth abstract uses same idea as the one with the waterdrop image: it purposely sub-samples the image in a sparse way, by choosing keypoints mainly on the image edges, then use the reconstruction algorithm to get the image back. With a low number of samples, the filter can only render a piecewise smooth image, i.e. a smooth abstraction of the input image.

Overview of the “Smooth abstract” filter in the G’MIC plug-in for GIMP.

  • Filter Rendering / Gradient [random] is able to synthetize random colored backgrounds. Here again, the filter initializes a set of colors keypoints randomly chosen over the image, then interpolate them with the new reconstruction algorithm. We end up with a psychedelic background composed of randomly oriented color gradients.

Overview of the “Gradient [random]” filter in the G’MIC plug-in for GIMP.

  • Simulation of analog films : the new reconstruction algorithm also allowed a major improvement for all the analog film emulation filters that have been present in G’MIC for years. The section Film emulation/ proposes a wide variety of filters for this purpose. Their goal is to apply color transformations to simulate the look of a picture shot by an analogue camera with a certain kind of film. Below, you can see for instance a few of the 300 colorimetric transformations that are available in G’MIC.

A few of the 300+ color transformations available in G’MIC.

From an algorithmic point of view, such a color mapping is extremely simple to implement : for each of the 300+ presets, G’MIC actually has an HaldCLUT, that is a function defining for each possible color (R,G,B) (of the original image), a new color (R’,G’,B’) color to set instead. As this function is not necessarily analytic, a HaldCLUT is stored in a discrete manner as a lookup table that gives the result of the mapping for all possible colors of the RGB cube (that is 2^24 = 16777216 values if we work with a 8bits precision per color component). This HaldCLUT-based color mapping is illustrated below for all values of the RGB color cube.


Principle of an HaldCLUT-based colorimetric transformation.

This is a large amount of data: even by subsampling the RGB space (e.g. with 6 bits per component) and compressing the corresponding HaldCLUT file, you ends up with approximately 200 and 300 kB for each mapping file. Multiply this number by 300+ (the number of available mappings in G’MIC), and you get a total of 85MB of data, to store all these color transformations. Definitely not convenient to spread and package!

The idea was then to develop a new lossy compression technique focused on HaldCLUT files, that is volumetric discretised vector-valued functions which are piecewise smooth by nature. And that what has been done in G’MIC, thanks to the new sparse reconstruction algorithm. Indeed, the reconstruction technique also works with 3D image data (such as a HaldCLUT!), so one simply has to extract a sufficient number of significant keypoints in the RGB cube and interpolate them afterwards to allow the reconstruction of a whole HaldCLUT (taking care to have a reconstruction error small enough to be sure that the color mapping we get with the compressed HaldCLUT is indistinguishable from the non-compressed one).


How the decompression of an HaldCLUT now works in G’MIC, from a set of colored keypoints located in the RGB cube.

Thus, G’MIC doesn’t need to store all the color data from a HaldCLUT, but only a sparse sampling of it (i.e. a sequence of { rgb_keypoint, new_rgb_color }). Depending on the geometric complexity of the HaldCLUTs to encode, more or less keypoints are necessary (roughly from 30 to 2000). As a result, the storage of the 300+ HaldCLUTs in G’MIC requires now only 850 KiB of data (instead of 85 MiB), that is a compression gain of 99% ! That makes the whole HaldCLUT data storable in a single file that is easy to ship with the G’MIC package. Now, a user can then apply all the G’MIC color transformations while being offline (previously, each HaldCLUT had to be downloaded separately from the G’MIC server when requested).

It looks like this new reconstruction algorithm from sparse samples is really great, and no doubts it will be used in other filters in the future.

Make textures tileable

Filter Arrays & tiles / Make seamless [patch-based] tries to transform an input texture to make it tileable, so that it can be duplicated as tiles along the horizontal and vertical axes without visible seams on the borders of adjacent tiles. Note that this is something that can be extremely hard to achieve, if the input texture has few auto-similarity or glaring luminosity changes spatially. That is the case for instance with the “Salmon” texture shown below as four adjacent tiles (configuration 2×2) with a lighting that goes from dark (on the left) to bright (on the right). Here, the algorithm modifies the texture so that the tiling shows no seams, but where the aspect of the original texture is preserved as much as possible (only the texture borders are modified).


Overview of the “Make Seamless” filter in the G’MIC plug-in for GIMP.

We can imagine some great uses of this filter, for instance in video games, where texture tiling is common to render large virtual worlds.


Result of the “Make seamless” filter of G’MIC to make a texture tileable.

Image decomposition into several levels of details

A “new” filter Details / Split details [wavelets] has been added to decompose an image into several levels of details. It is based on the so-called “à trous” wavelet decomposition. For those who already know the popular Wavelet Decompose plug-in for GIMP, there won’t be so much novelty here, as it is mainly the same kind of decomposition technique that has been implemented. Having it directly in G’MIC is still a great news: it offers now a preview of the different scales that will be computed, and the implementation is parallelized to take advantage of multiple cores.


Overview of the wavelet-based image decomposition filter, in the G’MIC plug-in for GIMP.

The filter outputs several layers, so that each layer contains the details of the image at a given scale. All those layers blended together gives the original image back. Thus, one can work on those output layers separately and modify the image details only for a given scale. There are a lot of applications for this kind of image decomposition, one of the most spectacular being the ability to retouch the skin in portraits : the flaws of the skin are indeed often present in layers with middle-sized scales, while the natural skin texture (the pores) are present in the fine details. By selectively removing the flaws while keeping the pores, the skin aspect stays natural after the retouch (see this wonderful link for a detailed tutorial about skin retouching techniques, with GIMP).


Using the wavelet decomposition filter in G’MIC for removing visible skin flaws on a portrait.

Image denoising based on “Patch-PCA”

G’MIC is also well known to offer a wide range of algorithms for image denoising and smoothing (currently more than a dozen). And he got one more ! Filter Repair / Smooth [patch-pca] proposed a new image denoising algorithm that is both efficient and computationally intensive (despite its multi-threaded implementation, you probably should avoid it on a machine with less than 8 cores…). In return, it sometimes does magic to suppress noise while preserving small details.


Result of the new patch-based denoising algorithm added to G’MIC.

The “Droste” effect

The Droste effect (also known as “mise en abyme“ in art) is the effect of a picture appearing within itself recursively. To achieve this, a new filter Deformations / Continuous droste has been added into G’MIC. It’s actually a complete rewrite of the popular Mathmap’s Droste filter that has existed for years. Mathmap was a very popular plug-in for GIMP, but it seems to be not maintained anymore. The Droste effect was one of its most iconic and complex filter. Martin “Souphead”, one former user of Mathmap then took the bull by the horns and converted the complex code of this filter specifically into a G’MIC script, resulting in a parallelized implementation of the filter.


Overview of the converted “Droste” filter, in the G’MIC plug-in for GIMP.

This filter allows all artistic delusions. For instance, it becomes trivial to create the result below in a few steps: create a selection around the clock, move it on a transparent background, run the Droste filter, et voilà!.


A simple example of what the G’MIC “Droste” filter can do.

Equirectangular to nadir-zenith transformation

The filter Deformations / Equirectangular to nadir-zenith is another filter converted from Mathmap to G’MIC. It is specifically used for the processing of panoramas: it reconstructs both the Zenith and the
Nadir regions of a panorama so that they can be easily modified (for instance to reconstruct missing parts), before being reprojected back into the input panorama.


Overview of the “Deformations / Equirectangular to nadir-zenith” filter in the G’MIC plug-in for GIMP.

Morgan Hardwood has wrote a quite detailled tutorial, on pixls.us, about the reconstruction of missing parts in the Zenith/Nadir of an equirectangular panorama. Check it out!

Other various improvements

Finally, here are other highlights about the G’MIC project:

  • Filter Rendering / Kitaoka Spin Illusion is another Mathmap filter converted to G’MIC by Martin “Souphead”. It generates a certain kind of optical illusions as shown below (close your eyes if you are epileptic!)

Result of the “Kitaoka Spin Illusion” filter.

  • Filter Colors / Color blindness transforms the colors of an image to simulate different types of color blindness. This can be very helpful to check the accessibility of a web site or a graphical document for colorblind people. The color transformations used here are the same as defined on Coblis, a website that proposes to apply this kind of simulation online. The G’MIC filter gives strictly identical results, but it ease the batch processing of several images at once.

Overview of the colorblindness simulation filter, in the G’MIC plug-in for GIMP.

  • Since a few years now, G’MIC has its own parser of mathematical expression, a really convenient module to perform complex calculations when applying image filters This core feature gets new functionalities: the ability to manage variables that can be complex, vector or matrix-valued, but also the creation of user-defined mathematical functions. For instance, the classical rendering of the Mandelbrot fractal set (done by estimating the divergence of a sequence of complex numbers) can be implemented like this, directly on the command line:
    $ gmic 512,512,1,1,"c = 2.4*[x/w,y/h] - [1.8,1.2]; z = [0,0]; for (iter = 0, cabs(z)

Using the G’MIC math evaluator to implement the rendering of the Mandelbrot set, directly from the command line!_

This clearly enlarge the math evaluator ability, as you are not limited to scalar variables anymore. You can now create complex filters which are able to solve linear systems or compute eigenvalues/eigenvectors, and this, for each pixel of an input image. It’s a bit like having a micro-(micro!)-Octave inside G’MIC. Note that the Brushify filter described earlier uses these new features extensively. It’s also interesting to know that the G’MIC math expression evaluator has its own JIT compiler to achieve a fast evaluation of expressions when applied on thousands of image values simultaneously.

  • Another great contribution has been proposed by Tobias Fleischer, with the creation of a new API to invoke the functions of the libgmic library (which is the library containing all the G’MIC features, initially available through a C++ API only). As the C ABI is standardized (unlike C++), this basically means G’MIC can be interfaced more easily with languages other than C++. In the future, we can imagine the development of G’MIC APIs for languages such as Python for instance. Tobias is currently using this new C API to develop G’MIC-based plug-ins compatible with the OpenFX standard. Those plug-ins should be usable indifferently in video editing software such as After effects, Sony Vegas Pro or Natron. This is still an on-going work though.

Overview of some G’MIC-based OpenFX plug-ins, running under Natron.


Overview of a dedicated G’MIC script running within the Blender VSE.

  • You can find out G’MIC filters also in the opensource nonlinear video editor Flowblade, thanks to the hard work of Janne Liljeblad (Flowblade project leader). Here again, the goal is to allow the application of G’MIC effects and filters directly on image sequences, mainly for artistic purposes (as shown in this video or this one).

Overview of a G’MIC filter applied under Flowblade, a nonlinear video editor.

What’s next ?

As you see, the G’MIC project is doing well, with an active development and cool new features added months after months. You can find and use interfaces to G’MIC in more and more opensource software, as GIMPKritaBlenderPhotoflowFlowbladeVeejayEKD and Natron in a near future (at least we hope so!).

At the same time, we can see more and more external resources available for G’MIC : tutorials, blog articles (hereherehere,…), or demonstration videos (herehereherehere,…). This shows the project becoming more useful to users of opensource software for graphics and photography.

The development of version 1.7.2 already hit the ground running, so stay tuned and visit the official G’MIC forum on pixls.us to get more info about the project developement and get answers to your questions. Meanwhile, feel the power of free software for image processing!

Christmas is already here for image processing folks…

… Because the latest version 1.6.8 « XMas 2015 Edition » of G’MIC has been released last week 🙂 ! G’MIC (GREYC’s Magic for Image Computing) is an open-source framework for image processing that I’ve started to develop in August 2008. This new release is a good occasion for me to discuss some of the advances and new features added to the project since my last digest, published here 8 months ago. Seven versions have been published since then.

I’ll talk about the few improvements done on the G’MIC plug-in for GIMP (which is currently the most used interface of the project). But I’ll also describe more technical developments of the framework, which already permitted to design some cool new filters and above all, which promise great things for the future.

1. Overview of the G’MIC project

(Skip this section, if you already know what G’MIC is !)


Fig .1.1. Mascot and logo of the G’MIC project, a full-featured open-source framework for image processing.

G’MIC is an open-source project developed in the IMAGE team at the GREYC laboratory (a public research unit based in Caen / France). It is distributed under the free software license CeCILL. It provides several user interfaces for the manipulation of generic image data. Currently, its most used interface is a plugin for GIMP (the plug-in alone seems to be quite popular, as it has been downloaded more than 2 million times so far).

G'MIC plugin for GIMP Overview

Fig. 1.2. The G’MIC plug-in (version 1.6.8) running inside GIMP 2.8.

G’MIC also proposes other worthy interfaces, as a command line tool (similar to the CLI tools from ImageMagick or GraphicsMagick) which fully exploits its capabilities. There is also a nice web service G’MIC Online to apply effects on images directly from a web browser (check it out, it has been re-designed recently!). Currently, the G’MIC core library (used by all those interfaces) provides more than 900 different functions to perform various image processing tasks. The library alone takes approximately 5.5 MB and has more than 100,000 lines of source code. There is a wide diversity of commands available for geometric and color manipulation, image filtering (denoising and enhancement by spectral, variational or non-local methods), motion estimation / image alignment, primitive drawing (including 3D meshes), edge detection, segmentation, artistic rendering, etc.  It’s a very generic tool, as you can imagine. To learn more about the motivations and goals of the G’MIC project, I suggest you take a look at the presentation slides. You can also meet the developers on the new forum, or on the IRC channel #pixls.us (on Freenode).

2. Improvements of the G’MIC plug-in for GIMP

Three major improvements have been done on the GIMP plug-in interface, these last months.

2.1. Import and export of image data via GEGL buffers

That’s probably the greatest news to announce about the plug-in itself. The G’MIC plug-in is now able to import and export image data from/to GIMP using GEGL buffers, which are the foundations of the current development version (2.9) of GIMP. In practice, the plug-in can process high-bit depth images (for example with pixels stored as 32 bits-floats per channel), without loss of numerical accuracy. Actually, the application of a single filter on an image rarely generates serious visual artifacts only because of the 8 bits quantization of an input image. But people applying many filters and effects sequentially on their images will appreciate that the numerical accuracy is preserved during their entire workflow. The animation below (Fig.2.1) shows the subtle quantization phenomenon occurring when certain types of filters are applied on an image (here, an anisotropic smoothing filter).

Where the application of the same G'MIC filter on an image by 8 bits and 32 bits per channel

Fig. 2.1. Comparison of output histograms generated by the same G’MIC filter applied on an image encoded with 8 bits or 32 bits-per-channel values.

The histograms in Fig.2.1 illustrate the fact we get a greater diversity of pixel values ​​when the filter is applied on an image stored as 32-bits floats rather than 8 bits integers. Applying such filters several times in 8 bits-mode could slowly create undesirable quantization / posterization / banding effects on the processed image. To say it briefly, the G’MIC plug-in seems to be ready for the next major stable release GIMP 2.10 🙂 (a big THANKS to Mitch from the GIMP team, who took the time to explain me how to achieve this).

2.2. Support for UTF-8 strings in widgets

The plugin now handles strings encoded in UTF-8 for all interface widgets, which will ease its internationalization. For instance, we have now a Japanese version of the interface (as well as Japanese translations for some of the filters), as shown in the screenshot below (Fig. 2.2).

The GIMP plugin G'MIC partially translated into Japanese

Fig.2.2. The G’MIC plug-in for GIMP, partially translated in Japanese.

2.3. A more responsive interface

The plug-in now reacts faster to a parameter change, particularly when an image preview is being computed. A cancellation mechanism has been added, and we don’t have to wait the rendering of the filter preview anymore before having the possibility to act on the interface. It might sound obvious, but this greatly improves the user experience.

Also check out the interesting initiative of Jean-Philippe Fleury – a nice contributor – who displays on his web page a gallery of most of the filters and image effects available in the G’MIC plug-in. It’s a very quick and easy way to get an overview of the filters, and to observe how they perform on different images (sometimes with different parameters).

3. The PatchMatch algorithm

PatchMatch is a fast algorithm for computing image correspondences using a block-matching technique. Since a few years, it raises a high interested in the image processing community, thanks to its computation speed and the fact it can be parallelized relatively easily. It has become then a key step in many recent algorithms that need to compute fast comparisons between image patches, including methods for reconstructing missing parts of an imagesynthesizing textures, or performing super-resolutionThus, I’m happy to announce that G’MIC has now a parallelized implementation of PatchMatch (through the native command -patchmatch) both for 2D and 3D images. Based on this recent implementation, I’ve been already able to develop and add three interesting filters in G’MIC, described hereafter.

3.1. Another filter for image Inpainting

A new image inpainting filter – using a multi-scale approach – has been included in G’MIC to reconstruct missing or corrupted image regions. Very convenient to pull your stepmother out of the holidays photos you shot in Ibiza! Note that this is not the first patch-based Inpainting algorithm available in G’MIC (this post on Pat David’s blog already presented such a tool in February 2014). It’s just a different algorithm, thus giving different results. But in this field, having the choice to generate alternative results is definitely not a luxury, given the ill-posed nature of the problem. And if – like me – you have a machine with a lot of CPU cores just waiting to be heated, then this new multi-threaded filter is for you! Here are two examples of results (Fig.3.1 and Fig.3.2) we can obtain with this Inpainting filter.


Fig. 3.1. Removing a bear from an image, using the PatchMatch-based Inpainting filter of G’MIC.


Fig. 3.2. Removing the Eiffel Tower, using the PatchMatch-based Inpainting filter of G’MIC.

A video showing how this latest example has been obtained (in only 1 minute 07) is visible below. This is a real-time video, no tricks were used, except using a PC with 24 cores! Even to me (the developer), it’s a bit magical to see the algorithm regenerating the whole trees at the feet of the Eiffel Tower. Actually, its does nothing more than cloning an existing tree from elsewhere in the picture. But that is still impressive. Of course, it does not work as well on all images where I’ve experimented the algorithm 🙂 .

This new Inpainting filter is roughly the same as the one Adobe introduced in Photoshop CS5 in 2010, under the (very windy) name “Content-Aware Fill”. The main strength of this kind of algorithms is their ability to reconstruct large coherent textured areas. Note also that my PhD student Maxime Daisy (who defended his thesis successfully two weeks ago, well done pal!) has worked on some nice extensions of such Inpainting methods to remove moving objects in video sequences (see his demo page for examples). These extensions for video Inpainting are not yet available in G’MIC (and are still very demanding in terms of computation time), but it may happen one day, who knows?

3.2. Texture re-synthesis

By using a very similar multi-scale approach, I’ve also added a new texture synthesis filter in G’MIC. It allows to synthesize a texture of arbitrary size from an input template texture (which is usually smaller). Of course, the filter does something smarter than just tiling the template texture: It actually regenerates a whole new texture having similar characteristics by copying / pasting some bits of the template texture so that the synthesized result does not contain visible seams. Fig.3.3 shows such a 512x512 re-synthesized texture (top-right) from an input template image having a smaller size 280x350 (left). A comparison of this result with a more basic tiling process is also shown (bottom-right).


Figure 3.3. Example of re-synthesizing a complex texture using G’MIC and comparison with basic tiling.

Note that we already had a texture re-synthesis algorithm in G’MIC, but it worked well only with micro-textures. On the contrary, the example on Fig.3.3 shows that the new algorithm is capable of regenerating more complex macro-textures.

3.3 Make seamless textures

And here is the third (and last) PatchMatch-based filter added so far in G’MIC. This one allows to transform an input texture to make it appear seamless when tiled. The idea is a bit different than with the previous filter, as most of the original input texture is actually untouched here. The filter only performs some global color normalization (to remove low-frequency gradients due to the non-homogeneous illumination) and locally adds an inner (or outer) frame that makes the texture appear seamless. Look at what the filter can do on the two examples below (textures shown as 2x2 tiled). Not bad at all ! I have to admit anyway that the parameters are still a bit hard to tweak.


Fig.3.4. Make a seamless texture with the third PatchMatch-based filter in G’MIC.


Fig.3.5 Make a seamless texture with the third PatchMatch-based filter in G’MIC.

Believe me, having this PatchMatch algorithm now implemented in G’MIC is a really great news! No doubts many filters could benefit from this method in the future.

4. A more powerful expression evaluator

In my mind, G’MIC has always been a very convenient tool to quickly design and build complex image processing pipelines (possibly with loops and conditionals). However, when I do algorithm prototyping, it frequently happens I want to experiment with quite “low-level” operations. For instance, I may want to write nested (x,y) loops over an image, where each loop iteration performs quite “exotic” things at the pixel scale. This kind of low-level algorithms cannot be expressed easily as pipelines of macro-operators (assuming we don’t already have a macro-operator that does the job, of course!). Trying to express such an algorithm anyway as a G’MIC pipeline was a bit tedious, either because it required some tricks to write, or because it ended up in a very slow implementation (due to the nested loops over all the image pixels that keep being interpreted). Note that this problem is not specific to G’MIC : try to discuss with image processing folks working with Matlab to see how creative they are when it comes to avoid writing explicit loops in a Matlab script ! It’s a fact: image processing often requires a lot of (pixel) data to process, and prototyping heavy IP algorithms only with interpreted languages introduces a speed bottleneck.

My goal anyway was to be to able to prototype most of my algorithms (including those low-level ones) directly in G’MIC, with the constraint they never become ridiculously slow to run (compared to an equivalent C++ implementation, for instance). As you might have guessed, a conventional solution to this kind of problems would consist in embedding a Just-in-Time compiler into my interpreter. So that’s what I’ve done – but only for a subset of G’MIC – namely the mathematical expression evaluator. As one might expect, evaluating math expressions is indeed a key feature for an image processing framework such as G’MIC.

To be perfectly honest, this JIT has been already included in G’MIC for years. But it has been also greatly optimized and improved these last months to allow the parallelized evaluation of more complex expressions. These expressions can now be in fact considered as small programs themselves (containing loops, variables, conditional tests, pixel access, etc.), rather than just plain mathematical formulas. As a result, we now have a very efficient expression evaluator in G’MIC, that appears to be at least 10x faster than the equivalent feature proposed in ImageMagick (with its generic command -fx) !

Thus, G’MIC is now a tool good enough to prototype almost any kind of image processing algorithms, without sacrificing the computation speed. I wrote a quite comprehensive post about this improved expression evaluator a few months ago, so please take some time to read it if you want more details. Here again, this is very encouraging news for the future of the project.

5. An abundance of filters!

This section proposes a bulk list of some other features/effects/filters added in G’MIC since last April.The screenshots below often show those filters running  in the G’MIC plug-in for GIMP, but of course they may be applied also from all the other interfaces (the CLI tool, in particular).

  • Filters search engine: this has been a long-time request from users, so finally we have proposed a first draft for such a tool (thanks Andy!). Indeed, the number of filters in the plug-in keeps increasing and it is not always convenient to find a particular filter, especially if you forgot to put it in your favorites. Anyway, you have still to specify relevant keywords to be able to find you filters with this tool.

Fig.5.1. New filter search engine by keywords in the G’MIC plug-in for GIMP.

  • Vector Painting: This image filter is a direct consequence of the recent improvements made on the math expression evaluator. It basically generates a piecewise constant abstraction of an input color image. It’s quite funny to look at the full source code of this filter, because it is surprisingly short (19 lines) !

Fig. 5.2. The Vector Painting filter creates image abstractions.

  • Freaky B&W: This filter performs black & white conversion (grayscale to be precise) of an input color image. But it does this by solving a Poisson equation rather than applying a simple linear formula to compute the luminance. Here, the goal is to get a B&W image which contains all of the contrast details present in each color channel of the original image. This filter often generates images that have a kind of HDR look (but in grayscale).

Fig. 5.3. The Freaky B&W filter converts a color image into HDR-like grayscale.

  • Bokeh: This one tries to generate synthetic Bokeh backgrounds on color images (artistic blur). It is highly configurable and can generate Bokeh with various kind of shapes (circles, pentagons, octagons, stars ..). It can be used for instance to add some light effects to a color image, as illustrated below.

Fig. 5.4. Application of the new Bokeh filter on a color image.

  • Rain & snow: as the name suggests, this filter adds rain or snow on your images (example below is a zoom of the previous image).

Fig. 5.5. Adding rain on a color image with the filter Rain & snow.

  • Neon lightning: not much to say about this one. It generates random curves starting from a region A to another region B and stylizes those curves as neon lights. Convenient for generating cool wallpapers probably 🙂

Fig. 5.6. Effect Neon Lightning in action!

  • Stroke: This filter decorate simple monchrome shapes on a transparent layer with color gradients. The figure below illustrates the transformation of a simple text (initially drawn with a single color) by this filter.

Fig. 5.7. Applying the Stroke filter to decorate a plain text.

  • Light leaks: this one simulates the presence of undesired light artifacts in pictures. In general, that is rather the kind of effects we want to remove! But the simulation of image degradation can be useful in some cases (for those who want to make their synthesized images more realistic for instance).

Fig. 5.8. Simulation of image degradation with the Light leaks filter.

  • Grid [triangular]: This filter converts an image into a grid composed of colored triangles, allowing many choices for the type of grids to apply.

Fig. 5.9. Transforming an image in a triangular grid.

  • Intarsia: This is a quite original filter. It allows to turn an image into a knitting diagram for Intarsia. The idea came from a user of the GimpChat forum to help her distributing custom knitting patterns (apparently there are websites that offer such patterns in exchange for hard cash). The filter itself does not modify the picture. It only generate a knitting diagram as an output HTML page. An example of such a diagram is visible here.
  • Drop water: I must admit I particularly like this one. The filter simulates the appearance of water droplets over an image. In its most basic version (with the default parameters), it looks like this:

Fig. 5.10. Adding water droplets on an image with the Drop water filter.

But it gets really interesting when the user defines its own droplet shapes by adding a transparent layer containing a few colored strokes on it. For instance, if you add this layer (here represented with pink strokes) to the leaf image:


Fig. 5.11. Adding a layer to define the desired shape of water drops.

Then, the filter Drop water will generate this image, with your custom water drops:


Fig. 5.12. Synthesis of custom shaped water drops.

Moreover, the filter has the good sense to generate its result as a stack of several ouput layers, each corresponding to the simulation of a different physical phenomena, namely: the specular light spots, the drop shadows, the own shadows, and the light refraction. It is then easy to modify these layers indivisually afterwards to create interesting effects with the image. For instance, if we apply a color-to-monochrome filter on the base image layer while preserving the refraction layer (computed from the initial color image), we end up with this:


Fig. 5.13. Applying the Drop water filter, followed by a colorimetric change on the initial layer only.

The short video tutorial below explains how to achieve this effect, step by step, using the G’MIC plug-in for GIMP. It takes less than 2 minutes, even for a complete beginner.

Even better, you can superimpose the result of the Drop water filter on another image. Here is an example where two portraits have been aligned then “merged” together, separated by a kind of liquid interface. It’s really easy to do, and the result looks pretty cool to me.


Fig. 5.14. The G’MIC Drop water filter applied to create a liquid interface between two distinct portraits.

Well, that’s it for the newest filters!

6. Other notable points

To end this long post, let me list some other interesting news about the project:

  • First of all, we have been contacted by Tobias Fleischer. Tobias is a professional plug-in developer for Adobe After Effects and Premiere Pro (among other video editing software). He has made ​​a tremendous job in developing a DLL for Windows which encapsulates libgmic (the G’MIC core library). And, he has used this DLL to implement a prototype of a G’MIC plug-in for After Effects. He was kind enough to share some screenshots with us of what he has done recently: you can see below an example of the G’MIC skeleton filter running in After Effects.

Fig. 6.1. Prototype of a G’MIC plug-in running in Adobe After Effects.

I can already hear grumpy guys saying: “But why didn’t he try doing that for a free software, as Natron, rather than a 100% proprietary software? “. But in fact, he did it also! And not just for Natron, but for any other video processing software compatible with the OpenFX API. The screenshot below shows a prototype of an OpenFX-compliant G’MIC plug-in running in Sony Vegas Pro. There are probably no reasons why this couldn’t work with Natron too.


Fig. 6.2. Prototype of an OpenFX-compliant G’MIC plug-in running under Sony Vegas Pro.

Of course, all this should be still considered as Work In Progress. There are probably still bugs to fix and improvements to do, but this is really promising. I can’t wait to see these plug-ins released and running on Natron 🙂

  • Let me continue with some other good news: Andrea, a nice contributor (who is also the developer of PhotoFlow) managed to understand why G’MIC had frequent crashes under MacOSX when computations were done using multiple threads. He proposed a simple patch to solve this issue (it was actually related to the tiny stack size allocated for each thread). G’MIC under MacOSX should be then fully functional now.
  • The webcam/video interface ZArt of G’MIC has been also improved a lot, with new added filters, automatic detection of the webcam resolutions, and the possibility of having a dual-window (one for monitoring + one for display, on a second screen).

Fig 6.3. The ZArt interface running the G’MIC filter Drop water in dual-window mode.

  • A new animated demo has been also added to G’MIC, through the command -x_landscape. The demo itself is of little interest for the user, but was gainful for me to test/improve the speed of the G’MIC interpreter (and isn’t it funny to have this kind of things in G’MIC 😉 ). Note that all animated and/or interactive G’MIC scripts are available via this invokation of the CLI tool:
$ gmic -demo

Fig. 6.4. Animated virtual landscape in the G’MIC CLI tool gmic with command -x landscape.

As I had more and more small and funny animations implemented into G’MIC, I’ve decided to put them together as a little intro, entirely coded as a single G’MIC script (750 lines of code). The result smells the divine fragrance of old 8/16 bits machines (the corresponding video is shown below). Remember that all have been generated 100% with the G’MIC interpreter. Isn’t that cool ?

8. So, what’s next?

Now it’s probably time to rest a little bit 🙂 . 2015 has already been a very busy year for the G’MIC project. Christmas is rapidly approaching and I’ll take a (small) break.

To be honest, I don’t have yet a clear idea about the things I’ll focus on. In any case, it appears that G’MIC keeps improving gradually and can potentially raise the interest of more and more people. I wanted to thank especially all the contributors of the project and users of the software who give regular feedback to make it better and better. Also, be aware that each significant addition is usually subject to an announcement on the Google+ page of G’MIC, so it’s easy to stay informed about the project progression, day after day. Dear reader, would you have any proposal for major improvements ?

Let me finally conclude this post by proclaiming it mightily: “Long live open-source image processing ! And Happy XMas everyone !



Image processing made easier with a powerful math expression evaluator.

Warning: This post contains personal thoughts about my research work in image processing. I’ll discuss about some of the issues I’m facing as an active developer of two open-source image processing frameworks (namely CImg and G’MIC). So keep in mind this will be a bit self-centered. There are high chances you find all this really boring if you are not a developer of image processing software yourself (and even if so). Anyhow, feel free to give your impressions after the reading!

1. Context and issues

In imaging science, image processing is processing of images using mathematical operations by using any form of signal processing for which the input is an image, such as a photograph or video frame.

That’s what Wikipedia says about  image processing. Selecting and ordering those mathematical operations is what actually defines algorithms, and implementing ready-to-use and interesting image processing algorithms is actually one of my goals, as well as making them available for interested users afterwards.

After all those years (>10) as a researcher in the image processing field (and like most of my colleagues), I can say I’ve already implemented a lot of these different algorithms. Mostly in C++ as far as I’m concerned. To be more precise, an important part of my work is even to design (and hopefully publish) my own image processing methods. Most of the time of course, my trials end up with clunky, ineffective and slow operators which give nothing interesting else than knowing the approach is not good enough to be followed. Someone who says everything he tries works right the first time is a liar. Step by step, I try to refine/optimize my prototypes or sometimes even take a completely different direction. Quickly, you realize that it is crucial in this job not to waste time when doing algorithm prototyping because the rate of success is in fact very low.


Don’t waste your time, in any occasion! (photo by (OvO), under CC-by-nc-sa.)

That’s actually one of the reason why I’ve started the G’MIC project. It was primarily designed as a helper to create and run custom image processing pipelines quickly (from the shell, basically). It saves me time, everyday. But the world of image processing algorithms is broad and sometimes you need to experiment with very low-level routines working at a pixel scale, trying such weird and unexpected stuffs that none of the “usual” image processing algorithms you already have in your toolbox can be of use as it is. Or it is used in a so diverted way that it gets hard to even think about using it adequately. In a word, your pixel-level algorithm won’t be expressed as a simple pipeline (or graph if you want to call it so) of macro-scale image processing operators. That’s the case for instance with most of the well known patch-based image processing algorithms (e.g. Non-Local-Means, or PatchMatch and plenty of variants), where each pixel value of the resulting image is computed from (a lot of) other pixel values whose spatial locations are sometimes not evenly distributed (but not random as well!).

Until now, when I was trying to implement this kind of algorithms, I was resigned to go back coding them in C++: It is one language I feel comfortable with, and I’m sure it will run fast enough most of the time. Indeed, computation time is often a bottleneck in image processing. Some of my colleagues are using scripting languages as Matlab or Python for algorithm prototyping. But they often need some tricks to avoid writing explicit code loops, or need to write at least some fast C/C++ modules that will be compiled and run from those higher-level interfaces, to ensure they get something fast enough (even for prototyping, I’m definitely not talking about optimized production code here!).


But, I’m not really satisfied with my C++ solution: Generally, I end up with several small pieces of C++ sources I need to compile and maintain. I can hardly re-use them in a bigger pipeline, or redistribute them as clean packages without a lot of extra work. Because they are just intended to be prototypes: They often have only basic command-line interfaces and thus cannot be directly integrated into bigger and user-friendly image processing frameworks. Making a prototype algorithm really usable by others requires at least to wrap it as a plug-in/module for [..copy the name of your favorite image processing tool or language here..]. This generally represents a lot of boring coding workthat may even require more time and efforts than writing the algorithm itself!  And I don’t talk about maintenance. If you’ve ever tried to maintain a 10-year old C++ prototype code, lost in one of your sub-sub-sub-sub folder in your $HOME, you know what I mean. I’d definitely prefer a simpler solution that let me spend more time on writing the algorithm itself than packaging it or making it usable. After all, the primary purpose of my work is to create cool algorithms, not really coding user interfaces for them. On the other way, I am a scientist and I’m also happy to share my discoveries with users (and possibly get feedback from them!). How to make those prototyped algorithms finally usable without spending too much time on making them usable ? 🙂

Ideally, I’d like something that could nicely integrate into G’MIC (my favorite framework for doing image processing stuffs, of course 🙂 ). Even if at the end, those algorithms run a bit slower than they are in C++. One could suggest to make them as Octave or Scilab scripts/modules. But I’m the developer of G’MIC, so of course, I’d prefer a solution that help to extend my own project.

So finally, how could I code prototypes for new algorithms working at a pixel-level and make them readily available in G’MIC ? This question has worried me for a long time.

2. Algorithm code viewed as a complex math expression

In G’MIC, the closest thing to what I was looking for, is the command

-fill 'expression'

This command fills each pixel of a given image with the value evaluated from a “mathematical expression”. A mathematical expression being a quite vague concept, it appears you can already write some complex formulas. For instance, typing this on the command line:

$ gmic 400,400,1,3 -fill "X=x-w/2; Y=y-h/2; R=sqrt(X^2+Y^2); a=atan2(Y,X); if(R<=180,255*abs(cos(c+200*(x/w-0.5)*(y/h-0.5))),850*(a%(0.1*(c+1))))"

creates this weird-looking 400×400 color image (I advise you to put sunglasses):


Fig.1. One synthetic color image obtained by one application of the G’MIC command -fill.

Of course, the specified expression can refer to pixels of an existing input image. And so, it can modify the pixels of an image as well, as in the following example:

$ gmic leno.png -fill "(abs(i(x+1,y)-i(x-1,y)))^0.25"

which computes gamma-corrected differences of neighboring pixels along the X-axis, as shown below:


Fig.2.1. Original image leno.png


Fig.2.2. Result of the -fill command described above.

(As an aside, let me tell you I’ve recently received e-mails and messages from people who claim that using the image of our beloved Lena to illustrate an article or a blog post is “sexist” (someone even used the term “pornographic”…). I invite you reading the Lena story page if you don’t know why we commonly use this image. As I don’t want to hurt the over-sensibility of these people, I’ll be using a slight variation I’ve made by mixing a photograph of the blessed Jay Leno with the usual image of Lena. Let me call this the Leno image and everyone will be happy (but seriously, get a life!)).

So, as you can imagine, the command -fill already allows me to do a lof of complex and exotic things on images at a pixel level. Technically speaking, it uses the embedded math parser I’ve written for the CImg Library, a C++ open-source image processing library I’ve been developing since 1999 (and on which G’MIC is based). This math parser is quite small (around 1500 lines of C++ code) and quite fast as well, when it is applied on a whole image. That’s mainly because:

1. It uses parallelization (thanks to the use of OpenMP directives) to evaluate expressions on blocs of image pixels in a multi-threaded way.

2. Before being evaluated, the given math expression is pre-compiled on-the-fly by CImg, into a sequence of bytecodes. Then, the evaluation procedure (which is done for the whole image, pixel by pixel) requires only that bytecode sequence to be interpreted, which is way faster than parsing the input mathematical expression itself.

Anyway, I thought the complexity of the pixel-level algorithms I’d like to implement was really higher than just the evaluation of a mathematical formula. But wait… What is missing actually ? Not much more than loops and updatable variables… I already had variables (though non-updatable) and conditionals. Only loops were really missing. That looks like something I could try adding during my summer holidays, isn’t it ? 😉 So, that is where my efforts were focused on during these last weeks: I’ve added new functions to the CImg math parser that allow users to write their own loops in mathematical expressions, namely the functions dowhile(expr,_cond)whiledo(cond,expr) and for(init,cond,expr_end,_expr_body). Of course, it made me also review and re-implement large parts of the math parser code, and I took this opportunity to optimize the whole thing. A new version of the math parser has been made available for the release of G’MIC at the end of August. I’m still working on this expression evaluator in CImg and new improvements and optimizations are ready for the upcoming version of G’MIC (soon to be released).

3. A toy example: Julia fractals

So, what can we do now with these new math features in G’MIC ? Let me illustrate this with a toy example. The following custom G’MIC command renders a Julia fractal. To test it, you just have to copy/paste the following lines in a regular text file user.gmic:

julia_expr :
  -fill "
    zr = -1.2 + 2.4*x/w;
    zi = -1.2 + 2.4*y/h;
    for (iter = 0, zr^2+zi^2<=4 && iter<256, ++iter,
      t = zr^2 - zi^2 + 0.4;
      (zi *= 2*zr) += 0.2;
      zr = t
  -map 7

and invoke the new command -juliar_expr it defines by typing this in the terminal:

$ gmic user.gmic -julia_expr

Then, you’ll get this 1024×1024 color image:

Rendering of the Julia fractal set only from filling an image with a math expression.

Fig.3. Rendering of a Julia fractal only by filling an image with a complex math expression, containing an iteration loop.

As you see, this custom user command -julia_expr is very short and is mainly based on the invokation of the -fill command of G’MIC. But the coolest thing of all happens when we look at the rendering time of that function. The timing measure has been performed on an ASUS laptop with a dual-core HT i7 2Ghz. This is what I get:

Edit : This post has been edited, on 09/20/2015 to reflect the new timings due to math parser optimization done after the initial post of this article.

$ gmic user.gmic -tic -julia -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'user.gmic' (added 1 command, total 6195).
[gmic]-0./ Initialize timer.
[gmic]-0./julia/ Input black image at position 0 (1 image 1024x1024x1x1).
[gmic]-1./julia/ Fill image [0] with expression ' zr = -1.2 + 2.4*x/w; zi = -1.2 + 2.4*(...)( zi *= 2*zr) += 0.2; zr = t ); iter '.
[gmic]-1./julia/ Map cube color LUT on image [0], with dirichlet boundary conditions.
[gmic]-1./ Elapsed time : 0.631 s.

Less than 0.7 second to fill a 1024×1024 image where each of the 1,048,576 pixels may require up to 256 iterations of a computation loop ? Definitely not bad for a prototyped code written in 5 minutes and which does not require compilation to run! Note that all my CPUs have been active during the computation. Trying the same G’MIC code on my machine at work (a powerful 4x 3-core HT Xeon 2.6Ghz) makes the same render in 0.176 second only!

But of course, one could say:

Why not using the native G’MIC command -mandelbrot instead (here, native means hard-coded as a C++ function) ? It is probably way faster!

Let me compare my previous code with the following G’MIC invokation (which renders exactly the same image):

$ gmic 1024,1024 -tic -mandelbrot -1.2,-1.2,1.2,1.2,256,1,0.4,0.2 -map 7 -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input black image at position 0 (1 image 1024x1024x1x1).
[gmic]-1./ Initialize timer.
[gmic]-1./ Draw julia fractal on image [0], from complex area (-1.2,-1.2)-(1.2,1.2) with c0 = (0.4,0.2) and 256 iterations.
[gmic]-1./ Map cube color LUT on image [0], with dirichlet boundary conditions.
[gmic]-1./ Elapsed time : 0.055 s.

That’s 12 times faster, than the previous command -julia_expr run on my laptop, indeed! A bit reassuring to know that C++ compiled into assembly code is faster than CImg home-made bytecode compiled on the fly 😉

But the point is: Suppose now I want to slightly modify the rendering of the fractal, i.e. I don’t want to display the maximum iteration anymore for each pixel (variable iter), but the latest value of the variable zi before the divergence test occurs. Look how simple this is to create a slightly modified command -julia_expr2 that does exactly what I want to do. I have the full control on what the function does at a pixel level:

julia_expr2 :
  -fill "
    zr = -1.2 + 2.4*x/w;
    zi = -1.2 + 2.4*y/h;
    for (iter = 0, zr^2+zi^2<=4 && iter<256, ++iter,
      t = zr^2 - zi^2 + 0.4;
      (zi *= 2*zr) += 0.2;
      zr = t
  -normalize 0,255 -map 7

and this modified algorithm renders the image below (still in 0.7 second of course):

Fig.4. Slightly modified version of the Julia fractal by displaying another variable zi in the rendering algorithm.

Without these new loop features introduced in the math parser, I would have been forced to do one of these two things in G’MIC, to get the same result:

  1. Either, add some new options to the native command -mandelbrot to allow this new type of visualization. This basically means: Writing new pieces of C++ code, compiling a new version of G’MIC with these added features, package it and release it to make this available for everyone. Even if I already have some decent release scripts, this implies a lot of extra-work and packaging time. This is not like the user can get the new feature in a few minutes (if you’ve already used the filter update mechanism present in the G’MIC plug-in for GIMP, you know what I mean). And I don’t speak about all the possibilities I couldn’t think of (but the user will obviously need one day 🙂 ), when adding such new display options to a native command like -mandebrot
  2. Or, write a new G’MIC custom script able to compute the same kind of result. This would be indeed the best way to make it available for other people quickly. But here, as the algorithm is very specific and works at a pixel level, doing it as a pipeline of macro-operators is quite a pain. It means I would have to use 3 nested -repeat...-done loops (which are basically the loops commands used in G’MIC pipelines) and it would probably take ages to render, as a G’MIC pipeline is always purely interpreted without any pre-compilation steps. Even by using multi-threading, it would have been a nightmare to compute here.

In fact, the quite long math expression we are using in command -julia_expr2 defines one complex algorithm as a whole, and we know it will be pre-compiled into a sequence of bytecodes by CImg before being evaluated for each of the 1024×1024=1,048,576 pixels that compose the image. Of course, we are not as fast as a native C++ implementation of the same command, but at the same time, we gain so much flexibility and genericity in the stuffs we can do now, that this disadvantage is easily forgiven. And the processing time stays reasonable. For fast algorithm prototyping, this feature seems to be incredibly nice! I won’t be forced to unsheathe my C ++ compiler every time I want to experiment some very specific image processing algorithms working at the pixel level.

4. A more serious example: the “Non-Local Means”

The Non-Local-Means is a quite famous patch-based denoising/smoothing algorithm in image processing, introduced in 2005 by A. Buades (beware, his home page contains images of Lena, please do not click if you are too sensitive!). I won’t enter in all the implementation details, as several different methods has been proposed in the literature just for implementing it. But one of the most simple (and slowest) technique requires 4 nested loops per image pixel. What a good opportunity to try writing this “slow” algorithm using the G’MIC -fill function! It took me less than 10 minutes, to be honest:

nlmeans_expr : -check "${1=10}>0 && isint(${2=3}) && $2>0 && isint(${3=1}) && $3>0"
  -fill "
    const sigma = $1;  # Denoising strength.
    const hl = $2;     # Lookup half-size.
    const hp = $3;     # Patch half-size.
    value = 0;
    sum_weights = 0;
    for (q = -hl, q<=hl, ++q,
      for (p = -hl, p<=hl, ++p,
        diff = 0;
        for (s = -hp, s<=hp, ++s,
          for (r = -hp, r<=hp, ++r,
            diff += (i(x+p+r,y+q+s) - i(x+r,y+s))^2
        weight = exp(-diff/(2*sigma)^2);
        value += weight*i(x+p,y+q);
        sum_weights += weight
    value/(1e-5 + sum_weights)

Now, let’s test it on a noisy version of the Leno image:

$ gmic user.gmic leno.png -noise 20 -c 0,255 -tic --nlmeans_expr 35,3,1 -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'user.gmic'(added 1 command, total 6195).
[gmic]-0./ Input file 'leno.png' at position 0 (1 image 512x512x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 20.
[gmic]-1./ Cut image [0] in range [0,255].
[gmic]-1./ Initialize timer.
[gmic]-1./nlmeans_expr/ Set local variable sigma='35'.
[gmic]-1./nlmeans_expr/ Set local variable hl='3'.
[gmic]-1./nlmeans_expr/ Set local variable hp='1'.
[gmic]-1./nlmeans_expr/ Fill image [0] with expression ' value=0; sum_weights=0; for(q = -3,q<(...)ight ) ); value/(1e-5 + sum_weights) '.
[gmic]-2./ Elapsed time : 3.156 s.

which results on these two images displayed on the screen, the noisy version (left), and the denoised one using the Non-Local-Means algorithm (right). Of course, the timing may differ from one machine to another. I guess my 3 seconds run here is seemly (tested on my powerful PC at the lab). It still takes less than 20 seconds on my laptop. A crop of the results are presented below. The initial Leno image is a 512×512 RGB image, and the timing has been measured for processing the whole image, of course.


Fig.5.1. Crop of a noisy version of the Leno image, degraded with gaussian noise, std=20.


Fig.5.2. Denoised version using the NL-means algorithm (custom command -nlmeans_expr).

Here again, you could argue that the native G’MIC command -denoise does the same thing and is faster to run. It is, definitely. Jérome Boulanger (one very active G’MIC contributor) has written a nice custom command -nlmeans that implements the NL-means with a smarter algorithm (avoiding the need for 4 nested loops per pixel) which runs even faster (and it is already available in the plug-in for GIMP). But that’s not the point. What I show here is I’m now able to do some (relatively fast) prototyping of algorithms working at a pixel level in G’MIC, without having to write and compile C++ code. But the best of all is about integration: if the algorithm appears to be interesting/effective enough, I can add it to the G’MIC standard library in a few minutes, and quickly create a filter for the GIMP plug-in as well. Let say it right away, probably 5 minutes after I’ve finished writing the first version of the algorithm, the plug-in users should be able to get it and use it on their own images (and give positive/negative feedback to help for future improvements). That’s what I call a smooth, quick and painless integration! And that is exactly the kind of algorithms I couldn’t implement before as custom G’MIC commands running at a decent speed.

To me, it clearly opens exciting perspectives to quickly prototype and integrate new custom image processing algorithms into G’MIC in the future!

5. The Vector Painting filter

In fact, this has happened earlier than expected. I’ve been able to add one of my latest image filter (named Vector painting) in the G’MIC plug-in for GIMP recently. It has been somehow unexpected, because I was just doing some debugging for improving the CImg math expression evaluator. Briefly, suppose you want to determine for each pixel of an image, the discrete spatial orientation of the maximal value variation, with an angular precision of 45°: For each pixel centered in a 3×3 neighborhood, I want to estimate which pixel of the neighborhood has the highest difference with the center pixel (measured as the squared difference between the two pixel values). To make things simpler, I’ve considered doing this on the image luminance only instead of using all the RGB color channels. At the end, I transform each pixel value into a label (an integer in range [1,8]) that represents one of the possible 45°-orientations of the plane. That was typically the kind of problems that would require the use of custom loops working at a pixel level, so something I couldn’t do easily before the loop feature introduced in my math parser (or I would have done the prototype in C++).

The solution to this problem was surprisingly easy to write. Here again, it didn’t take much more than 5 minutes of work:

foo :
  -fill "dmax = -1; nmax = 0;
         for (n = 0, ++n<=8,
           p = arg(n,-1,0,1,-1,1,-1,0,1);
           q = arg(n,-1,-1,-1,0,0,1,1,1);
           d = (j(p,q,0,0,0,1)-i)^2;
           if(d>dmax,dmax = d; nmax = n,nmax)

And if we apply this new custom command -foo on our Leno image,

$ gmic user.gmic leno.png -foo

We get this result (after re-normalization of the label image in range [0,255]). Keep in mind that each pixel of the resulting image is an integer label originally in range [1,8]. And by the way, the computation time is ridiculous here (178ms for this 512×512 image).


Fig.6. Each pixel of the Leno image is replaced by a label saying about which of its 3×3 neighbors is the most different from the central pixel.

It actually looks a bit ugly. But that’s not surprising, as the original image contains noise and you’ll get a lot of small random variations in flat regions. As a result, the labels you get in those regions are noisy as well. Now, what happens when we blur the image before computing the labels? That should regularize the resulting image of labels as well. Indeed:

$ gmic user.gmic leno.png --blur 1% -foo

returns this:


Fig.7. Each pixel of the blurred Leno image is replaced by a label saying about which of its 3×3 neighbors is the most different.

That’s interesting! Blurring the input image creates larger portions of constant labels, i.e. regions where the orientation of the maximal pixel variations is the same. And the original image contours keep appearing as natural frontiers of these labelled regions. Then, a natural idea would be to replace each connected region by the average color it overlays in the original color image. In G’MIC, this can be done easily with the command -blend shapeaverage.

$ gmic user.gmic leno.png --blur 1% -foo[-1] -blend shapeaverage

And what we get at the end is a nice piecewise constant abstraction of our initial image. Looks like a “vector painting”, no ? 😉


Fig.8. Result of the “shape average” blending between the original Leno image, and its map of labels, as obtained with command -foo.

As you may imagine, changing the amplitude of the blurring makes the result more or less abstract. Having this, It didn’t take too much time to create a filter that could be run directly from the G’MIC plug-in interface for GIMP. That’s the exact code I wrote to integrate my initial algorithm prototype in G’MIC and make it usable by everyone. It was done in less than 5 minutes, really:

#@gimp Vector painting : gimp_vector_painting, gimp_vector_painting_preview(1)
#@gimp : Details = float(9,0,10)
#@gimp : sep = separator(), Preview type = choice("Full","Forward horizontal","Forward vertical","Backward horizontal","Backward vertical","Duplicate horizontal","Duplicate vertical")
#@gimp : sep = separator(), note = note("<small>Author: <i>David Tschumperl&#233;</i>.\nLatest update: <i>08/25/2015</i>.</small>")
gimp_vector_painting :
  -repeat $! -l[$>]
    --luminance -b[-1] {10-$1}%,1,1
    -f[-1] "dmax = -1; nmax = 0;
            for (n = 0, ++n<=8,
              p = arg(n,-1,0,1,-1,1,-1,0,1);
              q = arg(n,-1,-1,-1,0,0,1,1,1);
              d = (j(p,q,0,0,0,1)-i)^2;
              if (d>dmax, dmax = d; nmax = n,nmax)
    -blend shapeaverage
  -endl -done

gimp_vector_painting_preview :
  -gimp_split_preview "-gimp_vector_painting $*",$-1

Here is the resulting filter, as it can be seen in the G’MIC plug-in for GIMP (requires version Just after I’ve pushed it in the G’MIC standard library:


Fig.9. The G’MIC plug-in for GIMP, running the “Vector Painting” filter.

Here again, that is how I conceive things should be done properly: 1. I create a quick algorithm prototype to transform an image into something else. 2. I decide that the algorithm is cool enough to be shared. 3. I add few lines to make it available immediately in the G’MIC image processing framework. What a gain of time compared to the time it would have taken by doing this in C++!

6. Comparison with ImageMagick’s -fx operator

While working on the improvement of my math expression evaluator in CImg, I’ve been wondering if what I was doing was not already existing in ImageMagick. Indeed, ImageMagick is one of the most well established open-source image processing framework, and I was almost sure they had already cope with the kind of questions I had for G’MIC. And of course, they had 🙂

So, they have a special operator -fx expression in convert that seems to be equivalent to what the G’MIC command -fill expression does. And yes, they probably had it for years, long before G’MIC even existed. But I admit I’ve almost completely stopped using ImageMagick tools when I’ve started developing my own C++ image processing library CImg, years ago. All the information you need to use this -fx operator in convert can be found on this documentation page, and even more examples on this page. Reading these pages was very instructive: I’ve noticed some interesting functions and notations they have in their own expression parser that I didn’t already have in mine (so, I’ve added some of them in my latest version of CImg!). Also I was particularly interested by this quote from their pages:

As people developed new types of image operations, they usually prototype it using a slow “-fx” operator first. When they have it worked out that ‘method’ is then converted into a new fast built-in operator in the ImageMagick Core library. Users are welcome to contribute their own “-fx” expressions (or other defined functions) that they feel would be a useful addition to IM, but which are not yet covered by other image operators, if they can be handled by one of the above generalized operators, it should be reasonably easy to add it.(…). What is really needed at this time is a FX expression compiler, that will pre-interpret the expression into a tighter and faster executable form. Someone was going to look into this but has since disappeared.

So it seems their -fx operator is quite slow as it re-parses the specified math expression for each image pixel. And when someone writes an interesting operator with -fx, they are willing to convert it into a C code and integrate this new built-in operator directly in the core ImageMagick library. It seems they don’t really mind adding new native hard-coded operators into IM, maybe even for very specific/unusual operators (at least they don’t mention it). That’s interesting, because that is precisely what I’m trying to avoid in G’MIC. My impression is that it’s often acceptable to be less efficient if the code we have to write for adding one feature is smaller, easier to maintain/upgrade and does not require releasing a new version to make this particular feature available. Personally, I’d always prefer to write a G’MIC custom command (so, a script that I can directly put in the G’MIC standard library) if possible, instead of adding the same feature as a new “native” built-in command (in C++). But maybe their -fx operator was that slow it was cumbersome to use in practice? I had to try!

And I’m a bit sorry to say this, but yes, it’s quite slow (and I have tested this on my pretty fast machine with 12 HT cores at 2.6Ghz). The ImageMagick -fx operator is able to use multiple cores which is clearly a good thing, but even with that, it is cumbersome to use on reasonably big images, with complex math expressions. In a sense, that reassures me about the usefulness of having developed my own math expression compiler in CImg. This pre-compilation step of the math expression into a shorter bytecode sequence seems then to be almost mandatory. I’ve done a quick timing comparison for some simple image effects that can be achieved similarly with both expression evaluators of G’MIC and ImageMagick. Most of the examples below have been actually taken from the -fx documentation pages. I’m dividing and multiplying my image values by 255 in the G’MIC examples below because ImageMagick formulas assume that RGB values of the pixels are defined in range [0,1]. These tests have been done with a high-resolution input image (of a motorbike) with size 3072×2048, in RGB mode. I’ve checked that both the ImageMagick and G’MIC invokations render the same images.

# Test1: Apply a sigmoid contrast function on the image colors.

$ time convert motorbike.jpg -fx "(1.0/(1.0+exp(10.0*(0.5-u)))-0.006693)*1.0092503" im_sigmo.jpg

real	0m9.033s
user	3m18.527s
sys	0m2.604s

$ time gmic -verbose - motorbike.jpg -/ 255 -fill "(1.0/(1.0+exp(10.0*(0.5-i)))-0.006693)*1.0092503" -* 255 -o gmic_sigmo.jpg,75

real    0m0.474s
user    0m3.183s
sys     0m0.111s
# Test2: Create a radial gradient from scratch.
$ time convert -size 3072x2048 canvas: -fx "Xi=i-w/2; Yj=j-h/2; 1.2*(0.5-hypot(Xi,Yj)/70.0)+0.5" im_radial.jpg

real	0m29.895s
user	8m11.320s
sys	2m59.184s

$ time gmic -verbose - 3072,2048 -fill "Xi=x-w/2; Yj=y-h/2; 1.2*(0.5-hypot(Xi,Yj)/70.0)+0.5" -cut 0,1 -* 255 -o gmic_radial.jpg

real    0m0.234s
user    0m0.990s
sys     0m0.045s

# Test3: Create a keftales pattern gradient from scratch.
$ time convert -size 3072x2048 xc: -channel G -fx  'sin((i-w/2)*(j-h/2)/w)/2+.5' im_gradient.jpg

real	0m2.951s
user	1m2.310s
sys	0m0.853s

$ time gmic -verbose - 3072,2048 -fill "sin((x-w/2)*(y-h/2)/w)/2+.5" -* 255 -o gmic_gradient.jpg

real    0m0.302s
user    0m1.164s
sys     0m0.061s
# Test4: Compute mirrored image along the X-axis.
$ time convert motorbike.jpg -fx 'p{w-i-1,j}' im_mirror.jpg 2>&1

real	0m4.409s
user	1m33.702s
sys	0m1.254s

$ time gmic -verbose - motorbike.jpg -fill "i(w-x-1,y)" -o gmic_mirror.jpg

real    0m0.495s
user    0m1.367s
sys     0m0.106s

The pre-compilation of the math expressions clearly makes a difference!

I would be really interested to compare the expression evaluators for more complex expressions, as the one I’ve used to compute Julia fractals with G’MIC for instance. I don’t have a deep knowledge of the ImageMagick syntax, so I don’t know what would be the equivalent command line. If you have any idea on how to do that, please let me know! I’d be interested also to get an idea on how Matlab is performing for the same kind of equations.

7. Conclusion and perspectives

What I conclude from all of this ? Well, I’m actually pretty excited by what the latest version of my expression evaluator integrated in G’MIC / CImg can finally do. It looks like it runs at a decent speed, at least compared to the one used in ImageMagick (which is definitely a reference project for image processing). I had also the idea of comparing it with GraphicsMagick, but I must admit I didn’t find the same -fx operator in it. And I didn’t find something similar (Maybe you could help teaching me how it works for GraphicsMagick?).

I’ve been already able to propose one (simple) artistic filter that I find interesting (Vector painting), and I’m very confident that these improvements of the math expression evaluator will open a lot of new possibilities for G’MIC. For allowing the design of new filters for everyone of course, but also to make my algorithm prototyping work easier and faster.

Could it be the beginning of a new boost for G’MIC? What do you think?