G’MIC 2.2 : New features and filters!

The IMAGE team of the GREYC laboratory (UMR CNRS 6072, Caen, France) is pleased to announce the release of a new 2.2 version of G’MIC, its open-source, generic, and extensible framework for image processing. As we already did in the past, we take this opportunity to look at the latest notable features added since the previous major release (2.0, last June).

Note 1: click on a picture to view a larger version.
Note 2: This is a translation of an original article, in French, published on Linuxfr.

1. Context and recent evolutions

G’MIC is a free and open-source software developed since August 2008 (distributed under the CeCILL license), by folks in the IMAGE team at the GREYC, a French public research laboratory located in Caen and supervised by three institutions: the CNRS, the University of Caen, and the ENSICAEN engineering school. This team is made up of researchers and lecturers specialized in the fields of algorithms and mathematics for image processing.
As one of the main developer of G’MIC, I wanted to sum up the work we’ve made on this software during these last months.

G'MIC logo
Fig. 1.1: The G’MIC project logo, and its cute little mascot “Gmicky” (designed by David Revoy).

G’MIC is multi-platform (GNU/Linux, MacOS, Windows …) and provides many ways of manipulating generic image data, i.e. still images or image sequences acquired as hyperspectral 2D or 3D floating-point arrays (including usual color images). More than 950 different image processing functions are already available in the G’MIC framework, this number being expandable through the use of the G’MIC scripting capabilities.

G'MIC plugin for GIMP
Fig.1.2: The G’MIC-Qt plugin for GIMP, currently the most popular G’MIC interface.

Since the last major version release there have been two important events in the project life:

1.1. Port of the G’MIC-Qt plugin to Krita

When we released version 2.0 of G’MIC a few months ago, we were happy to announce a complete rewrite (in Qt) of the plugin code for GIMP. An extra step has been taken, since this plugin has been extended to fit into the open-source digital painting software Krita.
This has been made possible thanks to the development work of Boudewijn Rempt (maintainer of Krita) and S√©bastien Fourey (developer of the plugin). The G’MIC-Qt plugin is now available for Krita versions 3.3+ and, although it does not yet implement all the I/O functionality of its GIMP counterpart, the feedback we’ve had so far is rather positive.
This new port replaces the old G’MIC plugin for Krita which has not been maintained for some time. The good news for Krita users (and developers) is that they now have an up-to-date plugin whose code is common with the one running in GIMP and for which we will be able to ensure the maintenance and further developments.
Note this port required the writing of a source file host_krita.cpp (in C++) implementing the communication between the host software and the plugin, and it is reasonable to think that a similar effort would allow other programs to get their own version of the G’MIC plugin (and the 500 image filters that come with it!).

G'MIC for Krita
Fig. 1.3: Overview of the G’MIC-Qt plugin running on Krita.

1.2. CeCILL-C, a more permissive license

Another major event concerns the new license of use : The CeCILL-C license (that is in the spirit of the LGPL) is now available for some components of the G’MIC framework. This license is more permissive than the previously proposed CeCILL license (which is GPL-compatible) and is more suitable for the distribution of software libraries. This license extension (now double licensing) applies precisely to the core files of G’MIC, i.e. its C++ library libgmic. Thus, the integration of the libgmic features (therefore, all G’MIC image filters) is now allowed in software that are not themselves licensed under GPL/CeCILL (including closed source products).
The source code of the G’MIC-Qt plugin, meanwhile, remains distributed under the single CeCILL license (GPL-like).

2. Fruitful collaboration with David Revoy

If you’ve followed us for a while, you may have noticed that we very often refer to the work of illustrator David Revoy for his multiple contributions to G’MIC: mascot design, ideas of filters, articles or video tutorials, tests of all kinds, etc. More generally, David is a major contributor to the world of free digital art, as much with the comic Pepper & Carrot he produces (distributed under free license CC -BY), as with his suggestions and ongoing bug reports for the open-source software he uses.
Therefore, it seems quite natural to devote a special section to him in this article, summarizing the different ideas, contributions and experiments he has brought to G’MIC just recently. A big thank you, David for your availability, the sharing of your ideas, and for all your work in general!

2.1. Improving the lineart colorization filter

Let’s first mention the progress made on the Black & White / Colorize lineart (smart-coloring) filter that had appeared at the time of the 2.0 G’MIC release.
This filter is basically a lineart colorization assistant which was developed in collaboration with David. It tries to automatically generate a colorization layer for a given lineart, from the analysis of the contours and the geometry of that lineart. Following David‘s suggestions, we were able to add a new colorization mode, named “Autoclean“. The idea is to try to automatically “clean” a coloring layer (made roughly by the user) provided in addition to the lineart layer, using the same geometric analysis as for the previous colorization modes.
The use of this new mode is illustrated below, where a given lineart (left) has been colorized approximately by the user. From the two layers line art + color layer, our “Autoclean” algorithm generates an image (right), where the colors do not overflow the lineart contours (even for “virtual” contours that are not closed). The result is not always perfect, but nevertheless reduces the time spent in the tedious process of colorization.

Fig. 2.1: The new “Autoclean” mode of the lineart colorization filter can automatically “clean” a rough colorization layer.

Note that this filter is also equipped with a new hatch detection module, which makes it possible to avoid generating too many small areas when using the previously available random colorization mode, particularly when the input lineart contains a large number of hatches (see figure below).

Fig. 2.2: The new hatching detection module limits the number of small colored areas generated by the automatic random coloring mode.

2.2. Color equalizer in HSI, HSL and HSV spaces

More recently, David suggested the idea of a filter to separately vary the hue and saturation of colors having certain levels of luminosity. The underlying idea is to give the artist the ability to draw or paint digitally using only grayscale, then colorize his masterpiece afterwards by re-assigning specific colors to the different gray values of the image. The obtained result has of course a limited color range, but the overall color mood is already in place. The artist only has to retouch the colors locally rather than having to colorize the entire painting by hand.
The figure below illustrates the use of this new filter Colors/Equalize HSI/HSL/HSV available in the G’MIC plugin : each category of values can be finely adjusted, resulting in preliminary colorizations of black and white paintings.

Equalize HSI1
Equalize HSI2
Equalize HSI3
Fig. 2.3: Equalization in HSI/HSL/HSV colorspaces allows to easily set the global color mood for B&W paintings.

Note that the effect is equivalent to applying a color gradient to the different gray values of the image. This is something that could already be done quite easily in GIMP. But the main interest here is we can ensure that the pixel brightness remains unchanged during the color transformation, which is not an obvious property to preserve when using a gradient map.
What is nice about this filter is that it can apply to color photographs as well. You can change the hue and saturation of colors with a certain brightness, with an effect that can sometimes be surprising, like with the landscape photography shown below.

Equalize HSI4
Fig. 2.4: The filter “Equalize HSI/HSL/HSV” applied on a color photograph makes it possible to change the colorimetric environment, here in a rather extreme way.

2.3. Angular deformations

Another one of the David‘s ideas concerned the development of a random local deformation filter, having the ability to generate angular deformations. From an algorithmic point of view, it seemed relatively simple to achieve.
Note that once the implementation has been done (in concise style: 12 lines!) and pushed into the official filter updates, David just had to press the “Update Filters” button of his G’MIC-Qt plug-in, and the new effect Deformations/Crease was there immediately for testing. This is one of the practical side of developing new filters using the G’MIC script language!

G'MIC Crease
Fig. 2.5: New effect “Crease” for local angular deformations.

However, I must admit I didn’t really have an idea on what this could be useful for in practice. But the good thing about cooperating with David is that HE knows exactly what he’s going to do with it! For instance, to give a crispy look to the edges of his comics, or for improving the render of his alien death ray.

G'MIC Crease 2
G'MIC Crease 3
Fig. 2.6: Using the G’MIC “Crease” filter for two real cases of artistic creation.

3. Filters, filters, filters…

David Revoy is not the only user of G’MIC: we sometimes count up to 900 daily downloads from the main project website. So it happens, of course, that other enthusiastic users inspire us new effects, especially during those lovely discussions that take place on our forum, kindly made available by the PIXLS.US community.

3.1. Bring out the details without creating “halos”

Many photographers will tell you that it is not always easy to enhance the details in digital photographs without creating naughty artifacts that often have to be masked manually afterwards. Conventional contrast enhancement algorithms are most often based on increasing the local variance of pixel lightness, or on the equalization of their local histograms. Unfortunately, these operations are generally done by considering neighborhoods with a fixed size and geometry, where each pixel of a neighborhood is always considered with the same weight in the statistical calculations related to these algorithms.
It is simpler and faster, but from a qualitative point of view it is not an excellent idea: we often get “halos” around contours that were already very contrasted in the image. This classic phenomenon is illustrated below with the application of the Unsharp mask filter (the one present by default in GIMP) on a part of a landscape image. This generates an undesirable “halo” effect at the frontier between the mountain and the sky (this is particularly visible in full resolution images).

G'MIC details filters
Fig. 3.1: Unwanted “halo” effects often occur with conventional contrast enhancement filters.

The challenge of the detail enhancement algorithms is to be able to analyze the geometry of the local image structures in a more fine way, to take into account geometry-adaptive local weights for each pixel of a given neighborhood. To make it simple, we want to create anisotropic versions of the usual enhancement methods, orienting them by the edges detected in the images.
Following this logic, we have added two new G’MIC filters recently, namely Details/Magic details and Details/Equalize local histograms, which try to better take the geometric content of the image into account for local detail enhancement (e.g. using the bilateral filter).

G'MIC magic details
G'MIC equalize local histograms
G'MIC equalize local histograms
Fig. 3.2: The new G’MIC detail enhancement filters.

Thus, the application of the new G’MIC local histogram equalization on the landscape image shown before gives something slightly different : a more contrasted result both in geometric details and colors, and reduced halos.

G'MIC magic details
G'MIC magic details
Fig. 3.3: Differences of results between the standard GIMP Unsharp Mask filter and the local histogram equalization of G’MIC, for details enhancement.

3.2. Different types of image deformations

New filters to apply geometric deformations on images are added to G’MIC on a regular basis, and this new major version 2.2 offers therefore a bunch of new deformation filters.
So let’s start with Deformations/Spherize, a filter which allows to locally distort an image to give the impression that it is projected on a 3D sphere or ellipsoid. This is the perfect filter to turn your obnoxious office colleague into a Mr. Potato Head!

G'MIC spherize
G'MIC spherize
Fig .3.4: Two examples of 3D spherical deformations obtained with the G’MIC “Spherize” filter.

On the other hand, the filter Deformations/Square to circle implements the direct and inverse transformations from a square domain (or rectangle) to a disk (as mathematically described on this page), which makes it possible to generate this type of deformations.

G'MIC square to circle
Fig. 3.5: Direct and inverse transformations from a square domain to a disk.

The effect Degradations/Streak replaces an image area masked by the user (filled with a constant color) with one or more copies of a neighboring area. It works mainly as the GIMP clone tool but prevents the user to fill the entire mask manually.

G'MIC streak
Fig. 3.6: The “Streak” filter clones parts of the image into a user-defined color mask.

3.3. Artistic Abstractions

You might say that image deformations are nice, but sometimes you want to transform an image in a more radical way. Let’s introduce now the new effects that turn an image into a more abstract version (simplification and re-rendering). These filters have in common the analysis of the local image geometry, followed by a step of image synthesis.

For example, G’MIC filter Contours/Super-pixels locally gathers the image pixels with the same color to form a partitioned image, like a puzzle, with geometric shapes that stick to the contours. This partition is obtained using the SLIC method (Simple Linear Iterative Clustering), a classic image partitioning algorithm, which has the advantage of being relatively fast to compute.

G'MIC super pixels 1
G'MIC super pixels 2
Fig. 3.7: Decomposition of an image in super-pixels by the Simple Linear Iterative Clustering algorithm (SLIC).

The filter Artistic/Linify tries to redraw an input image by superimposing semi-transparent colored lines on an initially white canvas, as shown in the figure below. This effect is the re-implementation of the smart algorithm initially proposed on the site http://linify.me (initially implemented in JavaScript).

G'MIC linify 1
G'MIC linify 2
Fig. 3.8: The “Linify” effect tries to redraw an image by superimposing only semi-transparent colored lines on a white canvas.

The effect Artistic/Quadtree variations first decomposes an image as a quadtree, then re-synthesize it by drawing oriented and plain ellipses on a canvas, one ellipse for each quadtree leaf. This renders a rather interesting “painting” effect. It is likely that with more complex shapes, even more attractive renderings could be synthesized. Surely an idea to keep in mind for the next filters update ūüôā

G'MIC quadtree 1
G'MIC quadtree 2
Fig. 3.9: Decomposing an image as a quadtree allows to re-synthesize it by superimposing only plain colored ellipses.

3.4. “Are there any more?”

And now that you have processed so many beautiful pictures, why not arrange them in the form of a superb photo montage? This is precisely the role of the filter Arrays & tiles/Drawn montage, which allows to create a juxtaposition of photographs very quickly, for any kind of shapes.
The idea is to provide the filter with a colored template in addition to the serie of photographs (Fig.3.10a), and then to associate each photograph with a different color of the template (Fig.3.10b). Next, the arrangement is done automatically by G’MIC, by resizing the images so that they appear best framed within the shapes defined in the given montage template (Fig.3.10c).
We made a video tutorial illustrating the use of this specific filter.

G'MIC drawn montage
Fig. 3.10a: Step 1: The user draws the desired organization of the montage with shapes of different colors.

G'MIC drawn montage
Fig. 3.10b: Step 2: G’MIC’s “Drawn Montage” filter allows you to associate a photograph for each template color.

G'MIC drawn montage
Fig. 3.10c: Step 3: The photo montage is then automatically synthetized by the filter.

But let’s go back to more essential questions: have you ever needed to draw gears? No?! It’s quite normal, that’s not something we do everyday! But just in case, the new G’MIC filter Rendering/Gear will be glad to help, with different settings to adjust gear size, colors and number of teeth. Perfectly useless, so totally indispensable!

G'MIC drawn montage
Fig. 3.11: The Gear filter, running at full speed.

Need a satin texture right now? No?! Too bad, the filter Patterns / Satin could have been of a great help!

G'MIC satin
Fig. 3.12: G’MIC’s satin filter will make your life more silky.

And finally, to end up with the series of these “effects that are useless until we need them”, note the apparition of the new filter Degradations/JPEG artifacts which simulates the appearance of JPEG compression artifacts due to the quantization of the DCT coefficients encoding 8√ó8 image blocks (yes, you will get almost the same result saving your image as a JPEG file with the desired quality).

Simulate JPEG Artifacts
Simulate JPEG Artifacts
Fig. 3.13: The “JPEG artifacts” filter simulates the image degradation due to 8√ó8 block DCT compression.

4. Other notable improvements

This review of these new available G’MIC filters should not overshadow the various improvements that have been made “under the hood” and that are equally important, even if they are less visible in practice for the user.

4.1. A better G’MIC-Qt plugin interface

A big effort of cleaning and restructuring the G’MIC-Qt plugin code has been realized, with a lot of little inconsistencies fixed in the GUI. Let’s also mention in bulk order some new interesting features that have appeared in the plugin:

  • The ability to set a timeout when trying to preview some computationnaly intensive filters.
  • A better management of the input-output parameters for each filter (with persistence, better menu location, and a reset button).
  • Maximizing the size of the preview area is now easier. Editing its zoom level manually is now possible, as well as chosing the language of the interface (regardless of the language used for the system), etc.

All these little things gathered together globally improves the user experience.

G'MIC Preferences
Fig. 4.1: Overview of the G’MIC-Qt plugin interface in its latest version 2.2.

4.2. Improvements in the G’MIC core

Even less visible, but just as important, many improvements have appeared in the G’MIC computational core and its associated G’MIC script language interpreter. You have to know that all of the available filters are actually written in the form of scripts in the G’MIC language, and each small improvement brought to the interpreter may have a beneficial consequence for all filters at once. Without going too much into the technical details of these internal improvements, we can highlight those points:

  • The notable improvement in the syntax of the language itself, which goes along with better performances for the analysis of the language syntax (therefore for the script executions), all this with a smaller memory footprint.
  • The G’MIC built-in mathematical expression evaluator is also experiencing various optimizations and new features, to consider even more possibilities for performing non-trivial operations at the pixel level.

  • A better support of raw video input/outputs (.yuv format) with support for4:2:2 and 4:4:4 formats, in addition to4:2:0 which was the only mode supported before.

  • Finally, two new animations have been added to the G’MIC demos menu (which is displayed e.g. when invoking gmic without arguments from the command-line):

    • First, a 3D starfield animation:

    Starfield demo
    Fig.4.2: New 3D starfield animation added to the G’MIC demo menu.

    Hanoi Demo
    Fig. 4.3: The playable 3D version of the “Tower of Hanoi”, available in G’MIC.

  • Finally, let us mention the introduction of the command tensors3d dedicated to the 3D representation of second order tensor fields. In practice, it does not only serve to make you want to eat Smarties¬ģ! It can be used for example to visualize certain regions of MRI volumes of diffusion tensors:

Fig. 4.4: G’MIC rendering of a 3D tensor field, with command tensors3d.

4.3. New design for G’MIC Online

To finish this tour, let us also mention the complete redesign of G’MIC Online during the year 2017, done by Christophe Couronne and V√©ronique Robert from the development departement of the GREYC laboratory.
G’MIC Online is a web service allowing you to apply a subset of G’MIC filters on your images, directly inside a web browser. These web pages now have a responsive design, which makes them more enjoyable than before on mobile devices (smartphones and tablets). Shown below is a screenshot of this service running in Chrome/Android, on a 10” tablet.

Fig. 4.5: New responsive design of the G’MIC Online web service, running here on a 10″ tablet.

5. Conclusion and perspectives

The overview of this new version 2.2 of G’MIC is now over.
One possible conclusion could be: “There are plenty of perspectives!“.

G’MIC is a free project that can be considered as mature: the first lines of code were composed almost ten years ago, and today we have a good idea of the possibilities (and limits) of the beast. We hope to see more and more interest from FOSS users and developers, for example for integrating the G’MIC-Qt generic plugin in various software focused on image or video processing.

The possibility of using the G’MIC core under a more permissive CeCILL-C license can also be a source of interesting collaborations in the future (some companies have already approached us about this). While waiting for potential collaborations, we will do our best to continue developping G’MIC and feed it with new filters and effects, according to the suggestions of our enthusiastic users. A big thanks to them for their help and constant encouragement (the motivation to write code or articles, past 11pm, would not be the same without them!).

“Long live to open-source image processing and artistic creation!”

G’MIC 2.0: A second breath for open-source image processing

Disclaimer: This article is a duplicate of this post, originally published on the Pixls.us website, by the same authors.

The IMAGE team of the research laboratory GREYC in Caen/France is pleased to announce the release of a new major version (numbered 2.0) of its project G’MIC: a generic, extensible, and open source framework for image processing. Here, we present the main advances made in the software since our last article. The new features presented here include the work carried out over the last twelve months (versions 2.0.0 and 1.7.x, for x varying from 2 to 9).


1. G’MIC: A brief overview

G’MIC is an open-source project started in August 2008, by the IMAGE team. This French research team specializes in the fields of algorithms and mathematics for image processing. G’MIC is distributed under the CeCILL license (which is GPL compatible) and is available for multiple platforms (GNU/Linux, MacOS and Windows). It provides a variety of user interfaces for manipulating generic image data, that is to say, 2D or 3D multispectral images (or sequences) with floating-point pixel values. This includes, of course, “classic” color images.

G'MIC logo Fig.1.1: Logo of the G’MIC project, an open-source framework for image processing, and its mascot Gmicky.

The popularity of G’MIC mostly comes from the plug-in it provides for GIMP (since 2009). To date, there are more than 480 different filters and effects to apply to your images, which considerably enlarges the list of image processing filters available by default in GIMP.

G’MIC also provides a powerful and autonomous command-line interface, which is complementary to the CLI tools you can find in the famous ImageMagick or GraphicsMagick projects. There is also a web service G’MIC Online, allowing to apply image processing effects directly from a browser. Other (but less well known) G’MIC-based interfaces exist: a webcam streaming tool ZArt, a plug-in for Krita, a subset of filters available in Photoflow, Blender or Natron… All these interfaces are based on the CImg and libgmic libraries, that are portable, thread-safe and multi-threaded, via the use of OpenMP.

G’MIC has more than 950 different and configurable processing functions, for a library of only 6.5Mio, representing a bit more than 180 kloc. The processing functions cover a wide spectrum of the image processing field, offering algorithms for geometric manipulations, colorimetric changes, image filtering (denoising and detail enhancement by spectral, variational, non-local methods, etc.), motion estimation and registration, display of primitives (2D or 3D mesh objects), edge detection, object segmentation, artistic rendering, etc. It is therefore a very generic tool for various uses, useful on the one hand for converting, visualizing and exploring image data, and on the other hand for designing complex image processing pipelines and algorithms (see these project slides for details).

2. A new versatile interface, based on Qt

One of the major new features of this version 2.0 is the re-implementation of the plug-in code, from scratch. The repository G’MIC-Qt developed by S√©bastien (an experienced member of the team) is a Qt-based version of the plug-in interface, being as independent as possible of the widget API provided by GIMP.

G'MIC-Qt plug-in 1 Fig.2.1: Overview of version 2.0 of the G’MIC-Qt plug-in running for GIMP.

This has several interesting consequences:

  • The plug-in uses its own widgets (in Qt) which makes it possible to have a more flexible and customizable interface than with the GTK widgets used by the GIMP plug-in API: for instance, the preview window becomes resizable at will, manages zooming by mouse wheel, and can be freely moved to the left or to the right. A filter search engine by keywords has been added, as well as the possibility of choosing between a light or dark theme. The management of favorite filters has been also improved and the interface even offers a new mode for setting the visibility of the filters. Interface personalization is now a reality.

  • The plug-in also defines its own API, which is used to facilitate its integration in third party software (other than GIMP). In practice, a software developer has to write a single file host_software.cpp implementing the functions of the API to make the link between the plug-in and the host application. Currently, the file host_gimp.cpp does this for GIMP as a host. But there is now also a stand-alone version available (file host_none.cpp that runs this Qt interface in solo mode, from a shell (with command gmic_qt).

  • Boudewijn Rempt, project manager and developer of the marvelous painting software Krita, has also started writing such a file host_krita.cpp to make this “new generation” plug-in communicate with Krita. In the long term, this should replace the previous G’MIC plug-in implementation they made (currently distributed with Krita), which is aging and poses maintenance problems for developers.

Minimizing the integration effort for developers, sharing the G’MIC plug-in code between different applications, and offering a user interface that is as comfortable as possible, have been the main objectives of this complete redesign. As you can imagine, this rewriting required a long and sustained effort, and we can only hope that this will raise interest among other software developers, where having a consistent set of image processing filters could be useful (a file host_blender.cpp available soon ? We can dream!). The animation below illustrates some of the features offered by this new Qt-based interface.

G'MIC-Qt plug-in 2 Fig.2.2: The new G’MIC-Qt interface in action.

Note that the old plug-in code written in GTK was updated also to work with the new version 2.0 of G’MIC, but has fewer features and probably will not evolve anymore in the future, unlike the Qt version.

3. Easing the work of cartoonists…

One of G’MIC’s purposes is to offer more filters and functions to process images. And that is precisely something where we have not relaxed our efforts, despite the number of filters already available in the previous versions!

In particular, this version comes with new and improved filters to ease the colorization of line-art. Indeed, we had the chance to host the artist David Revoy for a few days at the lab. David is well known to lovers of art and free software by his multiple contributions in these fields (in particular, his web comic Pepper & Carrot is a must-read!). In collaboration with David, we worked on the design of an original automatic line-art coloring filter, named Smart Coloring.

Smart coloring 1 Fig.3.1: Use of the “Colorize line-art [smart coloring]” filter in G’MIC.

When drawing comics, the colorization of line-art is carried out in two successive steps: The original drawing in gray levels (Fig.3.2.1) is first pre-colored with solid areas, i.e. by assigning a unique color to each region or distinct object in the drawing (Fig.3.2.3). In a second step, the colourist reworks this pre-coloring, adding shadows, lights and modifying the colorimetric ambiance, in order to obtain the final colorization result (Fig.3.2.4). Practically, flat coloring results in the creation of a new layer that contains only piecewise constant color zones, thus forming a colored partition of the plane. This layer is then merged with the original line-art to get the colored rendering (merging both in multiplication mode, typically).

Smart coloring 2 Fig.3.2: The different steps of a line-art coloring process (source: David Revoy).

Artists admit it themselves: flat coloring is a long and tedious process, requiring patience and precision. Classical tools available in digital painting or image editing software do not make this task easy. For example, even most filling tools (bucket fill) do not handle discontinuities in drawn lines very well (Fig.3.3.a), and even worse when lines are anti-aliased. It is then common for the artist to perform flat coloring by painting the colors manually with a brush on a separate layer (Fig.3.3.b), with all the precision problems that this supposes (especially around the contour lines, Fig.3.3.c). See also this link for more details.

Smart coloring 3 Fig.3.3: Classical problems encountered when doing flat coloring (source: David Revoy).

It may even happen that the artist decides to explicitly constrain his style of drawing, for instance by using aliased brushes in a higher resolution image, and/or by forcing himself to draw only connected contours, in order to ease the flat colorization work that has to be done afterwards.

The Smart Coloring filter developed in version 2.0 of G’MIC allows to automatically pre-color an input line-art without much work. First, it analyses the local geometry of the contour lines (estimating their normals and curvatures). Second, it (virtually) does contour auto-completion using spline curves. This virtual closure allows then the algorithm to fill objects with disconnected contour plots. Besides, this filter has the advantage of being quite fast to compute and gives coloring results of similar quality to more expensive optimization techniques used in some proprietary software. This algorithm smoothly manages anti-aliased contour lines, and has two modes of colorization: by random colors (Fig.3.2.2 and Fig.3.4) or guided by color markers placed beforehand by the user (Fig.3.5).

Smart coloring 4 Fig.3.4: Using the G’MICSmart Coloring” filter in random color mode, for line-art colorization (source: David Revoy).

In “random” mode, the filter generates a piecewise constant layer that is very easy to recolor with correct hues afterwards. This layer indeed contains only flat color regions, and the classic bucket fill tool is effective here to quickly reassign a coherent color to each existing region synthesized by the algorithm.

In the user-guided markers mode, color spots placed by the user are extrapolated in such a way that it respects the geometry of the original drawing as much as possible, taking into account the discontinuities in the pencil lines, as this is clearly illustrated by the figure below:

Smart coloring 5 Fig.3.5: Using the G’MICSmart Coloring” filter in user-guided color markers mode, for line-art colorization (source: David Revoy).

This innovative, flat coloring algorithm has been pre-published on HAL (in French): A semi-guided high-performance flat coloring algorithm for line-arts. Curious people could find there all the technical details of the algorithm used. The recurring discussions we had with David Revoy on the development of this filter enabled us to improve the algorithm step by step, until it became really usable in production. This method has been used successfully (and therefore validated) for the pre-colorization of the whole episode 22 of the webcomic Pepper & Carrot.

The wisest of you know that G’MIC already had a line-art colorization filter! True, but unfortunately it did not manage disconnected contour lines so well (such as the example in Fig.3.5), and could then require the user to place a large number of color spots to guide the algorithm properly. In practice, the performance of the new flat coloring algorithm is far superior.

And since it does not see any objection to anti-aliased lines, why not create ones? That is the purpose of another new filter “Repair / Smooth [antialias]” able to add anti-aliasing to lines in cartoons that would have been originally drawn with aliased brushes.

Smooth [antialias] Fig.3.6: Filter “Smooth [antialias]” smooths contours to reduce aliasing effect in cartoons (source: David Revoy).

4. …Not to forget the photographers!

“Colorizing drawings is nice, but my photos are already in color!”, kindly remarks the impatient photographer. Don’t be cruel! Many new filters related to the transformation and enhancement of photos have been also added in G’MIC 2.0. Let’s take a quick look of what we have.

4.1. CLUTs and colorimetric transformations

CLUTs (Color Lookup Tables) are functions for colorimetric transformations defined in the RGB cube: for each color (Rs,Gs,Bs) of a source image Is, a CLUT assigns a new color (Rd,Gd,Bd) transferred to the destination image Id at the same position. These processing functions may be truly arbitrary, thus very different effects can be obtained according to the different CLUTs used. Photographers are therefore generally fond of them (especially since these CLUTs are also a good way to simulate the color rendering of certain old films).

In practice, a CLUT is stored as a 3D volumetric color image (possibly “unwrapped” along the z = B axis to get a 2D version). This may quickly become cumbersome when several hundreds of CLUTs have to be managed. Fortunately, G’MIC has a quite efficient CLUT compression algorithm (already mentioned in a previous article), which has been improved version after version. So it was finally in a quite relax atmosphere that we added more than 60 new CLUT-based transformations in G’MIC, for a total of 359 CLUTs usable, all stored in a data file that does exceed 1.2 Mio. By the way, let us thank Pat David, Marc Roovers and Stuart Sowerby for their contributions to these color transformations.

CLUTs Fig.4.1.1: Some of the new CLUT-based transformations available in G’MIC (source: Pat David).

But what if you already have your own CLUT files and want to use them in GIMP? No problem‚ÄĮ! The new filter “Film emulation / User-defined” allows to apply such transformations from CLUT data file, with a partial support of files with extension .cube (CLUT file format proposed by Adobe, and encoded in ASCII o_O!).

And for the most demanding, who are not satisfied with the existing pre-defined CLUTs, we have designed a very versatile filter “Colors / Customize CLUT“, that allows the user to build their own custom CLUT from scratch: the user places colored keypoints in the RGB color cube and these markers are interpolated in 3D (according to a Delaunay triangulation) in order to rebuild a complete CLUT, i.e. a dense function in RGB. This is extremely flexible, as in the example below, where the filter has been used to change the colorimetric ambiance of a landscape, mainly altering the color of the sky. Of course, the synthesized CLUT can be saved as a file and reused later for other photographs, or even in other software supporting this type of color transformations (for example RawTherapee or Darktable).

Customize CLUT 1 Fig.4.1.2: Filter “Customize CLUT” used to design a custom color transform in the RGB cube.

Customize CLUT 2 Fig.4.1.3: Result of the custom colorimetric transformation applied to a landscape.

To stay in the field of color manipulation, let us also mention the appearance of the filter “Colors / Retro fade” which creates a “retro” rendering of an image with grain generated by successive averages of random quantizations of an input color image.

Retro fade Fig.4.1.4: Filter “Retro fade” in the G’MIC plug-in.

4.2. Making the details pop out

Many photographers are looking for ways to process their digital photographs so as to bring out the smallest details of their images, sometimes even to exaggeration, and we can find some of them in the pixls.us forum. Looking at how they perform allowed us to add several new filters for detail and contrast enhancement in G’MIC. In particular, we can mention the filters “Artistic / Illustration look” and “Artistic / Highlight bloom“, which are direct re-implementations of the tutorials and scripts written by S√©bastien Guyader as well as the filter “Light & Shadows / Pop shadows” suggested by Morgan Hardwood. Being immersed in such a community of photographers and cool guys always gives opportunities to implement interesting new effects!

Illustration look Fig.4.2.1: Filters “Illustration look” and “Highlight bloom” applied to a portrait image.

In the same vein, G’MIC gets its own implementation of the Multi-scale Retinex algorithm, something that was already present in GIMP, but here enriched with additional controls to improve the luminance consistency in images.

Retinex Fig.4.2.2: Filter “Retinex” for improving luminance consistency.

Our friend and great contributor to G’MIC, J√©rome Boulanger, also implemented and added a dehazing filter “Details / Dcp dehaze” to attenuate the fog effect in photographs, based on the Dark Channel Prior algorithm. Setting the parameters of this filter is kinda hard, but the filter gives sometimes spectacular results.

DCP dehaze 1 DCP dehaze 2 Fig.4.2.3: Filter “DCP Dehaze” to attenuate the fog effect.

And to finish with this subsection, let us mention the implementation in G’MIC of the Rolling Guidance algorithm, a method to simplify images that has become a key step used in many newly added filters. This was especially the case in this quite cool filter for image sharpening, available in “Details / Sharpen [texture]“. This filter works in two successive steps: First, the image is separated into a texture component + a color component, then the details of the texture component only are enhanced before the image is recomposed. This approach makes it possible to highlight all the small details of an image, while minimizing the undesired halos near the contours, a recurring problem happening with more classical sharpening methods (such as the well known Unsharp Mask).

Sharpen [texture] Fig.4.2.4: The “Sharpen [texture]“” filter shown for two different enhancement amplitudes.

4.3. Masking by color

As you may know, a lot of photograph retouching techniques require the creation of one or several “masks”, that is, the isolation of specific areas of an image to receive differentiated processing. For example, the very common technique of luminosity masks is a way to treat differently shadows and highlights in an image. G’MIC 2.0 introduces a new interesting filter “Colors / Color mask [interactive]” that implements a relatively sophisticated algorithm (albeit computationally demanding) to help creating complex masks. This filter asks the user to hover the mouse over a few pixels that are representative of the region to keep. The algorithm learns in real time the corresponding set of colors or luminosities and deduces then the set of pixels that composes the mask for the whole image (using Principal Component Analysis on the RGB samples).

Once the mask has been generated by the filter, the user can easily modify the corresponding pixels with any type of processing. The example below illustrates the use of this filter to drastically change the color of a car

Color mask [interactive] Fig.4.3.1: Changing the color of a car, using the filter “Color mask [interactive]“.

It takes no more than a minute and a half to complete, as shown in the video below:

Fig.4.3.2: Changing the color of a car, using filter “Color mask [interactive]” (video tutorial).

This other video exposes an identical technique to change the color of the sky in a landscape.

Fig.4.3.3: Changing the color of the sky in a landscape, using filter “Color mask [interactive]” (video tutorial).

5. And for the others…

Since illustrators and photographers are now satisfied, let’s move on to some more exotic filters, recently added to G’MIC, with interesting outcomes!

5.1. Average and median of a series of images

Have you ever wondered how to easily estimate the average or median frame of a sequence of input images? The libre aficionado Pat David, creator of the site pixls.us often asked the question. First of all when he tried to denoise images by combining several shots of a same scene. Then he wanted to simulate a longer exposure time by averaging photographs taken successively. And finally, calculating averages of various kind of images for artistic purposes (for example, frames of music video clips, covers of Playboy magazine or celebrity portraits).

Hence, with his cooperation, we added new commands -median_files,-median_videos, -average_files and-average_videos to compute all these image features very easily using the CLI tool gmic. The example below shows the results obtained from a sub-sequence of the ¬ę Big Buck Bunny” video. We have simply invoked the following commands from the Bash shell:

$ gmic -average_video bigbuckbunny.mp4 -normalize 0.255 -o average.jpg
$ gmic -median_video bigbuckbunny.mp4 -normalize 0.255 -o median.jpg`

Big buck bunny 1 Fig.5.1.1: Sequence in the ¬ę Big Buck Bunny” video, directed by the Blender foundation.

Big buck bunny 2 Fig.5.1.2: Result: Average image of the ¬ę Big Buck Bunny” sequence above.

Big buck bunny 3 Fig.5.1.3: Result: Median image of the ¬ę Big Buck Bunny” sequence above.

And to stay in the field of video processing, we can also mention the addition of the commands -morph_files and -morph_video that render temporal interpolations of video sequences, taking the estimated intra-frame object motion into account, thanks to a quite smart variational and multi-scale estimation algorithm.

The video below illustrates the rendering difference obtained for the retiming of a sequence using temporal interpolation, with (right) and without (left) motion estimation.

Fig.5.1.4: Video retiming using G’MIC temporal morphing technique.

5.2. Deformations and “Glitch Art”

Those who like to mistreat their images aggressively will be delighted to learn that a bunch of new image deformation and degradation effects have appeared in G’MIC.

First of all, the filter “Deformations / Conformal maps” allows one to distort an image using conformal maps. These deformations have the property of preserving the angles locally, and are most often expressed as functions of complex numbers. In addition to playing with predefined deformations, this filter allows budding mathematicians to experiment with their own complex formulas.

Conformal maps Fig.5.2.1: Filter “Conformal maps” applying a angle-preserving transformation to the image of Mona Lisa.

Fans of Glitch Art may also be concerned by several new filters whose rendering look like image encoding or compression artifacts. The effect “Degradations / Pixel sort” sorts the pixels of a picture by row or by column according to different criteria and to possibly masked regions, as initially described on this page.

Pixel sort Fig.5.2.2: Filter “Pixel sort” for rendering a kind of “Glitch Art” effect.

Degradations / /Pixel sort also has two little brothers, filters “Degradations / Flip & rotate blocks” and “Degradations / Warp by intensity“. The first divides an image into blocks and allows to rotate or mirror them, potentially only for certain color characteristics (like hue or saturation, for instance).

Flip and rotate blocks Fig.5.2.3: Filter “Flip & rotate blocks” applied to the hue only to obtain a “Glitch Art” effect.

The second locally deforms an image with more or less amplitude, according to its local geometry. Here again, this can lead to the generation of very strange images.

Warp by intensity Fig.5.2.4: Filter “Warp by intensity” applied to the image of Mona Lisa (poor Mona!).

It should be noted that these filters were largely inspired by the Polyglitch plug-in, available for Paint.NET, and have been implemented after a suggestion from a friendly user (yes, yes, we try to listen to our most friendly users!).

5.3. Image simplification

What else do we have in store? A new image abstraction filter, Artistic / Sharp abstract, based on the Rolling Guidance algorithm mentioned before. This filter applies contour-preserving smoothing to an image, and its main consequence is to remove the texture. The figure below illustrates its use to generate several levels of abstraction of the same input image, at different smoothing scales.

Sharp abstract Fig.5.3.1: Creating abstractions of an image via the filter “Sharp abstract“.

In the same vein, G’MIC also gets a filter Artistic / Posterize which degrades an image to simulate posterization. Unlike the filter with same name available by default in GIMP (which mainly tries to reduce the number of colors, i.e. do color quantization), our version adds spatial simplification and filtering to approach a little more the rendering of old posters.

Posterize Fig.5.3.2: Filter “Posterize” of G’MIC, compared to the filter with same name available by default in GIMP.

5.4. Other filters

If you still want more (and in this case one could say you are damn greedy!), we will end this section by discussing some of the new, but unclassifiable filters.

We start with the filter “Artistic / Diffusion tensors“, which displays a field of diffusion tensors, calculated from the structure tensors of an image (structure tensors are symmetric and positive definite matrices, classically used for estimating the local image geometry). To be quite honest, this feature had not been originally developed for an artistic purpose, but users of the plug-in came across it by chance and asked to make a GIMP filter from it. And yes, this is finally quite pretty, isn’t it?

Diffusion tensors Fig.5.4.1: Filter “Diffusion Tensors” filter and its multitude of colored ellipses.

From a technical point of view, this filter was actually an opportunity to introduce new drawing features into the G’MIC mathematical evaluator, and it has now become quite easy to develop G’MIC scripts for rendering custom visualizations of various image data. This is what has been done for instance, with the command -display_quiver reimplemented from scratch, and which allows to generate this type of rendering:

-display_quiver Fig. 5.4.2: Rendering vector fields with the G’MIC command -display_quiver.

For lovers of textures, we can mention the apparition of two new fun effects: First, the “Patterns / Camouflage” filter. As its name suggests, this filter produces a military camouflage texture.

Camouflage Fig. 5.4.3: Filter “Camouflage“, to be printed on your T-shirts to go unnoticed in parties!

Second, the filter “Patterns / Crystal background” overlays several randomly colored polygons in order to synthesize a texture that vaguely looks like a crystal seen under a microscope. Pretty useful to quickly render colored image backgrounds.

Crystal background Fig.5.4.4: Filter “Crystal background” in action.

And to end this long overview of new G’MIC filters developed since last year, let us mention “Rendering / Barnsley fern“. This filter renders the well-known Barnsley fern fractal. For curious people, note that the related algorithm is available on Rosetta Code, with even a code version written in the G’MIC script language, namely:

# Put this into a new file 'fern.gmic' and invoke it from the command line, like this:
# $ gmic fern.gmic -barnsley_fern

barnsley_fern :
  -skip {"
    f1 = [ 0,0,0,0.16 ]; g1 = [ 0,0 ];
    f2 = [ 0.2,-0.26,0.23,0.22 ]; g2 = [ 0,1.6 ];
    f3 = [ -0.15,0.28,0.26,0.24 ]; g3 = [ 0,0.44 ];
    f4 = [ 0.85,0.04,-0.04,0.85 ]; g4 = [ 0,1.6 ];
    xy = [ 0,0 ];
    for (n = 0, n<2e6, ++n,
      r = u(100);
      xy = r<=1?((f1**xy)+=g1):
      uv = xy*200 + [ 480,0 ];
      uv[1][1] = h - uv[1][1];
      I(uv) = 0.7*I(uv) + 0.3*255;
  -r 40%,40%,1,1,2

And here is the rendering generated by this function:

Barnsley Fern Fig.5.4.5: Fractal “Barnsley fern“, rendered by G’MIC.

6. Overall project improvements

All filters presented throughout this article constitute only the visible part of the G’MIC iceberg. They are in fact the result of many developments and improvements made “under the hood”, i.e., directly on the code of the G’MIC script language interpreter. This interpreter defines the basic language used to write all G’MIC filters and commands available to users. Over the past year, a lot of work has been done to improve the performances and the capabilities of this interpreter:

  • The mathematical expressions evaluator has been considerably enriched and optimized, with more functions available (especially for matrix calculus), the support of strings, the introduction of const variables for faster evaluation, the ability to write variadic macros, to allocate dynamic buffers, and so on.

  • New optimizations have been also introduced in the CImg library, including the parallelization of new functions (via the use of OpenMP). This C++ library provides the implementations of the “critical” image processing algorithms and its optimization has a direct impact on the performance of G’MIC (in this respect, note that CImg is also released with a major version 2.0).

  • Compiling G’MIC on Windows now uses a more recent version of g++ (6.2 rather than 4.5), with the help of Sylvie Alexandre. This has actually a huge impact on the performances of the compiled executables: some filters run up to 60 times faster than with the previous binaries (this is the case for example, with the Deformations / Conformal Maps filter, discussed in section 5.2).

  • The support of large .tiff images (format BigTIFF, with files that can be larger than 4Gb) is now enabled (read and write), as it is for 64-bit floating-point TIFF images

  • The 3D rendering engine built into G’MIC has also been slightly improved, with the support for bump mapping. No filter currently uses this feature, but we never know, and prepare ourselves for the future!

    Bump mapping Fig.6.1: Comparison of 3D textured rendering with (right) and without “Bump mapping” (left).

  • And as it is always good to relax after a hard day’s work, we added the game of Connect Four to G’MIC :). It can be launched via the shell command $ gmic -x_connect4 or via the plug-in filter “Various / Games & demos / Connect-4“. Note that it is even possible to play against the computer, which has a decent but not unbeatable skill (the very simple AI uses the Minimax algorithm with a two-level decision tree).

    Connect four Fig.6.2: The game of “Connect Four“, as playable in G’MIC.

Finally, let us mention the undergoing redesign work of the G’MIC Online web service, with a beta version already available for testing. This re-development of the site, done by Christophe Couronne and V√©ronique Robert (both members of the GREYC laboratory), has been designed to better adapt to mobile devices. The first tests are more than encouraging. Feel free to experiment and share your impressions!

7. What to remember?

First, the version 2.0 of G’MIC is clearly an important step in the project life, and the recent improvements are promising for the future developments. It seems that the number of users are increasing (and they are apparently satisfied!), and we hope that this will encourage open-source software developers to integrate our new G’MIC-Qt interface as a plug-in for their own software. In particular, we are hopeful to see the new G’MIC in action under Krita soon, this would be already a great step!

Second, G’MIC continues to be an active project, and evolve through meetings and discussions with members of artists and photographers communities (particularly those who populate the forums and IRC of pixls.us and GimpChat). You will likely able to find us there if you need more information, or just if you want to discuss things related to (open-source) image processing.

And while waiting for a future hypothetical article about a future release of G’MIC, you can always follow the day-after-day progress of the project via our Twitter feed.

Until then, long live open-source image processing!

Credit: Unless explicitly stated, the various non-synthetic images that illustrate this post come from Pixabay.

G’MIC 1.7.1: When the flowers are blooming, image filters abound!

Disclaimer: This article is a duplicate of this post, originally published on the Pixls.us website, by the same authors.

Then we shall all burn together by Philipp Haegi.

ÔŅľ¬†A new version 1.7.1 ‚ÄúSpring 2016‚ÄĚ of G‚ÄôMIC (GREYC‚Äôs Magic for Image Computing),
the open-source framework for image processing, has been released recently (26 April 2016). This is a great opportunity to summarize some of the latest advances and features over the last 5 months.

G’MIC: A brief overview

G‚ÄôMIC is an open-source project started in August 2008. It has been developed in the IMAGE team of the GREYC laboratory from the CNRS (one of the major French public research institutes). This team is made up of researchers and teachers specializing in the algorithms and mathematics of image processing. G‚ÄôMIC is released under the free software licence CeCILL (GPL-compatible) for various platforms (Linux, Mac and Windows). It provides a set of various user interfaces for the manipulation of generic image data, that is images or image sequences of multispectral data being 2D or 3D, and with high-bit precision (up to 32bits floats per channel). Of course, it manages ‚Äúclassical‚ÄĚ color images as well.


Logo and (new) mascot of the G’MIC project, the open-source framework for image processing.

Note that the project just got a redesign of its mascot Gmicky, drawn by David Revoy, a French illustrator well-known to free graphics lovers for being responsible for the great libre webcomics Pepper&Carott. G’MIC is probably best known for it’s GIMP plug-in, first released in 2009. Today, this popular GIMP extension proposes more than 460 customizable filters and effects to apply on your images.


Overview of the G’MIC plug-in for GIMP.

But G’MIC is not a plug-in for GIMP only. It also offers a command-line interface, that can be used in addition with the CLI tools from ImageMagick or GraphicsMagick (this is undoubtly the most powerful and flexible interface of the framework). G’MIC also has a web service G’MIC Online to apply effects on your images directly from a web browser. Other G’MIC-based interfaces also exist (ZArt, a plug-in for Krita, filters for Photoflow…). All these interfaces are based on the generic C++ libraries CImg and libgmic which are portable, thread-safe and multi-threaded (through the use of OpenMP). Today, G’MIC has more than 900 functions to process images, all being fully configurable, for a library of only approximately 150 kloc of source code. It’s features cover a wide spectrum of the image processing field, with algorithms for geometric and color manipulations, image filtering (denoising/sharpening with spectral, variational or patch-based approaches…), motion estimation and registration, drawing of graphic primitives (up to 3d vector objects), edge detection, object segmentation, artistic rendering, etc. This is a versatile tool, useful to visualize and explore complex image data, as well as elaborate custom image processing pipelines (see these slides to get more information about the motivations and goals of the G’MIC project).

A selection of some new filters and effects

Here we look at the descriptions of some of the most significant filters recently added. We illustrate their usage from the G’MIC plug-in for GIMP. All of these filters are of course available from other interfaces as well (in particular within the CLI tool gmic).

Painterly rendering of photographs

The filter Artistic / Brushify tries to transform an image into a painting.¬†Here, the idea is to simulate the process of painting with brushes on a white canvas. One provides a template image¬†and the algorithm first analyzes the image geometry (local contrasts and orientations of the contours), then¬†attempt to reproduce the image with a single brush that will be locally rotated and scaled accordingly to the¬†contour geometry.¬†By simulating enough of brushstrokes, one gets a ‚Äúpainted‚ÄĚ version of the template image, which is more or less close to the original one,¬†depending on the brush shape, its size, the number of allowed orientations, etc.¬†All these settings being customizable by the user as parameters of the algorithm:¬†This filter allows thus to render a wide variety of painting effects.


Overview of the filter ‚ÄúBrushify‚ÄĚ in the G‚ÄôMIC plug-in GIMP. The brush that will be used by the algorithmis visible on the top left.

The animation below illustrates the diversity of results one can get with this filter, applied on the same input picture of a lion. Various brush shapes and geometries have been supplied to the algorithm. Brushify is computationally expensive so its implementation is parallelized (each core gives several brushstrokes simultaneously).


A few examples of renderings obtained with ‚ÄúBrushify‚ÄĚ from the same template image, but with different brushes and parameters.

Note that it’s particularly fun to invoke this filter from the command line interface (using the option -brushify available in gmic) to process a sequence of video frames (see this example of “ brushified “ video):

Reconstructing missing data from sparse samples

G’MIC gets a new algorithm to reconstruct missing data in images. This is a classical problem in image processing, often named “Image Inpainting“, and G’MIC already had a lot of useful filters to solve this problem. Here, the newly added interpolation method assumes only a sparse set of image data is known, for instance a few scattered pixels over the image (instead of continuous chuncks of image data). The analysis and the reconstruction of the global image geometry is then particularly tough.

The new option -solidify in G’MIC allows the reconstruction of dense image data from such a sparse sampling, based on a multi-scale diffusion PDE’s-based technique. The figure below illustrates the ability of the algorithm with an example of image reconstruction. We start from an input image of a waterdrop, and we keep only 2.7% of the image data (a very little amount of data!). The algorithm is able to reconstruct a whole image that looks like the input, even if all the small details have not been fully reconstructed (of course!). The more samples we have, the finer details we can recover.


Reconstruction of an image from a sparse sampling.

As this reconstruction technique is quite generic, several new G’MIC filters takes advantage of it:

  • Filter Repair / Solidify applies the algorithm in a direct manner, by reconstructing transparent areas¬†from the interpolation of opaque regions.¬†The animation below shows how this filter can be used to create an artistic blur on the image borders.

Overview of the ‚ÄúSolidify‚ÄĚ filter, in the G‚ÄôMIC plug-in for GIMP.

From an artistic point of view, there are many possibilities offered by this filters. For instance, it becomes really easy to generate color gradients with complex shapes, as shown with the two examples below (also in this video that details the whole process).


Using the ‚ÄúSolidify‚ÄĚ filter of G‚ÄôMIC to easily create color gradients with complex shapes (input images on the left, filter results on the right).

  • Filter Artistic / Smooth abstract uses same idea as the one with the waterdrop image:¬†it purposely sub-samples the image in a sparse way, by choosing keypoints mainly on the image edges, then use the reconstruction¬†algorithm to get the image back. With a low number of samples, the filter can only render a piecewise smooth image,¬†i.e. a smooth abstraction of the input image.

Overview of the ‚ÄúSmooth abstract‚ÄĚ filter in the G‚ÄôMIC plug-in for GIMP.

  • Filter Rendering / Gradient [random] is able to synthetize random colored backgrounds. Here again, the filter initializes¬†a set of colors keypoints randomly chosen over the image, then interpolate them with the new reconstruction algorithm.¬†We end up with a psychedelic background composed of randomly oriented color gradients.

Overview of the ‚ÄúGradient [random]‚ÄĚ filter in the G‚ÄôMIC plug-in for GIMP.

  • Simulation of analog films : the new reconstruction algorithm also allowed a major improvement¬†for all the analog film emulation filters that have been present in G‚ÄôMIC for years.¬†The section Film emulation/ proposes a wide variety of filters for this purpose. Their goal is to apply color transformations¬†to simulate the look of a picture shot by an analogue camera with a certain kind of film.¬†Below, you can see for instance a few of the 300 colorimetric transformations that are available in G‚ÄôMIC.

A few of the 300+ color transformations available in G’MIC.

From an algorithmic point of view, such a color mapping is extremely simple to implement : for each of the 300+ presets, G’MIC actually has an HaldCLUT, that is a function defining for each possible color (R,G,B) (of the original image), a new color (R’,G’,B’) color to set instead. As this function is not necessarily analytic, a HaldCLUT is stored in a discrete manner as a lookup table that gives the result of the mapping for all possible colors of the RGB cube (that is 2^24 = 16777216 values if we work with a 8bits precision per color component). This HaldCLUT-based color mapping is illustrated below for all values of the RGB color cube.


Principle of an HaldCLUT-based colorimetric transformation.

This is a large amount of data: even by subsampling the RGB space (e.g. with 6 bits per component) and compressing the corresponding HaldCLUT file, you ends up with approximately 200 and 300 kB for each mapping file. Multiply this number by 300+ (the number of available mappings in G’MIC), and you get a total of 85MB of data, to store all these color transformations. Definitely not convenient to spread and package!

The idea was then to develop a new lossy compression technique focused on HaldCLUT files, that is volumetric discretised vector-valued functions which are piecewise smooth by nature. And that what has been done in G’MIC, thanks to the new sparse reconstruction algorithm. Indeed, the reconstruction technique also works with 3D image data (such as a HaldCLUT!), so one simply has to extract a sufficient number of significant keypoints in the RGB cube and interpolate them afterwards to allow the reconstruction of a whole HaldCLUT (taking care to have a reconstruction error small enough to be sure that the color mapping we get with the compressed HaldCLUT is indistinguishable from the non-compressed one).


How the decompression of an HaldCLUT now works in G’MIC, from a set of colored keypoints located in the RGB cube.

Thus, G’MIC doesn’t need to store all the color data from a HaldCLUT, but only a sparse sampling of it (i.e. a sequence of { rgb_keypoint, new_rgb_color }). Depending on the geometric complexity of the HaldCLUTs to encode, more or less keypoints are necessary (roughly from 30 to 2000). As a result, the storage of the 300+ HaldCLUTs in G’MIC requires now only 850 KiB of data (instead of 85 MiB), that is a compression gain of 99% ! That makes the whole HaldCLUT data storable in a single file that is easy to ship with the G’MIC package. Now, a user can then apply all the G’MIC color transformations while being offline (previously, each HaldCLUT had to be downloaded separately from the G’MIC server when requested).

It looks like this new reconstruction algorithm from sparse samples is really great, and no doubts it will be used in other filters in the future.

Make textures tileable

Filter Arrays & tiles / Make seamless [patch-based] tries to transform an input texture to make it tileable, so that it can be duplicated as tiles along the horizontal and vertical axes¬†without visible seams on the borders of adjacent tiles.¬†Note that this is something that can be extremely hard to achieve, if the input texture has few auto-similarity or glaring luminosity changes spatially.¬†That is the case for instance with the ‚ÄúSalmon‚ÄĚ texture shown below as four adjacent tiles (configuration 2×2) with a lighting that goes from dark (on the left) to bright (on the right).¬†Here, the algorithm modifies the texture so that the tiling shows no seams, but where the aspect of the original texture is preserved as much as possible¬†(only the texture borders are modified).


Overview of the ‚ÄúMake Seamless‚ÄĚ filter in the G‚ÄôMIC plug-in for GIMP.

We can imagine some great uses of this filter, for instance in video games, where texture tiling is common to render large virtual worlds.


Result of the ‚ÄúMake seamless‚ÄĚ filter of G‚ÄôMIC to make a texture tileable.

Image decomposition into several levels of details

A ‚Äúnew‚ÄĚ filter Details / Split details [wavelets] has been added to decompose an image into several levels of details.¬†It is based on the so-called ‚Äú√† trous‚ÄĚ wavelet decomposition.¬†For those who already know the popular Wavelet Decompose plug-in for GIMP, there won‚Äôt be so much novelty here, as it is mainly the same kind of¬†decomposition technique that has been implemented.¬†Having it directly in G‚ÄôMIC is still a great news: it offers now a preview of the different scales that will be computed, and the implementation is parallelized to take advantage of multiple cores.


Overview of the wavelet-based image decomposition filter, in the G’MIC plug-in for GIMP.

The filter outputs several layers, so that each layer contains the details of the image at a given scale. All those layers blended together gives the original image back. Thus, one can work on those output layers separately and modify the image details only for a given scale. There are a lot of applications for this kind of image decomposition, one of the most spectacular being the ability to retouch the skin in portraits : the flaws of the skin are indeed often present in layers with middle-sized scales, while the natural skin texture (the pores) are present in the fine details. By selectively removing the flaws while keeping the pores, the skin aspect stays natural after the retouch (see this wonderful link for a detailed tutorial about skin retouching techniques, with GIMP).


Using the wavelet decomposition filter in G’MIC for removing visible skin flaws on a portrait.

Image denoising based on ‚ÄúPatch-PCA‚ÄĚ

G’MIC is also well known to offer a wide range of algorithms for image denoising and smoothing (currently more than a dozen). And he got one more ! Filter Repair / Smooth [patch-pca] proposed a new image denoising algorithm that is both efficient and computationally intensive (despite its multi-threaded implementation, you probably should avoid it on a machine with less than 8 cores…). In return, it sometimes does magic to suppress noise while preserving small details.


Result of the new patch-based denoising algorithm added to G’MIC.

The ‚ÄúDroste‚ÄĚ effect

The Droste effect (also known as ‚Äúmise en abyme‚Äú in art) is the effect of a picture appearing within itself recursively.¬†To achieve this, a new filter Deformations / Continuous droste has been added into G‚ÄôMIC. It‚Äôs actually a complete rewrite of the popular Mathmap‚Äôs¬†Droste filter that has existed for years.¬†Mathmap was a very popular plug-in for GIMP, but it seems to be not maintained anymore. The Droste effect was one of its most iconic and complex filter.¬†Martin ‚ÄúSouphead‚ÄĚ, one former user of Mathmap then took the bull by the horns and converted the complex code of this filter specifically into a G‚ÄôMIC script,¬†resulting in a parallelized implementation of the filter.


Overview of the converted ‚ÄúDroste‚ÄĚ filter, in the G‚ÄôMIC plug-in for GIMP.

This filter allows all artistic delusions. For instance, it becomes trivial to create the result below in a few steps: create a selection around the clock, move it on a transparent background, run the Droste filter, et voilà!.


A simple example of what the G‚ÄôMIC ‚ÄúDroste‚ÄĚ filter can do.

Equirectangular to nadir-zenith transformation

The filter Deformations / Equirectangular to nadir-zenith is another filter converted from Mathmap to G’MIC. It is specifically used for the processing of panoramas: it reconstructs both the Zenith and the
Nadir regions of a panorama so that they can be easily modified (for instance to reconstruct missing parts), before being reprojected back into the input panorama.


Overview of the ‚ÄúDeformations / Equirectangular to nadir-zenith‚ÄĚ filter in the G‚ÄôMIC plug-in for GIMP.

Morgan Hardwood has wrote a quite detailled tutorial, on pixls.us, about the reconstruction of missing parts in the Zenith/Nadir of an equirectangular panorama. Check it out!

Other various improvements

Finally, here are other highlights about the G’MIC project:

  • Filter Rendering / Kitaoka Spin Illusion is another Mathmap filter converted to G‚ÄôMIC by Martin ‚ÄúSouphead‚ÄĚ. It generates a certain kind of¬†optical illusions as shown below (close your eyes if you are epileptic!)

Result of the ‚ÄúKitaoka Spin Illusion‚ÄĚ filter.

  • Filter Colors / Color blindness transforms the colors of an image to simulate different types of color blindness.¬†This can be very helpful to check the accessibility of a web site or a graphical document for colorblind people.¬†The color transformations used here are the same as defined on Coblis,¬†a website that proposes to apply this kind of simulation online. The G‚ÄôMIC filter gives strictly identical results, but it ease¬†the batch processing of several images at once.

Overview of the colorblindness simulation filter, in the G’MIC plug-in for GIMP.

  • Since a few years now, G‚ÄôMIC has its own parser of mathematical expression, a really convenient module to perform complex calculations when applying image filters¬†This core feature gets new functionalities: the ability to manage variables that can be complex, vector or matrix-valued, but also the creation of¬†user-defined mathematical functions. For instance, the classical rendering of the Mandelbrot fractal set¬†(done by estimating the divergence of a sequence of complex numbers) can be implemented like this, directly on the command line:
    $ gmic 512,512,1,1,"c = 2.4*[x/w,y/h] - [1.8,1.2]; z = [0,0]; for (iter = 0, cabs(z)

Using the G’MIC math evaluator to implement the rendering of the Mandelbrot set, directly from the command line!_

This clearly enlarge the math evaluator ability, as you are not limited to scalar variables anymore. You can now create complex filters which are able to solve linear systems or compute eigenvalues/eigenvectors, and this, for each pixel of an input image. It’s a bit like having a micro-(micro!)-Octave inside G’MIC. Note that the Brushify filter described earlier uses these new features extensively. It’s also interesting to know that the G’MIC math expression evaluator has its own JIT compiler to achieve a fast evaluation of expressions when applied on thousands of image values simultaneously.

  • Another great contribution has been proposed by Tobias Fleischer, with the creation of a new C¬†API to invoke the functions of the libgmic library¬†(which is the library containing all the G‚ÄôMIC features, initially available through a C++ API only).¬†As the C ABI is standardized (unlike C++),¬†this basically means G‚ÄôMIC can be interfaced more easily with languages other than C++.¬†In the future, we can imagine the development of G‚ÄôMIC APIs for languages such as Python for instance.¬†Tobias is currently using this new C API to develop G‚ÄôMIC-based plug-ins compatible with the OpenFX standard.¬†Those plug-ins should be usable indifferently in video editing software such as After effects, Sony Vegas Pro¬†or Natron. This is still an on-going work though.

Overview of some G’MIC-based OpenFX plug-ins, running under Natron.


Overview of a dedicated G’MIC script running within the Blender VSE.

  • You can find out G‚ÄôMIC filters also in the opensource nonlinear video editor Flowblade, thanks to the hard work of¬†Janne Liljeblad (Flowblade project leader).¬†Here again, the goal is to allow the application of G‚ÄôMIC effects and filters directly on image sequences, mainly for artistic purposes¬†(as shown in this video or this one).

Overview of a G’MIC filter applied under Flowblade, a nonlinear video editor.

What’s next ?

As you see, the G’MIC project is doing well, with an active development and cool new features added months after months. You can find and use interfaces to G’MIC in more and more opensource software, as GIMP, Krita, Blender, Photoflow, Flowblade, Veejay, EKD and Natron in a near future (at least we hope so!).

At the same time, we can see more and more external resources available for G’MIC : tutorials, blog articles (here, here, here,…), or demonstration videos (here, here, here, here,…). This shows the project becoming more useful to users of opensource software for graphics and photography.

The development of version 1.7.2 already hit the ground running, so stay tuned and visit the official G’MIC forum on pixls.us to get more info about the project developement and get answers to your questions. Meanwhile, feel the power of free software for image processing!

Christmas is already here for image processing folks…

… Because the latest version 1.6.8 ¬ę¬†XMas 2015 Edition¬†¬Ľ¬†of G‚ÄôMIC¬†has been released last week ūüôā !¬†G’MIC (GREYC‚Äôs Magic for Image Computing) is an open-source framework for image processing that I’ve started to develop in¬†August¬†2008. This new release is¬†a good occasion for me to discuss some of the advances and new features added to the project¬†since my last digest, published here 8 months ago.¬†Seven versions have been published since then.

I’ll talk about the few improvements done¬†on the G’MIC plug-in¬†for¬†GIMP¬†(which is currently the most used¬†interface of the project). But I’ll also describe more technical developments¬†of the framework, which already permitted to design some cool new filters and above all, which promise great things for the future.

1. Overview of the¬†G’MIC project

(Skip this section, if you already know what¬†G’MIC is !)


Fig .1.1. Mascot and logo of the G’MIC project, a full-featured open-source framework for image processing.

G’MIC is an open-source project developed in the¬†IMAGE team¬†at the¬†GREYC laboratory (a public research unit based in Caen / France).¬†It¬†is distributed under the free software license CeCILL.¬†It¬†provides several user interfaces for the¬†manipulation of¬†generic image data. Currently, its¬†most used interface is a¬†plugin for GIMP (the plug-in alone seems to be quite popular, as it has been downloaded more¬†than 2 million times so far).

G'MIC plugin for GIMP Overview

Fig. 1.2. The G’MIC plug-in (version 1.6.8) running inside GIMP 2.8.

G’MIC¬†also proposes other worthy interfaces, as¬†a command line tool¬†(similar to the CLI tools from ImageMagick or GraphicsMagick)¬†which fully exploits its capabilities. There is also a nice web service G’MIC Online to apply effects on images directly from a web browser (check it out, it has been re-designed recently!).¬†Currently, the¬†G’MIC¬†core library (used by all those interfaces) provides more than¬†900¬†different functions¬†to perform various image processing tasks. The library alone takes approximately¬†5.5 MB¬†and has more than¬†100,000¬†lines of source code. There is a wide diversity of commands available for geometric and color manipulation, image filtering (denoising and enhancement by spectral, variational or non-local methods), motion estimation / image alignment, primitive drawing (including 3D meshes), edge detection, segmentation, artistic rendering, etc.¬†¬†It’s a very generic tool, as you can imagine.¬†To learn more about the motivations and goals of the G’MIC project,¬†I suggest you take a look at the presentation slides. You can also meet the developers on the¬†new forum, or on the IRC channel¬†#pixls.us¬†(on¬†Freenode).

2. Improvements of the G’MIC plug-in for GIMP

Three major improvements have been done on the GIMP plug-in interface, these last months.

2.1. Import and export of image data via GEGL buffers

That’s probably the greatest news to announce about the plug-in itself. The¬†G’MIC¬†plug-in is now able to import and export image data from/to GIMP¬†using GEGL¬†buffers, which are the foundations¬†of¬†the current development version (2.9)¬†of¬†GIMP. In practice, the plug-in can process high-bit depth images (for example with pixels stored as 32 bits-floats per channel), without loss of numerical accuracy.¬†Actually, the application of a single filter on an image rarely generates serious visual artifacts only because of the 8 bits quantization of an input image.¬†But people applying many filters and effects sequentially on their images will appreciate that the numerical accuracy is preserved during their entire workflow.¬†The animation below (Fig.2.1) shows the subtle quantization phenomenon occurring when certain types of filters are applied on an image (here, an anisotropic smoothing filter).

Where the application of the same G'MIC filter on an image by 8 bits and 32 bits per channel

Fig. 2.1. Comparison of output histograms generated by the same G’MIC filter applied on an image encoded¬†with¬†8 bits or 32 bits-per-channel values.

The histograms in Fig.2.1¬†illustrate the fact we get a greater diversity of pixel values ‚Äč‚Äčwhen the¬†filter is applied on an image stored as¬†32-bits floats rather than 8 bits integers.¬†Applying such filters several times in 8 bits-mode¬†could slowly¬†create undesirable¬†quantization / posterization¬†/¬†banding effects on the processed image.¬†To say it briefly, the¬†G’MIC¬†plug-in seems to be ready for the next major stable release GIMP 2.10 ūüôā¬†(a big THANKS to Mitch from the¬†GIMP team, who took the time to explain me how to achieve this).

2.2. Support for UTF-8 strings in widgets

The plugin now handles strings encoded in UTF-8 for all interface widgets, which will ease its internationalization. For instance, we have now a Japanese version of the interface (as well as Japanese translations for some of the filters), as shown in the screenshot below (Fig. 2.2).

The GIMP plugin G'MIC partially translated into Japanese

Fig.2.2. The G’MIC plug-in for GIMP, partially translated in Japanese.

2.3. A more responsive interface

The plug-in now reacts faster to a parameter change, particularly when an image preview is being computed. A cancellation mechanism has been added, and we don’t have to wait the rendering of the filter preview anymore before having the possibility to act on the interface. It might sound obvious, but this¬†greatly improves the user experience.

Also check out the interesting initiative of Jean-Philippe Fleury –¬†a nice contributor – who displays on his web page a gallery of most¬†of the filters and image effects available in the G’MIC¬†plug-in.¬†It’s a very quick and easy way¬†to get an¬†overview of the filters, and to observe how they perform on different images (sometimes with different parameters).

3. The PatchMatch algorithm

PatchMatch is a fast algorithm for computing image correspondences using a¬†block-matching technique.¬†Since a few years, it raises a high interested in the image processing community, thanks to its computation speed and the fact it can be parallelized relatively easily.¬†It has become then a key step in many recent algorithms that need to compute fast comparisons between image patches, including methods for¬†reconstructing missing parts of an image,¬†synthesizing textures, or performing super-resolution.¬†Thus, I’m happy to announce that G’MIC¬†has now a parallelized implementation of PatchMatch¬†(through the native command -patchmatch) both for 2D and 3D images.¬†Based on this recent implementation, I’ve been already able to develop and add three interesting filters in¬†G’MIC,¬†described hereafter.

3.1. Another filter for image Inpainting

A new image inpainting¬†filter – using a multi-scale approach – has been included in G’MIC¬†to reconstruct missing or corrupted image regions. Very convenient to pull your stepmother out of the holidays photos you shot in Ibiza!¬†Note that this is not the first patch-based¬†Inpainting algorithm available in G’MIC (this post on Pat David’s blog¬†already presented such a tool in February 2014). It’s just a different algorithm, thus giving different results.¬†But¬†in this field, having the choice to generate alternative results is definitely not a luxury, given the ill-posed nature of the problem. And if – like me – you have a machine with a lot of CPU cores¬†just waiting to be heated,¬†then this new multi-threaded filter is for you! Here are two examples of results (Fig.3.1 and Fig.3.2)¬†we can obtain with this Inpainting filter.


Fig. 3.1. Removing a bear from an image, using the PatchMatch-based Inpainting filter of G’MIC.


Fig. 3.2. Removing the Eiffel Tower, using the PatchMatch-based Inpainting filter of G’MIC.

A video showing how this latest example has been obtained (in only 1 minute 07)¬†is visible below. This is a real-time video, no tricks were used, except using¬†a PC with¬†24 cores!¬†Even to me (the developer), it’s a bit¬†magical to see the algorithm regenerating the whole trees at the feet of the Eiffel Tower. Actually, its does nothing more than cloning an existing tree from elsewhere in the picture. But that is still impressive.¬†Of course, it does not work as well on all images where I’ve experimented the algorithm ūüôā .

This new¬†Inpainting¬†filter is roughly the same as the one Adobe introduced in Photoshop CS5 in 2010, under the (very windy) name¬†“Content-Aware Fill”. The main strength of this kind of algorithms is their ability¬†to reconstruct large coherent textured areas. Note also that my PhD student¬†Maxime Daisy (who defended his thesis successfully two weeks ago, well done pal!) has worked on some nice extensions of such Inpainting¬†methods to remove moving objects in video sequences (see his demo page for examples). These extensions for video Inpainting¬†are not yet available in¬†G’MIC (and are still very demanding in terms of computation time), but it may happen one day, who knows?

3.2. Texture re-synthesis

By using a very similar multi-scale approach, I’ve also added a new texture synthesis filter in G’MIC.¬†It allows to synthesize a texture of arbitrary size from an input template texture (which is usually smaller).¬†Of course, the filter does something smarter than just tiling the template texture:¬†It actually regenerates a whole new texture having similar characteristics by copying / pasting some bits of the template¬†texture so that the synthesized result does not contain visible seams.¬†Fig.3.3 shows such a 512x512¬†re-synthesized texture (top-right) from an input¬†template¬†image having a¬†smaller size 280x350¬†(left). A comparison of this result with a more basic tiling process is also shown (bottom-right).


Figure 3.3. Example of re-synthesizing a complex texture using¬†G’MIC and comparison with basic tiling.

Note that we already had a texture re-synthesis algorithm in G’MIC, but it worked well only¬†with micro-textures.¬†On the contrary, the example on Fig.3.3 shows that the new algorithm is capable of regenerating more complex macro-textures.

3.3 Make seamless textures

And here is the third (and last)¬†PatchMatch-based filter added so far in G’MIC. This one allows to transform an input texture to make it appear seamless when tiled. The idea is a bit different than with the previous filter, as most of the original input texture is actually untouched here. The filter only performs some global color normalization (to remove low-frequency gradients due to the non-homogeneous illumination) and locally adds an inner (or outer) frame that makes the texture appear seamless. Look at what the filter can do on the two examples below (textures shown as 2x2 tiled). Not bad at all ! I have to admit anyway that the parameters are still a bit hard to tweak.


Fig.3.4. Make a seamless texture with the third PatchMatch-based filter in G’MIC.


Fig.3.5 Make a seamless texture with the third PatchMatch-based filter in G’MIC.

Believe me, having this PatchMatch algorithm now implemented in G’MIC¬†is a really great news! No doubts many filters could benefit from this method in the future.

4. A more powerful expression evaluator

In my mind, G’MIC has always been a very convenient tool to quickly design and build complex image processing pipelines¬†(possibly with loops and conditionals).¬†However, when I do algorithm prototyping, it frequently happens I want to experiment with quite “low-level” operations. For instance, I may want to write nested (x,y)¬†loops over an image, where each loop iteration performs quite “exotic” things¬†at the pixel scale. This kind of low-level algorithms cannot be expressed easily¬†as pipelines of macro-operators (assuming we don’t already have a macro-operator that does the job, of course!).¬†Trying to express such an algorithm anyway as a¬†G’MIC pipeline was a bit tedious, either because it required some tricks¬†to write, or because it ended up in a very slow implementation (due to the nested loops¬†over all the image pixels that keep being interpreted). Note that this problem is not specific to¬†G’MIC : try to discuss with image processing folks working with¬†Matlab¬†to see how creative they are when it comes to avoid writing explicit loops in a Matlab¬†script !¬†It’s a fact: image processing often requires a lot of (pixel) data to process, and prototyping heavy IP algorithms only with interpreted languages introduces a speed bottleneck.

My goal anyway was to be to able to prototype most of my algorithms (including those low-level ones) directly in¬†G’MIC,¬†with the constraint they never become ridiculously slow to run (compared to an equivalent¬†C++¬†implementation, for instance).¬†As you might have guessed, a¬†conventional solution to this kind of problems would consist in embedding a¬†Just-in-Time compiler¬†into my¬†interpreter. So that’s what I’ve done – but only for a subset of G’MIC – namely¬†the mathematical expression evaluator. As one might expect, evaluating math expressions is indeed a key feature for an image processing¬†framework such as G’MIC.

To be perfectly honest,¬†this JIT¬†has been already included in G’MIC¬†for years. But it has been also greatly optimized and improved these last months to allow the parallelized evaluation of more complex expressions. These expressions can now be in fact considered as small programs themselves (containing loops, variables, conditional tests, pixel access, etc.), rather than just plain mathematical formulas. As a result, we now have a very efficient expression evaluator in G’MIC, that appears to be at least 10x faster than the equivalent feature proposed in¬†ImageMagick¬†(with its generic command -fx)¬†!

Thus, G’MIC is now a tool good enough to prototype almost any kind of image processing algorithms, without sacrificing the computation speed. I wrote a quite comprehensive post¬†about this improved¬†expression evaluator a few months ago, so please take some time to read it if you want more details. Here again, this is very encouraging news for the future of the project.

5. An abundance of filters!

This section proposes a bulk list of some other features/effects/filters added in G’MIC since last April.The screenshots below often show those¬†filters running ¬†in the G’MIC plug-in for GIMP,¬†but of course they may be applied also from all the other interfaces (the CLI¬†tool, in particular).

  • Filters search engine: this has been a long-time request from users, so finally we have proposed a first draft for such a tool (thanks Andy!). Indeed, the number of filters in the plug-in keeps increasing¬†and¬†it is not always convenient to find a particular filter, especially if you forgot to put it in your favorites. Anyway, you have still to specify relevant keywords to be able to find you filters with this tool.

Fig.5.1. New filter search engine by keywords in the G’MIC plug-in for GIMP.

  • Vector Painting: This image filter is a direct consequence of the recent improvements made on the math expression evaluator. It basically generates a piecewise constant abstraction of an input color image. It’s quite funny to look at the full source code of this filter, because it¬†is surprisingly short¬†(19 lines) !

Fig. 5.2. The Vector Painting filter creates image abstractions.

  • Freaky B&W: This filter performs black & white¬†conversion¬†(grayscale to be precise) of an input color image. But it does this by solving a Poisson equation rather than applying a simple linear formula to compute the luminance. Here, the goal is to get a B&W image which contains all of the contrast details present in each color channel of the original image. This filter often generates images that have a kind of HDR¬†look (but in grayscale).

Fig. 5.3. The Freaky B&W filter converts a color image into HDR-like grayscale.

  • Bokeh: This one tries to generate synthetic¬†Bokeh¬†backgrounds on color images (artistic blur). It is highly configurable and can generate Bokeh with various kind of shapes (circles, pentagons, octagons, stars ..). It can be used for instance to add some light effects to a color image, as illustrated below.

Fig. 5.4. Application of the new Bokeh filter on a color image.

  • Rain & snow: as the name suggests, this filter adds rain or snow on your images (example below is a zoom of the previous image).

Fig. 5.5. Adding rain on a color image with the filter Rain & snow.

  • Neon lightning: not much to say about this one. It generates random curves starting from a region A to another region B and stylizes those curves as neon lights. Convenient for generating cool wallpapers probably ūüôā

Fig. 5.6. Effect Neon Lightning in action!

  • Stroke:¬†This filter decorate simple monchrome shapes on a transparent layer with color gradients. The figure below illustrates the transformation of a simple text (initially drawn with a single color)¬†by this filter.

Fig. 5.7. Applying the Stroke filter to decorate a plain text.

  • Light leaks: this one simulates the presence of undesired light artifacts in pictures. In general, that is rather the kind of effects we want to remove! But the simulation of image degradation can be useful in some cases (for those who want to make their synthesized images more realistic for instance).

Fig. 5.8. Simulation of image degradation with the Light leaks filter.

  • Grid [triangular]: This filter converts an image into a grid composed of colored triangles, allowing many choices for the type of grids to apply.

Fig. 5.9. Transforming an image in a triangular grid.

  • Intarsia: This is a quite original filter. It allows to turn an image into a knitting diagram for Intarsia. The idea came from¬†a user of the¬†GimpChat¬†forum to help her distributing custom knitting patterns (apparently there are websites that offer such patterns in exchange for hard cash). The filter itself does not modify the picture. It only generate a knitting diagram as an output¬†HTML page. An example of such a diagram¬†is visible here.
  • Drop water: I must admit I particularly like this one. The filter simulates the appearance of water droplets over an image. In its most basic version (with the default parameters), it looks like this:

Fig. 5.10. Adding water droplets on an image with the Drop water filter.

But it gets really interesting when the user defines its own droplet shapes by adding a transparent layer containing a few colored strokes on it. For instance, if you add this layer (here represented with pink strokes) to the leaf image:


Fig. 5.11. Adding a layer to define the desired shape of water drops.

Then, the filter Drop water will generate this image, with your custom water drops:


Fig. 5.12. Synthesis of custom shaped water drops.

Moreover, the filter has the good sense to generate its result as a stack of several ouput layers, each corresponding to the simulation of a different physical phenomena, namely: the specular light spots, the drop shadows, the own shadows, and the light refraction. It is then easy to modify these layers indivisually afterwards to create interesting effects with the image. For instance, if we apply a color-to-monochrome filter on the base image layer while preserving the refraction layer (computed from the initial color image), we end up with this:


Fig. 5.13. Applying the Drop water filter, followed by a colorimetric change on the initial layer only.

The¬†short video tutorial¬†below explains how to achieve this effect, step by step, using the G’MIC plug-in for¬†GIMP. It takes less than 2 minutes, even for a complete beginner.

Even better, you can superimpose the result of the Drop water¬†filter on another image. Here is an example where two portraits have been aligned then “merged” together, separated by a kind of liquid interface.¬†It’s really easy to do, and the result looks pretty cool to me.


Fig. 5.14. The G’MIC Drop water filter applied to create a liquid interface between two distinct portraits.

Well, that’s it for the newest filters!

6. Other notable points

To end this long post, let me list some other interesting news about the project:

  • First of all, we have been contacted by Tobias Fleischer. Tobias is a professional plug-in developer for Adobe After Effects and Premiere Pro (among other video editing software). He has made ‚Äč‚Äča tremendous job in developing a DLL¬†for Windows which encapsulates libgmic¬†(the G’MIC core library). And, he has used this¬†DLL to implement a prototype of a G’MIC plug-in for After Effects. He was kind enough to share some screenshots with us of what he has done recently: you can see below an example of the¬†G’MIC¬†skeleton filter running in After Effects.

Fig. 6.1. Prototype of a G’MIC plug-in running in Adobe After Effects.

I can already hear grumpy guys saying: “But why didn’t he try doing that for a free software, as Natron, rather than a 100% proprietary software? “. But in fact, he did it also! And not just for Natron, but for any other video processing software compatible with the OpenFX API. The screenshot below shows a prototype of an OpenFX-compliant G’MIC plug-in running in Sony Vegas Pro. There are probably no reasons why this couldn’t work with Natron too.


Fig. 6.2. Prototype of an OpenFX-compliant G’MIC plug-in running under Sony Vegas Pro.

Of course, all this should be still considered as Work In Progress. There are probably still bugs to fix and improvements to do, but this is really promising. I can’t wait to see these plug-ins released and running on Natron ūüôā

  • Let me continue with some other good news: Andrea, a nice contributor (who is also the developer of PhotoFlow) managed to understand why G’MIC¬†had frequent crashes under MacOSX when computations were done using multiple threads. He¬†proposed a simple patch to solve this issue (it was actually related to the tiny stack size allocated for each thread). G’MIC under MacOSX should be then fully functional now.
  • The webcam/video interface ZArt of G’MIC has been also improved a lot, with new added filters, automatic detection of the webcam resolutions, and the possibility of having a dual-window (one for monitoring + one for display, on a second screen).

Fig 6.3. The ZArt interface running the G’MIC filter Drop water in dual-window mode.

  • A new animated demo has been also added to G’MIC, through the command -x_landscape. The demo itself is of little interest for the user, but was gainful for me to test/improve the speed of the G’MIC interpreter (and isn’t it funny to have this kind of things in G’MIC ūüėČ ). Note that all animated and/or interactive G’MIC scripts are available via this invokation of the CLI tool:
$ gmic -demo

Fig. 6.4. Animated virtual landscape in the G’MIC CLI tool gmic with command -x landscape.

As I had more and more small and funny animations implemented into G’MIC,¬†I’ve decided to put them together as a little intro, entirely coded as a single G’MIC script (750 lines of code). The result smells the divine fragrance of old 8/16 bits machines (the corresponding video is shown below). Remember that all have been generated 100% with the G’MIC interpreter. Isn’t that cool ?

8. So, what’s next?

Now it’s probably time to rest a little bit ūüôā . 2015 has already been a very busy year for the G’MIC project. Christmas is rapidly approaching and I’ll take a (small) break.

To be honest, I don’t have yet a clear idea about the things I’ll focus on. In any case, it appears that G’MIC keeps improving gradually and can potentially raise the interest of more and more people. I wanted to thank especially all the contributors of the project and users of the software who give regular feedback to make it better and better. Also, be aware that each significant addition is usually subject to an announcement on the¬†Google+ page of G’MIC, so it’s easy to stay informed¬†about¬†the project progression, day after day. Dear reader, would you have any proposal for major improvements ?

Let me finally conclude this post by proclaiming it mightily: “Long live open-source image processing ! And Happy XMas everyone !



Image processing made easier with a powerful math expression evaluator.

Warning: This post contains personal thoughts about my research work in image processing. I’ll discuss about some of the issues I’m facing as an active developer of two open-source image processing frameworks (namely CImg and G’MIC). So keep in mind this will be a bit self-centered. There are high chances you find all this really boring if you are not a developer of image processing software yourself (and even if so). Anyhow, feel free to give your impressions after the reading!

1. Context and issues

In imaging science, image processing is processing of images using mathematical operations by using any form of signal processing for which the input is an image, such as a photograph or video frame.

That’s what¬†Wikipedia¬†says about ¬†image processing. Selecting and ordering those mathematical operations is what actually defines¬†algorithms, and implementing ready-to-use and interesting image processing algorithms is actually one of my goals, as well as making them available for interested users afterwards.

After all those years (>10) as a researcher in the image processing field (and like most of my colleagues), I can say I’ve already implemented¬†a lot of these different algorithms. Mostly in¬†C++ as far as I’m concerned. To be more precise, an important part of my work is even to design (and hopefully publish) my own image processing methods. Most of the time of course, my trials end up with clunky, ineffective and slow operators which give nothing interesting else than knowing the¬†approach is not good enough to be followed. Someone who says everything he tries works right the first time is a liar. Step by step, I try to refine/optimize my prototypes or sometimes even take a completely different direction. Quickly, you realize that it is crucial¬†in this job not to waste time when doing algorithm prototyping because the rate of success is in fact very low.


Don’t waste your time, in any occasion! (photo by (OvO), under CC-by-nc-sa.)

That’s actually one of the reason why I’ve started the G’MIC project. It was primarily designed as a helper to create and run custom image processing pipelines quickly (from the shell, basically).¬†It saves me time, everyday. But the world of image processing algorithms is broad and sometimes you need to experiment with very low-level routines working at a pixel scale, trying such weird and unexpected stuffs that none of the “usual” image processing algorithms you already have in your toolbox can be of use as it is. Or it is used in a so diverted way that it gets hard to even think about using it adequately. In a word, your pixel-level algorithm won’t be expressed as a simple pipeline (or graph if you want to call it so) of macro-scale image processing operators. That’s the case for instance with most of the well known patch-based image processing algorithms (e.g. Non-Local-Means, or PatchMatch¬†and plenty of variants), where each pixel value of the resulting image is computed from (a lot of) other pixel values whose spatial locations are sometimes not evenly distributed (but not random as well!).

Until now, when I was trying to implement this kind of algorithms, I was resigned to go back coding them in C++: It is one language I feel comfortable with, and I’m sure it will run fast enough most of the time. Indeed, computation time is often a bottleneck in image processing. Some of my colleagues are using scripting languages as Matlab or Python for algorithm prototyping. But they often need some tricks to avoid writing explicit code loops, or need to write at least some fast¬†C/C++ modules that will be compiled and run from those higher-level interfaces, to ensure they get something fast enough (even for prototyping, I’m definitely not talking about optimized production code here!).


But, I’m not really satisfied with my¬†C++ solution: Generally, I end up with several small pieces of C++ sources I need to compile and maintain. I can hardly re-use them in a bigger pipeline, or redistribute them as clean packages without a lot of extra work. Because they are just intended to be prototypes: They often have only basic command-line interfaces and thus cannot be directly integrated into bigger and user-friendly image processing frameworks. Making a prototype algorithm really usable by others requires at least to¬†wrap it as a plug-in/module for¬†[..copy the name of your favorite image processing tool or language here..].¬†This generally represents a lot of boring coding work,¬†that may even require more time and efforts than writing the algorithm itself! ¬†And I don’t talk about maintenance.¬†If you’ve ever tried to maintain a 10-year old C++ prototype code, lost in one of your sub-sub-sub-sub folder in your $HOME,¬†you know what I mean. I’d definitely prefer a simpler solution that let me spend more time on writing the algorithm itself than packaging it or making it usable. After all, the primary purpose of my work is to create cool algorithms, not really coding user interfaces for them. On the other way, I am a scientist and I’m also happy to share my discoveries with users (and possibly get feedback from them!). How to make those prototyped algorithms finally usable without spending too much time on making them usable ? ūüôā

Ideally, I’d like something that could nicely integrate into G’MIC (my favorite framework for doing image processing stuffs, of course ūüôā ). Even if at the end, those algorithms run a bit slower than they are in C++. One could suggest to make them as¬†Octave¬†or Scilab¬†scripts/modules. But I’m the developer of G’MIC, so of course, I’d prefer a solution that help to extend my own project.

So finally, how could I code prototypes for new algorithms working at a pixel-level and make them readily available in G’MIC ? This question has worried me for a long time.

2. Algorithm code viewed as a complex math expression

In G’MIC, the closest thing to what I was looking for, is the command

-fill 'expression'

This command fills each pixel of a given image with the value evaluated from a “mathematical expression”. A mathematical expression being a quite vague concept, it appears you can already write some complex formulas. For instance, typing this on the command line:

$ gmic 400,400,1,3 -fill "X=x-w/2; Y=y-h/2; R=sqrt(X^2+Y^2); a=atan2(Y,X); if(R<=180,255*abs(cos(c+200*(x/w-0.5)*(y/h-0.5))),850*(a%(0.1*(c+1))))"

creates this weird-looking 400×400 color image (I advise you to put sunglasses):


Fig.1. One synthetic color image obtained by one application of the G’MIC command -fill.

Of course, the specified expression can refer to pixels of an existing input image. And so, it can modify the pixels of an image as well, as in the following example:

$ gmic leno.png -fill "(abs(i(x+1,y)-i(x-1,y)))^0.25"

which computes gamma-corrected differences of neighboring pixels along the X-axis, as shown below:


Fig.2.1. Original image leno.png


Fig.2.2. Result of the -fill command described above.

(As an aside, let me tell you I’ve recently received e-mails and messages from people who claim that using the image of our beloved Lena to illustrate an article or a blog post is “sexist” (someone even used the term “pornographic”…). I invite you reading the Lena story page if you don’t know why we commonly use this image. As I don’t want to hurt the over-sensibility of these people, I’ll be using a slight variation I’ve made by mixing a photograph of the blessed Jay Leno with the usual image of Lena. Let me call this the Leno image and everyone will be happy (but seriously, get a life!)).

So, as you can imagine, the command¬†-fill already allows me to do a lof of complex¬†and exotic things on images at a pixel level. Technically speaking, it uses the embedded math parser I’ve written for the CImg Library, a C++ open-source image processing library I’ve been developing since 1999 (and on which G’MIC is based). This math parser is quite small (around 1500 lines of C++ code) and quite fast as well, when it is applied on a whole image. That’s mainly because:

1. It uses parallelization (thanks to the use of OpenMP directives) to evaluate expressions on blocs of image pixels in a multi-threaded way.

2. Before being evaluated, the given math expression is pre-compiled on-the-fly by CImg, into a sequence of bytecodes. Then, the evaluation procedure (which is done for the whole image, pixel by pixel) requires only that bytecode sequence to be interpreted, which is way faster than parsing the input mathematical expression itself.

Anyway, I thought the complexity of the pixel-level algorithms I’d like to implement was really higher than just the evaluation of a mathematical formula.¬†But wait… What is missing actually ? Not much more than loops and updatable variables… I already had variables (though non-updatable) and conditionals. Only loops were really missing. That looks like something I could try adding during my summer holidays, isn’t it ? ūüėČ So, that is where my efforts were focused on during these last weeks: I’ve added new functions to the CImg math parser that allow users to write their own loops in mathematical expressions, namely the functions dowhile(expr,_cond),¬†whiledo(cond,expr) and for(init,cond,expr_end,_expr_body). Of course, it made me also review and re-implement large parts of the math parser code, and I took this opportunity to optimize the whole thing. A new version of the math parser has been made available for the release of G’MIC at the end of August. I’m still working on this expression evaluator in CImg and new improvements and optimizations are ready for the upcoming version of G’MIC¬†(soon to be released).

3. A toy example: Julia fractals

So, what can we do now with these new math features in G’MIC ? Let me illustrate this¬†with a toy example. The following custom G’MIC command renders a Julia fractal. To test it, you just have to copy/paste the following lines in a regular text file user.gmic:

julia_expr :
  -fill "
    zr = -1.2 + 2.4*x/w;
    zi = -1.2 + 2.4*y/h;
    for (iter = 0, zr^2+zi^2<=4 && iter<256, ++iter,
      t = zr^2 - zi^2 + 0.4;
      (zi *= 2*zr) += 0.2;
      zr = t
  -map 7

and invoke the new command -juliar_expr it defines by typing this in the terminal:

$ gmic user.gmic -julia_expr

Then, you’ll get this 1024×1024 color image:

Rendering of the Julia fractal set only from filling an image with a math expression.

Fig.3. Rendering of a Julia fractal only by filling an image with a complex math expression, containing an iteration loop.

As you see, this custom user command¬†-julia_expr¬†is very short and is mainly based on the invokation of the¬†-fill command of G’MIC. But the coolest thing of all happens when we look at the rendering time¬†of that function. The timing measure has been performed on an ASUS laptop with a dual-core HT i7 2Ghz. This is what I get:

Edit : This post has been edited, on 09/20/2015 to reflect the new timings due to math parser optimization done after the initial post of this article.

$ gmic user.gmic -tic -julia -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'user.gmic' (added 1 command, total 6195).
[gmic]-0./ Initialize timer.
[gmic]-0./julia/ Input black image at position 0 (1 image 1024x1024x1x1).
[gmic]-1./julia/ Fill image [0] with expression ' zr = -1.2 + 2.4*x/w; zi = -1.2 + 2.4*(...)( zi *= 2*zr) += 0.2; zr = t ); iter '.
[gmic]-1./julia/ Map cube color LUT on image [0], with dirichlet boundary conditions.
[gmic]-1./ Elapsed time : 0.631 s.

Less than 0.7 second to fill a 1024×1024 image where each of the 1,048,576 pixels may require up to 256 iterations of a computation loop ? Definitely not bad for a prototyped code written in 5 minutes and which does not require compilation to run! Note that all my CPUs have been active during the computation. Trying the same G’MIC code on my machine at work (a powerful 4x 3-core HT Xeon 2.6Ghz) makes the same render in 0.176 second¬†only!

But of course, one could say:

Why not using the native G’MIC command -mandelbrot instead (here, native means hard-coded as a C++ function) ? It is probably way faster!

Let me compare my previous code with the following G’MIC invokation (which renders exactly the same image):

$ gmic 1024,1024 -tic -mandelbrot -1.2,-1.2,1.2,1.2,256,1,0.4,0.2 -map 7 -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input black image at position 0 (1 image 1024x1024x1x1).
[gmic]-1./ Initialize timer.
[gmic]-1./ Draw julia fractal on image [0], from complex area (-1.2,-1.2)-(1.2,1.2) with c0 = (0.4,0.2) and 256 iterations.
[gmic]-1./ Map cube color LUT on image [0], with dirichlet boundary conditions.
[gmic]-1./ Elapsed time : 0.055 s.

That’s 12 times faster, than the previous command¬†-julia_expr run on my laptop, indeed! A bit reassuring to know that C++ compiled into assembly code is faster than CImg home-made bytecode compiled on the fly ūüėČ

But the point is: Suppose now I want to slightly modify the rendering of the fractal, i.e. I don’t want to display the maximum iteration anymore for each pixel (variable iter), but the latest value of the variable zi before the divergence test occurs. Look how simple this is to create a slightly modified command¬†-julia_expr2 that does exactly what I want to do. I have the full control on what the function does at a pixel level:

julia_expr2 :
  -fill "
    zr = -1.2 + 2.4*x/w;
    zi = -1.2 + 2.4*y/h;
    for (iter = 0, zr^2+zi^2<=4 && iter<256, ++iter,
      t = zr^2 - zi^2 + 0.4;
      (zi *= 2*zr) += 0.2;
      zr = t
  -normalize 0,255 -map 7

and this modified algorithm renders the image below (still in 0.7 second of course):

Fig.4. Slightly modified version of the Julia fractal by displaying another variable zi in the rendering algorithm.

Without these new loop features introduced in the math parser, I would have been forced to do one of these two things in G’MIC, to get the same result:

  1. Either, add some new options to the native command¬†-mandelbrot to allow this new type of visualization. This basically means: Writing new pieces of C++ code, compiling a new version of G’MIC with these added features, package it and release it to make this available for everyone. Even if I already have some decent release scripts, this implies a lot of extra-work and packaging time. This is not like the user can get the new feature in a few minutes (if you’ve already used the filter update mechanism present in the G’MIC plug-in for GIMP, you know what I mean). And I don’t speak about all the possibilities I couldn’t think of (but the user will obviously need one day ūüôā ), when adding such new display options to a native command like -mandebrot
  2. Or, write a new G’MIC custom script able to compute the same kind of result. This would be indeed the best way to make it available for other people quickly. But here, as the algorithm is very specific and works at a pixel level, doing it as a pipeline of macro-operators is quite a pain. It means I would have to use 3 nested -repeat...-done loops (which are basically the loops commands used in G’MIC pipelines) and it would probably take ages to render, as a¬†G’MIC pipeline is always purely interpreted without any pre-compilation steps. Even by using multi-threading, it would have been a nightmare to compute here.

In fact, the quite long math expression we are using in command -julia_expr2¬†defines one complex algorithm as a whole, and we know it will be pre-compiled into a sequence of bytecodes by CImg before being evaluated for each of the 1024×1024=1,048,576 pixels that compose the image. Of course, we are not as fast as a native C++ implementation of the same command, but at the same time, we gain so much flexibility and genericity in the stuffs we can do now, that¬†this disadvantage is easily forgiven. And the processing time stays reasonable. For fast algorithm prototyping, this feature seems to be incredibly nice! I won’t be forced to unsheathe my C ++ compiler every time I want to experiment some very specific image processing algorithms working at the pixel level.

4. A more serious example: the “Non-Local Means”

The Non-Local-Means is a quite famous patch-based denoising/smoothing algorithm in image processing, introduced in 2005 by A. Buades (beware, his home page contains images of Lena, please do not click if you are too sensitive!). I won’t enter in all the implementation details, as several different methods has been proposed in the literature just for implementing it. But one of the most simple (and slowest) technique requires 4 nested loops per image pixel. What a good opportunity to try writing this “slow” algorithm using the G’MIC¬†-fill function! It took me less than 10 minutes, to be honest:

nlmeans_expr : -check "${1=10}>0 && isint(${2=3}) && $2>0 && isint(${3=1}) && $3>0"
  -fill "
    const sigma = $1;  # Denoising strength.
    const hl = $2;     # Lookup half-size.
    const hp = $3;     # Patch half-size.
    value = 0;
    sum_weights = 0;
    for (q = -hl, q<=hl, ++q,
      for (p = -hl, p<=hl, ++p,
        diff = 0;
        for (s = -hp, s<=hp, ++s,
          for (r = -hp, r<=hp, ++r,
            diff += (i(x+p+r,y+q+s) - i(x+r,y+s))^2
        weight = exp(-diff/(2*sigma)^2);
        value += weight*i(x+p,y+q);
        sum_weights += weight
    value/(1e-5 + sum_weights)

Now, let’s test it on a noisy version of the Leno image:

$ gmic user.gmic leno.png -noise 20 -c 0,255 -tic --nlmeans_expr 35,3,1 -toc
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'user.gmic'(added 1 command, total 6195).
[gmic]-0./ Input file 'leno.png' at position 0 (1 image 512x512x1x3).
[gmic]-1./ Add gaussian noise to image [0], with standard deviation 20.
[gmic]-1./ Cut image [0] in range [0,255].
[gmic]-1./ Initialize timer.
[gmic]-1./nlmeans_expr/ Set local variable sigma='35'.
[gmic]-1./nlmeans_expr/ Set local variable hl='3'.
[gmic]-1./nlmeans_expr/ Set local variable hp='1'.
[gmic]-1./nlmeans_expr/ Fill image [0] with expression ' value=0; sum_weights=0; for(q = -3,q<(...)ight ) ); value/(1e-5 + sum_weights) '.
[gmic]-2./ Elapsed time : 3.156 s.

which results on these two images displayed on the screen, the noisy version (left), and the denoised one using the Non-Local-Means algorithm (right). Of course, the timing may differ from one machine to another. I guess my 3 seconds run here is seemly (tested on my powerful PC at the lab). It still takes less than 20 seconds on my laptop. A crop of the results are presented below. The initial Leno image is a 512×512 RGB image, and the timing has been measured for processing the whole image, of course.


Fig.5.1. Crop of a noisy version of the Leno image, degraded with gaussian noise, std=20.


Fig.5.2. Denoised version using the NL-means algorithm (custom command -nlmeans_expr).

Here again, you could argue that the native G’MIC command¬†-denoise does the same thing and is faster to run. It is, definitely. J√©rome Boulanger (one very active G’MIC contributor) has written a nice custom command¬†-nlmeans that implements the NL-means with a smarter algorithm (avoiding the need for 4 nested loops per pixel) which runs even faster (and it is already available in the plug-in for GIMP). But that’s not the point. What I show here is I’m now able to do some (relatively fast) prototyping of algorithms working at a pixel level in G’MIC, without having to write and compile C++ code. But the best of all is about integration: if the algorithm appears to be interesting/effective enough, I can add it to the G’MIC standard library in a few minutes, and quickly create a filter for the GIMP plug-in as well. Let say it right away, probably 5 minutes after I’ve finished writing the first version of the algorithm, the plug-in users should be able to get it and use it on their own images (and give positive/negative feedback to help for future improvements). That’s what I call a smooth, quick and painless integration! And that is exactly the kind of algorithms I couldn’t implement before as custom G’MIC commands running at a decent speed.

To me, it clearly opens exciting perspectives to quickly prototype and integrate new custom image processing algorithms into G’MIC in the future!

5. The Vector Painting filter

In fact, this has happened earlier than expected. I’ve been able to add one of my latest image filter (named Vector painting) in the G’MIC plug-in for GIMP recently. It has been somehow unexpected, because I was just doing some debugging for improving the CImg math expression evaluator. Briefly, suppose you want to determine for each pixel of an image, the discrete spatial orientation of the maximal value variation, with an angular precision of 45¬į: For each pixel centered in a 3×3 neighborhood, I want to estimate which pixel of the neighborhood has the highest difference with the center pixel (measured as the squared difference between the two pixel values). To make things simpler, I’ve considered doing this on the image luminance only instead of using all the RGB color channels. At the end, I transform each pixel value into a label (an integer in range [1,8]) that represents one of the possible 45¬į-orientations of the plane. That was typically the kind of problems that would require the use of custom loops working at a pixel level, so something I couldn’t do easily before the loop feature introduced in my math parser (or I would have done the prototype in C++).

The solution to this problem was surprisingly easy to write. Here again, it didn’t take much more than 5 minutes of work:

foo :
  -fill "dmax = -1; nmax = 0;
         for (n = 0, ++n<=8,
           p = arg(n,-1,0,1,-1,1,-1,0,1);
           q = arg(n,-1,-1,-1,0,0,1,1,1);
           d = (j(p,q,0,0,0,1)-i)^2;
           if(d>dmax,dmax = d; nmax = n,nmax)

And if we apply this new custom command -foo on our Leno image,

$ gmic user.gmic leno.png -foo

We get this result (after re-normalization of the label image in range [0,255]). Keep in mind that each pixel of the resulting image is an integer label originally in range [1,8]. And by the way, the computation time is ridiculous here (178ms for this 512×512 image).


Fig.6. Each pixel of the Leno image is replaced by a label saying about which of its 3×3 neighbors is the most different from the central pixel.

It actually looks a bit ugly. But that’s not surprising, as the original image contains noise and you’ll get a lot of small random variations in flat regions. As a result, the labels you get in those regions are noisy as well. Now, what happens when we blur the image before computing the labels? That should regularize the resulting image of labels as well. Indeed:

$ gmic user.gmic leno.png --blur 1% -foo

returns this:


Fig.7. Each pixel of the blurred Leno image is replaced by a label saying about which of its 3×3 neighbors is the most different.

That’s interesting! Blurring the input image creates larger portions of constant labels, i.e. regions where the orientation of the maximal pixel variations is the same. And the original image contours keep appearing as natural frontiers of these labelled regions. Then, a natural idea would be to replace each connected region by the average color it overlays in the original color image. In G’MIC, this can be done easily with the command -blend shapeaverage.

$ gmic user.gmic leno.png --blur 1% -foo[-1] -blend shapeaverage

And what we get at the end is a nice piecewise constant abstraction of our initial image. Looks like a “vector painting”, no ? ūüėČ


Fig.8. Result of the “shape average” blending between the original Leno image, and its map of labels, as obtained with command -foo.

As you may imagine, changing the amplitude of the blurring makes the result more or less abstract. Having this, It didn’t take too much time to create a filter that could be run directly from the G’MIC plug-in interface for GIMP. That’s the exact code I wrote to integrate my initial algorithm prototype in G’MIC and make it usable by everyone. It was done in less than 5 minutes, really:

#@gimp Vector painting : gimp_vector_painting, gimp_vector_painting_preview(1)
#@gimp : Details = float(9,0,10)
#@gimp : sep = separator(), Preview type = choice("Full","Forward horizontal","Forward vertical","Backward horizontal","Backward vertical","Duplicate horizontal","Duplicate vertical")
#@gimp : sep = separator(), note = note("<small>Author: <i>David Tschumperl&#233;</i>.\nLatest update: <i>08/25/2015</i>.</small>")
gimp_vector_painting :
  -repeat $! -l[$>]
    --luminance -b[-1] {10-$1}%,1,1
    -f[-1] "dmax = -1; nmax = 0;
            for (n = 0, ++n<=8,
              p = arg(n,-1,0,1,-1,1,-1,0,1);
              q = arg(n,-1,-1,-1,0,0,1,1,1);
              d = (j(p,q,0,0,0,1)-i)^2;
              if (d>dmax, dmax = d; nmax = n,nmax)
    -blend shapeaverage
  -endl -done

gimp_vector_painting_preview :
  -gimp_split_preview "-gimp_vector_painting $*",$-1

Here is the resulting filter, as it can be seen in the G’MIC plug-in for GIMP (requires version Just after I’ve pushed it in the G’MIC standard library:


Fig.9. The G’MIC plug-in for GIMP, running the “Vector Painting” filter.

Here again, that is how I conceive things should be done properly: 1. I create a quick algorithm prototype to transform an image into something else. 2. I decide that the algorithm is cool enough to be shared. 3. I add few lines to make it available immediately in the G’MIC image processing framework. What a gain of time compared to the time it would have taken by doing this in C++!

6. Comparison with ImageMagick’s -fx operator

While working on the improvement of my math expression evaluator in CImg, I’ve been wondering if what I was doing was not already existing in ImageMagick. Indeed, ImageMagick is one of the most well established open-source image processing framework, and I was almost sure they had already cope with the kind of questions I had for G’MIC. And of course, they had ūüôā

So, they have a special operator -fx expression in¬†convert that seems to be equivalent to what the G’MIC command -fill expression does. And yes, they probably had it for years, long before G’MIC even existed. But I admit I’ve almost completely stopped using ImageMagick tools when I’ve started developing my own C++ image processing library CImg, years ago. All the information you need to use this -fx operator in¬†convert can be found on this documentation page, and even more examples on this page. Reading these pages was very instructive: I’ve noticed some interesting functions and notations they have in their own expression parser that I didn’t already have in mine (so, I’ve added some of them in my latest version of CImg!). Also I was particularly interested by this quote from their pages:

As people developed new types of image operations, they usually prototype it using a slow “-fx” operator first. When they have it worked out that ‘method’ is then converted into a new fast built-in operator in the ImageMagick Core library. Users are welcome to contribute their own “-fx” expressions (or other defined functions) that they feel would be a useful addition to IM, but which are not yet covered by other image operators, if they can be handled by one of the above generalized operators, it should be reasonably easy to add it.(…). What is really needed at this time is a FX expression compiler, that will pre-interpret the expression into a tighter and faster executable form. Someone was going to look into this but has since disappeared.

So it seems their -fx operator is quite slow as it re-parses the specified math expression for each image pixel. And when someone writes an interesting operator with¬†-fx, they are willing to convert it into a C code and integrate this new built-in operator directly in the core ImageMagick library. It seems they don’t really mind adding new native hard-coded operators into IM, maybe even for very specific/unusual operators (at least they don’t mention it). That’s interesting, because that is precisely what I’m trying to avoid in G’MIC. My impression is that it’s often acceptable to be less efficient if the code we have to write for adding one feature is smaller, easier to maintain/upgrade and does not require releasing a new version to make this particular feature available. Personally, I’d always prefer to write a G’MIC custom command (so, a script that I can directly put in the G’MIC standard library) if possible, instead of adding the same feature as a new “native” built-in command (in C++). But maybe their -fx operator was that slow it was cumbersome to use in practice? I had to try!

And I’m a bit sorry to say this, but yes, it’s quite slow (and I have tested this on my pretty fast machine with 12 HT cores at 2.6Ghz). The ImageMagick¬†-fx operator is able to use multiple cores which is clearly a good thing, but even with that, it is cumbersome to use on reasonably big images, with complex math expressions. In a sense, that reassures me about the usefulness of having developed my own math expression compiler in CImg. This pre-compilation step of the math expression into a shorter bytecode sequence seems then to be almost mandatory. I’ve done a quick timing comparison for some simple image effects that can be achieved similarly with both expression evaluators of G’MIC and ImageMagick. Most of the examples below have been actually taken from the¬†-fx documentation pages. I’m dividing and multiplying my image values by 255 in the G’MIC examples below because ImageMagick formulas assume that RGB values of the pixels are defined in range [0,1]. These tests have been done with a high-resolution input image (of a motorbike) with size 3072×2048, in RGB mode. I’ve checked that both the ImageMagick and G’MIC invokations render the same images.

# Test1: Apply a sigmoid contrast function on the image colors.

$ time convert motorbike.jpg -fx "(1.0/(1.0+exp(10.0*(0.5-u)))-0.006693)*1.0092503" im_sigmo.jpg

real	0m9.033s
user	3m18.527s
sys	0m2.604s

$ time gmic -verbose - motorbike.jpg -/ 255 -fill "(1.0/(1.0+exp(10.0*(0.5-i)))-0.006693)*1.0092503" -* 255 -o gmic_sigmo.jpg,75

real    0m0.474s
user    0m3.183s
sys     0m0.111s
# Test2: Create a radial gradient from scratch.
$ time convert -size 3072x2048 canvas: -fx "Xi=i-w/2; Yj=j-h/2; 1.2*(0.5-hypot(Xi,Yj)/70.0)+0.5" im_radial.jpg

real	0m29.895s
user	8m11.320s
sys	2m59.184s

$ time gmic -verbose - 3072,2048 -fill "Xi=x-w/2; Yj=y-h/2; 1.2*(0.5-hypot(Xi,Yj)/70.0)+0.5" -cut 0,1 -* 255 -o gmic_radial.jpg

real    0m0.234s
user    0m0.990s
sys     0m0.045s

# Test3: Create a keftales pattern gradient from scratch.
$ time convert -size 3072x2048 xc: -channel G -fx  'sin((i-w/2)*(j-h/2)/w)/2+.5' im_gradient.jpg

real	0m2.951s
user	1m2.310s
sys	0m0.853s

$ time gmic -verbose - 3072,2048 -fill "sin((x-w/2)*(y-h/2)/w)/2+.5" -* 255 -o gmic_gradient.jpg

real    0m0.302s
user    0m1.164s
sys     0m0.061s
# Test4: Compute mirrored image along the X-axis.
$ time convert motorbike.jpg -fx 'p{w-i-1,j}' im_mirror.jpg 2>&1

real	0m4.409s
user	1m33.702s
sys	0m1.254s

$ time gmic -verbose - motorbike.jpg -fill "i(w-x-1,y)" -o gmic_mirror.jpg

real    0m0.495s
user    0m1.367s
sys     0m0.106s

The pre-compilation of the math expressions clearly makes a difference!

I would be really interested to compare the expression evaluators for more complex expressions, as the one I’ve used to compute Julia fractals with G’MIC for instance. I don’t have a deep knowledge of the ImageMagick syntax, so I don’t know what would be the equivalent command line. If you have any idea on how to do that, please let me know! I’d be interested also to get an idea on how Matlab is performing for the same kind of equations.

7. Conclusion and perspectives

What I conclude from all of this ? Well, I’m actually pretty excited by what the latest version of my expression evaluator integrated in G’MIC / CImg can finally do. It looks like it runs at a decent speed, at least compared to the one used in ImageMagick (which is definitely a reference project for image processing). I had also the idea of comparing it with GraphicsMagick, but I must admit I didn’t find the same -fx operator in it. And I didn’t find something similar (Maybe you could help teaching me how it works for GraphicsMagick?).

I’ve been already able to propose one (simple) artistic filter that I find interesting (Vector painting), and I’m very confident that these improvements of the math expression evaluator will open a lot of new possibilities for G’MIC. For allowing the design of new filters for everyone of course, but also to make my algorithm prototyping work easier and faster.

Could it be the beginning of a new boost for G’MIC? What do you think?

My latest ten months working on G’MIC

A few days ago, I¬†released a new version of G’MIC (GREYC’s Magic for Image Computing, numbered This is a free and generic image processing framework I’ve been¬†developing since 2008.¬†For this particular occasion, I thought it would be good (for me) to write a quick summary of what features I’ve been working on these last¬†ten months (since the last release of G’MIC in the 1.5.x.x branch). Hopefully, you’ll be¬†interested too ! This project takes¬†a large amount of my free time (basically every week-end and evenings, except Wednesday, table tennis time ūüôā ), so it’s probably good for me to take a little break and analyze what has been done recently around the G’MIC project.

I will¬†summarize the important features added to G’MIC recently, which are: Various color-related filters, a foreground extraction algorithm, new artistic filters, and other (more technical) cool features. As you will see this quick summary has ended up a not-so-quick summary, so you are warned. Just before I start, I’d like to thank Patrick David again a thousand times for his proof reading. Kudos Pat! And now, let’s start !

1. The G’MIC Project : Context and Presentation

The G’MIC project was born in August 2008, in the IMAGE Team of the GREYC¬†laboratory (a public research unit, located in Caen / France, affiliated to the CNRS, the largest public research institute in France), a team to which I belong. Since then, I’ve developed it apace. G’MIC is an open-source software, distributed under the¬†CeCILL¬†license (a French¬†GPL-compatible license).


Fig.1.1 Mascot and logo of the G’MIC project, an open-source framework for image processing.

What G’MIC¬†proposes is basically several different user interfaces to manipulate generic image data (sequences or still 2D/3D images, with an arbitrary number of channels, float-valued , etc., so definitely not only 2D color images). To date, the most popular G’MIC¬†interface is its plug-in for GIMP.


Fig.1.2. Overview of the G’MIC plug-in for GIMP.

But G’MIC¬†also provides a command-line interface¬†(similar to what ImageMagick¬†has, but more flexible, perhaps?), and a web interface G’MIC Online.¬†Other¬†ways of using G’MIC exist, such as ZArt, a qt-based image stream processor, or a nice plug-in for Krita¬†(developed by Lukas Tvrdy), or through some features implemented in the recent (and nice!) software¬†Photoflow. These latter interfaces¬†are still a bit more confidential, but not for long I guess. Actually, all of these interfaces are based on¬†libgmic. This is the C++,¬†portable, thread-safe and multi-threaded library (in particular through the use of OpenMP) which implements all of the G’MIC¬†core image processing routines and structures. It also embeds its own script language interpreter, so that advanced users are allowed to write and integrate their customized image processing functions into G’MIC easily. Today, G’MIC¬†already understands more than 900 different commands, all configurable, for a libgmic¬†library file that takes a bit less than 5 MB. The available commands cover a wide range of the image processing field, with algorithms for the manipulation of image geometry and colors, for image filtering (denoising, sharpening with spectral, variational or non-local methods, …), for image registration and motion estimation, for the drawing of graphical primitives, for image segmentation and contour extraction, for 3D object rendering, artistic filters, and so on…

The set of tools G’MIC provides is really useful, for both converting, visualizing and exploring image data, and for creating, and applying complex image processing pipelines. In my opinion, this is one piece of free software that image processing savvy folks should have tried at least once. At least for me, there is an everyday use. ūüôā

2. New G’MIC features for color processing

So, here is¬†some of the stuff I’ve added to G’MIC recently, related to the manipulation (creation of modification) of the colors in an image.

2.1. Color curves in an arbitrary color space

The concept of color curves is well known for any artist of photograph who wish to modify the colors of their images. Basically, a color curve tool allows you to define and apply on each R,G,B component of your image, a 1d continuous function f:[0,255] -> [0,255]. Each original red, green or blue value x of a pixel (assumed to be in the range 0..255) is then mapped into a new value f(x) (in 0..255 too). To define these transfer functions (one for each of the R,G,B components), the user classically defines a set of key-points that are interpolated by spline functions.

But what happens if you want to define and apply color curves in a color space different than RGB ? Well… mostly nothing, as most of the image retouching tools (GIMP included) only propose to color curves for RGB, unfortunately.

So, one of the things I’ve done recently in G’MIC has been to implement an interactive color curves filter that works in other color spaces than RGB, that is, CMY, CMYK, HSI, HSL, HSV, Lab, Lch and YCbCr. The new command -x_color_curves is in charge of doing that (using the CLI interface). Plug-in users may use it via the filter Colors / Curves [interactive]. Here, using the plug-in is even nicer because you can easily save your favorite curves, as new¬†Favorite filters, and apply the same color curves again and again on hundred of images afterwards. Here is how it looks:

Fig.2.1. Interactive definition of color curves in the Lab colorspace and application on a color image. On the left, the three user-defined curves for each of the components L (lightness), a and b (chrominances). On the left, the corresponding color transformation.

Fig.2.1. Interactive definition of color curves in the Lab color space and application on a color image. On the left, the three user-defined curves for each of the components L (lightness), a and b (chrominances). On the right, the corresponding color transformation it creates on the color image.

I’ve also recorded a quick video to show the action of this filter live, using it from the G’MIC plug-in for GIMP:

2.2. Comics colorization

I also had the chance to talk with David Revoy, a talented French illustrator who is the man behind the nice Pepper & Carrot webcomics (among other great things). He told me that the process of comics colorization is a really boring and tedious task, even using a computer, as he explains it on his blog. He also told me that there were some helpers for colorizing comics, but all proprietary software. Nice! There was obviously something worth trying for open-source graphics here.

So I took my keyboard and started coding a color interpolation algorithm that I hoped to be smart enough for this task. The idea is very simple: Instead of forcing the artist to do all the colorization job by himself, we just ask him to put some colored key-points here and here, inside the different image regions to fill-in. Then, the algorithm tries to guess a probable colorization of the drawing, by analyzing the contours in the image and by interpolating the given colored key-points with respect to these contours. Revoy and I also discussed with Timoth√©e Giet¬†(on irc, channel #krita :)),¬†another French artist, about what could be a nice “minimal” interface to make the process of putting these colored key-points comfortable enough for the artist. And finally, the result of these discussion is the new G’MIC command -x_colorize and the corresponding filter Black & White / Colorize [interactive]¬†in the¬†G’MIC¬†plug-in. Apparently, this seemed to be fair enough to do the job!

From what I’ve seen, this has raised quite a bit of interest¬†among the users of Krita (not so surprising as there are many comics creators among them!). Maybe I’m wrong, but I feel this is one of the reasons the Krita team decided to put some additional efforts on their own G’MIC plug-in¬†(which is something to become an awesome piece of code IMHO, probably even better than the current G’MIC plug-in for GIMP I’ve done).

Here is a quick illustration on how this works for real. The images below are borrowed from the David Revoy‘s web site. He has already written very nice articles about this particular feature, so consider reading them for more details about it.

Using G'MIC for comics colorization. Step 1 : Open your lineart image.

Fig.2.2. Using G’MIC for Comics colorization. Step 1 : Open your lineart image (image borrowed from the David Revoy web site).


Fig.2.3. Using G’MIC for Comics colorization. Step 2 : Place some colored key-points and let the colorization algorithm interpolate the colors for you (image borrowed from the David Revoy web site)

I believe¬†Revoy is using this colorizing algorithm in a regular manner for his creations (in particular for¬†Pepper & Carrot), as illustrated in Fig.2.2-2.3.¬†One can clearly see the different colored key-points put on the image and the outcome of the algorithm, all this being done under the G’MIC plug-in for Krita.¬†After this first colorization step, the artist may add shadows and lights on the flat color regions generated by the algorithm. The nice thing is he can easily work on each color region separately because it is easy to select exactly one single flat color in the output color layer. And for those who prefer to work on more layers, the algorithm is able to output multiple layers too, one for each single reconstructed color region.

So, again, thanks a lot to David Revoy and Timoth√©e Giet¬†for the fruitful discussions and their feedback. We have now a very cool Comics colorization tool in a free software. This is the kind of academic / artistic collaboration I’m fond of.

Below¬†you can see a small video I’ve made that shows how this filter runs inside the¬†G’MIC plug-in for GIMP :

2.3. B&W picture colorization

But could this also work for old B&W photographs ? The answer is¬†Yes!¬†By slightly modifying the colorization algorithm, I’ve been able to propose a way to reconstruct the color chrominance of a B&W image, exactly the same way, from user-defined colored key-points. This is illustrated in the figure below, with the colorization of an old B&W portrait. The pixel-by-pixel content of a photograph being obviously more detailed than a Comics, we often need to put more key-points to get an acceptable result. So, I’m not sure this speeds up the colorization process so much in this case, but still, this is cool stuff. These two colorization techniques are available from the same filter, namely¬†Black & white / Colorize [interactive].

Fig.2.4. Colorisation d'une vieille photo Noir & Blanc avec l'algorithme de colorisation de G'MIC._ (Ugh, vieil indien, excuse moi pour la couleur improbable de tes cheveux, qui te fait ressembler à [Kim Kardashian](http://media.melty.fr/article-2637254-ajust_930-f205979/kim-a-t-elle-le-meme-sex-appeal-en-blonde.jpg)...)

Fig.2.4. Colorization of an old B&W photograh with the G’MIC colorization algorithm.
(Ugh, old man, sorry for the improbable color of you hairs, which makes you look like Kim Kardashian…)

2.4. Color transfer

Here, what I call ¬ę‚ÄĮcolor transfer‚ÄĮ¬Ľ is the process of¬†modifying the colors of an image A¬†by replacing them by the colors of another image B (reference image) so that the modified image A’ gets the same “color ambiance” than the reference image B, and this is being done in a fully automatic way. This is an ill-posed problem, quite complex to solve, and there is already a lot of scientific papers published on this topic (like this one for instance).¬†The main issue consists in generating a new image A’ that keeps a “natural” aspect, without creating synthetic-looking flat color areas, or on the contrary high color discontinuities that are not initially present in the original image A. In short, this is far from being trivial to solve.

I’ve worked on this topic recently (with one of my colleagues¬†Julien Rabin), and we’ve designed a pretty cool algorithm for transferring colors from an image to another. It’s not perfect of course, but this is really a good start. I’ve implemented the algorithm and put it in G’MIC, available from the plug-in filter¬†Colors / Transfer colors [advanced]. This filter requires several input layers, one being the reference image B containing the colors to transfer to the other layers. Here is how it looks when run from the G’MIC plug-in for GIMP,¬†with the original color image (on the left in the preview window), the reference image (bottom-left) and the filter outcome (on the right).

_Fig.2.5. Aperçu du filtre de transfert de couleurs dans le greffon G'MIC pour GIMP._

Fig.2.5. Overview of the new Color transfer filter¬†in the G’MIC plug-in for GIMP.

Sometimes, this filter gives awesome results! Two color transfer examples are illustrated below. Of course, don’t expect it to work nicely with pathological cases (as transferring a colorful image to a monochrome one for instance). But it’s not lying to say it works already quite well. And the best of all, it is quite fast to render.


_Fig.2.6. Deux exemples de résultats de transfert de couleurs d'une photo de référence (image du milieu) vers une photo source (image de gauche). Les images de droite sont les résultats générés par l'algorithme de transfert de G'MIC._

Fig.2.6. Two different color transfer examples from a reference picture (middle) to a color image (left). The images on the right give the outcome of the G’MIC color transfer algorithm.

I’ve also recorded a quick video tutorial that shows how this kind of results can be obtained easily from the G’MIC plug-in for GIMP.¬†We can also think of other usages for this interesting new algorithm, like the¬†homogenization of colors between successive frames of a video, or ensuring the color stability of a stereoscopic pair for instance. Here is the video :

2.5. Website for analog film emulation

I have the pleasure¬†of being a friend with Pat David, an american photograph who is¬†very active in the libre graphics community, sharing his experience and tutorials on his must-read blog. I met him at¬†LGM’2014¬†in Leipzig, and honestly, many useful G’MIC filters have been suggested or improved by him. Particularly, all the filters in the¬†Film emulation¬†category could not have¬†been done without his work on the design of color transfer functions (a.k.a. CLUTs). Below is a figure that shows a small subset of the 300+ color transformations concocted by Pat that has been made available in G’MIC.

_Fig.2.7. Aperçu de quelques résultats de simulation de films argentiques, disponibles dans G'MIC._ (Cette image provient du site web de Pat David : http://blog.patdavid.net/2013/08/film-emulation-presets-in-gmic-gimp.html)

Fig.2.7. Overview of some analog film emulation outcomes, available in G’MIC.
(This image comes from the Pat David‘s blog)

Once added (two years ago), these filters raised much interest in the photo retouching community, and have been re-implemented in RawTherapee. To make these filters even more accessible, we have recently set up a web page dedicated to Film Emulation, where you can see/compare all the available presets and download the corresponding CLUT files. Note that you can also find the G’MIC Film Emulation filters in the G’MIC Online web site, and try them live directly in your web browser. Here again, analog film emulation was somehow a missing feature in the libre software world and the G’MIC infrastructure made it easy to fill this gap (who needs¬†DXO FilmPack anymore now ? ;)).

3. An algorithm for foreground/background extraction.

In photo retouching, it is not uncommon to differentiate the processing of¬†foreground and background objects. So one often needs to select the foreground object first (using the “lasso” tool for instance) before doing anything else. For objects with a complex geometry, this is a tedious task. G’MIC now integrates a user-guided segmentation algorithm to speed up this foreground extraction work. The G’MIC command -x_segment (using the CLI interface) and the plug-in filter Contours / Extract foreground [interactive]¬†let the user play with this algorithm.

This works exactly the same way as the Comics colorization filter, except that the colored key-points are replaced by labelled key-points, with labels being either “foreground” or “background”. Then, the extraction algorithm interpolates those labels all inside the image, taking care of the contours, and deduce a foreground/background binary map. Thus, the filter is able to decompose an image into two layers, one with the foreground pixels only, and the other with the remaining pixels (background). The figure below shows a typical use of this filter : Starting from a color image of a flower (top-left), the user put few keypoints on it (top-right) and the extraction algorithm is able to generate the two background / foreground layers on the last row, from this very sparse set of points.

Doing this takes only few seconds, while the same operation made manually would have been much much longer to perform (the flower’s contour being not simple to cut).

_Fig.3.1. D√©composition avant-plan / arri√®re plan avec le filtre ¬ę Contours / Extract foreground ¬Ľ de G'MIC._

Fig.3.1. Foreground / background user-guided decomposition of an image, with the filter ¬ę Contours / Extract foreground ¬Ľ of G’MIC.

After that, it is quite easy to process the background and foreground separately. Here for instance, I’ve just changed the hue and saturation of the foreground layer a little bit, in order to get a more purple flower without changing the colors of the background. Selecting objects in images is a common operation¬†in photo retouching, I’ll let you imagine in how many cases this filter can be useful.

_Fig.3.2. Après modification de la teinte et de la saturation des couleurs de l'avant-plan uniquement._

Fig.3.2. The same image, after changes on the hue and saturation applied on the extracted foreground only.

I’ve done another G’MIC¬†video tutorial¬† to illustrate the whole process in real-time, using the G’MIC plug-in for GIMP:

4. Some new artistic filters.

G’MIC has always been a source of dozens of artistic filters. Below is an overview of some of the newest additions.

4.1. Engrave effect.

The filter Black & White / Engrave tries to transform an image into an etching. The high number of parameters lets you precisely tune the rendering done by the algorithm, and many different effects may be obtained with this single filter.

Fig.4.1. Overview of the ¬ę Engrave ¬Ľ filter, available in the G’MIC plug-in for¬†GIMP.

What I find particularly interesting with this filter is the ability it has (with properly chosen parameters) to convert photographs into comics-like renderings. This is illustrated with the two images below.

_Fig.4.2. Conversion de photo (en haut à gauche) sous forme de dessin façon Comics, avec le filtre "Engrave" de G'MIC._

Fig.4.2. Photo to Comics conversion, using the Engrave filter from G’MIC.

_Fig.4.3. Conversion d'une photo de chimpanzé (à gauche) sous forme de dessin façon Comics (à droite), avec le filtre "Engrave" de G'MIC._

Fig.4.3. Another example of photo to Comics conversion, with the Engrave filter from¬†G’MIC.

I’ve also recorded a video tutorial¬†to show the different steps used to generate this kind of results. It takes only few seconds to achieve (maybe minutes for the slowest of you!).

4.2. Delaunay triangulation.

The algorithm of Delaunay triangulation has been added to¬†G’MIC (through the -delaunay3d command). A new filter, namely¬†Artistic / Polygonize [delaunay] inside the plug-in for¬†GIMP uses¬†it to transform color images into nice geometric abstraction. Each generated triangle color can be random, constant or related to the pixels under the triangle. The triangulation tries to stick to the image contours (more or less, depending on the chosen parameters).

_Fig.4.4. Aperçu du filtre "Polygone [delaunay]" et application sur une image pour un rendu de type "vitrail"._

Fig.4.4. Overview of the Polygone [delaunay] filter, and application on a color image to simulate a stained glass effect.

Applying this filter on image sequences gives also nice outcomes. I wonder if importing¬†these triangulated images into Blender could be of¬†any interest (even better, a full G’MIC plug-in for Blender, maybe ? :p )

_Fig.4.5. Application de la triangulation de Delaunay sur une séquence d'images._

Fig.4.5. Applying the Delaunay triangulation on an image sequence.

4.3. Other artistic filters.

As you can see, G’MIC is a very¬†active project, and the number of available artistic filters increases every day. Thus, I can’t describe in details all the new filters added these ten last months, and because¬†a picture¬†is worth a thousand words, you will find below a quick overview of some of them.

_Fig.4.6. Aperçu du filtre "Arrays & Tiles / Grid [hexagonal]", potentiellement utile pour les créateurs de cartes des jeux de type Wargames ?_

Fig.4.6. Overview of the filter Arrays & Tiles / Grid [hexagonal], maybe useful for wargames map makers ?

_Fig.4.7. Aperçu du filtre "Patterns / Crystal" qui transforme vos images en cristaux multicolores._

Fig.4.7. Overview of the filter Patterns / Crystal which convert your images into colorful crystals.

_Fig.4.8. Aperçu du filtre "Rendering / Lightning" qui dessine un éclair dans une image._

Fig.4.8. Overview of the filter Rendering / Lightning which draws lightnings in your images.

_Fig.4.9. Aperçu du filtre "Arrays & Tiles / Ministeck" qui transforme vos images en représentation de type [Ministeck](http://de.wikipedia.org/wiki/Ministeck) (jeu pour enfants)._

Fig.4.9. Overview of the filter Arrays & Tiles / Ministeck which transforms your images into Ministeck games (child games).

_Fig.4.10. Aperçu du filtre "Sequences / Spatial transition", qui prend en entrée plusieurs calques et qui génère une séquence d'images de transition entre chaque calque consécutif, avec des motifs de transition réglables._

Fig.4.10. Overview of the filter Sequences / Spatial transition, which takes several layers as input, and generates an image sequence corresponding to a custom transition between each consecutive layers.

5. A quick view of the other improvements

Of course, the few new filters I’ve presented above are only a small part of the overall work that has been done on G’MIC¬†(the most visible part). Here is a list of other notable (mostly technical) improvements:

5.1. Global improvements and the libgmic library.

  • A lot of work has been done to clean and optimize the entire¬†source code. The size of the¬†libgmic¬†library has been drastically reduced (currently less than¬†5 MB), with an improved C++ API. Using some of the¬†C++11¬†features (rvalue-references) allows¬†me to avoid temporary buffer copies which is really nice in the context of image processing as the allocated memory may become huge quickly. With the help of¬†Lukas Tvrdy,¬†one of the¬†Krita developers, we have been able to improve the compilation of the project a little bit on Windows (with Visual Studio), and run various code analysis tools to detect and fix memory leaks and strange behavior. Many¬†sessions of valgrind, gprof, g++ with the -fsanitize=address option, PVS Studio has successfully led to effective code improvements. Not very visible or¬†exciting for me, but probably very satisfying for the final user ūüôā As a consequence, I’ve decided the code was clean enough to regularly propose G’MIC¬†pre-releases¬†that are considered as stable enough while getting the latest features developed. The method¬†of releasing¬†G’MIC is now closer to a Rolling Release scheme.
  • The compilation of the default¬†G’MIC binaries on Windows has been improved. It now uses the latest¬†g++-4.9.2 / MinGW¬†as the default compiler, and new installers for Windows 32 bits / 64 bits are now available.
  • G’MIC¬†has new commands to compress/uncompress arbitrary data on fly (through the use of the zlib). So now, the embedded G’MIC commands are stored in a compressed form and the Internet updates are now faster.
  • The¬†G’MIC¬†project got its own domain name¬†http://gmic.eu, independent from¬†Sourceforge. We have improved the web pages a lot, particularly thanks to the wonderful tutorials on the use of the G’MIC command-line tool, written by Garry Osgood (again, thanks Garry!). This is clearly a page to bookmark for people who slowly want to learn the basics of G’MIC.

5.2. New possibilities for image filtering.

A bunch of new commands dedicated to image filtering, and their associated filters in the¬†G’MIC¬†plug-in for¬†GIMP have been added, for image sharpening (Mighty Details), deconvolution by an arbitrary kernel (Richardson-Lucy¬†algorithm), guided filtering, Fast Non-Local Means, Perona-Malik anisotropic diffusion, box filtering, DCT transforms, and so on. Honestly, G’MIC now implements so many different image filtering techniques that it is a must-have for any image processing person¬†(even only for the denoising techniques it provides). For the users of the plug-in, this means potentially more diverse filters in the future. The figure below shows an application of the¬†Mighty Details filter on a portrait, to simulate a kind of Dragan effect.

_Fig.5.1 Application du filtre ¬ę Mighty Details ¬Ľ sur un portrait pour le rehaussement de d√©tails._

Fig.5.1 Application of the Mighty Details filter on a portrait to enhance details.

5.3. Improvements of the G’MIC plug-in for GIMP.

The¬†plug-in for¬†GIMP¬†being the most used interface of the¬†G’MIC project, it was necessary to make a lot of improvements:

  • The plug-in now has an¬†automatic update system which ensures you always get the latest new filters and bug fixes. At the date of the¬† release, the plug-in counts¬†430 different filters, plus¬†209 filters still considered in development (in the¬†Testing/¬†category). For a plug-in taking 5.5 Mb on the disk, let us say this makes an awesome ratio of¬†number of filters / disk space. A complete list of the filters¬†available in the plug-in for GIMP can be seen here.
  • During the execution of a filter that takes a bit of time, the plug-in now displays information about the elapsed time and the used memory in the progress bar.
_Fig.5.2. Affichage des ressources prises par un filtre dans le greffon G'MIC pour GIMP._

Fig.5.2. Display of the used resources in the progress bar when running a G’MIC filter in¬†GIMP.

  • The plug-in GUI has been slightly improved: the preview window is more accurate when dealing with multi-layer filters. The filters of the plug-in can be interactive now: they can open their own display window, and handle user events. A filter can also decide to modify the value of its parameters. All of this allowed me to code the new interactive filters presented below (as the one for the Comics colorization, for the color curves in an arbitrary color space, or for the interactive foreground extraction). This¬†is something that was not possible before, except when using the command-line interface of¬†G’MIC.
  • A¬†G’MIC¬†filter now has knowledge about all the input layer properties, such as their position, their opacities and their blending mode. A filter can modify these properties for the output layers too.
  • A new filter¬†About / User satisfaction survey¬†has been added. You can give some anonymous information about your use of¬†G’MIC. The result of this survey is visible here (and updated in almost real-time). Of course, this result image is generated by a¬†G’MIC¬†command itself :).

5.4. G’MIC without any limits : Using the command-line interface.

gmic is the command-line interface of¬†G’MIC,¬†usable from a shell. That is undoubtly the most powerful G’MIC¬†interface, without any limitations inherent to the input/output constraints of the other interfaces (like being limited to 2D images, 8bits / channels, 4 channels max on¬†GIMP, or a single input image in the¬†G’MIC Online¬†site, etc.). With the CLI interface, you can load/save/manipulate sequences of 3D volumetric images, with an arbitrary number of channels, float-valued, without limits other than the available memory. The CLI interface of¬†G’MIC¬†has also gotten a lot of improvements:

  • It now uses by default¬†OpenCV¬†to load/save image sequences. We can then apply image processing algorithms frame by frame on video files (command¬†-apply_video), on webcam streams (command¬†-apply_camera), or generate video files from still images for instance. New commands¬†-video2files and¬†-files2video have been added to easily decompose/recompose video files into/from several frames. Processing video files is almost childs play¬†with¬†G’MIC now. The G’MIC manual (command -help)¬†has been also improved, with a colored output, proposed corrections in case of typos, links to tutorials when available, etc..
_Fig.5.3. Aperçu du fonctionnement de l'aide en ligne de commande._

Fig.5.3. Overview of the help command with the command line interface gmic.

  • The invokation of¬†gmic¬†on the command line without any arguments makes it enter a¬†demo¬†mode, where one can select between a lot of different small interactive animations from a menu, to get an idea of what¬†G’MIC is capable of. A good occasion to play¬†Pacman, Tetris¬†or the minesweeper on the pretext to test a new serious image processing software!
_Fig.5.4. Aperçu de G'MIC, lancé en mode démo !_

Fig.5.4. Overview of the demo mode¬†of¬†G’MIC.

5.5. Other G’MIC interfaces being developed.

  • ZArt is another available¬†G’MIC interface initially developed as a demonstration platform, for processing images from the webcam. We use it classically for public exhibitions like the “Science Festival” we have once a year in France. It is very convenient to illustrate what image processing are, and what they can do, for the general public.¬†ZArt has been significantly improved. First, it is now able to import any kind of video files (instead of supporting only webcam streams). Second, most of the filters from the¬†GIMP¬†plug-in has been imported to¬†ZArt. This video shows these new possibilities, for instance with real-time emulation of analog films of a video stream. This could be really interesting to get a¬†G’MIC plug-in working for video editing software, as the brand new Natron¬†for instance. I’m already in touch with some of the¬†Natron developers, so maybe this is something doable in the future. Note also that the code of¬†ZArt have been upgraded to be compatible with the newest¬†Qt 5.
_Fig.5.5. Aperçu du logiciel ZArt traitant une vidéo en temps réel._

Fig.5.5. Overview of ZArt, a G’MIC-based video processing software.

  • It has been a long time now since¬†Lukas Tvrdy,¬†member of the Krita devs, has started to develop a¬†G’MIC plug-in for¬†Krita. This plug-in is definitely promising. It contains all of the elements of the current¬†G’MIC plug-in for¬†GIMP, and it already works quite well on GNU/Linux. We are discussing to try solving some problems remaining for the compilation of the plug-in on¬†Windows. Using the¬†Krita¬†plug-in is really nice because it is able to generate 16bits/channel images, or even more (32bits float-valued), whereas the plug-in for¬†GIMP¬†is currently limited to¬†8bits/channel for the plug-in API. This may change in the future with¬†GEGL, but right now, there is¬†not enough documentation available to ease the support of¬†GEGL for the¬†G’MIC plug-in. So now, if you want to process 16bits/channel images, you can do this only through the command-line tool¬†gmic,¬†or the¬†G’MIC¬†plug-in for¬†Krita.
_Fig.5.6. Aperçu du greffon G'MIC pour Krita en action._ (Cette image provient du site web de Krita : https://krita.org/wp-content/uploads/2014/11/gmic-preview.png)

Fig.5.6. Overview of the G’MIC plug-in running on¬†Krita.
(This image has been borrowed from the Krita website : https://krita.org/wp-content/uploads/2014/11/gmic-preview.png)

  • The integration of some of the¬†G’MIC image processing algorithms has also begun¬†in¬†PhotoFlow. This is a quite recent and promising software which focuses on the development of¬†RAW images into¬†JPEG, as well as on photo retouching. This is a project done by Carmelo DrRaw, check out this project, it is really nice.
_Fig.5.7. Aperçu du filtre "Dream Smoothing" de G'MIC inclus dans PhotoFlow._ (Cette image provient du site web de PhotoFlow : http://photoflowblog.blogspot.fr/2014/12/news-gmic-dream-smoothing-filter-added.html)

Fig.5.7. Overview of the Dream Smoothing¬†filter from¬†G’MIC, included in¬†PhotoFlow.
(This image has been borrowed from the PhotoFlow website : http://photoflowblog.blogspot.fr/2014/12/news-gmic-dream-smoothing-filter-added.html)

6. Perspectives and Conclusions.

Here we are! That was a quite complete summary of what happened the ten last months around the G’MIC project. All this has been made possible also thanks to many different contributors (coders, artists, and users in general), whose number is increasing everyday. Thanks again to them!

In the future, I still dream of more¬†G’MIC integration in other open-source software. A¬†plug-in for¬†Natron could be really great to explore the possibilities of video processing with¬†G’MIC.¬†A plug-in for¬†Blender would be really great too! Having a¬†G’MIC-based¬†GEGL¬†node is again a great idea, for the upcoming¬†GIMP 3.0, but we have probably a few decades to decide ūüôā If you feel you can help in these topics, feel free to contact me. One thing I’m sure: I won’t be bored by¬†G’MIC in the next few years!

Having said that, I think it’s time to go back working on¬†G’MIC¬†again. See you next time, for other exciting news about the¬†G’MIC project! (and if you appreciate what we are doing or just the time spent, you are free to send us a nice postcard from your place, or give a few bucks for the hot chocolates I need to keep my mind clear working on¬†G’MIC, through the donation page). Thank you!

How to code a nice user-guided foreground extraction algorithm? (Addendum)

After writing my last article on the¬†Easy (user-guided) foreground extraction¬†algorithm, I realized that you could maybe think I was exaggeratedly arguing the whole algorithm can be re-coded very quickly from scratch. After all, I’ve just illustrated how the things work by using G’MIC command lines, but there is already a lot of image processing algorithms implemented in G’MIC, so it is somehow a biased demonstration. So let me just give you the corresponding C++ code for the algorithm. Here again, you may find I cheat a little bit, because I use some functions of a C++ image processing library (CImg) that actually doe most of the hard implementation work for me. But:
1. You can use this library too in your own code. The CImg Library I use here works on multiple platforms and has a very permissive license, so you probably don’t have to re-code the key image processing algorithms by yourself. And even if you want to do so, you can still look at the source code of CImg. It is quite clearly organized and function codes are easy to find (the whole library is defined in a single header file CImg.h).
2. I never said the whole algorithm could be done in few lines, only that it was easy to implement ūüôā So, dear C++ programmer, here is the simple prototype you need to implement the algorithm:

#include "CImg.h"
using namespace cimg_library;

int main() {

  // Load input color image.
  const CImg image("image.png");

  // Load input label image.
  CImg labels("labels.png");
  labels.resize(image.width(),image.height(),1,1,0);  // Be sure labels has the correct size.

  // Compute gradients.
  const float sigma = 0.002f*cimg::max(image.width(),image.height());
  const CImg blurred = image.get_blur(sigma);
  CImgList gradient = blurred.get_gradient("xy");
    // gradient[0] and gradient[1] are two CImg images which contain
    // respectively the X and Y derivatives of the blurred RGB image.

  // Compute the potential map P.
  CImg P(labels);
    P(x,y) = 1/(1 +
                cimg::sqr(gradient(0,x,y,0)) + // Rx^2
                cimg::sqr(gradient(0,x,y,1)) + // Gx^2
                cimg::sqr(gradient(0,x,y,2)) + // Bx^2
                cimg::sqr(gradient(1,x,y,0)) + // Ry^2
                cimg::sqr(gradient(1,x,y,1)) + // Gy^2
                cimg::sqr(gradient(1,x,y,2))); // By^2

  // Run the label propagation algorithm.

  // Display the result and exit.
  (image,P,labels).display("Image - Potential - Labels");

  return 0;

To compile it, using g++ on Linux for instance, you have to type something like this in a shell (I assume you have all the necessarily headers and libraries installed):

$ g++ -o foo foo.cpp -Dcimg_use_png -lX11 -lpthread -lpng -lzsses

Now, execute it:

$ ./foo

and you get this kind of result:

cimg_ef Once you have the resulting image of propagated labels, it is up to you to use it the way you want to split the original image into foreground/background layers or keep it as a mask associated to the color image. The point is that even if you add the code of the important CImg methods I’ve used here, i.e. CImg::get_blur(), CImg::get_gradient() and CImg::watershed(), you won’t probably exceed about 500 lines of C++ code. Yes, instead of a single line of G’MIC code. Now you see why I’m also a big fan of coding image processing stuffs directly in G’MIC instead of C++ ;). A last remark before ending this addendum: Unfortunately, the label propagation algorithm itself is hardly¬†parallelizable. It is based on a priority queue and basically, the choice of a new label to set depends on how the latest label has been set. This is a sequential algorithm by nature. So, when processing large images, a good idea is to do the computations and preview the foreground extraction result only on a smaller preview of the original image. Then compute the full-res mask only once, after all key points have been placed by the user. Well that’s it, happy coding!

How to code a nice user-guided foreground extraction algorithm?

Principle of the algorithm: About three months ago, I’ve added a new filter in the

G’MIC plug-in for GIMP, which allows to interactively extract a foreground object¬†present in a color image, from its background (considering a single-layered bitmap image as the input). But instead of doing this the “usual” way, e.g. by letting¬†the user crop the object manually with any kind of selection tools, this filter relies on a semi-automatic (user-guided) algorithm to do the job. Basically, the only thing the user needs to do is placing key points which tell about the nature of the underlying pixels: either they belong to the foreground (symbolized by green dots) or to the background (red dots). The algorithm tries then to propagate¬†those labels all over the image, so that every¬†image pixel get a¬†binary label (either foreground or background). The needed interaction is quite limited, and the user doesn’t need to be an image processing guru to control what he does: left/right mouse buttons to add a new foreground/background key point (or move an existing one), middle button and mouse wheel to zoom in/out and pan shift. Nothing really complicated, easy mode.


]1 Fig.1. Principle of the user-guided foreground extraction algorithm: the user puts some foreground/background key points over a color image (left), and the algorithm propagates those labels for all pixels, then splits the initial image into two foreground/background layers (middle and right).

Of course, the algorithm which performs the propagation of these binary labels is quite dumb (as most image processing algorithms actually) despite the fact it tries to follow the contours detected in the image as much as possible. So, sometimes it just assign the wrong label to a pixel. But from the user point of view, this is not so annoying because it is really easy to locally correct the algorithm estimation by putting additional key points in the wrongly labelled regions, and run a new iteration of the algorithm. Here is a quick video that shows how foreground extraction can be done when using this algorithm. Basically we start by placing one or two key points, run the algorithm (i.e. press the Space bar), then correct the erroneous areas by adding or moving key points, run the algorithm again, etc.. until the contours of the desired object to extract are precise enough. As the label propagation algorithm is reasonably fast (hence dumb), the corresponding workflow runs smoothly. https://www.youtube.com/watch?v=DUy_FqUCl_A At this stage, two observations have to be done: 1. As a user, you can easily experiment this manipulation in GIMP, if you have the G’MIC plug-in installed (and if this is not the case, do not wait one second longer and go install it! ūüėČ ). This foreground extraction filter is located in Contours / Extract foreground [interactive]. When clicking on the Apply button, it opens a new interactive window where you can start adding your foreground/background key points. Press Space to update the extracted mask (i.e. run a new iteration of the algorithm) until it fits your requirements, then quit the window and your foreground (and optionally background) selection will be transferred back to GIMP. I must say I have been quite surprised by the good feedback I got for this particular filter. I’m aware this can be probably a time-saver in some cases, but I can’t get out of my head the overall simplicity of the underlying algorithm.

]2 Fig.2. Location of the “Extract foreground” filter in the G’MIC plug-in for GIMP.

2. As a developer, you need to convince yourself that this label propagation algorithm is¬†trivial¬†to code. Dear reader, if by chance you are a developer of an open-source software for image processing or photo retouching, and you think this kind of foreground extraction tool could be a cool feature to have, then be aware this can be re-coded from scratch in a very short amount of time.¬†I’m not lying, the hardest part is actually coding the GUI. I illustrate this fact in the followings, and give some details about how this propagation algorithm is currently implemented in G’MIC.

Details of the implementation: Now I assume that the hardest work has been already done, which is the design of the GUI for letting the user place foreground/background key points. Now, we have then two known images as the input of our algorithm.


Fig.3.a. The input color image (who’s that guy ?)


Fig.3.b. An image of labels, containing the FG/BG key points

At this stage, the image of user-defined labels (in Fig.3.b.) contain pixels whose value can be only

0 (unlabeled pixels = black), 1 (labeled as background = gray) or 2 (labeled as foreground = white). The goal of the label propagation algorithm is then to set a value of 1 or 2¬†to pixels that have a current label of 0. This label propagation is performed by a so-called Watershed algorithm, which is in fact nothing more than the usual Dijkstra’s algorithm applied on a regular grid (the support of the image) instead of a generic graph. The toughest parts of the label propagation algorithm are:
1. Managing a priority queue¬†(look at the pseudo-code on the Wikipedia page of the Dijkstra’s algorithm¬†to see what I mean). This is a very classical algorithmic structure, so nothing terrible for a programmer.
2. Defining a scalar field P of potentials (or priorities) to drive the label propagation. We have to define a potential/priority value for each image pixel, that will tells if the current pixel should be labeled in priority or not (hence the need for a priority queue). So P has the same size as the input image, but has only one channel (see it as a matrix of floating-point values if you prefer). Suppose that you tells the propagation algorithm that all pixels have the same priority (so you feed it with a constant potential image

P(x,y)=1). Then, your labels will propagate such that each reconstructed label will take the value of its nearest known (user-defined) label. This will not take the image data into account, so there is no chance¬†to get a satisfactory result. Instead, the key idea is to define the potential image P such that it depends on the contour variations of the color image you want to segment. More precisely, you want pixels with low variations to have a high priority, while pixels on image structures and contours should be labeled last. In G’MIC, I’ve personally defined the potential image P from the input¬†RGB image with the following (heuristic) formula: potentials where gradient is the usual definition of the image gradient, here appearing in the potential map P for each image channel R, G and B. To be more precise, we estimate those gradients from a slightly blurred version of the input¬†image (hence the sigma added to R,G and B, which stands for the standard variation of the Gaussian blur applied on the RGB color image). Defined like this, the values of this potential map¬†P¬†are low for pixels on image contours, and high (with a maximum of¬†1) for pixels located on flat regions. If you have the command-line interface gmic of G’MIC¬†installed on your machine, you can easily compute and visualize such a propagation map P¬†for your own image. Just type this simple command line in a shell:

$ gmic image.jpg -blur 0.2% -gradient_norm -fill '1/(1+i^2)'

For our sample image above, the priority map

P looks like this (color white correspond to the value 1, and black to something close to 0):


]5 Fig.4. Estimated potential/priority map P.

Now, you have all the ingredients to extract your foreground. Run the label propagation algorithm on your image of user-defined labels, with the priority map P estimated from the input color image. It will output the desired image of propagated labels:


Fig.5.a. Input color image.


Fig.5.b. Result of the labels propagation algorithm.

And you won’t believe it, but you can actually do all these things in a single command line with¬†


$ gmic image.jpg labels.png --blur[0] 0.2% -gradient_norm[-1] -fill[-1] '1/(1+i^2)' -watershed[1] [2] -reverse[-2,-1]

In Fig.5.b., I let the foreground/background frontier pixels keep their values 

0, to better see the estimated background (in grey) and foreground (in white). Those pixels are of course also labeled in the final version of the algorithm. And once you have computed this image of propagated labels, the extraction work is almost done. Indeed, this label image is a binary mask that you can use to easily split the input color image into two foreground/background layers with alpha channels:


Fig.6.a. Estimated foreground layer, after label propagation.


Fig.6.b. Estimated background layer, after label propagation.

So, now you have a better idea on how all this works… Isn’t that a completely stupid algorithm ? Apologies if I broke the magic of the thing! ūüėÄ Of course, you can run the full¬†G’MIC foreground extraction filter, not only from the plug-in, but also from the command line, by typing:

$ gmic image.jpg -x_segment , -output output.png

The input file should be a regular

RGB image. This command generates an output file in RGBA mode, where only a binary alpha channel has been added to the image (the RGB colors of each pixel remain unchanged).

And now ? You know the secret of this nice (but simple) foreground extraction algorithm. So you have two options:

  1. If you are not a developer, harass the developers of your favorite graphics software to include this “so-simple-to-code” algorithm into it!
  2. If you are a developer interested by the algorithm, just try to code a prototype for it. You will be surprised by how fast this could be actually done! In any case, please don’t hate me if none of these suggestions worked ;). See you in a next post!

See also the Addendum I wrote, that describes how this can be implemented in C++.

Visualizing the 3D point cloud of RGB colors

Preliminary note:

This is the very first post of my blog on Open Source Graphics. So, dear reader, welcome! I hope you will find some useful (or at least distracting) articles here. Please feel free to comment, and tell me about your advice and experience on graphics creation, photo retouching and image processing.

Understanding how the image colors are distributed:

According to Wikipedia, the histogram of an image is “a graphical representation of the tonal distribution in a digital image. It plots the number of pixels for each tonal value. By looking at the histogram for a specific image a viewer will be able to judge the entire tonal distribution at a glance.”

This is a very usual way to better understand the global statistics of the pixel values in an image. Often, when we look at an image histogram (which is a feature that most image editors and photo retouching programs propose), we see something like this:


Fig.1. Lena image (left), and histogram of the luminances (right).

Here, the histogram (on the right) tells about the distribution of the image luminances, for each possible value of the luminance, from 0 to 255 (assuming a 8-bits per channel color image as an input). Sometimes, the displayed histogram rather contains the distribution of the pixel values simultaneously for the 3 image channels RGB like in the figure below:


Fig.2. Lena image (left), and its RGB histogram (right).

For instance, this¬†RGB histogram clearly shows that the image¬†contains a lot of pixels with high values of Red, which is in fact not so surprising for this particular image of¬†Lena. But what we still miss is the relationship between the three color components: do the pixels with a lot of Red cover a wide range of different colors, or is it just the same reddish color repeated all over the image ? Look at¬†the image below: it has exactly the same RGB¬†histogram as the image above. I’ve just mirrored the Green and Blue channels respectively along the x and y-axes. But, the perception of the image colors is in fact very different. We see a lot more greens in it.


Fig.3. Partially mirrored Lena image (left), and its RGB histogram.

So clearly, it could be interesting to get more clues about the correlation between the different color components of the pixels in an image. This requires to visualize something different, as the 3D distribution (or histogram) of the colors in the RGB space: each pixel (x,y) of a color image can be indeed viewed as a point with 3D coordinates (R(x,y),G(x,y),B(x,y)) located inside a cube of size 256x256x256, in the 3D color space RGB. The whole set of image pixels forms then a 3D point cloud in this 3D space. To visualize this pointcloud, each displayed point takes a color that can be either its actual RGB value (to get the 3D colors distribution), or a color expressing the number of occurrences of this RGB color in the initial image (to get the 3D colors histogram). For the Lena image, it looks like this. A wireframe representation of the RGB cube boundaries has been also added to better locate the image colors:


Fig.4. Lena image (left), colors distribution in 3D RGB space (middle) and colors histogram in 3D RGB space (right).

With this way of representing colors distributions, we get a better feeling about:

  1. The global variety of the colors in an image.
  2. Locally, the amount of dispersion of color tones and shades around a given color.

I personally prefer the 3D colors distribution view (Fig.4.middle), as it is easier to see which color corresponds to which point, even if we lost the information about the color occurrences (but this is still hardly visible in the 3D colors histogram). Now, if we compare the 3D colors distributions of the Lena image and its partially-mirrored version (remember, they have the same RGB histogram), we can see a big difference !


Fig.5. Comparisons of the colors distributions in the 3D RGB space, for the Lena image, and its partially-mirrored version.

Let me show you some other examples, to illustrate that visualizing 3D colors distributions is a nice additional way to get more global information about how the colors of an image are arranged:


Fig.6. Las Vegas image (left) and its colors distribution in the 3D RGB space (right).

The image above has been shot by Patrick David in Las Vegas. We can see in the 3D colors distribution view, which colors are present and the input image (which is already quite colorful), and which are not (purples are almost non-existent). Below is another image (a portrait of Mairi) shot by Patrick:


Fig.7. Portrait of Mairi (left) and its colors distribution in the 3D RGB space (right).

Here, the 3D colors distribution reflects the two dominant colors of the image (greys and skin/hair colors) and the continuity of their shading into the black. Note that this is not something we can see from the RGB histogram of the same input image, so this kind of visualization really says something new about the global statistics of the image colors.


Fig.8. Portrait of Mairi (left) and its RGB histogram (right).

Let me show you some more extreme cases:


Fig.9. Bottles image (left) and its colors distribution in the 3D RGB space (right).

We can clearly see three main color branches, corresponding to each of the three main colors of the bottles (blue, pink and yellow-green), with all the shades (to white) the corresponding pixels take. Now, let us take the same image which has been desaturated:


Fig.10. Desaturated bottles image (left) and its colors distribution in 3D RGB space (right).

No surprise, the only 3D color points you get belong to the diagonal of the RGB cube, i.e. the points where R = G = B. This last example shows the opposite case with a (synthetic) image containing a lot of different colors :


Fig.11. Synthetic plasma image (left) and its colors distribution in 3D RGB space (right).

I think all these examples illustrate quite well that we can often learn a lot about the colors dispersion of an image by looking at his 3D colors distribution, in addition to the usual histograms. This is somehow something I miss in many image editors and image retouching software.

So, my next step will be to show how I’ve create those views, and how you can do the same with your own images. And of course, only with¬†open-source imaging tools ūüėČ ¬†I chose to use the command-line interface of G’MIC for this particular task, because it is quite flexible to manage 3D vector objects (and also because I know it quite well!).

Generating 3D colors distribution views with G’MIC:

I assume you have already the command-line interface of¬†G’MIC¬†(which is called gmic) installed on your system. The commands I show below should be typed in a shell (I use bash¬†on a machine running Linux, personally). The first thing to do after installing gmic is to ensure we get the latest update of the commands proposed in the G’MIC framework:

$ gmic -update

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Update commands from the latest definition file on the G'MIC server.
[gmic]-0./ End G'MIC interpreter.

Now, the main trick consists in using the specific G’MIC command -distribution3d, which converts an input color image as a 3D vector object corresponding to the desired colors distribution we want to visualize. I will also add two things to improve it a little bit:

  1. if the image is too big, I will reduce it to a reasonnable size,¬†because the number of points in the generated 3D object is always equal to the number of pixels in the image, and we don’t want it to be unnecessarily large, for performance reasons.
  2. I will also merge the generated 3D pointcloud with the boundaries of the RGB color cube, in wireframe mode, as in the examples you’ve seen in the previous section.

This long (but comprehensive) command line below does the job:

$ gmic input_image.jpg -resize2dy "{min(h,512)}" -distribution3d -colorcube3d -primitives3d[-1] 1 -+3d

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input_image.jpg' at position 0 (1 image 3283x4861x1x3).
[gmic]-1./ Resize 2d image [0] to 512 pixels along the y-axis, preserving 2d ratio.
[gmic]-1./ Get 3d color distribution of image [0].
[gmic]-1./ Input 3d RGB-color cube.
[gmic]-2./ Convert primitives of 3d object [1] to segments.
[gmic]-2./ Merge 3d objects [0,1].
[gmic]-1./ Display 3d object [0] = '[3d distribution]*' (177160 vertices, 177176 primitives).

It opens an interactive visualization window with the 3D colors distribution that you can rotate with your mouse:


Nice! But let me go one step further.

Now, I want to generate an animation (a video file basically) showing the 3D colors distribution object rotating, just beside the input image. For this, I can use the command -snapshot3d¬†inside a -do…-while loop in order to get all the images of my animation. I won’t enter into much technical details because this post is not about the G’MIC programming language (for this, I refer the interested reader to these nicely written tutorial pages), but basically we can do this task easily by defining our own custom command for G’MIC. Just copy/paste the code snippet below into your¬†$HOME/.gmic configuration file (or %APPDATA%/gmic for Windows users) in order to create a new command named¬†-animate_color_distribution3d,¬†that will be recognized by the G’MIC interpreter next time you invoke it:

animate_color_distribution3d :

  # Create 3d object from input image.
    -resize2dy {min(h,512)} -distribution3d
    -colorcube3d -primitives3d[-1] 1
    -+3d -center3d -normalize3d -*3d 180
    -rotate3d 1,1,0,60

  # Resize input image so that its height is exactly 300.
  -resize2dy[0] 300

  # Render animated frames (with height 300).
    --rotate3d[1] 0,1,0,$angle
    -snapshot3d[-1] 300,0,0,0,0,0 --a[0,-1] x
    -w[-1] -rm[-2]
  -while {$angle<360}
  -rm[0,1] -resize2dx 640

Once you’ve done that, the command is ready for use. It takes no arguments, so you can simply call it like this:

$ gmic input_image.jpg -animate_color_distribution3d -output output_video.avi

And after a while, your video file should be saved in your current path. That’s it !

Below, I show some examples which have been generated by this custom G’MIC command. The first picture is a beautiful (and colorful) drawing from David Revoy. I find its 3D color distribution particularly interesting.

The landscape picture below is also interesting, because of the almost bi-tonal nature of the image:

With the Joconde painting, we can also see that the number of effective color tones is quite reduced:

And if you apply the command -animate_color_distribution3d on a synthetic plasma image, as done below, you will get a really nice 3D pointcloud, just like the ones we can see in some sci-fi movies !

And now ?

Well, that’s it ! Nothing more to say for this post :). Surprisingly, it is already much longer than expected when I’ve started writing it. Next time you see a color image, maybe you could try to visualize its color distribution in the 3D RGB space along with the classical histograms. Maybe it will help you to better understand the dynamics of the image colors, who knows ?. At least we know that this is something we can do easily with FLOSS imaging tools. See you next time !