Visualizing the 3D point cloud of RGB colors


Preliminary note:

This is the very first post of my blog on Open Source Graphics. So, dear reader, welcome! I hope you will find some useful (or at least distracting) articles here. Please feel free to comment, and tell me about your advice and experience on graphics creation, photo retouching and image processing.


Understanding how the image colors are distributed:

According to Wikipedia, the histogram of an image is “a graphical representation of the tonal distribution in a digital image. It plots the number of pixels for each tonal value. By looking at the histogram for a specific image a viewer will be able to judge the entire tonal distribution at a glance.”

This is a very usual way to better understand the global statistics of the pixel values in an image. Often, when we look at an image histogram (which is a feature that most image editors and photo retouching programs propose), we see something like this:

histogram_luminance1

Fig.1. Lena image (left), and histogram of the luminances (right).

Here, the histogram (on the right) tells about the distribution of the image luminances, for each possible value of the luminance, from 0 to 255 (assuming a 8-bits per channel color image as an input). Sometimes, the displayed histogram rather contains the distribution of the pixel values simultaneously for the 3 image channels RGB like in the figure below:

histogram_colors

Fig.2. Lena image (left), and its RGB histogram (right).

For instance, this RGB histogram clearly shows that the image contains a lot of pixels with high values of Red, which is in fact not so surprising for this particular image of Lena. But what we still miss is the relationship between the three color components: do the pixels with a lot of Red cover a wide range of different colors, or is it just the same reddish color repeated all over the image ? Look at the image below: it has exactly the same RGB histogram as the image above. I’ve just mirrored the Green and Blue channels respectively along the x and y-axes. But, the perception of the image colors is in fact very different. We see a lot more greens in it.

histogram_colors2

Fig.3. Partially mirrored Lena image (left), and its RGB histogram.

So clearly, it could be interesting to get more clues about the correlation between the different color components of the pixels in an image. This requires to visualize something different, as the 3D distribution (or histogram) of the colors in the RGB space: each pixel (x,y) of a color image can be indeed viewed as a point with 3D coordinates (R(x,y),G(x,y),B(x,y)) located inside a cube of size 256x256x256, in the 3D color space RGB. The whole set of image pixels forms then a 3D point cloud in this 3D space. To visualize this pointcloud, each displayed point takes a color that can be either its actual RGB value (to get the 3D colors distribution), or a color expressing the number of occurrences of this RGB color in the initial image (to get the 3D colors histogram). For the Lena image, it looks like this. A wireframe representation of the RGB cube boundaries has been also added to better locate the image colors:

coldis_lena

Fig.4. Lena image (left), colors distribution in 3D RGB space (middle) and colors histogram in 3D RGB space (right).

With this way of representing colors distributions, we get a better feeling about:

  1. The global variety of the colors in an image.
  2. Locally, the amount of dispersion of color tones and shades around a given color.

I personally prefer the 3D colors distribution view (Fig.4.middle), as it is easier to see which color corresponds to which point, even if we lost the information about the color occurrences (but this is still hardly visible in the 3D colors histogram). Now, if we compare the 3D colors distributions of the Lena image and its partially-mirrored version (remember, they have the same RGB histogram), we can see a big difference !

coldis_lenaflip

Fig.5. Comparisons of the colors distributions in the 3D RGB space, for the Lena image, and its partially-mirrored version.

Let me show you some other examples, to illustrate that visualizing 3D colors distributions is a nice additional way to get more global information about how the colors of an image are arranged:

coldis_vegas

Fig.6. Las Vegas image (left) and its colors distribution in the 3D RGB space (right).

The image above has been shot by Patrick David in Las Vegas. We can see in the 3D colors distribution view, which colors are present and the input image (which is already quite colorful), and which are not (purples are almost non-existent). Below is another image (a portrait of Mairi) shot by Patrick:

coldis_mary

Fig.7. Portrait of Mairi (left) and its colors distribution in the 3D RGB space (right).

Here, the 3D colors distribution reflects the two dominant colors of the image (greys and skin/hair colors) and the continuity of their shading into the black. Note that this is not something we can see from the RGB histogram of the same input image, so this kind of visualization really says something new about the global statistics of the image colors.

mary_histo

Fig.8. Portrait of Mairi (left) and its RGB histogram (right).

Let me show you some more extreme cases:

coldis_bottles

Fig.9. Bottles image (left) and its colors distribution in the 3D RGB space (right).

We can clearly see three main color branches, corresponding to each of the three main colors of the bottles (blue, pink and yellow-green), with all the shades (to white) the corresponding pixels take. Now, let us take the same image which has been desaturated:

coldis_bottlesbw

Fig.10. Desaturated bottles image (left) and its colors distribution in 3D RGB space (right).

No surprise, the only 3D color points you get belong to the diagonal of the RGB cube, i.e. the points where R = G = B. This last example shows the opposite case with a (synthetic) image containing a lot of different colors :

coldis_plasma

Fig.11. Synthetic plasma image (left) and its colors distribution in 3D RGB space (right).

I think all these examples illustrate quite well that we can often learn a lot about the colors dispersion of an image by looking at his 3D colors distribution, in addition to the usual histograms. This is somehow something I miss in many image editors and image retouching software.

So, my next step will be to show how I’ve create those views, and how you can do the same with your own images. And of course, only with open-source imaging tools 😉  I chose to use the command-line interface of G’MIC for this particular task, because it is quite flexible to manage 3D vector objects (and also because I know it quite well!).


Generating 3D colors distribution views with G’MIC:

I assume you have already the command-line interface of G’MIC (which is called gmic) installed on your system. The commands I show below should be typed in a shell (I use bash on a machine running Linux, personally). The first thing to do after installing gmic is to ensure we get the latest update of the commands proposed in the G’MIC framework:

$ gmic -update

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Update commands from the latest definition file on the G'MIC server.
[gmic]-0./ End G'MIC interpreter.

Now, the main trick consists in using the specific G’MIC command -distribution3d, which converts an input color image as a 3D vector object corresponding to the desired colors distribution we want to visualize. I will also add two things to improve it a little bit:

  1. if the image is too big, I will reduce it to a reasonnable size, because the number of points in the generated 3D object is always equal to the number of pixels in the image, and we don’t want it to be unnecessarily large, for performance reasons.
  2. I will also merge the generated 3D pointcloud with the boundaries of the RGB color cube, in wireframe mode, as in the examples you’ve seen in the previous section.

This long (but comprehensive) command line below does the job:

$ gmic input_image.jpg -resize2dy "{min(h,512)}" -distribution3d -colorcube3d -primitives3d[-1] 1 -+3d

[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input file 'input_image.jpg' at position 0 (1 image 3283x4861x1x3).
[gmic]-1./ Resize 2d image [0] to 512 pixels along the y-axis, preserving 2d ratio.
[gmic]-1./ Get 3d color distribution of image [0].
[gmic]-1./ Input 3d RGB-color cube.
[gmic]-2./ Convert primitives of 3d object [1] to segments.
[gmic]-2./ Merge 3d objects [0,1].
[gmic]-1./ Display 3d object [0] = '[3d distribution]*' (177160 vertices, 177176 primitives).

It opens an interactive visualization window with the 3D colors distribution that you can rotate with your mouse:

gmic_view

Nice! But let me go one step further.

Now, I want to generate an animation (a video file basically) showing the 3D colors distribution object rotating, just beside the input image. For this, I can use the command -snapshot3d inside a -do…-while loop in order to get all the images of my animation. I won’t enter into much technical details because this post is not about the G’MIC programming language (for this, I refer the interested reader to these nicely written tutorial pages), but basically we can do this task easily by defining our own custom command for G’MIC. Just copy/paste the code snippet below into your $HOME/.gmic configuration file (or %APPDATA%/gmic for Windows users) in order to create a new command named -animate_color_distribution3d, that will be recognized by the G’MIC interpreter next time you invoke it:

animate_color_distribution3d :
    
  angle=0
  delta_angle=2

  # Create 3d object from input image.
  --local
    -resize2dy {min(h,512)} -distribution3d
    -colorcube3d -primitives3d[-1] 1
    -+3d -center3d -normalize3d -*3d 180
    -rotate3d 1,1,0,60
  -endl

  # Resize input image so that its height is exactly 300.
  -resize2dy[0] 300

  # Render animated frames (with height 300).
  -do
    --rotate3d[1] 0,1,0,$angle
    -snapshot3d[-1] 300,0,0,0,0,0 --a[0,-1] x
    -w[-1] -rm[-2]
    angle={$angle+$delta_angle}
  -while {$angle<360}
  -rm[0,1] -resize2dx 640

Once you’ve done that, the command is ready for use. It takes no arguments, so you can simply call it like this:

$ gmic input_image.jpg -animate_color_distribution3d -output output_video.avi

And after a while, your video file should be saved in your current path. That’s it !

Below, I show some examples which have been generated by this custom G’MIC command. The first picture is a beautiful (and colorful) drawing from David Revoy. I find its 3D color distribution particularly interesting.

The landscape picture below is also interesting, because of the almost bi-tonal nature of the image:

With the Joconde painting, we can also see that the number of effective color tones is quite reduced:

And if you apply the command -animate_color_distribution3d on a synthetic plasma image, as done below, you will get a really nice 3D pointcloud, just like the ones we can see in some sci-fi movies !


And now ?

Well, that’s it ! Nothing more to say for this post :). Surprisingly, it is already much longer than expected when I’ve started writing it. Next time you see a color image, maybe you could try to visualize its color distribution in the 3D RGB space along with the classical histograms. Maybe it will help you to better understand the dynamics of the image colors, who knows ?. At least we know that this is something we can do easily with FLOSS imaging tools. See you next time !

12 thoughts on “Visualizing the 3D point cloud of RGB colors

  1. Great article. I hadn’t heard of G’Mic before! Gotta check it out. Thanks!
    -Joe

  2. Congratulations on getting things setup!

  3. Very interesting!

    Some improvements needed: 1) all videos are same. 2) the code in first snippet is out of line and when a paste it in my terminal it produce a error. I paste something like this ‘gmic Image.jpg -resize2dy “{min(h,512)}” -distribution3d -colorcube3d -primitives3d[-1] 1 -+3’ And I got,
    *** Error *** Unknown command or filename ‘-+3’ (did you mean ‘-+’ ?)

  4. The post is very useful. Thank you David!

    That reminds me a program like 3D LUT for Photoshop, for example http://3dlutcreator.com/
    It would be nice if we had something like this in open-source.

  5. A very good start, David! The structure of the distributions are themselves interesting, qualifying an image in a very particular way that would be hard to grasp when considering an image by itself.

    And, yes, there always seem to be more words at the end of the day than what the start of the day promised… 😉

  6. Hey David. Some additional cool things that you might list are the generators out there such as Fractal programs. Anyway, found your link at GIMPChat. Look forward to seeing how this site develops. 🙂

  7. Hi David,
    Nice article – thank you for writing it!

    A little head-up — on my mexus 7, with Chrome, the code snippets are truncated, and there is no horizontal scrollbar. The main one is cut around -color…

    • Thanks Romano. You are right, I’ve changed the code snippets again, and it seems to work better now. Thanks for reporting.

  8. Thanks David! I enjoyed the article and wonder what visualizing my favorite developed raw photos in this way could tell me about my taste, and how application of this kind of color visualization could be used to make developing raws both faster and easier, since if I identify a pattern based on analyzing many previously developed photos maybe I could more easily replicate this pattern in further work. Maybe I like a particular set of shades and contrasts, and visualizing a 3D point cloud may point them out very easily. Knowing this, it would be easier to tweak photos as I process them to match the pattern I generally see in the distribution. I’ve learned that eyeballing the image preview as I tweak photos is not very reliable because the longer I stare at an image the more I adapt to it, and even if I end up with something which I think looks great, if I take a break and look at the monitor after some minutes, what I now see is very different to what I thought I saw before the break due to psycho-visual phenomena. I’ve always relied on the traditional histogram to be objective, but this type of visualization might prove more useful.

  9. Thanks for posting this David, this is an eye-opener. I’ve been wondering for some time now how to visualize the tone distributions of images and compare them. Now if only we could figure out a semi-automatic way of pushing an image closer to another one (color-wise) based on those distributions…