Computers Understand Art Same As Humans Do

Art appreciation is a tricky subject. But most would agree that this is an activity exclusive to mankind. Is that right? It is almost fashionable for society to push the envelope in modern age. So it is not surprising when computer scientists ultimately come together and design experiment that attempts to convince us the alternative view'. But this experiment is interesting nevertheless as the programmers have put together some algorithm to demonstrate computers do "understand" art, in a way not dissimilar to how arts buyers and valuators process their thinking in evaluating a piece of art.

Jane Tarakhovsky and Lior Shamir from Michigan's Lawrence Technological University are responsible for the experiment featured in the ACM Journal on Computing and Cultural Heritage recent edition. For the experiment, as much as 1,000 paintworks from 34 established artists were used. These paintings were then left to the computer algorithm to analyze. The objective is to discern as much similarity as possible across all these works. The experiment has been designed in such a way that no human intervention is allowed. The evaluation criterion is based strictly on the visual content of the exhibits. Impressively, the computer was able to recognize distinctive characteristics across the painters behind these painting. And the more amazing aspect of this experiment is that the computer shows uncanny resemblance on how art historians perceive every individual piece of artwork.

To start with, the computer had no problem to tell classical realism from modern artistic styles. It subsequently segregated the painters into 2 groups:

• 18 classical painters and,
• 16 modern painters, correctly!

The 2 board categories were then divided according the line of artistic movements, again by the computer exclusively! It was able to pick up Leonardo Da Vinci, Michelangelo and Raphael and parked them into a sub-group, a class of artists more associated with the High Renaissance era. The algorithm also selected Baroque painters like Rembrandt, Vermeer and Rubens together, yet again demonstrating its ability to identify the painters who share similar artistic styles.

Image [Click to enlarge]

The first pick of High Renaissance artists is immaculate and indisputable. The second pick of those three painters and associated them with the Baroque artistic movement would be the 'right thing to do', according to a reputable arts historian. It did not stop there, the computer went on to select Cézanne and Gauguin, both post-impressionist artists, appropriately decided that both share similar artistic styles. It was also right-on when it further grouped the work of Giorgio de Chirico, Salvador Dali and Max Ernst together, all happened to be strict sticklers of the surrealism school of art. All in all, the entire deduction process (between artistic movements and painters) was in almost entire agreement with critiques and historians alike.

Granted, anybody who is sufficiently exposed to the art scene can probably tell modern art from classical realism, but it is a different matter to appreciate and distinguish Early and High Renaissance or Mannerism and Romanticism. Indeed, this experiment demonstrated that specially prepared algorithm can do much better than untrained humans, as far as fine art analysis is concerned.

The reality is that computer does no more than computing. In order for the paintings to make sense to the computer, 4,027 numerical image context descriptors have been fed into the computer. In laymen speaks, every detail of the content (or image) has to be translated into a quantifiable numeric in order to provide context. For this experiment, shapes, colors and texture collectively make up the 4,027 numerical image context descriptors described above. Now the computer can turn on its statistical methods and pattern recognition capabilities to crunch these numbers; and makes itself a machine that understands complex patterns, discerns artistic styles and then quantifies these observations (or computations).