A beginner’s guide to AI Style TransferDigital@AdminMarch 6, 2020
In recent years AI has become intertwined in our everyday lives. From globally available, real-time maps to newer applications like self-driving cars, it seems AI is everywhere. A beginner’s guide to AI Style Transfer. AI is even one of humanity’s main trump card to solve enormous challenges like climate change, cancer research, agriculture, etc. However in creative fields like cinema, music or art AI has played only a supportive role, such as recognizing and sorting images. While this is very useful, it is not central to the creative process itself. A recently invented technique called “Style Transfer” seeks to change that.
Style Transfer, or Neural Style Transfer (or NST for short). Is an AI technique (within a branch of AI called Deep Learning) to “transfer” the style of a reference piece of art such as a painting, to a target piece of art (often a photograph) to transform the photograph into a painting-like piece of art. The result is a pastiche in the distinctive style of the reference art. This is a revolutionary step in art “creation”, both because the computer program independently makes the decision what aspects of the reference art to pick up to apply to the target art and as opposed to older techniques, does not need a pair of photo and art of the same subject to compare. Thus allowing the reference art to be pretty much anything.
Figure 1 below shows a photograph transformed into digital art. The inset in the picture on the left shows the reference style which is clearly visible in the final rendering.
Figure 1: Photograph stylized as digital art (courtesy NVidia research)
We tried it out for ourselves. Figure 1 is a photograph of a pair of swan-shaped flower vases. On the right is a painting of the two same vases and the background in the unmistakable style of Van Gogh’s masterpiece, “Starry Nights”. We used a free online application, and the new art was produced in less than a 10 seconds. Clearly neither the subject nor the style is original, and yet something is created here that’s never existed before and is therefore by definition, new art. The artistic value is clearly in choosing the right reference art for the photo in question to create the right effect.
Figure 2 (Left) and 2 (right): Neural Style Transfer of Van Gogh’s Starry Nights on a photograph
Very interesting and large arrays of pastiches can be created by ‘mixing’ styles from multiple reference art pieces:
Figure 3a (top) and b (bottom): Target photograph and the results of multi-reference mixing (courtesy: Google Brain)
Photo-to-art is one thing. Even more impressive is a photo-to-photo transfer. You see, the human mind is forgiving regarding lighting and dimensions if the art in question looks like a painting. However, that same human mind is rather unforgiving if it sees a photograph with subject features and lighting that looks anything less than real.
Figure 4: Example of Photo-to-photo transformation via Fastphotostyle algorithm (courtesy Petapixel)
Figure 5a (left) and b (right): Photorealistic style transfers by Deep Transfer (Cornell University)
All the big players in the image processing hardware and software field- Adobe, Google, Microsoft, NVidia and a host of small AI specialists as well as several universities are working on their own versions of Style Transfer for images, art and video. It isn’t difficult to understand why. and its important understand a beginner’s guide to AI Style Transfer
In an age where over three-fourths of all web traffic is audio-visual. A technique like Style Transfer is worth its weight in gold. Entire new stylizations for video, animation and still imagery can now become available to graphic and CGI artists, creative directors, advertisers as well as the rest of us in a variety of programs, from professional to freeware, opening all sorts of new possibilities for digital advertising.
Interesting new business opportunities have emerged. While most B2C value propositions revolve around creating interesting digital art and either printing it on a poster or acrylic to create affordable art, some companies have gone a step further. In one case, the website offers multiple stylizations in various artistry movement styles, from Renaissance to Cubism and allow unlimited experimentation. Once the user has found a style that creates the desired effect on the target photo. The website scans the result, recommends the medium (oil on canvas, watercolor on paper, etc.), and offers the user a chance to order a real painting in the style the user has just created. Thus creating participative art where the user creates the concept while the company executes.
Style Transfer also creates opportunities for growth for existing professional companies involved in commercial image processing, such as large outsourcing companies already proficient in image manipulation techniques. For instance, with minimal skill upgradation and workflow changes they can offer commercial style transfer services to their clients and can execute them based on the creative requirements, by tying up with a company proficient in computer vision and CNN-based Style Transfer techniques.
This new emergence also provides pure service providers to develop proficiency in particular types of niche stylizations and offer these to advertisers and marketers as creative ideas. Finally, professional companies in photo-editing, already familiar with advanced retouching techniques can utilize organizational knowledge to develop techniques for style and even feature transfers between commercial photographs. In any case, this new (r)evolution of AI offers a truly transformational opportunity in democratizing art.
