Introduction
Artificial intelligence (AI) has taken huge steps forward over the past decade. Many of the game-changing developments in the field are down to advances in machine learning (ML) and deep learning. Both fields have developed so much that the American Academy of Arts and Sciences recently referred to the past decade as the “golden decade of deep learning.”
Google’s Chimera Painter tool powered by deep learning technology allows users to bring their sketches to life. A doodle becomes a fully-fledged 3D creature model within a short space of time.
Table of contents
Here’s how it works.
Like thousands of software solutions, Chimera Painter started with the Google team recognizing a problem. Game developers were taking days to develop full 3D renderings of new creatures. Whilst that may not be problematic when there are only a handful of creatures required, character development could throw game launches off schedule when dozens were needed.
Enter Google’s Chimera Painter: Google launched this tool to accelerate game development with the help of AI. In a nutshell, that is what Chimera Painter does best – turn drawings into magical creatures, fast. But this demo accelerates game development in other ways, too.
One of the advantages of artificial intelligence over human intelligence is that computers can process far more data in far less time than the human brain. Applied to creature development, this capability means that Chimera Painter can take a handful of creatures and combine them in thousands of ways. The combinations spawn more creatures.
Combinations of creatures lie at the heart of the tool. It is not by accident that the tool takes its name from the mythological Greek fire-breathing hybrid creature. Composed of different animal parts, the original chimera was generally based on a lion with a goat’s head reaching up from its back. In many depictions, the creature’s tail ends with a snake’s head. The idea, both in mythology and in the 21st century, is to create stronger creatures by combining their traits.
Using AI to create different types of chimera allows game designers to multiply a handful of core creatures into an entire zoo – or the cast of a multi-layered game. Even if there need to be hundreds of animals in one single game, AI can develop them more or less overnight or at least much faster than an artist can draw.
Also Read: How to Make AI Generated Art
Prototyping the New Model
When development started for Google’s Chimera Painter, the idea was to produce game-ready creature images that could be combined into new hybrids. Development was based on a card game, in which players could merge the creatures on their cards to create new, stronger animals.
To make this happen, Google’s developers needed to build a tool that could generate creatures quickly. The creatures needed to retain elements of their “ancestors” so their heritage could be identified. That meant retaining some of the artistic assets from the original characters.
To allow for both artist input and high-quality digital images, the Google team used general adversarial networks (GANs) to create the creatures for its card game prototype. GANs feature two neural networks that helped Chimera Painter’s developers to hit two birds with one stone. A so-called generator network is responsible for creating new images, whilst a discriminator network ensures and checks that the new images relate to the original artists’ images.
The next step in the process was training the GAN-produced images. To achieve that, Google used 3D creature models and broke those down into their individual body parts. These segmentation maps of the creatures allowed the AI to animate each body part and generate chimeras combining multiple species.
Google’s developers tried their technology on several models and used the one that performed best to incorporate into Chimera Painter. Over time, the team moved from relatively simple combinations to more complex creatures.
Also Read: Famous Pieces of AI Generated Art
Training to Generate Creatures with Structure
The developers chose to work with GANs to be able to create new images whilst retaining the original artistic features. From that perspective, GANs were a perfect choice, but they also bring their own challenges to the process. Those challenges include the danger of losing some spatial and anatomical coherence during the rendering process.
The loss of coherence usually occurs with low-contrast or subtle parts of the images are being rendered. The problem is that these parts may seem subtle to the AI, but they are important to the human viewing experience. Fingers, eyes, or overlapping body parts can be especially problematic.
To resolve these issues, the developers took an artist-led approach to training the machine learning algorithms for Chimera Painter. By using a semi-automatic process, the team could scale and iterate rapidly.
Artists would use 3D creature models to produce two texture sets that could then be overlaid. One layer featured the creature’s full-color texture. The second layer provided a segmentation map with flat colors for each body part. The second layer allowed the ML model to understand the different body parts and their structure, shape, and proportions for each creature.
With each creature broken down into these parts, artists and developers then applied automated scripts to interpolate the layers between various poses. They also applied different zoom levels and perspectives. This process generated the training dataset for the GAN with a total of more than 10,000 images and segmentation maps for each creature model.
Developers estimate that this process might have saved millions of hours of artists’ time without removing artistic input from the development.
Also Read: Introduction to Generative Adversarial Networks (GANs)
Adjusting GANs for Better Results
GANs were essential to the development of Chimera Painter. But as we mentioned above, they also introduced additional challenges to the process. However, the advantages of GANs outweighed the downsides. One of those advantages is the sheer adjustability of their parameters.
To understand which parameters produced the best chimeras, the developers went back to the artists. Digital artists were asked to rate the versions of the creatures generated by the early versions of the tool’s models. They evaluated the realism of the creatures’ eyes and faces as well as their textures, and style.
Based on the artists’ feedback, the next version of the model could be trained. One of the main functions the training focused on was the perpetual loss component. During their iterative approach, the developers discovered that perpetual loss weights were instrumental in figuring out what final images needed to look like.
The developers noticed that choosing specific values delivered sharper, more recognizable facial features. Those features help even the most fantasy-inspired creatures appear more real. Understanding how to fine tune the GANs helped overcome their biggest drawback and lay the foundations for the development of realistic, yet fantastic chimeras.
Chimera Painter Step-By-Step User Guide
Getting started with Chimera Painter could not be easier. Here is a step-by-step guide to get you started
- Head to Google Chimera Painter and open the tool.
- Notice that there are three main areas in front of you. On the left, there is a panel providing access to the creature’s body parts. To the right of that is an image of the segmented creature. Right again is a depiction of the finished, animated chimera. It is a simple, easily accessible setup.
- You will see a work area with preloaded creatures to practice on and play with. If you do not already have your own drawings, these creatures are perfect for getting to know Chimera Painter. Alternatively, you can upload your drawings and transform them.
- Either way, the creatures are broken down into three sections: head, body, and legs. Notice that the creature’s body is already segmented into individual parts, allowing you to quickly select the area you want to change. The colors on the creature match the blocks on the left-hand panel. It is simply a matter of clicking on the correct part.
- There is no limit to how much you can transform the creature in front of you. Once you are done with a specific body part, all you need to do is hit the “transform” button at the top to see your changes incorporated into the final 3D image. Rendering only takes a few seconds.
Give it a try today! Chimera Painter is currently a demo, and it is a fantastic AI tool for artists to enjoy and incorporate into their process.
Chimera Painter is not here to replace digital artists. It is simply a tool to help make their work quicker and easier. In that way, this tool is no different from some of the leading AI applications. They are not designed to replace human intelligence. Instead, they were developed to augment human intelligence and support the work humans do in ways that only technology can.
Conclusion
Chimera Painter, a product of Google Research and Google AI, has emerged as a transformative tool in the world of digital video games and art software, redefining the way game artists create high-quality art. This demonstration tool harnesses the power of machine learning models, specifically Generative Adversarial Networks (GANs), to turn user-supplied or preset creature outlines into fantastical creatures in real-time. Whether it’s single-species creature outlines or complex multi-species chimeras, the model takes separate inputs of body part labels and creature shapes to generate high-quality creature card images with impressive spatial coherence and detailed textures.
In the fast-paced environment of game development, artists often face tight deadlines to generate a high volume of assets. The traditional art pipeline involves creating creatures from scratch or modifying a 3D model – a time-consuming and often iterative process. This is where Chimera Painter stands out. By generating a volume of images within moments, this tool allows game artists to focus more on the creative process and less on the technical knowledge usually required to design such detailed creatures. With a wide range of creature types at their disposal, artists can rapidly prototype a fantasy card game, for instance, accelerating the design process and boosting productivity.
One of the key features of Chimera Painter that sets it apart from other art generators is its ability to maintain the placement of creature parts in the final creature designs. This feature offers a degree of control that is often lacking in other GAN-generated images. Also, its model parameters and hyper-parameters are fine-tuned to ensure that the final output is not just unique, but also matches the original images’ quality and the creative vision of the artists. The creative environment of Chimera Painter allows for an exploration of creature categories, from basic creatures to more advanced fantasy creatures, paving the way for limitless creativity.
Chimera Painter is more than just an art tool; it’s an embodiment of how Google’s AI and Google Creative Lab are leveraging advanced machine learning to augment the creative capabilities of artists. While it’s currently focused on creature designs for digital video games, the potential applications of such technology are vast, ranging from the creation of unique art for books and films to bypassing licensing restrictions for generating new content. As the technology matures, the future holds the promise of a world where creating high-quality art is democratized, with everyone from professionals to novices being able to create and experiment in ways that were previously unimaginable.
