This eye does not exist (Oh no, not another GAN)

May 8, 2019 by Gonçalo Abreu

A couple of weeks ago I was talking to my girlfriend. She had to do a university assignment about a new and emerging machine learning technology which would be impactful in the future so she asked me:

What's poppin in machine learning right now?

I couldn't think of anything else that was visual enough so I said:

Actually there is this cool new thing called generative adversarial networks

My head started going a thousand miles - "how the hell am I going to explain this stuff?"

(WARNING: I do terrible analogies)

Well, I knew she liked makeup, I told her the following:

Imagine that you are a makeup artist of some sort and you need to conceptualize some new eye makeup brand. But you have a problem, you are out of ideas. Generative adversarial networks allow you to create stuff that doesn't exist, lets say, photographs of eyes with makeup. This algorithm essentially allows you to "dream up" new stuff

Imagine that you need to post another Instagram eye photo to promote your brand, but for some reason you don't have the possibility of hiring a photographer or a model. Why not dream up a new photo, or thousands of them?

Ok, that was weird, oversimplified and I can already hear some machine learning purists screaming (good luck on trying to explain to your family and friends what you understand philosophically about fitting a generative model on images and using it to sample "new" ones. "Dreaming" seems like a good way after all).

I thought that this was the perfect excuse to create my first GAN (I also miss working with images since I have been looking at a lot of tabular data lately. Aren't we all!?). Since I couldn't find an "eye makeup" dataset I had to create one. Check out the Methodology page which describes that process. Here is a teaser,

The following video shows the network capability of "dreaming":

Checkout the network learning how to generate eyes with makeup:

Because the dataset is not of the highest quality there are a lot of imperfections on the generated samples. Overall the results are quite rewarding.

I would love to publish the dataset I used to train the GAN. In case anyone understands the legality issues about this subject and wants to advise me for free, please feel free to do so at,

[email protected]

Notes:

Ian Goodfellow, who came up with the first GAN formulation recently appeared on Lex Fridman MIT Artificial Intelligence podcast and gave wonderful insights. In case you want to know more about the history of GANs you should watch it: https://youtu.be/Z6rxFNMGdn0

The architecture used to generate this images was the one described on "Karras, Tero, et al. "Progressive growing of gans for improved quality, stability, and variation." arXiv preprint arXiv:1710.10196(2017)."

To train the network this repository was used.

Heavily influenced by (you should check it out),

https://thispersondoesnotexist.com/