Generative adversarial nets as implicit distribution

[masa-su]

We would like to implement generative adversarial nets as "implicit distribution" as follows:

z = InputLayer((None, z_dim))
g  = batch_norm(DenseLayer(z,num_units=512,nonlinearity=activation))
g  = batch_norm(DenseLayer(g,num_units=512,nonlinearity=activation))
g_mean = DenseLayer(g,num_units=x_dim,nonlinearity=sigmoid)
g = Deterministic(g_mean,given=[z]) #p(x|z)

x = InputLayer((None, x_dim))
d_0  = DenseLayer(x,num_units=512,nonlinearity=leaky_rectify)
d_1  = DenseLayer(d_0,num_units=512,nonlinearity=leaky_rectify)
d_mean = DenseLayer(d_1,num_units=1,nonlinearity=sigmoid)
d = Bernoulli(d_mean,given=[x])

p = ImplicitDistribution(mean=g, loss=d)