## Variational AutoEncoder (VAE)

vae.VAE( latent_dim, weight_stddev=0.1, itr=5000, name="vae", hidden_encoder_dim=100,
hidden_decoder_dim=100, batch_size=None, KL_param=1, mode="learn"  )


vae.VAE is a module for dimensional compression. It sends compressed latent variables to the connected module. The variational lower bound of a normal VAE is as follows:

$\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\boldsymbol{o})=-D_{KL}(q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{o})||\mathcal{N}(0,\boldsymbol{I}))+\mathbb{E}_{q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{o})}[\log{p_{\boldsymbol{\theta}}(\boldsymbol{o}|\boldsymbol{z})}].$

On the other hand, in Serket, this is optimized using messages ($$\boldsymbol{\mu}$$) received from the connected module as follows:

$\mathcal{L}(\boldsymbol{\theta},\boldsymbol{\phi};\boldsymbol{o})=- \alpha D_{KL}(q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{o})||\mathcal{N}(\boldsymbol{\mu},\boldsymbol{I}))+\mathbb{E}_{q_{\boldsymbol{\phi}}(\boldsymbol{z}|\boldsymbol{o})}[\log{p_{\boldsymbol{\theta}}(\boldsymbol{o}|\boldsymbol{z})}],$

where $$D_{KL}$$ represents KL divergence and $$\alpha$$ is the weight of KL divergence.

### Parameters

Parameter Type Description
latent_dim int Number of dimensions of latent variables
weight_stddev float Standard deviation of weight
itr int Number of iterations
name string Module name
hidden_encoder_dim int Number of nodes in encoder
hidden_decoder_dim int Number of nodes in decoder
batch_size int Number of batch sizes
KL_param float Weight of KL divergence
mode string Choose from learning mode (“learn”) or recognition mode (“recog”)

### Methods

• .connect()
This method connects this module to an observation or a module and constructs the model.
• .update()
This method estimates model parameters and computes the latent variables. The module estimates model parameters in “learn” mode and computes the latent variables of novel data element using the parameters in “recog” mode. If training is successful, then the module{n}_vae directory is created. The following files are saved in the directory ({mode} contains the selected mode (learn or recog)):
• model.ckpt: The model parameters.
• loss.txt: The losses computed in the training phase.
• x_hat_{mode}.txt: Data reconstructed by the VAE decoder.
• z_{mode}.txt: The latent variables computed by the VAE encoder.

### Example

# import necessary modules
import serket as srk
import vae
import gmm
import numpy as np

# define the modules
obs = srk.Observation( data )  # send the observation to the connected module
vae1 = vae.VAE( 10 )  # compress to ten dimensions
gmm1 = gmm.GMM( 10, catogory=data_category )  # classify into ten classes

# construct the model
vae1.connect( obs )  # connect obs to vae1
gmm1.connect( vae1 )  # connect vae1 to gmm1

# optimize the model
for i in range(5):
vae1.update()  # train vae1
gmm1.update()  # train gmm1