Table 1 Log-likelihood comparisons across data sets.

From: Mode-assisted unsupervised learning of restricted Boltzmann machines

  S. Bar Inv. S. Bar Bars and stripes MNIST
CD-1 −20.42 −20.73 −61.08 −152.42
PCD-1 −21.71 −21.64 −57.01 −140.43
PT −20.57 −20.57 −51.99 −142.00
MT −19.85 −19.86 −50.79 (−41.82) 136.42
Exact −19.77 −19.77 −41.59
  1. We report the highest achieved log-likelihoods over 50,000 gradient updates on a 9 × 4 restricted Boltzmann machine (RBM) across various RBM types (standard, enhanced-RBM, centered-RBM) and training techniques (contrastive divergence (CD), persistent-CD (PCD), parallel tempering (PT)) as reported in previous work16 compared with mode-assisted training (MT) on a standard RBM. In the table, rows correspond to different training techniques and columns are different data sets. For each technique, the best achieved log-likelihood score across 25 runs is reported. In parenthesis are results for a 9 × 9 RBM. For these small datasets we can also compare with the exact result. For the MNIST dataset, the trained networks trained had 16 hidden nodes and PCD-1 was used as the gradient update, and average log-likelihood is reported.
  2. The highest log-likelihood achieved on a given data set is shown in bold.