- Tytuł:
- Probabilistic adaptive computation time
- Autorzy:
-
Figurnov, M.
Sobolev, A.
Vetrov, D. - Powiązania:
- https://bibliotekanauki.pl/articles/201248.pdf
- Data publikacji:
- 2018
- Wydawca:
- Polska Akademia Nauk. Czytelnia Czasopism PAN
- Tematy:
-
deep learning
probabilistic models
adaptive computation time
uczenie głębokie
modele probabilistyczne
adaptacyjny czas obliczeniowy - Opis:
- We present a probabilistic model with discrete latent variables that control the computation time in deep learning models such as ResNets and LSTMs. A prior on the latent variables expresses the preference for faster computation. The amount of computation for an input is determined via amortized maximum a posteriori (MAP) inference. MAP inference is performed using a novel stochastic variational optimization method. The recently proposed adaptive computation time mechanism can be seen as an ad-hoc relaxation of this model. We demonstrate training using the general-purpose concrete relaxation of discrete variables. Evaluation on ResNet shows that our method matches the speed-accuracy trade-off of adaptive computation time, while allowing for evaluation with a simple deterministic procedure that has a lower memory footprint.
- Źródło:
-
Bulletin of the Polish Academy of Sciences. Technical Sciences; 2018, 66, 6; 811-820
0239-7528 - Pojawia się w:
- Bulletin of the Polish Academy of Sciences. Technical Sciences
- Dostawca treści:
- Biblioteka Nauki
Artykuł