Note on the quadratic penalties in elastic weight consolidation [Biological Sciences]

Catastrophic forgetting is an undesired phenomenon which occurs when neural networks are trained on different tasks sequentially. Elastic weight consolidation (EWC; ref. 1), published in PNAS, is a novel algorithm designed to safeguard against this. Despite its satisfying simplicity, EWC is remarkably effective. Motivated by Bayesian inference, EWC adds quadratic...

http://ift.tt/2tJVsGe