Robot Learning

A3C (Asynchronous Advantage Actor-Critic)

A3C is a deep reinforcement learning algorithm, introduced by Mnih et al. in 2016, that runs many actor-learner threads in parallel, each interacting with its own copy of the environment and asynchronously updating shared policy and value networks. The advantage estimate reduces gradient variance, while parallel exploration decorrelates training data without a replay buffer. It was largely superseded by synchronous variants such as A2C and later by PPO.

Why it matters for physical AI

Parallel actor-learner architectures pioneered by A3C underpin modern large-scale simulation training for locomotion and manipulation, where thousands of environment instances feed a single policy update loop.