Parallax(chunk_size=128, local_proxy_variable=False, sync=True, staleness=0)¶
Generates the Parallax Strategy from https://arxiv.org/pdf/1808.02621.pdf.
The Parallax strategy mixes Parameter Server and AllReduce. The rationale is that a PS architecture is more suitable for sparse gradient updates, while AllReduce has reportedly better performance on dense gradient updates.
Generate the strategy.