Parallax Strategy

Parallax StrategyBuilder.

class Parallax(chunk_size=128, local_proxy_variable=False, sync=True, staleness=0)[source]

Bases: autodist.strategy.ps_lb_strategy.PSLoadBalancing, autodist.strategy.all_reduce_strategy.AllReduce

Generates the Parallax Strategy from

The Parallax strategy mixes Parameter Server and AllReduce. The rationale is that a PS architecture is more suitable for sparse gradient updates, while AllReduce has reportedly better performance on dense gradient updates.

build(graph_item, resource_spec)[source]

Generate the strategy.