Choose Strategy Builders

As in previous tutorials, the only required argument from

AutoDist(resource_spec_file, strategy_builder=None)

is resource_spec_file, the path of a yaml file describing resource specifications. Meanwhile, AutoDist also offers flexibility to provide built-in or customized strategy_builder as described below. If strategy_builder=None, AutoDist will automatically choose the default strategy builder for you.

What is a strategy builder? Intuitively speaking, a strategy in AutoDist is a representation on how to distribute the computational graph, while a strategy builder creates that representation. For more technical details, please refer to Architecture or Strategy Protocol or Developer API Reference.

New users of AutoDist can start by choosing one of the built-in strategy builders from autodist.strategy. To know more about different built-in strategy builders, and their configurations and pros & cons, please see the reference page for built-in strategy builders.

from autodist.strategy import PSLoadBalancing
from autodist import AutoDist
autodist = AutoDist(
    resource_spec_file='resource_spec.yml', 
    strategy_builder=PSLoadBalancing(sync=True)
)

For advanced users, one can customize a strategy besides the built-ins.