Graph Transformer

Graph Transformer.

class GraphTransformer(compiled_strategy, cluster, graph_item)[source]

Bases: object

Graph Transformer.

This is the bulk of the AutoDist backend logic, taking a single-node, single-GPU graph and transforming it into a distributed graph. This all happens based on the Strategy provided.

The transformation occurs over several steps:

  1. Partitions the necessary variables

  2. Replicates the graph the desired number of times

  3. Within the graph, synchronizes gradients with in-graph logic

  4. Adds between-graph gradient synchronization logic

transform()[source]

Call graph transformer to transform a graph item based on strategy and cluster.

property num_local_replicas[source]

Return the number of local replicas.