Collective Key

Utility functions for AllReduceSynchronizer.

get_collective_keys()

Return a singleton instance of CollectiveKey.

class CollectiveKey(group_key_start=1)

Bases: object

A hash that generates group key and instance key for AllReduce.

get_group_key(canonical_devices)

Generate or retrieve the group key based on a list of strings of the participating devices.

get_instance_key(var_op_name)

Generate or retrieve the instance key based on the original variable op name.