Resource Spec

Resource Specification.

class Connectivity(value)[source]

Bases: enum.Enum


ETHERNET = 0[source]
CPU_TO_GPU = 1[source]
GPU_TO_GPU_LOCAL = 2[source]
GPU_TO_GPU_REMOTE = 3[source]
SAME = 4[source]
class DeviceType(value)[source]

Bases: enum.Enum

Device Type.

CPU = 0[source]
GPU = 1[source]
class ResourceSpec(resource_file=None)[source]

Bases: object

Resource Spec.

Contains node and SSH information found by parsing a resource_spec.yml.

# TODO: Make it a real Graph (a clique), with edge weights being network bandwidth.

This would allow for even more intelligent strategy generation.

property chief[source]

Return chief address.

property devices[source]

Return all devices.

property nodes[source]

Return all node addresses.

property cpu_devices[source]

String-to-device_spec mapping of all cpu devices.

property num_cpus[source]

Number of all cpu devices.

property gpu_devices[source]

String-to-device_spec mapping of all gpu devices.

property node_gpu_devices[source]

Node_address-to-device_string mapping of all gpu devices.

property node_cpu_devices[source]

Node_address-to-device_string mapping of all cpu devices.

property num_gpus[source]

Number of all gpu devices.

property ssh_config_map[source]

Configurations for SSH.

property ssh_group[source]

SSH Group for each node.

property network_bandwidth[source]

Network bandwidth of each node.

class DeviceSpec(host_address, host_device=None, device_type=<DeviceType.CPU: 0>, device_index=None)[source]

Bases: object

Device specification.


Name string.

classmethod from_string(name_string)[source]

Construct an AutoDist DeviceSpec based on its name string.


name_string – AutoDist DeviceSpec name string


an instance

Return type


class SSHConfig(username: str, port: int, python_venv: str, key_file: str, pkey: Optional[paramiko.rsakey.RSAKey], env: dict)[source]

Bases: tuple

Contains any necessary SSH information (e.g. passwords, keyfiles, etc.).

property username[source]

Alias for field number 0

property port[source]

Alias for field number 1

property python_venv[source]

Alias for field number 2

property key_file[source]

Alias for field number 3

property pkey[source]

Alias for field number 4

property env[source]

Alias for field number 5

count(value, /)[source]

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)[source]

Return first index of value.

Raises ValueError if the value is not present.

class SSHConfigMap(info: Dict[str, Dict], node_groups: Dict[str, str], gpu_devices: Dict[str, str])[source]

Bases: dict

Contains all necessary SSH configs, grouped by config name.

clear() → None. Remove all items from D.[source]
copy() → a shallow copy of D[source]
fromkeys(value=None, /)[source]

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)[source]

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D’s items[source]
keys() → a set-like object providing a view on D’s keys[source]
pop(k[, d]) → v, remove specified key and return the corresponding value.[source]

If key is not found, d is returned if given, otherwise KeyError is raised

popitem() → (k, v), remove and return some (key, value) pair as a[source]

2-tuple; but raise KeyError if D is empty.

setdefault(key, default=None, /)[source]

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None. Update D from dict/iterable E and F.[source]

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D’s values[source]