Resource Spec

Resource Specification.

class Connectivity(value)

Bases: enum.Enum

Connectivity.

ETHERNET = 0

CPU_TO_GPU = 1

GPU_TO_GPU_LOCAL = 2

GPU_TO_GPU_REMOTE = 3

SAME = 4

class DeviceType(value)

Bases: enum.Enum

Device Type.

CPU = 0

GPU = 1

class ResourceSpec(resource_file=None)

Bases: object

Resource Spec.

Contains node and SSH information found by parsing a resource_spec.yml.

# TODO: Make it a real Graph (a clique), with edge weights being network bandwidth.

This would allow for even more intelligent strategy generation.

property chief

Return chief address.

property devices

Return all devices.

property nodes

Return all node addresses.

property cpu_devices

String-to-device_spec mapping of all cpu devices.

property num_cpus

Number of all cpu devices.

property gpu_devices

String-to-device_spec mapping of all gpu devices.

property node_gpu_devices

Node_address-to-device_string mapping of all gpu devices.

property node_cpu_devices

Node_address-to-device_string mapping of all cpu devices.

property num_gpus

Number of all gpu devices.

property ssh_config_map

Configurations for SSH.

property ssh_group

SSH Group for each node.

property network_bandwidth

Network bandwidth of each node.

class DeviceSpec(host_address, host_device=None, device_type=<DeviceType.CPU: 0>, device_index=None)

Bases: object

Device specification.

name_string()

Name string.

classmethod from_string(name_string)

Construct an AutoDist DeviceSpec based on its name string.

Parameters

name_string – AutoDist DeviceSpec name string

Returns

an instance

Return type

DeviceSpec

class SSHConfig(username: str, port: int, python_venv: str, key_file: str, pkey: Optional[paramiko.rsakey.RSAKey], env: dict)

Bases: tuple

Contains any necessary SSH information (e.g. passwords, keyfiles, etc.).

property username

Alias for field number 0

property port

Alias for field number 1

property python_venv

Alias for field number 2

property key_file

Alias for field number 3

property pkey

Alias for field number 4

property env

Alias for field number 5

count(value, /)

Return number of occurrences of value.

index(value, start=0, stop=9223372036854775807, /)

Return first index of value.

Raises ValueError if the value is not present.

class SSHConfigMap(info: Dict[str, Dict], node_groups: Dict[str, str], gpu_devices: Dict[str, str])

Bases: dict

Contains all necessary SSH configs, grouped by config name.

clear() → None. Remove all items from D.

copy() → a shallow copy of D

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

items() → a set-like object providing a view on D’s items

keys() → a set-like object providing a view on D’s keys

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem() → (k, v), remove and return some (key, value) pair as a

2-tuple; but raise KeyError if D is empty.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None. Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D’s values