lago package¶
Subpackages¶
Submodules¶
lago.brctl module¶
lago.cmd module¶
lago.config module¶
-
class
lago.config.
ConfigLoad
(root_section='lago')[source]¶ Bases:
object
Merges configuration parameters from 3 different sources: 1. Enviornment vairables 2. config files in .INI format 3. argparse.ArgumentParser
The assumed order(but not necessary) order of calls is: load() - load from config files and environment variables update_parser(parser) - update from the declared argparse parser update_args(args) - update from passed arguments to the parser
-
__getitem__
(key)[source]¶ Get a variable from the default section, good for fail-fast if key does not exists.
Parameters: key (str) – key Returns: config variable Return type: str
-
get
(*args)[source]¶ Get a variable from the default section :param *args: dict.get() args
Returns: config variable Return type: str
-
get_ini
(defaults_only=False, incl_unset=False)[source]¶ Return the config dictionary in INI format :param defaults_only: if set, will ignore arguments set by the CLI.
Returns: string of the config file in INI format Return type: str
-
get_section
(*args)[source]¶ get a section dictionary Args:
Returns: section config dictionary Return type: dict
-
load
()[source]¶ Load all configuration from INI format files and ENV, always preferring the last read. Order of loading is: 1) Custom paths as defined in constants.CONFS_PATH 2) XDG standard paths 3) Environment variables
Returns: dict of section configuration dicts Return type: dict
-
update_args
(args)[source]¶ Update config dictionary with parsed args, as resolved by argparse. Only root positional arguments that already exist will overridden.
Parameters: args (namespace) – args parsed by argparse
-
update_parser
(parser)[source]¶ Update config dictionary with declared arguments in an argparse.parser New variables will be created, and existing ones overridden.
Parameters: parser (argparse.ArgumentParser) – parser to read variables from
-
-
lago.config.
_get_configs_path
()[source]¶ Get a list of possible configuration files, from the following sources: 1. All files that exists in constants.CONFS_PATH. 2. All XDG standard config files for “lago.conf”, in reversed order of importance.
Returns: list – list of files Return type: str
-
lago.config.
get_env_dict
(root_section)[source]¶ Read all Lago variables from the environment. The lookup format is: LAGO_VARNAME - will land into ‘lago’ section LAGO__SECTION1__VARNAME - will land into ‘section1’ section, notice the double ‘__’. LAGO__LONG_SECTION_NAME__VARNAME - will land into ‘long_section_name’
Returns: dict of section configuration dicts Return type: dict Examples
>>> os.environ['LAGO_GLOBAL_VAR'] = 'global' >>> os.environ['LAGO__INIT__REPO_PATH'] = '/tmp/store' >>> >>> config.get_env_dict() {'init': {'repo_path': '/tmp/store'}, 'lago': {'global_var': 'global'}}
lago.constants module¶
-
lago.constants.
CONFS_PATH
= ['/etc/lago/lago.conf']¶ CONFS_PATH - default path to first look for configuration files.
-
lago.constants.
LIBEXEC_DIR
= '/usr/libexec/lago/'¶ LIBEXEC_DIR -
lago.dirlock module¶
-
lago.dirlock.
lock
(path, excl, key_path)[source]¶ Waits until the given directory can be locked
Parameters: Returns: None
lago.export module¶
-
class
lago.export.
DiskExportManager
(dst, disk_type, disk, do_compress)[source]¶ Bases:
object
ExportManager object is responsible on the export process of an image from the current Lago prefix.
DiskExportManger is the base class of specific ExportManagers. Each specific ExportManger is responsible on the export process of an image with a specific type (e.g template, file...)
-
src
¶ str
Path to the image that should be exported
-
name
¶ str
The name of the exported disk
-
dst
¶ str
The absolute path of the exported disk
-
disk_type
¶ str
The type of the image e.g template, file, empty...
-
disk
¶ dict
Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME
-
exported_metadata
¶ dict
A copy of the source disk metadata, this dict should be updated with new values during the export process. do_compress(bool): If true, apply compression to the exported disk.
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
calc_sha
(checksum)[source]¶ Calculate the checksum of the new exported disk, write it to a file, and update this managers ‘exported_metadata’.
Parameters: checksum (str) – The type of the checksum
-
export
()[source]¶ This method will handle the export process and should implemented in each subclass.
-
static
get_instance_by_type
(dst, disk, do_compress, *args, **kwargs)[source]¶ Parameters: - Returns
- An instance of a subclass of DiskExportManager which matches the disk type.
-
-
class
lago.export.
FileExportManager
(dst, disk_type, disk, do_compress, *args, **kwargs)[source]¶ Bases:
lago.export.DiskExportManager
FileExportManager is responsible exporting images of type file and empty.
-
standalone
¶ bool
If true, create a new image which is the result of merging all the layers of src (the image that we want to export).
-
src_qemu_info
¶ dict
Metadata on src which was generated by qemu-img.
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.export.
TemplateExportManager
(dst, disk_type, disk, do_compress, *args, **kwargs)[source]¶ Bases:
lago.export.DiskExportManager
TemplateExportManager is responsible exporting images of type template.
-
See superclass
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
lago.libvirt_utils module¶
Utilities to help deal with the libvirt python bindings
-
lago.libvirt_utils.
DOMAIN_STATES
= {<class 'sphinx.ext.autodoc.VIR_DOMAIN_SHUTOFF'>: 'shut off', <class 'sphinx.ext.autodoc.VIR_DOMAIN_BLOCKED'>: 'blocked', <class 'sphinx.ext.autodoc.VIR_DOMAIN_PMSUSPENDED'>: 'suspended', <class 'sphinx.ext.autodoc.VIR_DOMAIN_NOSTATE'>: 'no state', <class 'sphinx.ext.autodoc.VIR_DOMAIN_SHUTDOWN'>: 'beign shut down', <class 'sphinx.ext.autodoc.VIR_DOMAIN_CRASHED'>: 'crashed', <class 'sphinx.ext.autodoc.VIR_DOMAIN_RUNNING'>: 'running', <class 'sphinx.ext.autodoc.VIR_DOMAIN_PAUSED'>: 'paused'}¶ Mapping of domain statuses values to human readable strings
-
class
lago.libvirt_utils.
Domain
[source]¶ Bases:
object
Class to namespace libvirt domain related helpers
-
static
resolve_state
(state_number)[source]¶ Get a nice description from a domain state number
Parameters: state_number (list of int) – State number as returned by libvirt.virDomain.state()
Returns: small human readable description of the domain state, unknown if the state is not in the known list Return type: str
-
static
-
lago.libvirt_utils.
LIBVIRT_CONNECTIONS
= {}¶ Singleton with the cached opened libvirt connections
lago.log_utils module¶
This module defines the special logging tools that lago uses
-
lago.log_utils.
ALWAYS_SHOW_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
ALWAYS_SHOW_TRIGGER_MSG
= 'force-show:%s'¶ Message template that will always shoud the messago
-
class
lago.log_utils.
ColorFormatter
(fmt=None, datefmt=None)[source]¶ Bases:
logging.Formatter
Formatter to add colors to log records
-
CRITICAL
= '\x1b[31m'¶
-
CYAN
= '\x1b[36m'¶
-
DEBUG
= ''¶
-
DEFAULT
= '\x1b[0m'¶
-
ERROR
= '\x1b[31m'¶
-
GREEN
= '\x1b[32m'¶
-
INFO
= '\x1b[36m'¶
-
NONE
= ''¶
-
RED
= '\x1b[31m'¶
-
WARNING
= '\x1b[33m'¶
-
WHITE
= '\x1b[37m'¶
-
YELLOW
= '\x1b[33m'¶
-
classmethod
colored
(color, message)[source]¶ Small function to wrap a string around a color
Parameters: Returns: the colored string
Return type:
-
format
(record)[source]¶ Adds colors to a log record and formats it with the default
Parameters: record (logging.LogRecord) – log record to format Returns: The colored and formatted record string Return type: str
-
-
class
lago.log_utils.
ContextLock
[source]¶ Bases:
object
Context manager to thread lock a block of code
-
lago.log_utils.
END_TASK_MSG
= 'Success'¶ Message to be shown when a task is ended
-
lago.log_utils.
END_TASK_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
END_TASK_TRIGGER_MSG
= 'end task%s'¶ Message template that will trigger a task end
-
class
lago.log_utils.
LogTask
(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]¶ Bases:
object
Context manager for a log task
Example
>>> with LogTask('mytask'): ... pass
-
lago.log_utils.
START_TASK_MSG
= ''¶ Message to be shown when a task is started
-
lago.log_utils.
START_TASK_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
START_TASK_TRIGGER_MSG
= 'start task%s'¶ Message template that will trigger a task
-
class
lago.log_utils.
Task
(name, *args, **kwargs)[source]¶ Bases:
collections.deque
Small wrapper around deque to add the failed status and name to a task
-
name
¶ str
name for this task
-
failed
¶ bool
If this task has failed or not (if there was any error log shown during it’s execution)
-
force_show
¶ bool
If set, will show any log records generated inside this task even if it’s out of nested depth limit
-
-
class
lago.log_utils.
TaskHandler
(initial_depth=0, task_tree_depth=-1, buffer_size=2000, dump_level=40, level=0, formatter=<class 'lago.log_utils.ColorFormatter'>)[source]¶ Bases:
logging.StreamHandler
This log handler will use the concept of tasks, to hide logs, and will show all the logs for the current task if there’s a logged error while running that task.
It will hide any logs that belong to nested tasks that have more than
task_tree_depth
parent levels, and for the ones that are above that level, it will show only the logs that have a loglevel abovelevel
.You can force showing a log record immediately if you use the
log_always()
function bypassing all the filters.If there’s a log record with log level higher than
dump_level
it will be considered a failure, and all the logs for the current task that have a log level abovelevel
will be shown no matter at which depth the task belongs to. Also, all the parent tasks will be tagged as error.-
formatter
¶ logging.LogFormatter
formatter to use
-
initial_depth
¶ int
Initial depth to account for, in case this handler was created in a subtask
-
tasks_by_thread (dict of str
OrderedDict of str: Task): List of thread names, and their currently open tasks with their latest log records
-
dump_level
¶ int
log level from which to consider a log record as error
-
buffer_size
¶ int
Size of the log record deque for each task, the bigger, the more records it can show in case of error but the more memory it will use
-
task_tree_depth
¶ int
number of the nested level to show start/end task logs for, if -1 will show always
-
level
¶ int
Log level to show logs from if the depth limit is not reached
-
main_failed
¶ bool
used to flag from a child thread that the main should fail any current task
-
_tasks_lock
¶ ContextLock
Lock for the tasks_by_thread dict
-
_main_thread_lock
¶ ContextLock
Lock for the main_failed bool
-
TASK_INDICATORS
= ['@', '#', '*', '-', '~']¶ List of chars to show as task prefix, to ease distinguishing them
-
close_children_tasks
(parent_task_name)[source]¶ Closes all the children tasks that were open
Parameters: parent_task_name (str) – Name of the parent task Returns: None
-
emit
(record)[source]¶ Handle the given record, this is the entry point from the python logging facility
- Params:
- record (logging.LogRecord): log record to handle
Returns: None
-
get_task_indicator
(task_level=None)[source]¶ Parameters: task_level (int or None) – task depth level to get the indicator for, if None, will use the current tasks depth Returns: char to prepend to the task logs to indicate it’s level Return type: str
-
get_tasks
(thread_name)[source]¶ Parameters: thread_name (str) – name of the thread to get the tasks for Returns: list of task names and log records for each for the given thread Return type: OrderedDict of str, Task
-
handle_closed_task
(task_name, record)[source]¶ Do everything needed when a task is closed
- Params:
- task_name (str): name of the task that is finishing record (logging.LogRecord): log record with all the info
Returns: None
-
handle_new_task
(task_name, record)[source]¶ Do everything needed when a task is starting
- Params:
- task_name (str): name of the task that is starting record (logging.LogRecord): log record with all the info
Returns: None
-
mark_main_tasks_as_failed
()[source]¶ Flags to the main thread that all it’s tasks sholud fail
Returns: None
-
mark_parent_tasks_as_failed
(task_name, flush_logs=False)[source]¶ Marks all the parent tasks as failed
Parameters: Returns: None
-
pretty_emit
(record, is_header=False, task_level=None)[source]¶ Wrapper around the
logging.StreamHandler
emit method to add some decoration stuff to the messageParameters: - record (logging.LogRecord) – log record to emit
- is_header (bool) – if this record is a header, usually, a start or end task message
- task_level (int) – If passed, will take that as the current nested task level instead of calculating it from the current tasks
Returns: None
-
should_show_by_depth
(cur_level=None)[source]¶ Parameters: cur_level (int) – depth level to take into account Returns: True if the given depth level should show messages (not taking into account the log level) Return type: bool
-
should_show_by_level
(record_level, base_level=None)[source]¶ Parameters: - record_level (int) – log level of the record to check
- base_level (int or None) – log level to check against, will use the
object’s
dump_level
if None is passed
Returns: True if the given log record should be shown according to the log level
Return type:
-
tasks
¶ Returns: list of task names and log records for each for the current thread Return type: OrderedDict of str, Task
-
-
lago.log_utils.
end_log_task
(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')[source]¶ Ends a log task
Parameters: - task (str) – name of the log task to end
- logger (logging.Logger) – logger to use
- level (str) – log level to use
Returns: None
-
lago.log_utils.
hide_stevedore_logs
()[source]¶ Hides the logs of stevedore, this function was added in order to support older versions of stevedore
We are using the NullHandler in order to get rid from ‘No handlers could be found for logger...’ msg
Returns: None
-
lago.log_utils.
log_always
(message)[source]¶ Wraps the given message with a tag that will make it be always logged by the task logger
Parameters: message (str) – message to wrap with the tag Returns: tagged message that will get it shown immediately by the task logger Return type: str
-
lago.log_utils.
log_task
(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]¶ Parameterized decorator to wrap a function in a log task
Example
>>> @log_task('mytask') ... def do_something(): ... pass
-
lago.log_utils.
setup_prefix_logging
(logdir)[source]¶ Sets up a file logger that will create a log in the given logdir (usually a lago prefix)
Parameters: logdir (str) – path to create the log into, will be created if it does not exist Returns: None
-
lago.log_utils.
start_log_task
(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')[source]¶ Starts a log task
Parameters: - task (str) – name of the log task to start
- logger (logging.Logger) – logger to use
- level (str) – log level to use
Returns: None
lago.prefix module¶
-
class
lago.prefix.
Prefix
(prefix)[source]¶ Bases:
object
A prefix is a directory that will contain all the data needed to setup the environment.
-
_prefix
¶ str
Path to the directory of this prefix
-
_paths
¶ lago.path.Paths
Path handler class
-
_virt_env
¶ lago.virt.VirtEnv
Lazily loaded virtual env handler
-
_metadata
¶ dict
Lazily loaded metadata
-
VIRT_ENV_CLASS
¶ alias of
VirtEnv
-
_add_nic_to_mapping
(net, dom, nic)[source]¶ Populates the given net spec mapping entry with the nicks of the given domain
Parameters: Returns: None
-
_allocate_ips_to_nics
(conf)[source]¶ For all the nics of all the domains in the conf that have dynamic ip, allocate one and addit to the network mapping
Parameters: conf (dict) – Configuration spec to extract the domains from Returns: None
-
_allocate_subnets
(conf)[source]¶ Allocate all the subnets needed by the given configuration spec
Parameters: conf (dict) – Configuration spec where to get the nets definitions from Returns: tuple – allocated subnets and modified conf Return type: list, dict
-
static
_check_predefined_subnets
(conf)[source]¶ Checks if all of the nets defined in the config are inside the allowed range, throws exception if not
Parameters: conf (dict) – Configuration spec where to get the nets definitions from Returns: None Raises: RuntimeError
– If there are any subnets out of the allowed range
-
_config_net_topology
(conf)[source]¶ Initialize and populate all the network related elements, like reserving ips and populating network specs of the given confiiguration spec
Parameters: conf (dict) – Configuration spec to initalize Returns: None
-
_copy_delpoy_scripts
(scripts)[source]¶ Copy the given deploy scripts to the scripts dir in the prefix
Parameters: scripts (list of str) – list of paths of the scripts to copy to the prefix Returns: list with the paths to the copied scripts, with a prefixed with $LAGO_PREFIX_PATH so the full path is not hardcoded Return type: list of str
-
_copy_deploy_scripts_for_hosts
(domains)[source]¶ Copy the deploy scripts for all the domains into the prefix scripts dir
Parameters: domains (dict) – spec with the domains info as when loaded from the initfile Returns: None
-
_create_disk
(name, spec, template_repo=None, template_store=None)[source]¶ Creates a disc with the given name from the given repo or store
Parameters: Returns: Tuple – Path to the disk and disk metadata
Return type: str, dict
Raises: RuntimeError
– If the type of the disk is not supported or failed to create the disk
-
_create_ssh_keys
()[source]¶ Generate a pair of ssh keys for this prefix
Returns: None Raises: RuntimeError
– if it fails to create the keys
-
_create_virt_env
()[source]¶ Create a new virt env from this prefix
Returns: virt env created from this prefix Return type: lago.virt.VirtEnv
-
_get_metadata
()[source]¶ Retrieve the metadata info for this prefix
Returns: metadata info Return type: dict
-
_get_scripts
(host_metadata)[source]¶ Temporary method to retrieve the host scripts
- TODO:
- remove once the “ovirt-scripts” option gets deprecated
Parameters: host_metadata (dict) – host metadata to retrieve the scripts for Returns: deploy scripts for the host, empty if none found Return type: list
-
static
_init_net_specs
(conf)[source]¶ Given a configuration specification, initializes all the net definitions in it so they can be used comfortably
Parameters: conf (dict) – Configuration specification Returns: the adapted new conf Return type: dict
-
_ova_to_spec
(filename)[source]¶ Retrieve the given ova and makes a template of it. Creates a disk from network provided ova. Calculates the needed memory from the ovf. The disk will be cached in the template repo
Parameters: filename (str) – the url to retrive the data from - TODO:
- Add hash checking against the server for faster download and latest version
- Add config script running on host - other place
- Add cloud init support - by using cdroms in other place
- Handle cpu in some way - some other place need to pick it up
- Handle the memory units properly - we just assume MegaBytes
Returns: list with the disk specification int: VM memory, None if none defined int: Number of virtual cpus, None if none defined
Return type: list of dict
Raises: RuntimeError
– If the ova format is not supportedTypeError
– If the memory units in the ova are noot supported (currently only ‘MegaBytes’)
-
_register_preallocated_ips
(conf)[source]¶ Parse all the domains in the given conf and preallocate all their ips into the networks mappings, raising exception on duplicated ips or ips out of the allowed ranges
See also
Parameters: conf (dict) – Configuration spec to parse Returns: None Raises: RuntimeError
– if there are any duplicated ips or any ip out of the allowed range
-
_set_scripts
(host_metadata, scripts)[source]¶ Temporary method to set the host scripts
- TODO:
- remove once the “ovirt-scripts” option gets deprecated
Parameters: host_metadata (dict) – host metadata to set scripts in Returns: the updated metadata Return type: dict
-
_use_prototype
(spec, prototypes)[source]¶ Populates the given spec with the values of it’s declared prototype
Parameters: Returns: updated spec
Return type:
-
cleanup
(*args, **kwargs)[source]¶ Stops any running entities in the prefix and uninitializes it, usually you want to do this if you are going to remove the prefix afterwards
Returns: None
-
create_snapshots
(name)[source]¶ Creates one snapshot on all the domains with the given name
Parameters: name (str) – Name of the snapshots to create Returns: None
-
fetch_url
(url)[source]¶ Retrieves the given url to the prefix
Parameters: url (str) – Url to retrieve Returns: path to the downloaded file Return type: str
-
get_nets
()[source]¶ Retrieve info on all the nets from all the domains
Returns: dict of str->list – dictionary with net_name -> net list Return type: str
-
get_snapshots
()[source]¶ Retrieve info on all the snapshots from all the domains
Returns: list(str): dictionary with vm_name -> snapshot list Return type: dict of str
-
get_vms
()[source]¶ Retrieve info on all the vms
Returns: dict of str->list – dictionary with vm_name -> vm list Return type: str
-
initialize
(*args, **kwargs)[source]¶ Initialize this prefix, this includes creating the destination path, and creating the uuid for the prefix, for any other actions see
Prefix.virt_conf()
Will safely roll back if any of those steps fail
Returns: None Raises: RuntimeError
– If it fails to create the prefix dir
-
classmethod
is_prefix
(path)[source]¶ Check if a path is a valid prefix
Parameters: path (str) – path to be checked Returns: True if the given path is a prefix Return type: bool
-
resolve_parent
(disk_path, template_store, template_repo)[source]¶ Given a virtual disk, checks if it has a backing file, if so check if the backing file is in the store, if not download it from the provided template_repo.
After verifying that the backing-file is in the store, create a symlink to that file and locate it near the layered image.
Parameters: - disk_path (str) – path to the layered disk
- template_repo (TemplateRepository or None) – template repo instance to use
- template_store (TemplateStore or None) – template store instance to use
-
classmethod
resolve_prefix_path
(start_path=None)[source]¶ Look for an existing prefix in the given path, in a path/.lago dir, or in a .lago dir under any of it’s parent directories
Parameters: start_path (str) – path to start the search from, if None passed, it will use the current dir Returns: path to the found prefix Return type: str Raises: RuntimeError
– if no prefix was found
-
revert_snapshots
(name)[source]¶ Revert all the snapshots with the given name from all the domains
Parameters: name (str) – Name of the snapshots to revert Returns: None
-
start
(vm_names=None)[source]¶ Start this prefix
Parameters: vm_names (list of str) – List of the vms to start Returns: None
-
stop
(vm_names=None)[source]¶ Stop this prefix
Parameters: vm_names (list of str) – List of the vms to stop Returns: None
-
virt_conf
(conf, template_repo=None, template_store=None, do_bootstrap=True)[source]¶ Initializes all the virt infrastructure of the prefix, creating the domains disks, doing any network leases and creating all the virt related files and dirs inside this prefix.
Parameters: - conf (dict) – Configuration spec
- template_repo (TemplateRepository) – template repository intance
- template_store (TemplateStore) – template store instance
Returns: None
-
virt_conf_from_stream
(conf_fd, template_repo=None, template_store=None, do_bootstrap=True)[source]¶ Initializes all the virt infrastructure of the prefix, creating the domains disks, doing any network leases and creating all the virt related files and dirs inside this prefix.
Parameters: - conf_fd (File) – File like object to read the config from
- template_repo (TemplateRepository) – template repository intance
- template_store (TemplateStore) – template store instance
Returns: None
-
virt_env
¶ Getter for this instance’s virt env, creates it if needed
Returns: virt env instance used by this prefix Return type: lago.virt.VirtEnv
-
-
lago.prefix.
_create_ip
(subnet, index)[source]¶ Given a subnet or an ip and an index returns the ip with that lower index from the subnet (255.255.255.0 mask only subnets)
Parameters: - subnet (str) – Strign containing the three first elements of the decimal representation of a subnet (X.Y.Z) or a full ip (X.Y.Z.A)
- index (int or str) – Last element of a decimal ip representation, for example, 123 for the ip 1.2.3.123
Returns: The dotted decimal representation of the ip
Return type:
-
lago.prefix.
_ip_in_subnet
(subnet, ip)[source]¶ Checks if an ip is included in a subnet.
Note
only 255.255.255.0 masks allowed
Parameters: - subnet (str) – Strign containing the three first elements of the decimal representation of a subnet (X.Y.Z) or a full ip (X.Y.Z.A)
- ip (str or int) – Decimal ip representation
Returns: True
if ip is in subnet,False
otherwiseReturn type:
lago.service module¶
-
class
lago.service.
SysVInitService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/sbin/service'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.service.
SystemdContainerService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/usr/bin/docker'¶
-
HOST_BIN_PATH
= '/usr/bin/systemctl'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.service.
SystemdService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/usr/bin/systemctl'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
lago.ssh module¶
-
lago.ssh.
drain_ssh_channel
(chan, stdin=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)[source]¶
-
lago.ssh.
get_ssh_client
(ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456')[source]¶
-
lago.ssh.
interactive_ssh
(ip_addr, command=None, host_name=None, ssh_key=None, username='root', password='123456')[source]¶
-
lago.ssh.
interactive_ssh_channel
(chan, command=None, stdin=<open file '<stdin>', mode 'r'>)[source]¶
-
lago.ssh.
ssh
(ip_addr, command, host_name=None, data=None, show_output=True, propagate_fail=True, tries=None, ssh_key=None, username='root', password='123456')[source]¶
lago.subnet_lease module¶
Module that handles the leases for the subnets of the virtual network interfaces.
Note
Currently only /24 ranges are handled, and all of them under the 192.168.MIN_SUBNET to 192.168.MAX_SUBNET ranges
The leases are stored under LEASE_DIR
as json files with the form:
[
"/path/to/prefix/uuid/file",
"uuid_hash",
]
Where the uuid_hash is the 32 char uuid of the prefix (the contents of the uuid file at the time of doing the lease)
-
lago.subnet_lease.
MAX_SUBNET
= 209¶ Upper range for the allowed subnets
-
lago.subnet_lease.
MIN_SUBNET
= 200¶ Lower range for the allowed subnets
-
lago.subnet_lease.
_acquire
(*args, **kwargs)[source]¶ Lease a free network for the given uuid path
Parameters: uuid_path (str) – Path to the uuid file of a lago.Prefix
Returns: the third element of the dotted ip of the leased network or None
if no lease was availableReturn type: int or None
-
lago.subnet_lease.
_lease_owned
(path, current_uuid_path)[source]¶ Checks if the given lease is owned by the prefix whose uuid is in the given path
Note
The prefix must be also in the same path it was when it took the lease
Parameters: Returns: True
if the given lease in owned by the prefix,False
otherwiseReturn type:
-
lago.subnet_lease.
_lease_valid
(path)[source]¶ Checs if the given lease still has a prefix that owns it
Parameters: path (str) – Path to the lease Returns: True
if the uuid path in the lease still exists and is the same as the one in the leaseReturn type: bool
-
lago.subnet_lease.
_locked
(func)[source]¶ Decorator that will make sure that you have the exclusive lock for the leases
-
lago.subnet_lease.
_release
(*args, **kwargs)[source]¶ Free the lease of the given subnet index
Parameters: index (int) – Third element of a dotted ip representation of the subnet, for example, for 1.2.3.4 it would be 3 Returns: None
-
lago.subnet_lease.
_take_lease
(path, uuid_path)[source]¶ Persist to the given leases path the prefix uuid that’s in the uuid path passed
Parameters: Returns: None
-
lago.subnet_lease.
_validate_lease_dir_present
(func)[source]¶ Decorator that will ensure that the lease dir exists, creating it if necessary
-
lago.subnet_lease.
acquire
(uuid_path)[source]¶ Lease a free network for the given uuid path
Parameters: uuid_path (str) – Path to the uuid file of a lago.Prefix
Returns: the dotted ip of the gateway for the leased net Return type: str
lago.sysprep module¶
lago.templates module¶
This module contains any disk template related classes and functions, including the repository store manager classes and template providers, some useful definitions:
- Template repositories:
Repository where to fetch templates from, as an http server
- Template store:
Local store to cache templates
- Template:
Unititialized disk image to use as base for other disk images
- Template version:
Specific version of a template, to allow getting updates without having to change the template name everywhere
-
class
lago.templates.
FileSystemTemplateProvider
(root)[source]¶ Handles file type templates, that is, getting a disk template from the host’s filesystem
-
_prefixed
(*path)[source]¶ Join all the given paths prefixed with this provider’s base root path
Parameters: *path (str) – sections of the path to join, passed as positional arguments Returns: Joined paths prepended with the provider root path Return type: str
-
download_image
(handle, dest)[source]¶ Copies over the handl to the destination
Parameters: Returns: None
-
-
class
lago.templates.
HttpTemplateProvider
(baseurl)[source]¶ This provider allows the usage of http urls for templates
-
download_image
(handle, dest)[source]¶ Downloads the image from the http server
Parameters: Returns: None
-
get_hash
(handle)[source]¶ Get the associated hash for the given handle, the hash file must exist (
handle + '.hash'
).Parameters: handle (str) – Path to the template to get the hash from Returns: Hash for the given handle Return type: str
-
get_metadata
(handle)[source]¶ Returns the associated metadata info for the given handle, the metadata file must exist (
handle + '.metadata'
). If the given handle has an.xz
extension, it will get removed when calculating the handle metadata pathParameters: handle (str) – Path to the template to get the metadata from Returns: Metadata for the given handle Return type: dict
-
open_url
(url, suffix='', dest=None)[source]¶ Opens the given url, trying the compressed version first. The compressed version url is generated adding the
.xz
extension to theurl
and adding the given suffix after that.xz
extension. If dest passed, it will download the data to that path if ableParameters: Returns: response object to read from (lazy read), closed if no dest passed
Return type: urllib.addinfourl
Raises: RuntimeError
– if the url gave http error when retrieving it
-
-
class
lago.templates.
Template
(name, versions)[source]¶ Disk image template class
-
name
¶ str
Name of this template
-
_versions (dict(str
TemplateVersion)): versions for this template
-
get_latest_version
()[source]¶ Retrieves the latest version for this template, the latest being the one with the newest timestamp
Returns: TemplateVersion
-
get_version
(ver_name=None)[source]¶ Get the given version for this template, or the latest
Parameters: ver_name (str or None) – Version to retieve, None for the latest Returns: The version matching the given name or the latest one Return type: TemplateVersion
-
-
class
lago.templates.
TemplateRepository
(dom)[source]¶ A template repository is a single source for templates, that uses different providers to actually retrieve them. That means for example that the ‘ovirt’ template repository, could support the ‘http’ and a theoretical ‘gluster’ template providers.
-
_dom
¶ dict
Specification of the template
-
_providers
¶ dict
Providers instances for any source in the spec
-
_get_provider
(spec)[source]¶ Get the provider for the given template spec
Parameters: spec (dict) – Template spec Returns: A provider instance for that spec Return type: HttpTemplateProvider or FileSystemTemplateProvider
-
classmethod
from_url
(path)[source]¶ Instantiate a
TemplateRepository
instance from the data in a file or urlParameters: path (str) – Path or url to the json file to load Returns: A new instance Return type: TemplateRepository
-
-
class
lago.templates.
TemplateStore
(path)[source]¶ Local cache to store templates
The store uses various files to keep track of the templates cached, access and versions. An example template store looks like:
$ tree /var/lib/lago/store/ /var/lib/lago/store/ ├── in_office_repo:centos6_engine:v2.tmp ├── in_office_repo:centos7_engine:v5.tmp ├── in_office_repo:fedora22_host:v2.tmp ├── phx_repo:centos6_engine:v2 ├── phx_repo:centos6_engine:v2.hash ├── phx_repo:centos6_engine:v2.metadata ├── phx_repo:centos6_engine:v2.users ├── phx_repo:centos7_engine:v4.tmp ├── phx_repo:centos7_host:v4.tmp └── phx_repo:storage-nfs:v1.tmp
There you can see the files:
- *.tmp
Temporary file created while downloading the template from the repository (depends on the provider)
- ${repo_name}:${template_name}:${template_version}
This file is the actual disk image template
- *.hash
Cached hash for the template disk image
- *.metadata
Metadata for this template image in json format, usually this includes the distro and root-password
-
__contains__
(temp_ver)[source]¶ Checks if a given version is in this store
Parameters: temp_ver (TemplateVersion) – Version to look for Returns: True
if the version is in this storeReturn type: bool
-
_init_users
(temp_ver)[source]¶ Initializes the user access registry
Parameters: temp_ver (TemplateVersion) – template version to update registry for Returns: None
-
_prefixed
(*path)[source]¶ Join the given paths and prepend this stores path
Parameters: *path (str) – list of paths to join, as positional arguments Returns: all the paths joined and prepended with the store path Return type: str
-
download
(temp_ver, store_metadata=True)[source]¶ Retrieve the given template version
Parameters: - temp_ver (TemplateVersion) – template version to retrieve
- store_metadata (bool) – If set to
False
, will not refresh the local metadata with the retrieved one
Returns: None
-
get_path
(temp_ver)[source]¶ Get the path of the given version in this store
Parameters: TemplateVersion (temp_ver) – version to look for Returns: The path to the template version inside the store Return type: str Raises: RuntimeError
– if the template is not in the store
-
get_stored_hash
(temp_ver)[source]¶ Retrieves the hash for the given template version from the store
Parameters: temp_ver (TemplateVersion) – template version to retrieve the hash for Returns: hash of the given template version Return type: str
-
get_stored_metadata
(temp_ver)[source]¶ Retrieves the metadata for the given template version from the store
Parameters: temp_ver (TemplateVersion) – template version to retrieve the metadata for Returns: the metadata of the given template version Return type: dict
-
mark_used
(temp_ver, key_path)[source]¶ Adds or updates the user entry in the user access log for the given template version
Parameters: - temp_ver (TemplateVersion) – template version to add the entry for
- key_path (str) – Path to the prefix uuid file to set the mark for
-
class
lago.templates.
TemplateVersion
(name, source, handle, timestamp)[source]¶ Each template can have multiple versions, each of those is actually a different disk template file representation, under the same base name.
-
download
(destination)[source]¶ Retrieves this template to the destination file
Parameters: destination (str) – file path to write this template to Returns: None
-
get_hash
()[source]¶ Returns the associated hash for this template version
Returns: Hash for this version Return type: str
-
-
lago.templates.
_locked
(func)[source]¶ Decorator that ensures that the decorated function has the lock of the repo while running, meant to decorate only bound functions for classes that have lock_path method.
-
lago.templates.
find_repo_by_name
(name, repo_dir=None)[source]¶ Searches the given repo name inside the repo_dir (will use the config value ‘template_repos’ if no repo dir passed), will rise an exception if not found
Parameters: Returns: path to the repo
Return type: Raises: RuntimeError
– if not found
lago.utils module¶
-
class
lago.utils.
CommandStatus
[source]¶ Bases:
lago.utils.CommandStatus
-
exception
lago.utils.
LagoException
[source]¶ Bases:
exceptions.Exception
-
exception
lago.utils.
LagoUserException
[source]¶ Bases:
lago.utils.LagoException
-
class
lago.utils.
LockFile
(path, timeout=None, **kwargs)[source]¶ Bases:
object
Context manager that creates a lock around a directory, with optional timeout in the acquire operation
Parameters: -
__enter__
()[source]¶ Start the lock with timeout if needed in the acquire operation
Raises: TimerException
– if the timeout is reached before acquiring the lock
-
-
class
lago.utils.
RollbackContext
(*args)[source]¶ Bases:
object
A context manager for recording and playing rollback. The first exception will be remembered and re-raised after rollback
Sample usage: > with RollbackContext() as rollback: > step1() > rollback.prependDefer(lambda: undo step1) > def undoStep2(arg): pass > step2() > rollback.prependDefer(undoStep2, arg)
More examples see tests/utilsTests.py @ vdsm code
-
exception
lago.utils.
TimerException
[source]¶ Bases:
exceptions.Exception
Exception to throw when a timeout is reached
-
lago.utils.
_CommandStatus
¶ alias of
CommandStatus
-
lago.utils.
_run_command
(command, input_data=None, stdin=None, out_pipe=-1, err_pipe=-1, env=None, uuid=None, **kwargs)[source]¶ Runs a command
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- input_data (str) – If passed, will feed that data to the subprocess through stdin
- out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
- stdin (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdin
- err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- uuid (uuid) – If set the command will be logged with the given uuid converted to string, otherwise, a uuid v4 will be generated.
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
-
lago.utils.
deepcopy
(original_obj)[source]¶ Creates a deep copy of an object with no crossed referenced lists or dicts, useful when loading from yaml as anchors generate those cross-referenced dicts and lists
Parameters: original_obj (object) – Object to deep copy Returns: deep copy of the object Return type: object
-
lago.utils.
get_hash
(file_path, checksum='sha1')[source]¶ Generate a hash for the given file
Parameters: Returns: hash for that file
Return type:
-
lago.utils.
get_qemu_info
(path, backing_chain=False, fail_on_error=True)[source]¶ Get info on a given qemu disk
Parameters: - path (str) – Path to the required disk
- backing_chain (boo) – if true, include also info about
- image predecessors. (the) –
Returns: if backing_chain == True then a list of dicts else a dict
Return type:
-
lago.utils.
load_virt_stream
(virt_fd)[source]¶ Loads the given conf stream into a dict, trying different formats if needed
Parameters: virt_fd (str) – file like objcect with the virt config to load Returns: Loaded virt config Return type: dict
-
lago.utils.
qemu_rebase
(target, backing_file, safe=True, fail_on_error=True)[source]¶ changes the backing file of ‘source’ to ‘backing_file’ If backing_file is specified as “” (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file). (Taken from qemu-img man page)
Parameters:
-
lago.utils.
run_command
(command, input_data=None, out_pipe=-1, err_pipe=-1, env=None, **kwargs)[source]¶ Runs a command non-interactively
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- input_data (str) – If passed, will feed that data to the subprocess through stdin
- out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
- err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
-
lago.utils.
run_command_with_validation
(cmd, fail_on_error=True, msg='An error has occurred')[source]¶
-
lago.utils.
run_interactive_command
(command, env=None, **kwargs)[source]¶ Runs a command interactively, reusing the current stdin, stdout and stderr
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
lago.virt module¶
-
class
lago.virt.
BridgeNetwork
(env, spec)[source]¶ Bases:
lago.virt.Network
-
class
lago.virt.
NATNetwork
(env, spec)[source]¶ Bases:
lago.virt.Network
-
class
lago.virt.
Network
(env, spec)[source]¶ Bases:
object
-
class
lago.virt.
VirtEnv
(prefix, vm_specs, net_specs)[source]¶ Bases:
object
Env properties: * prefix * vms * net
- libvirt_con
-
_CPU_FAMILIES
= {'Westmere': 'Intel Westmere Family', 'Opteron_G2': 'AMD Opteron G2', 'Penryn': 'Intel Penryn Family', 'SandyBridge': 'Intel SandyBridge Family', 'Opteron_G3': 'AMD Opteron G3', 'Conroe': 'Intel Conroe Family', 'Opteron_G1': 'AMD Opteron G1', 'Opteron_G4': 'AMD Opteron G4', 'Opteron_G5': 'AMD Opteron G5', 'Nehalem': 'Intel Nehalem Family'}¶
-
_compatible_cpu_and_family
= None¶
-
get_snapshots
(domains=None)[source]¶ Get the list of snapshots for each domain
Parameters: - domanins (list of str) – list of the domains to get the snapshots
- all will be returned if none or empty list passed (for,) –
Returns: dict of str -> list – with the domain names and the list of snapshots for each
Return type:
lago.vm module¶
-
class
lago.vm.
DefaultVM
(env, spec)[source]¶ Bases:
lago.plugins.vm.VMPlugin
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 28¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.vm.
LocalLibvirtVMProvider
(vm)[source]¶ Bases:
lago.plugins.vm.VMProviderPlugin
-
export_disks
(standalone, dst_dir, compress, *args, **kwargs)[source]¶ Exports all the disks of self. For each disk type, handler function should be added.
Parameters:
-
extract_paths
(paths, ignore_nopath)[source]¶ Extract the given paths from the domain
Attempt to extract all files defined in
paths
with the method defined inextract_paths()
, if it fails, will try extracting the files with libguestfs.Parameters: - paths (list of str) – paths to extract
- ignore_nopath (boolean) – if True will ignore none existing paths.
Returns: None
Raises: ExtractPathNoPathError
– if a none existing path was found on the VM, and ignore_nopath is True.ExtractPathError
– on all other failures.
-
interactive_console
(*args, **kwargs)[source]¶ Opens an interactive console
Returns: result of the virsh command execution Return type: lago.utils.CommandStatus
-
lago.workdir module¶
A workdir is the base directory where lago will store all the files it needs and that are unique (not shared between workdirs).
It’s basic structure is a directory with one soft link and multiple directories, one per prefix. Where the link points to the default prefix to use.
-
exception
lago.workdir.
MalformedWorkdir
[source]¶ Bases:
lago.workdir.WorkdirError
-
exception
lago.workdir.
PrefixAlreadyExists
[source]¶ Bases:
lago.workdir.WorkdirError
-
exception
lago.workdir.
PrefixNotFound
[source]¶ Bases:
lago.workdir.WorkdirError
-
class
lago.workdir.
Workdir
(path, prefix_class=<class 'lago.prefix.Prefix'>)[source]¶ Bases:
object
This class reperesents a base workdir, where you can store multiple prefixes
- Properties:
- path(str): Path to the workdir perfixes(dict of str->self.prefix_class): dict with the prefixes in the workdir, by name current(str): Name of the current prefix prefix_class(type): Class to use when creating prefixes
-
_set_current
(new_current)[source]¶ Change the current default prefix, for internal usage
Parameters: new_current (str) – Name of the new current prefix, it must already exist Returns: None Raises: PrefixNotFound
– if the given prefix name does not exist in the workdir
-
add_prefix
(workdir, *args, **kwargs)[source]¶ Adds a new prefix to the workdir.
Parameters: - name (str) – Name of the new prefix to add
- *args – args to pass along to the prefix constructor
- *kwargs – kwargs to pass along to the prefix constructor
Returns: The newly created prefix
Raises: PrefixAlreadyExists
– if the prefix name already exists in the workdir
-
destroy
(workdir, *args, **kwargs)[source]¶ Destroy all the given prefixes and remove any left files if no more prefixes are left
Parameters: - prefix_names (list of str) – list of prefix names to destroy, if None
- passed (default) –
-
get_prefix
(workdir, *args, **kwargs)[source]¶ Retrieve a prefix, resolving the current one if needed
Parameters: name (str) – name of the prefix to retrieve, or current to get the current one Returns: instance of the prefix with the given name Return type: self.prefix_class
-
initialize
(prefix_name='default', *args, **kwargs)[source]¶ Initializes a workdir by adding a new prefix to the workdir.
Parameters: - prefix_name (str) – Name of the new prefix to add
- *args – args to pass along to the prefix constructor
- *kwargs – kwargs to pass along to the prefix constructor
Returns: The newly created prefix
Raises: PrefixAlreadyExists
– if the prefix name already exists in the workdir
-
classmethod
is_workdir
(path)[source]¶ Check if the given path is a workdir
Parameters: path (str) – Path to check Returns: True if the given path is a workdir Return type: bool
-
join
(*args)[source]¶ Gets a joined path prefixed with the workdir path
Parameters: *args (str) – path sections to join Returns: Joined path prefixed with the workdir path Return type: str
-
load
()[source]¶ Loads the prefixes that are available is the workdir
Returns: None Raises: MalformedWorkdir
– if the wordir is malformed
-
classmethod
resolve_workdir_path
(start_path='.')[source]¶ Look for an existing workdir in the given path, in a path/.lago dir, or in a .lago dir under any of it’s parent directories
Parameters: start_path (str) – path to start the search from, if None passed, it will use the current dir Returns: path to the found prefix Return type: str Raises: RuntimeError
– if no prefix was found
-
set_current
(workdir, *args, **kwargs)[source]¶ Change the current default prefix
Parameters: new_current (str) – Name of the new current prefix, it must already exist Returns: None Raises: PrefixNotFound
– if the given prefix name does not exist in the workdir
-
exception
lago.workdir.
WorkdirError
[source]¶ Bases:
exceptions.RuntimeError
Base exception for workdir errors, catch this one to catch any workdir error