lago package

Submodules

lago.brctl module

lago.brctl._brctl(command, *args)[source]
lago.brctl.create(name, stp=True)[source]
lago.brctl.destroy(name)[source]
lago.brctl.exists(name)[source]

lago.build module

class lago.build.Build(name, disk_path, paths)[source]

Bases: object

A Build object represents a build section in the init file. Each build section (which in turn belongs to a specific disk) should get his own Build object

In order to add support for a new build command, a new function with the name of the command should be implemented in this class. this function should accept a list of options and arguments and return a named tuple ‘Command’, where ‘Command.name’ is the name of the command and ‘Command.cmd’ is the a list containing the command and its args, for example: Command.name = ‘virt-customize’ Command.cmd = [‘virt-customize’, ‘-a’, PATH_TO_DISK, SOME_CMDS...]

name

str – The name of the vm this builder belongs

disk_path

str – The path to the disk that needs to be customized

paths

lago.paths.Paths – The paths of the current prefix

build_cmds

list of str – A list of commands that should be invoked on the disk located in disk_path

build()[source]

Run all the commands in self.build_cmds

Raises:lago.build.BuildException – If a command returned a non-zero code
get_cmd_handler(cmd)[source]

Return an handler for cmd. The handler and the command should have the same name. See class description for more info about handlers.

Parameters:cmd (str) – The name of the command
Returns:which handles cmd
Return type:callable
Raises:lago.build.BuildException – If an handler for cmd doesn’t exist
classmethod get_instance_from_build_spec(name, disk_path, build_spec, paths)[source]
Parameters:
  • name (str) – The name of the vm this builder belongs
  • disk_path (str) – The path to the disk that needs to be customized
  • paths (lago.paths.Paths) – The paths of the current prefix
  • build_spec (dict) – The build spec part, associated with the disk located at disk_path, from the init file.
Returns:

An instance of Build with a normalized build spec i.e ready to

be invoked.

normalize_build_spec(build_spec)[source]

Convert a build spec into a list of Command tuples. After running this command, self.build_cmds should hold all the commands that should be run on the disk in self.disk_path.

Parameters:build_spec (dict) – The buildspec part from the init file
static normalize_options(options)[source]

Turns a mapping of ‘option: arg’ to a list and prefix the options. arg can be a list of arguments.

for example:

dict = {
o1: a1, o2: , o3: [a31, a32] o4: []

}

will be transformed to:

[
prefix_option(o1), a1, prefix_option(o2), prefix_option(o3), a31, prefix_option(o3), a32 prefix_option(o4)

]

note that empty arguments are omitted

Parameters:options (dict) – A mapping between options and arguments
Returns:A normalized version of ‘options’ as mentioned above
Return type:lst
static prefix_option(option)[source]

Depends on the option’s length, prefix it with ‘-‘ or ‘–’ :param option: The option to prefix :type option: str

Returns:prefixed option
Return type:str
virt_customize(options)[source]

Handler for ‘virt-customize’ note: if ‘ssh-inject’ option was specified without a path to a key, the prefix’ key will be copied to the vm.

Parameters:options (lst of str) – Options and arguments for ‘virt-customize’
Returns:which handles cmd
Return type:callable
Raises:lago.build.BuildException – If an handler for cmd doesn’t exist
exception lago.build.BuildException[source]

Bases: lago.utils.LagoException

class lago.build.Command(name, cmd)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__getstate__()

Exclude the OrderedDict from pickling

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values

_fields = ('name', 'cmd')
classmethod _make(iterable, new=<built-in method __new__ of type object at 0x906d60>, len=<built-in function len>)

Make a new Command object from a sequence or iterable

_replace(_self, **kwds)

Return a new Command object replacing specified fields with new values

cmd

Alias for field number 1

name

Alias for field number 0

lago.cmd module

lago.config module

class lago.config.ConfigLoad(root_section='lago', defaults={})[source]

Bases: object

Merges configuration parameters from 3 different sources: 1. Enviornment vairables 2. config files in .INI format 3. argparse.ArgumentParser

The assumed order(but not necessary) order of calls is: load() - load from config files and environment variables update_parser(parser) - update from the declared argparse parser update_args(args) - update from passed arguments to the parser

__getitem__(key)[source]

Get a variable from the default section, good for fail-fast if key does not exists.

Parameters:key (str) – key
Returns:config variable
Return type:str
get(*args)[source]

Get a variable from the default section :param *args: dict.get() args :type *args: args

Returns:config variable
Return type:str
get_ini(incl_unset=False)[source]

Return the config dictionary in INI format :param incl_unset: include variables with no defaults. :type incl_unset: bool

Returns:string of the config file in INI format
Return type:str
get_section(*args)[source]

get a section dictionary Args:

Returns:section config dictionary
Return type:dict
load()[source]

Load all configurations from available resources, skip if empty:

  1. default` dict passed to ConfigLoad.__init__().
  2. Custom paths as defined in CONFS_PATH in
    constants.
  3. XDG standard paths.
  4. Environment variables.
Returns:dict of dicts.
Return type:dict
update_args(args)[source]

Update config dictionary with parsed args, as resolved by argparse. Only root positional arguments that already exist will overridden.

Parameters:args (namespace) – args parsed by argparse
update_parser(parser)[source]

Update config dictionary with declared arguments in an argparse.parser New variables will be created, and existing ones overridden.

Parameters:parser (argparse.ArgumentParser) – parser to read variables from
lago.config._get_configs_path()[source]

Get a list of possible configuration files, from the following sources: 1. All files that exists in constants.CONFS_PATH. 2. All XDG standard config files for “lago.conf”, in reversed order of importance.

Returns:list of files
Return type:list(str)
lago.config.get_env_dict(root_section)[source]

Read all Lago variables from the environment. The lookup format is: LAGO_VARNAME - will land into ‘lago’ section LAGO__SECTION1__VARNAME - will land into ‘section1’ section, notice the double ‘__’. LAGO__LONG_SECTION_NAME__VARNAME - will land into ‘long_section_name’

Returns:dict of section configuration dicts
Return type:dict

Examples

>>> os.environ['LAGO_GLOBAL_VAR'] = 'global'
>>> os.environ['LAGO__INIT__REPO_PATH'] = '/tmp/store'
>>>
>>> config.get_env_dict()
{'init': {'repo_path': '/tmp/store'}, 'lago': {'global_var': 'global'}}

lago.constants module

lago.constants.CONFS_PATH = ['/etc/lago/lago.conf']

CONFS_PATH - default path to first look for configuration files.

lago.constants.LIBEXEC_DIR = '/usr/libexec/lago/'

LIBEXEC_DIR -

lago.export module

class lago.export.DiskExportManager(dst, disk_type, disk, do_compress)[source]

Bases: object

ExportManager object is responsible on the export process of an image from the current Lago prefix.

DiskExportManger is the base class of specific ExportManagers. Each specific ExportManger is responsible on the export process of an image with a specific type (e.g template, file...)

src

str – Path to the image that should be exported

name

str – The name of the exported disk

dst

str – The absolute path of the exported disk

disk_type

str – The type of the image e.g template, file, empty...

disk

dict – Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME

exported_metadata

dict – A copy of the source disk metadata, this dict should be updated with new values during the export process. do_compress(bool): If true, apply compression to the exported disk.

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
calc_sha(checksum)[source]

Calculate the checksum of the new exported disk, write it to a file, and update this managers ‘exported_metadata’.

Parameters:checksum (str) – The type of the checksum
compress()[source]

Compress the new exported image, Block size was taken from virt-builder page

copy()[source]

Copy the disk using cp in order to preserves the ‘sparse’ structure of the file

export()[source]

This method will handle the export process and should implemented in each subclass.

static get_instance_by_type(dst, disk, do_compress, *args, **kwargs)[source]
Parameters:
  • dst (str) – The path of the new exported disk. can contain env variables.
  • disk (dict) – Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME
  • do_compress (bool) – If true, apply compression to the exported disk.
Returns
An instance of a subclass of DiskExportManager which matches the disk type.
sparse()[source]

Make the exported images more compact by removing unused space. Please refer to ‘virt-sparsify’ for more info.

update_lago_metadata()[source]
write_lago_metadata()[source]
class lago.export.FileExportManager(dst, disk_type, disk, do_compress, *args, **kwargs)[source]

Bases: lago.export.DiskExportManager

FileExportManager is responsible exporting images of type file and empty.

standalone

bool – If true, create a new image which is the result of merging all the layers of src (the image that we want to export).

src_qemu_info

dict – Metadata on src which was generated by qemu-img.

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
export()[source]

See DiskExportManager.export

class lago.export.TemplateExportManager(dst, disk_type, disk, do_compress, *args, **kwargs)[source]

Bases: lago.export.DiskExportManager

TemplateExportManager is responsible exporting images of type template.

See superclass
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
export()[source]

See DiskExportManager.export

rebase()[source]

Change the backing-file entry of the exported disk. Please refer to ‘qemu-img rebase’ manual for more info.

update_lago_metadata()[source]

lago.lago_ansible module

class lago.lago_ansible.LagoAnsible(prefix)[source]

Bases: object

A class for handling Ansible related tasks

prefix

lago.prefix.Prefix – The prefix that this object wraps

_generate_entry(vm)[source]

Generate host entry for the given VM :param vm: The VM for which the entry

should be created for.
Returns:An entry for vm
Return type:str
get_inventory(keys=None)[source]

Create an Ansible inventory based on python dicts and lists. The returned value is a dict in which every key represents a group and every value is a list of entries for that group.

Parameters:keys (list of str) – Path to the keys that will be used to create groups.
Returns:dict based Ansible inventory
Return type:dict
get_inventory_str(keys=None)[source]

Convert a dict generated by ansible.LagoAnsible.get_inventory to an INI-like file.

Parameters:keys (list of str) – Path to the keys that will be used to create groups.
Returns:INI-like Ansible inventory
Return type:str
get_inventory_temp_file(*args, **kwds)[source]

Context manager which returns the inventory written on a tempfile. The tempfile will be deleted as soon as this context manger ends.

Parameters:keys (list of str) – Path to the keys that will be used to create groups.
Yields:tempfile.NamedTemporaryFile – Temp file containing the inventory
static get_key(key, data_structure)[source]

Helper method for extracting values from a nested data structure.

Parameters:
  • key (str) – The path to the vales (a series of keys and indexes separated by ‘/’)
  • data_structure (dict or list) – The data structure from which the value will be extracted.
Returns:

The values associated with key

Return type:

str

lago.log_utils module

This module defines the special logging tools that lago uses

lago.log_utils.ALWAYS_SHOW_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.ALWAYS_SHOW_TRIGGER_MSG = 'force-show:%s'

Message template that will always shoud the messago

class lago.log_utils.ColorFormatter(fmt=None, datefmt=None)[source]

Bases: logging.Formatter

Formatter to add colors to log records

CRITICAL = '\x1b[31m'
CYAN = '\x1b[36m'
DEBUG = ''
DEFAULT = '\x1b[0m'
ERROR = '\x1b[31m'
GREEN = '\x1b[32m'
INFO = '\x1b[36m'
NONE = ''
RED = '\x1b[31m'
WARNING = '\x1b[33m'
WHITE = '\x1b[37m'
YELLOW = '\x1b[33m'
classmethod colored(color, message)[source]

Small function to wrap a string around a color

Parameters:
  • color (str) – name of the color to wrap the string with, must be one of the class properties
  • message (str) – String to wrap with the color
Returns:

the colored string

Return type:

str

format(record)[source]

Adds colors to a log record and formats it with the default

Parameters:record (logging.LogRecord) – log record to format
Returns:The colored and formatted record string
Return type:str
class lago.log_utils.ContextLock[source]

Bases: object

Context manager to thread lock a block of code

lago.log_utils.END_TASK_MSG = 'Success'

Message to be shown when a task is ended

lago.log_utils.END_TASK_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.END_TASK_TRIGGER_MSG = 'end task%s'

Message template that will trigger a task end

class lago.log_utils.LogTask(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]

Bases: object

Context manager for a log task

Example

>>> with LogTask('mytask'):
...     pass
lago.log_utils.START_TASK_MSG = ''

Message to be shown when a task is started

lago.log_utils.START_TASK_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.START_TASK_TRIGGER_MSG = 'start task%s'

Message template that will trigger a task

class lago.log_utils.Task(name, *args, **kwargs)[source]

Bases: collections.deque

Small wrapper around deque to add the failed status and name to a task

name

str – name for this task

failed

bool – If this task has failed or not (if there was any error log shown during it’s execution)

force_show

bool – If set, will show any log records generated inside this task even if it’s out of nested depth limit

elapsed_time()[source]
class lago.log_utils.TaskHandler(initial_depth=0, task_tree_depth=-1, buffer_size=2000, dump_level=40, level=0, formatter=<class 'lago.log_utils.ColorFormatter'>)[source]

Bases: logging.StreamHandler

This log handler will use the concept of tasks, to hide logs, and will show all the logs for the current task if there’s a logged error while running that task.

It will hide any logs that belong to nested tasks that have more than task_tree_depth parent levels, and for the ones that are above that level, it will show only the logs that have a loglevel above level.

You can force showing a log record immediately if you use the log_always() function bypassing all the filters.

If there’s a log record with log level higher than dump_level it will be considered a failure, and all the logs for the current task that have a log level above level will be shown no matter at which depth the task belongs to. Also, all the parent tasks will be tagged as error.

formatter

logging.LogFormatter – formatter to use

initial_depth

int – Initial depth to account for, in case this handler was created in a subtask

tasks_by_thread (dict of str

OrderedDict of str: Task): List of thread names, and their currently open tasks with their latest log records

dump_level

int – log level from which to consider a log record as error

buffer_size

int – Size of the log record deque for each task, the bigger, the more records it can show in case of error but the more memory it will use

task_tree_depth

int – number of the nested level to show start/end task logs for, if -1 will show always

level

int – Log level to show logs from if the depth limit is not reached

main_failed

bool – used to flag from a child thread that the main should fail any current task

_tasks_lock

ContextLock – Lock for the tasks_by_thread dict

_main_thread_lock

ContextLock – Lock for the main_failed bool

TASK_INDICATORS = ['@', '#', '*', '-', '~']

List of chars to show as task prefix, to ease distinguishing them

am_i_main_thread

**Returns* – bool* – if the current thread is the main thread

close_children_tasks(parent_task_name)[source]

Closes all the children tasks that were open

Parameters:parent_task_name (str) – Name of the parent task
Returns:None
cur_depth_level

**Returns* – int* – depth level for the current task

cur_task

**Returns* – str* – the current active task

cur_thread

**Returns* – str* – Name of the current thread

emit(record)[source]

Handle the given record, this is the entry point from the python logging facility

Params:
record (logging.LogRecord): log record to handle
Returns:None
get_task_indicator(task_level=None)[source]
Parameters:task_level (int or None) – task depth level to get the indicator for, if None, will use the current tasks depth
Returns:char to prepend to the task logs to indicate it’s level
Return type:str
get_tasks(thread_name)[source]
Parameters:thread_name (str) – name of the thread to get the tasks for
Returns:
list of task names and log records for
each for the given thread
Return type:OrderedDict of str, Task
handle_closed_task(task_name, record)[source]

Do everything needed when a task is closed

Params:
task_name (str): name of the task that is finishing record (logging.LogRecord): log record with all the info
Returns:None
handle_error()[source]

Handles an error log record that should be shown

Returns:None
handle_new_task(task_name, record)[source]

Do everything needed when a task is starting

Params:
task_name (str): name of the task that is starting record (logging.LogRecord): log record with all the info
Returns:None
mark_main_tasks_as_failed()[source]

Flags to the main thread that all it’s tasks sholud fail

Returns:None
mark_parent_tasks_as_failed(task_name, flush_logs=False)[source]

Marks all the parent tasks as failed

Parameters:
  • task_name (str) – Name of the child task
  • flush_logs (bool) – If True will discard all the logs form parent tasks
Returns:

None

pretty_emit(record, is_header=False, task_level=None)[source]

Wrapper around the logging.StreamHandler emit method to add some decoration stuff to the message

Parameters:
  • record (logging.LogRecord) – log record to emit
  • is_header (bool) – if this record is a header, usually, a start or end task message
  • task_level (int) – If passed, will take that as the current nested task level instead of calculating it from the current tasks
Returns:

None

should_show_by_depth(cur_level=None)[source]
Parameters:cur_level (int) – depth level to take into account
Returns:
True if the given depth level should show messages (not
taking into account the log level)
Return type:bool
should_show_by_level(record_level, base_level=None)[source]
Parameters:
  • record_level (int) – log level of the record to check
  • base_level (int or None) – log level to check against, will use the object’s dump_level if None is passed
Returns:

True if the given log record should be shown according to the

log level

Return type:

bool

tasks

Returns

OrderedDict of str, Task: list of task names and log records for
each for the current thread
lago.log_utils.end_log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')[source]

Ends a log task

Parameters:
  • task (str) – name of the log task to end
  • logger (logging.Logger) – logger to use
  • level (str) – log level to use
Returns:

None

lago.log_utils.get_default_log_formatter()[source]
lago.log_utils.hide_paramiko_logs()[source]
lago.log_utils.hide_stevedore_logs()[source]

Hides the logs of stevedore, this function was added in order to support older versions of stevedore

We are using the NullHandler in order to get rid from ‘No handlers could be found for logger...’ msg

Returns:None
lago.log_utils.log_always(message)[source]

Wraps the given message with a tag that will make it be always logged by the task logger

Parameters:message (str) – message to wrap with the tag
Returns:
tagged message that will get it shown immediately by the task
logger
Return type:str
lago.log_utils.log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]

Parameterized decorator to wrap a function in a log task

Example

>>> @log_task('mytask')
... def do_something():
...     pass
lago.log_utils.setup_prefix_logging(logdir)[source]

Sets up a file logger that will create a log in the given logdir (usually a lago prefix)

Parameters:logdir (str) – path to create the log into, will be created if it does not exist
Returns:None
lago.log_utils.start_log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')[source]

Starts a log task

Parameters:
  • task (str) – name of the log task to start
  • logger (logging.Logger) – logger to use
  • level (str) – log level to use
Returns:

None

lago.paths module

class lago.paths.Paths(prefix)[source]

Bases: object

images(*path)[source]
logs()[source]
metadata()[source]
prefix_lagofile()[source]

This file represents a prefix that’s initialized

prefixed(*args)[source]
scripts(*args)[source]
ssh_id_rsa()[source]
ssh_id_rsa_pub()[source]
uuid()[source]
virt(*path)[source]

lago.prefix module

lago.sdk module

lago.sdk_utils module

class lago.sdk_utils.SDKMethod(name)[source]

Bases: object

Metadata to store inside the decorated function

class lago.sdk_utils.SDKWrapper[source]

Bases: sphinx.ext.autodoc.ObjectProxy

A proxy object that exposes only methods which were decorated with expose() decorator.

lago.sdk_utils.expose(func)[source]

Decorator to be used with SDKWrapper. This decorator indicates that the wrapped method or class should be exposed in the proxied object.

Parameters:func (types.FunctionType/types.MethodType) – function to decorate
Returns:None
lago.sdk_utils.getattr_sdk(attr, name)[source]

Filter SDK attributes

Parameters:
  • attr (attribute) – Attribute as returned by getattr().
  • name (str) – Attribute name.
Returns:

attr if passed.

lago.sdk_utils.setup_sdk_logging(logfile=None, loglevel=20)[source]

Setup a NullHandler to the root logger. If logfile is passed, additionally add a FileHandler in loglevel level.

Parameters:
  • logfile (str) – A path to setup a log file.
  • loglevel (int) – logging log level.
Returns:

None

lago.service module

class lago.service.SysVInitService(vm, name)[source]

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/sbin/service'
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
_request_start()[source]
_request_stop()[source]
state()[source]
class lago.service.SystemdContainerService(vm, name)[source]

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/usr/bin/docker'
HOST_BIN_PATH = '/usr/bin/systemctl'
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
_request_start()[source]
_request_stop()[source]
state()[source]
class lago.service.SystemdService(vm, name)[source]

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/usr/bin/systemctl'
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
_request_start()[source]
_request_stop()[source]
state()[source]

lago.ssh module

lago.ssh._gen_ssh_command_id()[source]
lago.ssh.drain_ssh_channel(chan, stdin=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)[source]
lago.ssh.get_ssh_client(ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456')[source]
lago.ssh.interactive_ssh(ip_addr, command=None, host_name=None, ssh_key=None, username='root', password='123456')[source]
lago.ssh.interactive_ssh_channel(chan, command=None, stdin=<open file '<stdin>', mode 'r'>)[source]
lago.ssh.ssh(ip_addr, command, host_name=None, data=None, show_output=True, propagate_fail=True, tries=None, ssh_key=None, username='root', password='123456')[source]
lago.ssh.ssh_script(ip_addr, path, host_name=None, show_output=True, ssh_key=None, username='root', password='123456')[source]
lago.ssh.wait_for_ssh(ip_addr, host_name=None, connect_timeout=600, ssh_key=None, username='root', password='123456')[source]

lago.subnet_lease module

lago.sysprep module

lago.sysprep._guestfs_version(default={'major': 1L, 'minor': 20L})[source]
lago.sysprep._render_template(distro, loader, **kwargs)[source]
lago.sysprep.sysprep(disk, distro, loader=None, backend='direct', **kwargs)[source]

Run virt-sysprep on the disk, commands are built from the distro specific template and arguments passed in kwargs. If no template is available it will default to sysprep-base.j2.

Parameters:
  • disk (str) – path to disk
  • distro (str) – distro to render template for
  • loader (jinja2.BaseLoader) – Jinja2 template loader, if not passed, will search Lago’s package.
  • backend (str) – libguestfs backend to use
  • **kwargs (dict) – environment variables for Jinja2 template
Returns:

None

Raises:

RuntimeError – On virt-sysprep none 0 exit code.

lago.templates module

This module contains any disk template related classes and functions, including the repository store manager classes and template providers, some useful definitions:

  • Template repositories:
    Repository where to fetch templates from, as an http server
  • Template store:
    Local store to cache templates
  • Template:
    Unititialized disk image to use as base for other disk images
  • Template version:
    Specific version of a template, to allow getting updates without having to change the template name everywhere
class lago.templates.FileSystemTemplateProvider(root)[source]

Handles file type templates, that is, getting a disk template from the host’s filesystem

_prefixed(*path)[source]

Join all the given paths prefixed with this provider’s base root path

Parameters:*path (str) – sections of the path to join, passed as positional arguments
Returns:Joined paths prepended with the provider root path
Return type:str
download_image(handle, dest)[source]

Copies over the handl to the destination

Parameters:
  • handle (str) – path to copy over
  • dest (str) – path to copy to
Returns:

None

get_hash(handle)[source]

Returns the associated hash for the given handle, the hash file must exist (handle + '.hash').

Parameters:handle (str) – Path to the template to get the hash from
Returns:Hash for the given handle
Return type:str
get_metadata(handle)[source]

Returns the associated metadata info for the given handle, the metadata file must exist (handle + '.metadata').

Parameters:handle (str) – Path to the template to get the metadata from
Returns:Metadata for the given handle
Return type:dict
class lago.templates.HttpTemplateProvider(baseurl)[source]

This provider allows the usage of http urls for templates

download_image(handle, dest)[source]

Downloads the image from the http server

Parameters:
  • handle (str) – url from the self.baseurl to the remote template
  • dest (str) – Path to store the downloaded url to, must be a file path
Returns:

None

static extract_image_xz(path)[source]
get_hash(handle)[source]

Get the associated hash for the given handle, the hash file must exist (handle + '.hash').

Parameters:handle (str) – Path to the template to get the hash from
Returns:Hash for the given handle
Return type:str
get_metadata(handle)[source]

Returns the associated metadata info for the given handle, the metadata file must exist (handle + '.metadata'). If the given handle has an .xz extension, it will get removed when calculating the handle metadata path

Parameters:handle (str) – Path to the template to get the metadata from
Returns:Metadata for the given handle
Return type:dict
open_url(url, suffix='', dest=None)[source]

Opens the given url, trying the compressed version first. The compressed version url is generated adding the .xz extension to the url and adding the given suffix after that .xz extension. If dest passed, it will download the data to that path if able

Parameters:
  • url (str) – relative url from the self.baseurl to retrieve
  • suffix (str) – optional suffix to append to the url after adding the compressed extension to the path
  • dest (str or None) – Path to save the data to
Returns:

response object to read from (lazy read), closed

if no dest passed

Return type:

urllib.addinfourl

Raises:

RuntimeError – if the url gave http error when retrieving it

class lago.templates.Template(name, versions)[source]

Disk image template class

name

str – Name of this template

_versions (dict(str

TemplateVersion)): versions for this template

get_latest_version()[source]

Retrieves the latest version for this template, the latest being the one with the newest timestamp

Returns:TemplateVersion
get_version(ver_name=None)[source]

Get the given version for this template, or the latest

Parameters:ver_name (str or None) – Version to retieve, None for the latest
Returns:
The version matching the given name or the latest
one
Return type:TemplateVersion
class lago.templates.TemplateRepository(dom)[source]

A template repository is a single source for templates, that uses different providers to actually retrieve them. That means for example that the ‘ovirt’ template repository, could support the ‘http’ and a theoretical ‘gluster’ template providers.

_dom

dict – Specification of the template

_providers

dict – Providers instances for any source in the spec

_get_provider(spec)[source]

Get the provider for the given template spec

Parameters:spec (dict) – Template spec
Returns:A provider instance for that spec
Return type:HttpTemplateProvider or FileSystemTemplateProvider
classmethod from_url(path)[source]

Instantiate a TemplateRepository instance from the data in a file or url

Parameters:path (str) – Path or url to the json file to load
Returns:A new instance
Return type:TemplateRepository
get_by_name(name)[source]

Retrieve a template by it’s name

Parameters:name (str) – Name of the template to retrieve
Raises:KeyError – if no template is found
name

Getter for the template repo name

Returns:the name of this template repo
Return type:str
class lago.templates.TemplateStore(path)[source]

Local cache to store templates

The store uses various files to keep track of the templates cached, access and versions. An example template store looks like:

$ tree /var/lib/lago/store/
/var/lib/lago/store/
├── in_office_repo:centos6_engine:v2.tmp
├── in_office_repo:centos7_engine:v5.tmp
├── in_office_repo:fedora22_host:v2.tmp
├── phx_repo:centos6_engine:v2
├── phx_repo:centos6_engine:v2.hash
├── phx_repo:centos6_engine:v2.metadata
├── phx_repo:centos6_engine:v2.users
├── phx_repo:centos7_engine:v4.tmp
├── phx_repo:centos7_host:v4.tmp
└── phx_repo:storage-nfs:v1.tmp

There you can see the files:

  • *.tmp
    Temporary file created while downloading the template from the repository (depends on the provider)
  • ${repo_name}:${template_name}:${template_version}
    This file is the actual disk image template
  • *.hash
    Cached hash for the template disk image
  • *.metadata
    Metadata for this template image in json format, usually this includes the distro and root-password
__contains__(temp_ver)[source]

Checks if a given version is in this store

Parameters:temp_ver (TemplateVersion) – Version to look for
Returns:True if the version is in this store
Return type:bool
_prefixed(*path)[source]

Join the given paths and prepend this stores path

Parameters:*path (str) – list of paths to join, as positional arguments
Returns:all the paths joined and prepended with the store path
Return type:str
download(temp_ver, store_metadata=True)[source]

Retrieve the given template version

Parameters:
  • temp_ver (TemplateVersion) – template version to retrieve
  • store_metadata (bool) – If set to False, will not refresh the local metadata with the retrieved one
Returns:

None

get_path(temp_ver)[source]

Get the path of the given version in this store

Parameters:TemplateVersion (temp_ver) – version to look for
Returns:The path to the template version inside the store
Return type:str
Raises:RuntimeError – if the template is not in the store
get_stored_hash(temp_ver)[source]

Retrieves the hash for the given template version from the store

Parameters:temp_ver (TemplateVersion) – template version to retrieve the hash for
Returns:hash of the given template version
Return type:str
get_stored_metadata(temp_ver)[source]

Retrieves the metadata for the given template version from the store

Parameters:temp_ver (TemplateVersion) – template version to retrieve the metadata for
Returns:the metadata of the given template version
Return type:dict
class lago.templates.TemplateVersion(name, source, handle, timestamp)[source]

Each template can have multiple versions, each of those is actually a different disk template file representation, under the same base name.

download(destination)[source]

Retrieves this template to the destination file

Parameters:destination (str) – file path to write this template to
Returns:None
get_hash()[source]

Returns the associated hash for this template version

Returns:Hash for this version
Return type:str
get_metadata()[source]

Returns the associated metadata info for this template version

Returns:Metadata for this version
Return type:dict
timestamp()[source]

Getter for the timestamp

lago.templates._locked(func)[source]

Decorator that ensures that the decorated function has the lock of the repo while running, meant to decorate only bound functions for classes that have lock_path method.

lago.templates.find_repo_by_name(name, repo_dir=None)[source]

Searches the given repo name inside the repo_dir (will use the config value ‘template_repos’ if no repo dir passed), will rise an exception if not found

Parameters:
  • name (str) – Name of the repo to search
  • repo_dir (str) – Directory where to search the repo
Returns:

path to the repo

Return type:

str

Raises:

RuntimeError – if not found

lago.utils module

class lago.utils.CommandStatus[source]

Bases: lago.utils.CommandStatus

class lago.utils.EggTimer(timeout)[source]
elapsed()[source]
class lago.utils.ExceptionTimer(timeout)[source]

Bases: object

start()[source]
stop()[source]
exception lago.utils.LagoException[source]

Bases: exceptions.Exception

exception lago.utils.LagoInitException[source]

Bases: lago.utils.LagoException

exception lago.utils.LagoUserException[source]

Bases: lago.utils.LagoException

class lago.utils.LockFile(path, timeout=None, **kwargs)[source]

Bases: object

Context manager that creates a lock around a directory, with optional timeout in the acquire operation

Parameters:
  • path (str) – path to the dir to lock
  • timeout (int) – timeout in seconds to wait while acquiring the lock
  • **kwargs (dict) – Any other param to pass to lockfile.LockFile
__enter__()[source]

Start the lock with timeout if needed in the acquire operation

Raises:TimerException – if the timeout is reached before acquiring the lock
class lago.utils.RollbackContext(*args)[source]

Bases: object

A context manager for recording and playing rollback. The first exception will be remembered and re-raised after rollback

Sample usage: > with RollbackContext() as rollback: > step1() > rollback.prependDefer(lambda: undo step1) > def undoStep2(arg): pass > step2() > rollback.prependDefer(undoStep2, arg)

More examples see tests/utilsTests.py @ vdsm code

__exit__(exc_type, exc_value, traceback)[source]

If this function doesn’t return True (or raises a different exception), python re-raises the original exception once this function is finished.

clear()[source]
defer(func, *args, **kwargs)[source]
prependDefer(func, *args, **kwargs)[source]
exception lago.utils.TimerException[source]

Bases: exceptions.Exception

Exception to throw when a timeout is reached

class lago.utils.VectorThread(targets)[source]
join_all(raise_exceptions=True)[source]
start_all()[source]
lago.utils._CommandStatus

alias of CommandStatus

lago.utils._add_subparser_to_cp(cp, section, actions, incl_unset)[source]
lago.utils._ret_via_queue(func, queue)[source]
lago.utils._run_command(command, input_data=None, stdin=None, out_pipe=-1, err_pipe=-1, env=None, uuid=None, **kwargs)[source]

Runs a command

Parameters:
  • command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
  • input_data (str) – If passed, will feed that data to the subprocess through stdin
  • out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
  • stdin (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdin
  • err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
  • of str (env(dict) – str): If set, will use the given dict as env for the subprocess
  • uuid (uuid) – If set the command will be logged with the given uuid converted to string, otherwise, a uuid v4 will be generated.
  • **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.add_timestamp_suffix(base_string)[source]
lago.utils.argparse_to_ini(parser, root_section='lago', incl_unset=False)[source]
lago.utils.compress(input_file, block_size, fail_on_error=True)[source]
lago.utils.cp(input_file, output_file, fail_on_error=True)[source]
lago.utils.deepcopy(original_obj)[source]

Creates a deep copy of an object with no crossed referenced lists or dicts, useful when loading from yaml as anchors generate those cross-referenced dicts and lists

Parameters:original_obj (object) – Object to deep copy
Returns:deep copy of the object
Return type:object
lago.utils.filter_spec(spec, paths, wildcard='*', separator='/')[source]

Remove keys from a spec file. For example, with the following path: domains//disks//metadata all the metadata dicts from all domains disks will be removed.

Parameters:
  • spec (dict) – spec to remove keys from
  • paths (list) – list of paths to the keys that should be removed
  • wildcard (str) – wildcard character
  • separator (str) – path separator
Returns:

None

Raises:

utils.LagoUserException – If a malformed path was detected

lago.utils.func_vector(target, args_sequence)[source]
lago.utils.get_hash(file_path, checksum='sha1')[source]

Generate a hash for the given file

Parameters:
  • file_path (str) – Path to the file to generate the hash for
  • checksum (str) – hash to apply, one of the supported by hashlib, for example sha1 or sha512
Returns:

hash for that file

Return type:

str

lago.utils.get_qemu_info(path, backing_chain=False, fail_on_error=True)[source]

Get info on a given qemu disk

Parameters:
  • path (str) – Path to the required disk
  • backing_chain (boo) – if true, include also info about
  • image predecessors. (the) –
Returns:

if backing_chain == True then a list of dicts else a dict

Return type:

object

lago.utils.in_prefix(prefix_class, workdir_class)[source]
lago.utils.invoke_different_funcs_in_parallel(*funcs)[source]
lago.utils.invoke_in_parallel(func, *args_sequences)[source]
lago.utils.ipv4_to_mac(ip)[source]
lago.utils.json_dump(obj, f)[source]
lago.utils.load_virt_stream(virt_fd)[source]

Loads the given conf stream into a dict, trying different formats if needed

Parameters:virt_fd (str) – file like objcect with the virt config to load
Returns:Loaded virt config
Return type:dict
lago.utils.qemu_rebase(target, backing_file, safe=True, fail_on_error=True)[source]

changes the backing file of ‘source’ to ‘backing_file’ If backing_file is specified as “” (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file). (Taken from qemu-img man page)

Parameters:
  • target (str) – Path to the source disk
  • backing_file (str) – path to the base disk
  • safe (bool) – if false, allow unsafe rebase (check qemu-img docs for more info)
lago.utils.read_nonblocking(file_descriptor)[source]
lago.utils.rotate_dir(base_dir)[source]
lago.utils.run_command(command, input_data=None, out_pipe=-1, err_pipe=-1, env=None, **kwargs)[source]

Runs a command non-interactively

Parameters:
  • command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
  • input_data (str) – If passed, will feed that data to the subprocess through stdin
  • out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
  • err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
  • of str (env(dict) – str): If set, will use the given dict as env for the subprocess
  • **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.run_command_with_validation(cmd, fail_on_error=True, msg='An error has occurred')[source]
lago.utils.run_interactive_command(command, env=None, **kwargs)[source]

Runs a command interactively, reusing the current stdin, stdout and stderr

Parameters:
  • command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
  • of str (env(dict) – str): If set, will use the given dict as env for the subprocess
  • **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.service_is_enabled(name)[source]
lago.utils.sparse(input_file, input_format, fail_on_error=True)[source]
lago.utils.ver_cmp(ver1, ver2)[source]

Compare lago versions

Parameters:
  • ver1 (str) – version string
  • ver2 (str) – version string
Returns:

Return negative if ver1<ver2, zero if ver1==ver2, positive if ver1>ver2.

lago.utils.with_logging(func)[source]

lago.virt module

class lago.virt.VirtEnv(prefix, vm_specs, net_specs)[source]

Bases: object

Env properties: * prefix * vms * net

  • libvirt_con
_create_net(net_spec)[source]
_create_vm(vm_spec)[source]
_get_stop_shutdown_common_args(vm_names)[source]

Get the common arguments for stop and shutdown commands

Parameters:vm_names (list of str) – The names of the requested vms
Returns
list of plugins.vm.VMProviderPlugin:
vms objects that should be stopped

list of virt.Network: net objects that should be stopped str: log message

Raises:utils.LagoUserException – If a vm name doesn’t exist
_get_unused_nets(vms_to_stop)[source]

Return a list of nets that used only by the vms in vms_to_stop

Parameters:vms_to_stop (list of str) – The names of the requested vms
Returns
list of virt.Network: net objects that used only by
vms in vms_to_stop
Raises:utils.LagoUserException – If a vm name doesn’t exist
bootstrap()[source]
create_snapshots(*args, **kwargs)[source]
export_vms(vms_names, standalone, dst_dir, compress, init_file_name, out_format)[source]
classmethod from_prefix(prefix)[source]
generate_init(dst, out_format, filters=None)[source]

Generate an init file which represents this env and can be used with the images created by self.export_vms

Parameters:
  • dst (str) – path and name of the new init file
  • out_format (plugins.output.OutFormatPlugin) – formatter for the output (the default is yaml)
  • filters (list) – list of paths to keys that should be removed from the init file
Returns:

None

get_compat()[source]

Get compatibility level for this environment - which is the Lago version used to create this environment

get_env_spec(filters=None)[source]

Get the spec of the current env. The spec will hold the info about all the domains and networks associated with this env.

Parameters:filters (list) – list of paths to keys that should be removed from the init file
Returns:the spec of the current env
Return type:dict
get_net(name=None)[source]
get_nets()[source]
get_snapshots(domains=None)[source]

Get the list of snapshots for each domain

Parameters:
  • domanins (list of str) – list of the domains to get the snapshots
  • all will be returned if none or empty list passed (for,) –
Returns:

with the domain names and the list of snapshots for each

Return type:

dict of str -> list(str)

get_vm(name)[source]
get_vms(vm_names=None)[source]

Returns the vm objects associated with vm_names if vm_names is None, return all the vms in the prefix

Parameters:vm_names (list of str) – The names of the requested vms
Returns
dict: Which contains the requested vm objects indexed by name
Raises:utils.LagoUserException – If a vm name doesn’t exist
prefixed_name(unprefixed_name, max_length=0)[source]

Returns a uuid pefixed identifier

Parameters:
  • unprefixed_name (str) – Name to add a prefix to
  • max_length (int) – maximum length of the resultant prefixed name, will adapt the given name and the length of the uuid ot fit it
Returns:

prefixed identifier for the given unprefixed name

Return type:

str

revert_snapshots(*args, **kwargs)[source]
save(*args, **kwargs)[source]
shutdown(vm_names, reboot=False)[source]
start(vm_names=None)[source]
stop(vm_names=None)[source]
virt_path(*args)[source]
lago.virt._gen_ssh_command_id()[source]
lago.virt._guestfs_copy_path(g, guest_path, host_path)[source]
lago.virt._path_to_xml(basename)[source]

lago.vm module

class lago.vm.DefaultVM(env, spec)[source]

Bases: lago.plugins.vm.VMPlugin

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 33
_abc_registry = <_weakrefset.WeakSet object>
class lago.vm.SSHVMProvider(vm)[source]

Bases: lago.plugins.vm.VMProviderPlugin

bootstrap(*args, **kwargs)[source]
create_snapshot(name, *args, **kwargs)[source]
defined(*args, **kwargs)[source]
revert_snapshot(name, *args, **kwargs)[source]
start(*args, **kwargs)[source]
state(*args, **kwargs)[source]
stop(*args, **kwargs)[source]

lago.workdir module