lago package

Subpackages

• lago.plugins package
  • Submodules
  • lago.plugins.cli module
  • lago.plugins.output module
  • lago.plugins.service module
    • Service Plugin
  • lago.plugins.vm module
    • VM Plugins
• lago.providers package
  • Subpackages
    • lago.providers.libvirt package
      • Submodules
      • lago.providers.libvirt.cpu module
      • lago.providers.libvirt.network module
      • lago.providers.libvirt.utils module
      • lago.providers.libvirt.vm module

Submodules

lago.brctl module

lago.brctl._brctl(command, *args)

lago.brctl._set_link(name, state)

lago.brctl.create(name, stp=True)

lago.brctl.destroy(name)

lago.brctl.exists(name)

lago.build module

class lago.build.Build(name, disk_path, paths)

Bases: object

A Build object represents a build section in the init file. Each build section (which in turn belongs to a specific disk) should get his own Build object

In order to add support for a new build command, a new function with the name of the command should be implemented in this class. this function should accept a list of options and arguments and return a named tuple ‘Command’, where ‘Command.name’ is the name of the command and ‘Command.cmd’ is the a list containing the command and its args, for example: Command.name = ‘virt-customize’ Command.cmd = [‘virt-customize’, ‘-a’, PATH_TO_DISK, SOME_CMDS...]

name

str – The name of the vm this builder belongs

disk_path

str – The path to the disk that needs to be customized

paths

lago.paths.Paths – The paths of the current prefix

build_cmds

list of str – A list of commands that should be invoked on the disk located in disk_path

build()

Run all the commands in self.build_cmds

Raises:lago.build.BuildException – If a command returned a non-zero code

get_cmd_handler(cmd)

Return an handler for cmd. The handler and the command should have the same name. See class description for more info about handlers.

Parameters:cmd (str) – The name of the command Returns:which handles cmd Return type:callable Raises:lago.build.BuildException – If an handler for cmd doesn’t exist

classmethod get_instance_from_build_spec(name, disk_path, build_spec, paths)

Parameters:

• name (str) – The name of the vm this builder belongs
• disk_path (str) – The path to the disk that needs to be customized
• paths (lago.paths.Paths) – The paths of the current prefix
• build_spec (dict) – The build spec part, associated with the disk located at disk_path, from the init file.

Returns:

An instance of Build with a normalized build spec i.e ready to

be invoked.

normalize_build_spec(build_spec)

Convert a build spec into a list of Command tuples. After running this command, self.build_cmds should hold all the commands that should be run on the disk in self.disk_path.

Parameters:build_spec (dict) – The buildspec part from the init file

static normalize_options(options)

Turns a mapping of ‘option: arg’ to a list and prefix the options. arg can be a list of arguments.

for example:

dict = {

o1: a1, o2: , o3: [a31, a32] o4: []

}

will be transformed to:

[

prefix_option(o1), a1, prefix_option(o2), prefix_option(o3), a31, prefix_option(o3), a32 prefix_option(o4)

]

note that empty arguments are omitted

Parameters:options (dict) – A mapping between options and arguments Returns:A normalized version of ‘options’ as mentioned above Return type:lst

static prefix_option(option)

Depends on the option’s length, prefix it with ‘-‘ or ‘–’ :param option: The option to prefix :type option: str

Returns:prefixed option Return type:str

virt_customize(options)

Handler for ‘virt-customize’ note: if ‘ssh-inject’ option was specified without a path to a key, the prefix’ key will be copied to the vm.

Parameters:options (lst of str) – Options and arguments for ‘virt-customize’ Returns:which handles cmd Return type:callable Raises:lago.build.BuildException – If an handler for cmd doesn’t exist

exception lago.build.BuildException

Bases: lago.utils.LagoException

class lago.build.Command(name, cmd)

Bases: tuple

__getnewargs__()

Return self as a plain tuple. Used by copy and pickle.

__getstate__()

Exclude the OrderedDict from pickling

__repr__()

Return a nicely formatted representation string

_asdict()

Return a new OrderedDict which maps field names to their values

_fields = ('name', 'cmd')

classmethod _make(iterable, new=<built-in method __new__ of type object at 0x906d60>, len=<built-in function len>)

Make a new Command object from a sequence or iterable

_replace(_self, **kwds)

Return a new Command object replacing specified fields with new values

cmd

Alias for field number 1

name

Alias for field number 0

lago.cmd module

lago.config module

class lago.config.ConfigLoad(root_section='lago', defaults={})

Bases: object

Merges configuration parameters from 3 different sources: 1. Enviornment vairables 2. config files in .INI format 3. argparse.ArgumentParser

The assumed order(but not necessary) order of calls is: load() - load from config files and environment variables update_parser(parser) - update from the declared argparse parser update_args(args) - update from passed arguments to the parser

__getitem__(key)

Get a variable from the default section, good for fail-fast if key does not exists.

Parameters:key (str) – key Returns:config variable Return type:str

get(*args)

Get a variable from the default section :param *args: dict.get() args :type *args: args

Returns:config variable Return type:str

get_ini(incl_unset=False)

Return the config dictionary in INI format :param incl_unset: include variables with no defaults. :type incl_unset: bool

Returns:string of the config file in INI format Return type:str

get_section(*args)

get a section dictionary Args:

Returns:section config dictionary Return type:dict

load()

Load all configurations from available resources, skip if empty:

1. default` dict passed to ConfigLoad.__init__().

2. Custom paths as defined in CONFS_PATH in

   constants.

3. XDG standard paths.
4. Environment variables.

Returns:dict of dicts. Return type:dict

update_args(args)

Update config dictionary with parsed args, as resolved by argparse. Only root positional arguments that already exist will overridden.

Parameters:args (namespace) – args parsed by argparse

update_parser(parser)

Update config dictionary with declared arguments in an argparse.parser New variables will be created, and existing ones overridden.

Parameters:parser (argparse.ArgumentParser) – parser to read variables from

lago.config._get_configs_path()

Get a list of possible configuration files, from the following sources: 1. All files that exists in constants.CONFS_PATH. 2. All XDG standard config files for “lago.conf”, in reversed order of importance.

Returns:list of files Return type:list(str)

lago.config.get_env_dict(root_section)

Read all Lago variables from the environment. The lookup format is: LAGO_VARNAME - will land into ‘lago’ section LAGO__SECTION1__VARNAME - will land into ‘section1’ section, notice the double ‘__’. LAGO__LONG_SECTION_NAME__VARNAME - will land into ‘long_section_name’

Returns:dict of section configuration dicts Return type:dict

Examples

>>> os.environ['LAGO_GLOBAL_VAR'] = 'global'
>>> os.environ['LAGO__INIT__REPO_PATH'] = '/tmp/store'
>>>
>>> config.get_env_dict()
{'init': {'repo_path': '/tmp/store'}, 'lago': {'global_var': 'global'}}

lago.constants module

lago.constants.CONFS_PATH = ['/etc/lago/lago.conf']

CONFS_PATH - default path to first look for configuration files.

lago.constants.LIBEXEC_DIR = '/usr/libexec/lago/'

LIBEXEC_DIR -

lago.export module

class lago.export.DiskExportManager(dst, disk_type, disk, do_compress)

Bases: object

DiskExportManager object is responsible on the export process of an image from the current Lago prefix.

DiskExportManger is the base class of specific DiskExportManger. Each specific DiskExportManger is responsible on the export process of an image with a specific type (e.g template, file...)

src

str – Path to the image that should be exported

name

str – The name of the exported disk

dst

str – The absolute path of the exported disk

disk_type

str – The type of the image e.g template, file, empty...

disk

dict – Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME

exported_metadata

dict – A copy of the source disk metadata, this dict should be updated with new values during the export process.

do_compress

bool – If true, apply compression to the exported disk.

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

calc_sha(checksum)

Calculate the checksum of the new exported disk, write it to a file, and update this managers ‘exported_metadata’.

Parameters:checksum (str) – The type of the checksum

compress()

Compress the new exported image, Block size was taken from virt-builder page

copy()

Copy the disk using cp in order to preserves the ‘sparse’ structure of the file

export()

This method will handle the export process and should implemented in each subclass.

static get_instance_by_type(dst, disk, do_compress, *args, **kwargs)

Parameters:

• dst (str) – The path of the new exported disk. can contain env variables.
• disk (dict) – Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME
• do_compress (bool) – If true, apply compression to the exported disk.

Returns

An instance of a subclass of DiskExportManager which matches the disk type.

sparse()

Make the exported images more compact by removing unused space. Please refer to ‘virt-sparsify’ for more info.

update_lago_metadata()

write_lago_metadata()

class lago.export.FileExportManager(dst, disk_type, disk, do_compress, *args, **kwargs)

Bases: lago.export.DiskExportManager

FileExportManager is responsible exporting images of type file and empty.

standalone

bool – If true, create a new image which is the result of merging all the layers of src (the image that we want to export).

src_qemu_info

dict – Metadata on src which was generated by qemu-img.

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

export()

See DiskExportManager.export

class lago.export.TemplateExportManager(dst, disk_type, disk, do_compress, *args, **kwargs)

Bases: lago.export.DiskExportManager

TemplateExportManager is responsible exporting images of type template.

See superclass

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

export()

See DiskExportManager.export

rebase()

Change the backing-file entry of the exported disk. Please refer to ‘qemu-img rebase’ manual for more info.

update_lago_metadata()

class lago.export.VMExportManager(disks, dst, compress, with_threads=True, *args, **kwargs)

Bases: object

VMExportManager object is responsible on the export process of a list of disks.

disks

list of dicts – Disks to export.

dst

str – Where to place the exported disks.

compress

bool – If True compress each exported disk.

with_threads

bool – If True, run the export in parallel

*args

list – Extra args, will be passed to each DiskExportManager

**kwargs

dict – Extra args, will be passed to each DiskExportManager

_collect()

Returns:The disks that needed to be exported Return type:(generator of dicts)

_get_export_mgr()

Returns:Handler for each disk Return type:(DiskExportManager)

collect_paths()

Returns:The path of the disks that will be exported. Return type:(list of str)

export()

Run the export process :returns: The path of the exported disks. :rtype: (list of str)

exported_disks_paths()

Returns:The path of the exported disks. Return type:(list of str)

lago.guestfs_tools module

lago.guestfs_tools._copy_path(guestfs_conn, guest_path, host_path)

lago.guestfs_tools.extract_paths(disk_path, disk_root, paths, ignore_nopath)

Extract paths from a disk using guestfs

Parameters:

• disk_path (str) – path to the disk
• disk_root (str) – root partition
• paths (list of tuples) – files to extract in [(src1, dst1), (src2, dst2)...] format.
• ignore_nopath (bool) – If set to True, ignore source paths that do
• exist (not) –

Returns:

None

Raises:

• ExtractPathNoPathError – if a none existing path was found on the VM, and ignore_nopath is False.
• ExtractPathError – on all other failures.

lago.lago_ansible module

class lago.lago_ansible.LagoAnsible(prefix)

Bases: object

A class for handling Ansible related tasks

prefix

lago.prefix.Prefix – The prefix that this object wraps

_generate_entry(vm)

Generate host entry for the given VM :param vm: The VM for which the entry

should be created for.

Returns:An entry for vm Return type:str

get_inventory(keys=None)

Create an Ansible inventory based on python dicts and lists. The returned value is a dict in which every key represents a group and every value is a list of entries for that group.

Parameters:keys (list of str) – Path to the keys that will be used to create groups. Returns:dict based Ansible inventory Return type:dict

get_inventory_str(keys=None)

Convert a dict generated by ansible.LagoAnsible.get_inventory to an INI-like file.

Parameters:keys (list of str) – Path to the keys that will be used to create groups. Returns:INI-like Ansible inventory Return type:str

get_inventory_temp_file(*args, **kwds)

Context manager which returns the inventory written on a tempfile. The tempfile will be deleted as soon as this context manger ends.

Parameters:keys (list of str) – Path to the keys that will be used to create groups. Yields:tempfile.NamedTemporaryFile – Temp file containing the inventory

static get_key(key, data_structure)

Helper method for extracting values from a nested data structure.

Parameters:

• key (str) – The path to the vales (a series of keys and indexes separated by ‘/’)
• data_structure (dict or list) – The data structure from which the value will be extracted.

Returns:

The values associated with key

Return type:

str

lago.log_utils module

This module defines the special logging tools that lago uses

lago.log_utils.ALWAYS_SHOW_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.ALWAYS_SHOW_TRIGGER_MSG = 'force-show:%s'

Message template that will always shoud the messago

class lago.log_utils.ColorFormatter(fmt=None, datefmt=None)

Bases: logging.Formatter

Formatter to add colors to log records

CRITICAL = '\x1b[31m'

CYAN = '\x1b[36m'

DEBUG = ''

DEFAULT = '\x1b[0m'

ERROR = '\x1b[31m'

GREEN = '\x1b[32m'

INFO = '\x1b[36m'

NONE = ''

RED = '\x1b[31m'

WARNING = '\x1b[33m'

WHITE = '\x1b[37m'

YELLOW = '\x1b[33m'

classmethod colored(color, message)

Small function to wrap a string around a color

Parameters:

• color (str) – name of the color to wrap the string with, must be one of the class properties
• message (str) – String to wrap with the color

Returns:

the colored string

Return type:

str

format(record)

Adds colors to a log record and formats it with the default

Parameters:record (logging.LogRecord) – log record to format Returns:The colored and formatted record string Return type:str

class lago.log_utils.ContextLock

Bases: object

Context manager to thread lock a block of code

lago.log_utils.END_TASK_MSG = 'Success'

Message to be shown when a task is ended

lago.log_utils.END_TASK_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.END_TASK_TRIGGER_MSG = 'end task%s'

Message template that will trigger a task end

class lago.log_utils.LogTask(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)

Bases: object

Context manager for a log task

Example

>>> with LogTask('mytask'):
...     pass

lago.log_utils.START_TASK_MSG = ''

Message to be shown when a task is started

lago.log_utils.START_TASK_REG = <_sre.SRE_Pattern object>

Regexp that will match the above template

lago.log_utils.START_TASK_TRIGGER_MSG = 'start task%s'

Message template that will trigger a task

class lago.log_utils.Task(name, *args, **kwargs)

Bases: collections.deque

Small wrapper around deque to add the failed status and name to a task

name

str – name for this task

failed

bool – If this task has failed or not (if there was any error log shown during it’s execution)

force_show

bool – If set, will show any log records generated inside this task even if it’s out of nested depth limit

elapsed_time()

class lago.log_utils.TaskHandler(initial_depth=0, task_tree_depth=-1, buffer_size=2000, dump_level=40, level=0, formatter=<class 'lago.log_utils.ColorFormatter'>)

Bases: logging.StreamHandler

This log handler will use the concept of tasks, to hide logs, and will show all the logs for the current task if there’s a logged error while running that task.

It will hide any logs that belong to nested tasks that have more than task_tree_depth parent levels, and for the ones that are above that level, it will show only the logs that have a loglevel above level.

You can force showing a log record immediately if you use the log_always() function bypassing all the filters.

If there’s a log record with log level higher than dump_level it will be considered a failure, and all the logs for the current task that have a log level above level will be shown no matter at which depth the task belongs to. Also, all the parent tasks will be tagged as error.

formatter

logging.LogFormatter – formatter to use

initial_depth

int – Initial depth to account for, in case this handler was created in a subtask

tasks_by_thread (dict of str

OrderedDict of str: Task): List of thread names, and their currently open tasks with their latest log records

dump_level

int – log level from which to consider a log record as error

buffer_size

int – Size of the log record deque for each task, the bigger, the more records it can show in case of error but the more memory it will use

task_tree_depth

int – number of the nested level to show start/end task logs for, if -1 will show always

level

int – Log level to show logs from if the depth limit is not reached

main_failed

bool – used to flag from a child thread that the main should fail any current task

_tasks_lock

ContextLock – Lock for the tasks_by_thread dict

_main_thread_lock

ContextLock – Lock for the main_failed bool

TASK_INDICATORS = ['@', '#', '*', '-', '~']

List of chars to show as task prefix, to ease distinguishing them

am_i_main_thread

**Returns* – bool* – if the current thread is the main thread

close_children_tasks(parent_task_name)

Closes all the children tasks that were open

Parameters:parent_task_name (str) – Name of the parent task Returns:None

cur_depth_level

**Returns* – int* – depth level for the current task

cur_task

**Returns* – str* – the current active task

cur_thread

**Returns* – str* – Name of the current thread

emit(record)

Handle the given record, this is the entry point from the python logging facility

Params:

record (logging.LogRecord): log record to handle

Returns:None

get_task_indicator(task_level=None)

Parameters:task_level (int or None) – task depth level to get the indicator for, if None, will use the current tasks depth Returns:char to prepend to the task logs to indicate it’s level Return type:str

get_tasks(thread_name)

Parameters:thread_name (str) – name of the thread to get the tasks for Returns:

list of task names and log records for

each for the given thread

Return type:OrderedDict of str, Task

handle_closed_task(task_name, record)

Do everything needed when a task is closed

Params:

task_name (str): name of the task that is finishing record (logging.LogRecord): log record with all the info

Returns:None

handle_error()

Handles an error log record that should be shown

Returns:None

handle_new_task(task_name, record)

Do everything needed when a task is starting

Params:

task_name (str): name of the task that is starting record (logging.LogRecord): log record with all the info

Returns:None

mark_main_tasks_as_failed()

Flags to the main thread that all it’s tasks sholud fail

Returns:None

mark_parent_tasks_as_failed(task_name, flush_logs=False)

Marks all the parent tasks as failed

Parameters:

• task_name (str) – Name of the child task
• flush_logs (bool) – If True will discard all the logs form parent tasks

Returns:

None

pretty_emit(record, is_header=False, task_level=None)

Wrapper around the logging.StreamHandler emit method to add some decoration stuff to the message

Parameters:

• record (logging.LogRecord) – log record to emit
• is_header (bool) – if this record is a header, usually, a start or end task message
• task_level (int) – If passed, will take that as the current nested task level instead of calculating it from the current tasks

Returns:

None

should_show_by_depth(cur_level=None)

Parameters:cur_level (int) – depth level to take into account Returns:

True if the given depth level should show messages (not

taking into account the log level)

Return type:bool

should_show_by_level(record_level, base_level=None)

Parameters:

• record_level (int) – log level of the record to check
• base_level (int or None) – log level to check against, will use the object’s dump_level if None is passed

Returns:

True if the given log record should be shown according to the

log level

Return type:

bool

tasks

Returns –

OrderedDict of str, Task: list of task names and log records for

each for the current thread

lago.log_utils.end_log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')

Ends a log task

Parameters:

• task (str) – name of the log task to end
• logger (logging.Logger) – logger to use
• level (str) – log level to use

Returns:

None

lago.log_utils.get_default_log_formatter()

lago.log_utils.hide_paramiko_logs()

lago.log_utils.hide_stevedore_logs()

Hides the logs of stevedore, this function was added in order to support older versions of stevedore

We are using the NullHandler in order to get rid from ‘No handlers could be found for logger...’ msg

Returns:None

lago.log_utils.log_always(message)

Wraps the given message with a tag that will make it be always logged by the task logger

Parameters:message (str) – message to wrap with the tag Returns:

tagged message that will get it shown immediately by the task

logger

Return type:str

lago.log_utils.log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)

Parameterized decorator to wrap a function in a log task

Example

>>> @log_task('mytask')
... def do_something():
...     pass

lago.log_utils.setup_prefix_logging(logdir)

Sets up a file logger that will create a log in the given logdir (usually a lago prefix)

Parameters:logdir (str) – path to create the log into, will be created if it does not exist Returns:None

lago.log_utils.start_log_task(task, logger=<module 'logging' from '/usr/lib/python2.7/logging/__init__.pyc'>, level='info')

Starts a log task

Parameters:

• task (str) – name of the log task to start
• logger (logging.Logger) – logger to use
• level (str) – log level to use

Returns:

None

lago.paths module

class lago.paths.Paths(prefix)

Bases: object

images(*path)

logs()

metadata()

prefix_lagofile()

This file represents a prefix that’s initialized

prefixed(*args)

scripts(*args)

ssh_id_rsa()

ssh_id_rsa_pub()

uuid()

virt(*path)

lago.prefix module

lago.sdk module

lago.sdk_utils module

class lago.sdk_utils.SDKMethod(name)

Bases: object

Metadata to store inside the decorated function

class lago.sdk_utils.SDKWrapper

Bases: sphinx.ext.autodoc.ObjectProxy

A proxy object that exposes only methods which were decorated with expose() decorator.

lago.sdk_utils.expose(func)

Decorator to be used with SDKWrapper. This decorator indicates that the wrapped method or class should be exposed in the proxied object.

Parameters:func (types.FunctionType/types.MethodType) – function to decorate Returns:None

lago.sdk_utils.getattr_sdk(attr, name)

Filter SDK attributes

Parameters:

• attr (attribute) – Attribute as returned by getattr().
• name (str) – Attribute name.

Returns:

attr if passed.

lago.sdk_utils.setup_sdk_logging(logfile=None, loglevel=20)

Setup a NullHandler to the root logger. If logfile is passed, additionally add a FileHandler in loglevel level.

Parameters:

• logfile (str) – A path to setup a log file.
• loglevel (int) – logging log level.

Returns:

None

lago.service module

class lago.service.SysVInitService(vm, name)

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/sbin/service'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

_request_start()

_request_stop()

state()

class lago.service.SystemdContainerService(vm, name)

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/usr/bin/docker'

HOST_BIN_PATH = '/usr/bin/systemctl'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

_request_start()

_request_stop()

state()

class lago.service.SystemdService(vm, name)

Bases: lago.plugins.service.ServicePlugin

BIN_PATH = '/usr/bin/systemctl'

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

_request_start()

_request_stop()

state()

lago.ssh module

lago.ssh._gen_ssh_command_id()

lago.ssh.drain_ssh_channel(chan, stdin=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)

lago.ssh.get_ssh_client(ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456')

lago.ssh.interactive_ssh(ip_addr, command=None, host_name=None, ssh_key=None, username='root', password='123456')

lago.ssh.interactive_ssh_channel(chan, command=None, stdin=<open file '<stdin>', mode 'r'>)

lago.ssh.ssh(ip_addr, command, host_name=None, data=None, show_output=True, propagate_fail=True, tries=None, ssh_key=None, username='root', password='123456')

lago.ssh.ssh_script(ip_addr, path, host_name=None, show_output=True, ssh_key=None, username='root', password='123456')

lago.ssh.wait_for_ssh(ip_addr, host_name=None, connect_timeout=600, ssh_key=None, username='root', password='123456')

lago.subnet_lease module

lago.sysprep module

lago.sysprep._guestfs_version(default={'major': 1L, 'minor': 20L})

lago.sysprep._render_template(distro, loader, **kwargs)

lago.sysprep.sysprep(disk, distro, loader=None, backend='direct', **kwargs)

Run virt-sysprep on the disk, commands are built from the distro specific template and arguments passed in kwargs. If no template is available it will default to sysprep-base.j2.

Parameters:

• disk (str) – path to disk
• distro (str) – distro to render template for
• loader (jinja2.BaseLoader) – Jinja2 template loader, if not passed, will search Lago’s package.
• backend (str) – libguestfs backend to use
• **kwargs (dict) – environment variables for Jinja2 template

Returns:

None

Raises:

RuntimeError – On virt-sysprep none 0 exit code.

lago.templates module

This module contains any disk template related classes and functions, including the repository store manager classes and template providers, some useful definitions:

• Template repositories:

  Repository where to fetch templates from, as an http server

• Template store:

  Local store to cache templates

• Template:

  Unititialized disk image to use as base for other disk images

• Template version:

  Specific version of a template, to allow getting updates without having to change the template name everywhere

class lago.templates.FileSystemTemplateProvider(root)

Handles file type templates, that is, getting a disk template from the host’s filesystem

_prefixed(*path)

Join all the given paths prefixed with this provider’s base root path

Parameters:*path (str) – sections of the path to join, passed as positional arguments Returns:Joined paths prepended with the provider root path Return type:str

download_image(handle, dest)

Copies over the handl to the destination

Parameters:

• handle (str) – path to copy over
• dest (str) – path to copy to

Returns:

None

get_hash(handle)

Returns the associated hash for the given handle, the hash file must exist (handle + '.hash').

Parameters:handle (str) – Path to the template to get the hash from Returns:Hash for the given handle Return type:str

get_metadata(handle)

Returns the associated metadata info for the given handle, the metadata file must exist (handle + '.metadata').

Parameters:handle (str) – Path to the template to get the metadata from Returns:Metadata for the given handle Return type:dict

class lago.templates.HttpTemplateProvider(baseurl)

This provider allows the usage of http urls for templates

download_image(handle, dest)

Downloads the image from the http server

Parameters:

• handle (str) – url from the self.baseurl to the remote template
• dest (str) – Path to store the downloaded url to, must be a file path

Returns:

None

static extract_image_xz(path)

get_hash(handle)

Get the associated hash for the given handle, the hash file must exist (handle + '.hash').

Parameters:handle (str) – Path to the template to get the hash from Returns:Hash for the given handle Return type:str

get_metadata(handle)

Returns the associated metadata info for the given handle, the metadata file must exist (handle + '.metadata'). If the given handle has an .xz extension, it will get removed when calculating the handle metadata path

Parameters:handle (str) – Path to the template to get the metadata from Returns:Metadata for the given handle Return type:dict

open_url(url, suffix='', dest=None)

Opens the given url, trying the compressed version first. The compressed version url is generated adding the .xz extension to the url and adding the given suffix after that .xz extension. If dest passed, it will download the data to that path if able

Parameters:

• url (str) – relative url from the self.baseurl to retrieve
• suffix (str) – optional suffix to append to the url after adding the compressed extension to the path
• dest (str or None) – Path to save the data to

Returns:

response object to read from (lazy read), closed

if no dest passed

Return type:

urllib.addinfourl

Raises:

RuntimeError – if the url gave http error when retrieving it

class lago.templates.Template(name, versions)

Disk image template class

name

str – Name of this template

_versions (dict(str

TemplateVersion)): versions for this template

get_latest_version()

Retrieves the latest version for this template, the latest being the one with the newest timestamp

Returns:TemplateVersion

get_version(ver_name=None)

Get the given version for this template, or the latest

Parameters:ver_name (str or None) – Version to retieve, None for the latest Returns:

The version matching the given name or the latest

one

Return type:TemplateVersion

class lago.templates.TemplateRepository(dom)

A template repository is a single source for templates, that uses different providers to actually retrieve them. That means for example that the ‘ovirt’ template repository, could support the ‘http’ and a theoretical ‘gluster’ template providers.

_dom

dict – Specification of the template

_providers

dict – Providers instances for any source in the spec

_get_provider(spec)

Get the provider for the given template spec

Parameters:spec (dict) – Template spec Returns:A provider instance for that spec Return type:HttpTemplateProvider or FileSystemTemplateProvider

classmethod from_url(path)

Instantiate a TemplateRepository instance from the data in a file or url

Parameters:path (str) – Path or url to the json file to load Returns:A new instance Return type:TemplateRepository

get_by_name(name)

Retrieve a template by it’s name

Parameters:name (str) – Name of the template to retrieve Raises:KeyError – if no template is found

name

Getter for the template repo name

Returns:the name of this template repo Return type:str

class lago.templates.TemplateStore(path)

Local cache to store templates

The store uses various files to keep track of the templates cached, access and versions. An example template store looks like:

$ tree /var/lib/lago/store/
/var/lib/lago/store/
├── in_office_repo:centos6_engine:v2.tmp
├── in_office_repo:centos7_engine:v5.tmp
├── in_office_repo:fedora22_host:v2.tmp
├── phx_repo:centos6_engine:v2
├── phx_repo:centos6_engine:v2.hash
├── phx_repo:centos6_engine:v2.metadata
├── phx_repo:centos6_engine:v2.users
├── phx_repo:centos7_engine:v4.tmp
├── phx_repo:centos7_host:v4.tmp
└── phx_repo:storage-nfs:v1.tmp

There you can see the files:

• *.tmp

  Temporary file created while downloading the template from the repository (depends on the provider)

• ${repo_name}:${template_name}:${template_version}

  This file is the actual disk image template

• *.hash

  Cached hash for the template disk image

• *.metadata

  Metadata for this template image in json format, usually this includes the distro and root-password

__contains__(temp_ver)

Checks if a given version is in this store

Parameters:temp_ver (TemplateVersion) – Version to look for Returns:True if the version is in this store Return type:bool

_prefixed(*path)

Join the given paths and prepend this stores path

Parameters:*path (str) – list of paths to join, as positional arguments Returns:all the paths joined and prepended with the store path Return type:str

download(temp_ver, store_metadata=True)

Retrieve the given template version

Parameters:

• temp_ver (TemplateVersion) – template version to retrieve
• store_metadata (bool) – If set to False, will not refresh the local metadata with the retrieved one

Returns:

None

get_path(temp_ver)

Get the path of the given version in this store

Parameters:TemplateVersion (temp_ver) – version to look for Returns:The path to the template version inside the store Return type:str Raises:RuntimeError – if the template is not in the store

get_stored_hash(temp_ver)

Retrieves the hash for the given template version from the store

Parameters:temp_ver (TemplateVersion) – template version to retrieve the hash for Returns:hash of the given template version Return type:str

get_stored_metadata(temp_ver)

Retrieves the metadata for the given template version from the store

Parameters:temp_ver (TemplateVersion) – template version to retrieve the metadata for Returns:the metadata of the given template version Return type:dict

class lago.templates.TemplateVersion(name, source, handle, timestamp)

Each template can have multiple versions, each of those is actually a different disk template file representation, under the same base name.

download(destination)

Retrieves this template to the destination file

Parameters:destination (str) – file path to write this template to Returns:None

get_hash()

Returns the associated hash for this template version

Returns:Hash for this version Return type:str

get_metadata()

Returns the associated metadata info for this template version

Returns:Metadata for this version Return type:dict

timestamp()

Getter for the timestamp

lago.templates._locked(func)

Decorator that ensures that the decorated function has the lock of the repo while running, meant to decorate only bound functions for classes that have lock_path method.

lago.templates.find_repo_by_name(name, repo_dir=None)

Searches the given repo name inside the repo_dir (will use the config value ‘template_repos’ if no repo dir passed), will rise an exception if not found

Parameters:

• name (str) – Name of the repo to search
• repo_dir (str) – Directory where to search the repo

Returns:

path to the repo

Return type:

str

Raises:

RuntimeError – if not found

lago.utils module

class lago.utils.CommandStatus

Bases: lago.utils.CommandStatus

class lago.utils.EggTimer(timeout)

elapsed()

class lago.utils.ExceptionTimer(timeout)

Bases: object

start()

stop()

exception lago.utils.LagoException

Bases: exceptions.Exception

exception lago.utils.LagoInitException

Bases: lago.utils.LagoException

exception lago.utils.LagoUserException

Bases: lago.utils.LagoException

class lago.utils.LockFile(path, timeout=None, **kwargs)

Bases: object

Context manager that creates a lock around a directory, with optional timeout in the acquire operation

Parameters:

• path (str) – path to the dir to lock
• timeout (int) – timeout in seconds to wait while acquiring the lock
• **kwargs (dict) – Any other param to pass to lockfile.LockFile

__enter__()

Start the lock with timeout if needed in the acquire operation

Raises:TimerException – if the timeout is reached before acquiring the lock

class lago.utils.RollbackContext(*args)

Bases: object

A context manager for recording and playing rollback. The first exception will be remembered and re-raised after rollback

Sample usage: > with RollbackContext() as rollback: > step1() > rollback.prependDefer(lambda: undo step1) > def undoStep2(arg): pass > step2() > rollback.prependDefer(undoStep2, arg)

More examples see tests/utilsTests.py @ vdsm code

__exit__(exc_type, exc_value, traceback)

If this function doesn’t return True (or raises a different exception), python re-raises the original exception once this function is finished.

clear()

defer(func, *args, **kwargs)

prependDefer(func, *args, **kwargs)

exception lago.utils.TimerException

Bases: exceptions.Exception

Exception to throw when a timeout is reached

class lago.utils.VectorThread(targets)

join_all(raise_exceptions=True)

start_all()

lago.utils._CommandStatus

alias of CommandStatus

lago.utils._add_subparser_to_cp(cp, section, actions, incl_unset)

lago.utils._ret_via_queue(func, queue)

lago.utils._run_command(command, input_data=None, stdin=None, out_pipe=-1, err_pipe=-1, env=None, uuid=None, **kwargs)

Runs a command

Parameters:

• command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
• input_data (str) – If passed, will feed that data to the subprocess through stdin
• out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
• stdin (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdin
• err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
• of str (env(dict) – str): If set, will use the given dict as env for the subprocess
• uuid (uuid) – If set the command will be logged with the given uuid converted to string, otherwise, a uuid v4 will be generated.
• **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call

Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.add_timestamp_suffix(base_string)

lago.utils.argparse_to_ini(parser, root_section='lago', incl_unset=False)

lago.utils.compress(input_file, block_size, fail_on_error=True)

lago.utils.cp(input_file, output_file, fail_on_error=True)

lago.utils.deepcopy(original_obj)

Creates a deep copy of an object with no crossed referenced lists or dicts, useful when loading from yaml as anchors generate those cross-referenced dicts and lists

Parameters:original_obj (object) – Object to deep copy Returns:deep copy of the object Return type:object

lago.utils.filter_spec(spec, paths, wildcard='*', separator='/')

Remove keys from a spec file. For example, with the following path: domains//disks//metadata all the metadata dicts from all domains disks will be removed.

Parameters:

• spec (dict) – spec to remove keys from
• paths (list) – list of paths to the keys that should be removed
• wildcard (str) – wildcard character
• separator (str) – path separator

Returns:

None

Raises:

utils.LagoUserException – If a malformed path was detected

lago.utils.func_vector(target, args_sequence)

lago.utils.get_hash(file_path, checksum='sha1')

Generate a hash for the given file

Parameters:

• file_path (str) – Path to the file to generate the hash for
• checksum (str) – hash to apply, one of the supported by hashlib, for example sha1 or sha512

Returns:

hash for that file

Return type:

str

lago.utils.get_qemu_info(path, backing_chain=False, fail_on_error=True)

Get info on a given qemu disk

Parameters:

• path (str) – Path to the required disk
• backing_chain (boo) – if true, include also info about
• image predecessors. (the) –

Returns:

if backing_chain == True then a list of dicts else a dict

Return type:

object

lago.utils.in_prefix(prefix_class, workdir_class)

lago.utils.invoke_different_funcs_in_parallel(*funcs)

lago.utils.invoke_in_parallel(func, *args_sequences)

lago.utils.ipv4_to_mac(ip)

lago.utils.json_dump(obj, f)

lago.utils.load_virt_stream(virt_fd)

Loads the given conf stream into a dict, trying different formats if needed

Parameters:virt_fd (str) – file like objcect with the virt config to load Returns:Loaded virt config Return type:dict

lago.utils.qemu_rebase(target, backing_file, safe=True, fail_on_error=True)

changes the backing file of ‘source’ to ‘backing_file’ If backing_file is specified as “” (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file). (Taken from qemu-img man page)

Parameters:

• target (str) – Path to the source disk
• backing_file (str) – path to the base disk
• safe (bool) – if false, allow unsafe rebase (check qemu-img docs for more info)

lago.utils.read_nonblocking(file_descriptor)

lago.utils.rotate_dir(base_dir)

lago.utils.run_command(command, input_data=None, out_pipe=-1, err_pipe=-1, env=None, **kwargs)

Runs a command non-interactively

Parameters:

• command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
• input_data (str) – If passed, will feed that data to the subprocess through stdin
• out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
• err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
• of str (env(dict) – str): If set, will use the given dict as env for the subprocess
• **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call

Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.run_command_with_validation(cmd, fail_on_error=True, msg='An error has occurred')

lago.utils.run_interactive_command(command, env=None, **kwargs)

Runs a command interactively, reusing the current stdin, stdout and stderr

Parameters:

• command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
• of str (env(dict) – str): If set, will use the given dict as env for the subprocess
• **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call

Returns:

result of the interactive execution

Return type:

lago.utils.CommandStatus

lago.utils.service_is_enabled(name)

lago.utils.sparse(input_file, input_format, fail_on_error=True)

lago.utils.ver_cmp(ver1, ver2)

Compare lago versions

Parameters:

• ver1 (str) – version string
• ver2 (str) – version string

Returns:

Return negative if ver1<ver2, zero if ver1==ver2, positive if ver1>ver2.

lago.utils.with_logging(func)

lago.validation module

lago.validation.check_import(module_name)

Search if a module exists, and it is possible to try importing it

Parameters:module_name (str) – module to import Returns:True if the package is found Return type:bool

lago.virt module

class lago.virt.VirtEnv(prefix, vm_specs, net_specs)

Bases: object

Env properties: * prefix * vms * net

• libvirt_con

_create_net(net_spec)

_create_vm(vm_spec)

_get_stop_shutdown_common_args(vm_names)

Get the common arguments for stop and shutdown commands

Parameters:vm_names (list of str) – The names of the requested vms

Returns

list of plugins.vm.VMProviderPlugin:

vms objects that should be stopped

list of virt.Network: net objects that should be stopped str: log message

Raises:utils.LagoUserException – If a vm name doesn’t exist

_get_unused_nets(vms_to_stop)

Return a list of nets that used only by the vms in vms_to_stop

Parameters:vms_to_stop (list of str) – The names of the requested vms

Returns

list of virt.Network: net objects that used only by

vms in vms_to_stop

Raises:utils.LagoUserException – If a vm name doesn’t exist

bootstrap()

create_snapshots(*args, **kwargs)

export_vms(vms_names, standalone, dst_dir, compress, init_file_name, out_format, collect_only=False, with_threads=True)

classmethod from_prefix(prefix)

generate_init(dst, out_format, vms_to_include, filters=None)

Generate an init file which represents this env and can be used with the images created by self.export_vms

Parameters:

• dst (str) – path and name of the new init file
• out_format (plugins.output.OutFormatPlugin) – formatter for the output (the default is yaml)
• filters (list) – list of paths to keys that should be removed from the init file
• (list of (vms_to_include) – class:lago.plugins.vm.VMPlugin): list of vms to include in the init file

Returns:

None

get_compat()

Get compatibility level for this environment - which is the Lago version used to create this environment

get_env_spec(filters=None)

Get the spec of the current env. The spec will hold the info about all the domains and networks associated with this env.

Parameters:filters (list) – list of paths to keys that should be removed from the init file Returns:the spec of the current env Return type:dict

get_net(name=None)

get_nets()

get_snapshots(domains=None)

Get the list of snapshots for each domain

Parameters:

• domanins (list of str) – list of the domains to get the snapshots
• all will be returned if none or empty list passed (for,) –

Returns:

with the domain names and the list of snapshots for each

Return type:

dict of str -> list(str)

get_vm(name)

get_vms(vm_names=None)

Returns the vm objects associated with vm_names if vm_names is None, return all the vms in the prefix

Parameters:vm_names (list of str) – The names of the requested vms

Returns

dict: Which contains the requested vm objects indexed by name

Raises:utils.LagoUserException – If a vm name doesn’t exist

prefixed_name(unprefixed_name, max_length=0)

Returns a uuid pefixed identifier

Parameters:

• unprefixed_name (str) – Name to add a prefix to
• max_length (int) – maximum length of the resultant prefixed name, will adapt the given name and the length of the uuid ot fit it

Returns:

prefixed identifier for the given unprefixed name

Return type:

str

revert_snapshots(*args, **kwargs)

save(*args, **kwargs)

shutdown(vm_names, reboot=False)

start(vm_names=None)

stop(vm_names=None)

virt_path(*args)

lago.virt._gen_ssh_command_id()

lago.virt._guestfs_copy_path(g, guest_path, host_path)

lago.virt._path_to_xml(basename)

lago.vm module

class lago.vm.DefaultVM(env, spec)

Bases: lago.plugins.vm.VMPlugin

_abc_cache = <_weakrefset.WeakSet object>

_abc_negative_cache = <_weakrefset.WeakSet object>

_abc_negative_cache_version = 33

_abc_registry = <_weakrefset.WeakSet object>

class lago.vm.SSHVMProvider(vm)

Bases: lago.plugins.vm.VMProviderPlugin

bootstrap(*args, **kwargs)

create_snapshot(name, *args, **kwargs)

defined(*args, **kwargs)

revert_snapshot(name, *args, **kwargs)

start(*args, **kwargs)

state(*args, **kwargs)

stop(*args, **kwargs)

lago.workdir module