lago package¶
Subpackages¶
Submodules¶
lago.brctl module¶
lago.build module¶
-
class
lago.build.
Build
(name, disk_path, paths)[source]¶ Bases:
object
A Build object represents a build section in the init file. Each build section (which in turn belongs to a specific disk) should get his own Build object
In order to add support for a new build command, a new function with the name of the command should be implemented in this class. this function should accept a list of options and arguments and return a named tuple ‘Command’, where ‘Command.name’ is the name of the command and ‘Command.cmd’ is the a list containing the command and its args, for example: Command.name = ‘virt-customize’ Command.cmd = [‘virt-customize’, ‘-a’, PATH_TO_DISK, SOME_CMDS…]
-
paths
¶ The paths of the current prefix
Type: lago.paths.Paths
-
build_cmds
¶ A list of commands that should be invoked on the disk located in disk_path
Type: list of str
-
build
()[source]¶ Run all the commands in self.build_cmds
Raises: lago.build.BuildException
– If a command returned a non-zero code
-
get_cmd_handler
(cmd)[source]¶ Return an handler for cmd. The handler and the command should have the same name. See class description for more info about handlers.
Parameters: cmd (str) – The name of the command Returns: which handles cmd Return type: callable Raises: lago.build.BuildException
– If an handler for cmd doesn’t exist
-
classmethod
get_instance_from_build_spec
(name, disk_path, build_spec, paths)[source]¶ Parameters: - name (str) – The name of the vm this builder belongs
- disk_path (str) – The path to the disk that needs to be customized
- paths (lago.paths.Paths) – The paths of the current prefix
- build_spec (dict) – The build spec part, associated with the disk located at disk_path, from the init file.
Returns: - An instance of Build with a normalized build spec i.e ready to
be invoked.
-
normalize_build_spec
(build_spec)[source]¶ Convert a build spec into a list of Command tuples. After running this command, self.build_cmds should hold all the commands that should be run on the disk in self.disk_path.
Parameters: build_spec (dict) – The buildspec part from the init file
-
static
normalize_options
(options)[source]¶ Turns a mapping of ‘option: arg’ to a list and prefix the options. arg can be a list of arguments.
for example:
- dict = {
- o1: a1, o2: , o3: [a31, a32] o4: []
}
will be transformed to:
- [
- prefix_option(o1), a1, prefix_option(o2), prefix_option(o3), a31, prefix_option(o3), a32 prefix_option(o4)
]
note that empty arguments are omitted
Parameters: options (dict) – A mapping between options and arguments Returns: A normalized version of ‘options’ as mentioned above Return type: lst
-
static
prefix_option
(option)[source]¶ Depends on the option’s length, prefix it with ‘-‘ or ‘–’ :param option: The option to prefix :type option: str
Returns: prefixed option Return type: str
-
virt_customize
(options)[source]¶ Handler for ‘virt-customize’ note: if ‘ssh-inject’ option was specified without a path to a key, the prefix’ key will be copied to the vm.
Parameters: options (lst of str) – Options and arguments for ‘virt-customize’ Returns: which handles cmd Return type: callable Raises: lago.build.BuildException
– If an handler for cmd doesn’t exist
-
-
exception
lago.build.
BuildException
[source]¶ Bases:
lago.utils.LagoException
-
class
lago.build.
Command
(name, cmd)¶ Bases:
tuple
-
__getnewargs__
()¶ Return self as a plain tuple. Used by copy and pickle.
-
__getstate__
()¶ Exclude the OrderedDict from pickling
-
__repr__
()¶ Return a nicely formatted representation string
-
_asdict
()¶ Return a new OrderedDict which maps field names to their values
-
_fields
= ('name', 'cmd')¶
-
classmethod
_make
(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)¶ Make a new Command object from a sequence or iterable
-
_replace
(**kwds)¶ Return a new Command object replacing specified fields with new values
-
cmd
¶ Alias for field number 1
-
name
¶ Alias for field number 0
-
lago.cmd module¶
-
lago.cmd.
exit_handler
(signum, frame)[source]¶ Catch SIGTERM and SIGHUP and call “sys.exit” which raises “SystemExit” exception. This will trigger all the cleanup code defined in ContextManagers and “finally” statements.
For more details about the arguments see “signal” documentation.
Parameters: - signum (int) – The signal’s number
- frame (frame) – The current stack frame, can be None
lago.config module¶
-
class
lago.config.
ConfigLoad
(root_section='lago', defaults={})[source]¶ Bases:
object
Merges configuration parameters from 3 different sources: 1. Enviornment vairables 2. config files in .INI format 3. argparse.ArgumentParser
The assumed order(but not necessary) order of calls is: load() - load from config files and environment variables update_parser(parser) - update from the declared argparse parser update_args(args) - update from passed arguments to the parser
-
__getitem__
(key)[source]¶ Get a variable from the default section, good for fail-fast if key does not exists.
Parameters: key (str) – key Returns: config variable Return type: str
-
get
(*args)[source]¶ Get a variable from the default section :param *args: dict.get() args :type *args: args
Returns: config variable Return type: str
-
get_ini
(incl_unset=False)[source]¶ Return the config dictionary in INI format :param incl_unset: include variables with no defaults. :type incl_unset: bool
Returns: string of the config file in INI format Return type: str
-
get_section
(*args)[source]¶ get a section dictionary Args:
Returns: section config dictionary Return type: dict
-
load
()[source]¶ Load all configurations from available resources, skip if empty:
default`
dict passed toConfigLoad.__init__()
.- Custom paths as defined in
CONFS_PATH
in constants
.
- Custom paths as defined in
- XDG standard paths.
- Environment variables.
Returns: dict of dicts. Return type: dict
-
update_args
(args)[source]¶ Update config dictionary with parsed args, as resolved by argparse. Only root positional arguments that already exist will overridden.
Parameters: args (namespace) – args parsed by argparse
-
update_parser
(parser)[source]¶ Update config dictionary with declared arguments in an argparse.parser New variables will be created, and existing ones overridden.
Parameters: parser (argparse.ArgumentParser) – parser to read variables from
-
-
lago.config.
_get_configs_path
()[source]¶ Get a list of possible configuration files, from the following sources: 1. All files that exists in constants.CONFS_PATH. 2. All XDG standard config files for “lago.conf”, in reversed order of importance.
Returns: list of files Return type: list(str)
-
lago.config.
get_env_dict
(root_section)[source]¶ Read all Lago variables from the environment. The lookup format is: LAGO_VARNAME - will land into ‘lago’ section LAGO__SECTION1__VARNAME - will land into ‘section1’ section, notice the double ‘__’. LAGO__LONG_SECTION_NAME__VARNAME - will land into ‘long_section_name’
Returns: dict of section configuration dicts Return type: dict Examples
>>> os.environ['LAGO_GLOBAL_VAR'] = 'global' >>> os.environ['LAGO__INIT__REPO_PATH'] = '/tmp/store' >>> >>> config.get_env_dict() {'init': {'repo_path': '/tmp/store'}, 'lago': {'global_var': 'global'}}
lago.constants module¶
-
lago.constants.
CONFS_PATH
= ['/etc/lago/lago.conf']¶ CONFS_PATH - default path to first look for configuration files.
-
lago.constants.
LIBEXEC_DIR
= '/usr/libexec/lago/'¶ LIBEXEC_DIR -
lago.export module¶
-
class
lago.export.
DiskExportManager
(dst, disk_type, disk, do_compress)[source]¶ Bases:
object
DiskExportManager object is responsible on the export process of an image from the current Lago prefix.
DiskExportManger is the base class of specific DiskExportManger. Each specific DiskExportManger is responsible on the export process of an image with a specific type (e.g template, file…)
-
disk
¶ Disk attributes (of the disk that should be exported) as found in workdir/current/virt/VM-NAME
Type: dict
-
exported_metadata
¶ A copy of the source disk metadata, this dict should be updated with new values during the export process.
Type: dict
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
calc_sha
(checksum)[source]¶ Calculate the checksum of the new exported disk, write it to a file, and update this managers ‘exported_metadata’.
Parameters: checksum (str) – The type of the checksum
-
export
()[source]¶ This method will handle the export process and should implemented in each subclass.
-
static
get_instance_by_type
(dst, disk, do_compress, *args, **kwargs)[source]¶ Parameters: - Returns
- An instance of a subclass of DiskExportManager which matches the disk type.
-
-
class
lago.export.
FileExportManager
(dst, disk_type, disk, do_compress, *args, **kwargs)[source]¶ Bases:
lago.export.DiskExportManager
FileExportManager is responsible exporting images of type file and empty.
-
standalone
¶ If true, create a new image which is the result of merging all the layers of src (the image that we want to export).
Type: bool
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.export.
TemplateExportManager
(dst, disk_type, disk, do_compress, *args, **kwargs)[source]¶ Bases:
lago.export.DiskExportManager
TemplateExportManager is responsible exporting images of type template.
-
See superclass
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.export.
VMExportManager
(disks, dst, compress, with_threads=True, *args, **kwargs)[source]¶ Bases:
object
VMExportManager object is responsible on the export process of a list of disks.
-
disks
¶ Disks to export.
Type: list of dicts
-
*args
Extra args, will be passed to each DiskExportManager
Type: list
-
**kwargs
Extra args, will be passed to each DiskExportManager
Type: dict
-
_get_export_mgr
()[source]¶ Returns: Handler for each disk Return type: (DiskExportManager)
-
collect_paths
()[source]¶ Returns: The path of the disks that will be exported. Return type: (list of str)
-
lago.guestfs_tools module¶
-
exception
lago.guestfs_tools.
GuestFSError
[source]¶ Bases:
lago.utils.LagoException
-
lago.guestfs_tools.
extract_paths
(disk_path, disk_root, paths, ignore_nopath)[source]¶ Extract paths from a disk using guestfs
Parameters: - disk_path (str) – path to the disk
- disk_root (str) – root partition
- paths (list of tuples) – files to extract in
[(src1, dst1), (src2, dst2)…] format, if
srcN
is a directory in the guest, anddstN
does not exist on the host, it will be created. IfsrcN
is a file on the guest, it will be copied exactly todstN
- ignore_nopath (bool) – If set to True, ignore paths in the guest that do not exit
Returns: None
Raises: ExtractPathNoPathError
– if a none existing path was found on the guest, and ignore_nopath is False.ExtractPathError
– on all other failures.
-
lago.guestfs_tools.
find_rootfs
(conn, disk_root)[source]¶ Find the image’s device root filesystem, and return its path.
- Use
guestfs.GuestFS.inspect_os()
method. If it returns more than - one root filesystem or None, try:
- Use
- Find an exact match of disk_root from
guestfs.GuestFS.list_filesystems()
, if none is found, try:
- Return the device that has the substring disk_root contained in it,
- from the output of
guestfs.GuestFS.list_filesystems()
.
Parameters: - conn (guestfs.GuestFS) – Open GuestFS handle.
- disk_root (str) – Root device to search for. Note that by default, if guestfs can deduce the filesystem, it will not be used.
Returns: root device path
Return type: Raises: GuestFSError
if no root filesystem was found
-
lago.guestfs_tools.
guestfs_conn_mount_ro
(*args, **kwds)[source]¶ Open a GuestFS handle with disk_path and try mounting the root filesystem. disk_root is a hint where it should be looked and will only be used if GuestFS will not be able to deduce it independently.
Note that mounting a live guest, can lead to filesystem inconsistencies, causing the mount operation to fail. As we use readonly mode, this is safe, but the operation itself can still fail. Therefore, this method will watch for mount failures and retry 5 times before throwing an exception.
Parameters: Yields: guestfs.GuestFS – An open GuestFS handle.
Raises: GuestFSError
– On any guestfs operation error, including exceeding retries for themount_ro()
operation.
-
lago.guestfs_tools.
guestfs_conn_ro
(*args, **kwds)[source]¶ Open a GuestFS handle and add disk in read only mode.
Parameters: disk (disk path) – Path to the disk. Yields: guestfs.GuestFS – Open GuestFS handle Raises: GuestFSError
– On any guestfs operation failure
lago.lago_ansible module¶
-
class
lago.lago_ansible.
LagoAnsible
(prefix)[source]¶ Bases:
object
A class for handling Ansible related tasks
-
prefix
¶ The prefix that this object wraps
Type: lago.prefix.Prefix
-
_generate_entry
(vm)[source]¶ Generate host entry for the given VM :param vm: The VM for which the entry
should be created for.Returns: An entry for vm Return type: str
-
get_inventory
(keys=None)[source]¶ Create an Ansible inventory based on python dicts and lists. The returned value is a dict in which every key represents a group and every value is a list of entries for that group.
Parameters: keys (list of str) – Path to the keys that will be used to create groups. Returns: dict based Ansible inventory Return type: dict
-
get_inventory_str
(keys=None)[source]¶ Convert a dict generated by ansible.LagoAnsible.get_inventory to an INI-like file.
Parameters: keys (list of str) – Path to the keys that will be used to create groups. Returns: INI-like Ansible inventory Return type: str
-
get_inventory_temp_file
(**kwds)[source]¶ Context manager which returns the inventory written on a tempfile. The tempfile will be deleted as soon as this context manger ends.
Parameters: keys (list of str) – Path to the keys that will be used to create groups. Yields: tempfile.NamedTemporaryFile – Temp file containing the inventory
-
lago.log_utils module¶
-
lago.log_utils.
ALWAYS_SHOW_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
ALWAYS_SHOW_TRIGGER_MSG
= 'force-show:%s'¶ Message template that will always shoud the messago
-
class
lago.log_utils.
ColorFormatter
(fmt=None, datefmt=None)[source]¶ Bases:
logging.Formatter
Formatter to add colors to log records
-
CRITICAL
= '\x1b[31m'¶
-
CYAN
= '\x1b[36m'¶
-
DEBUG
= ''¶
-
DEFAULT
= '\x1b[0m'¶
-
ERROR
= '\x1b[31m'¶
-
GREEN
= '\x1b[32m'¶
-
INFO
= '\x1b[36m'¶
-
NONE
= ''¶
-
RED
= '\x1b[31m'¶
-
WARNING
= '\x1b[33m'¶
-
WHITE
= '\x1b[37m'¶
-
YELLOW
= '\x1b[33m'¶
-
classmethod
colored
(color, message)[source]¶ Small function to wrap a string around a color
Parameters: Returns: the colored string
Return type:
-
format
(record)[source]¶ Adds colors to a log record and formats it with the default
Parameters: record (logging.LogRecord) – log record to format Returns: The colored and formatted record string Return type: str
-
-
class
lago.log_utils.
ContextLock
[source]¶ Bases:
object
Context manager to thread lock a block of code
-
lago.log_utils.
END_TASK_MSG
= 'Success'¶ Message to be shown when a task is ended
-
lago.log_utils.
END_TASK_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
END_TASK_TRIGGER_MSG
= 'end task%s'¶ Message template that will trigger a task end
-
class
lago.log_utils.
LogTask
(task, logger=<module 'logging' from '/home/docs/.pyenv/versions/2.7.16/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]¶ Bases:
object
Context manager for a log task
Example
>>> with LogTask('mytask'): ... pass
-
lago.log_utils.
START_TASK_MSG
= ''¶ Message to be shown when a task is started
-
lago.log_utils.
START_TASK_REG
= <_sre.SRE_Pattern object>¶ Regexp that will match the above template
-
lago.log_utils.
START_TASK_TRIGGER_MSG
= 'start task%s'¶ Message template that will trigger a task
-
class
lago.log_utils.
Task
(name, *args, **kwargs)[source]¶ Bases:
collections.deque
Small wrapper around deque to add the failed status and name to a task
-
failed
¶ If this task has failed or not (if there was any error log shown during it’s execution)
Type: bool
-
-
class
lago.log_utils.
TaskHandler
(initial_depth=0, task_tree_depth=-1, buffer_size=2000, dump_level=40, level=0, formatter=<class 'lago.log_utils.ColorFormatter'>)[source]¶ Bases:
logging.StreamHandler
This log handler will use the concept of tasks, to hide logs, and will show all the logs for the current task if there’s a logged error while running that task.
It will hide any logs that belong to nested tasks that have more than
task_tree_depth
parent levels, and for the ones that are above that level, it will show only the logs that have a loglevel abovelevel
.You can force showing a log record immediately if you use the
log_always()
function bypassing all the filters.If there’s a log record with log level higher than
dump_level
it will be considered a failure, and all the logs for the current task that have a log level abovelevel
will be shown no matter at which depth the task belongs to. Also, all the parent tasks will be tagged as error.-
formatter
¶ formatter to use
Type: logging.LogFormatter
-
tasks_by_thread (dict of str
OrderedDict of str: Task): List of thread names, and their currently open tasks with their latest log records
-
buffer_size
¶ Size of the log record deque for each task, the bigger, the more records it can show in case of error but the more memory it will use
Type: int
-
task_tree_depth
¶ number of the nested level to show start/end task logs for, if -1 will show always
Type: int
-
_tasks_lock
¶ Lock for the tasks_by_thread dict
Type: ContextLock
-
_main_thread_lock
¶ Lock for the main_failed bool
Type: ContextLock
-
TASK_INDICATORS
= ['@', '#', '*', '-', '~']¶ List of chars to show as task prefix, to ease distinguishing them
-
close_children_tasks
(parent_task_name)[source]¶ Closes all the children tasks that were open
Parameters: parent_task_name (str) – Name of the parent task Returns: None
-
emit
(record)[source]¶ Handle the given record, this is the entry point from the python logging facility
- Params:
- record (logging.LogRecord): log record to handle
Returns: None
-
get_task_indicator
(task_level=None)[source]¶ Parameters: task_level (int or None) – task depth level to get the indicator for, if None, will use the current tasks depth Returns: char to prepend to the task logs to indicate it’s level Return type: str
-
get_tasks
(thread_name)[source]¶ Parameters: thread_name (str) – name of the thread to get the tasks for Returns: - list of task names and log records for
- each for the given thread
Return type: OrderedDict of str, Task
-
handle_closed_task
(task_name, record)[source]¶ Do everything needed when a task is closed
- Params:
- task_name (str): name of the task that is finishing record (logging.LogRecord): log record with all the info
Returns: None
-
handle_new_task
(task_name, record)[source]¶ Do everything needed when a task is starting
- Params:
- task_name (str): name of the task that is starting record (logging.LogRecord): log record with all the info
Returns: None
-
mark_main_tasks_as_failed
()[source]¶ Flags to the main thread that all it’s tasks sholud fail
Returns: None
-
mark_parent_tasks_as_failed
(task_name, flush_logs=False)[source]¶ Marks all the parent tasks as failed
Parameters: Returns: None
-
pretty_emit
(record, is_header=False, task_level=None)[source]¶ Wrapper around the
logging.StreamHandler
emit method to add some decoration stuff to the messageParameters: - record (logging.LogRecord) – log record to emit
- is_header (bool) – if this record is a header, usually, a start or end task message
- task_level (int) – If passed, will take that as the current nested task level instead of calculating it from the current tasks
Returns: None
-
should_show_by_depth
(cur_level=None)[source]¶ Parameters: cur_level (int) – depth level to take into account Returns: - True if the given depth level should show messages (not
- taking into account the log level)
Return type: bool
-
should_show_by_level
(record, base_level=None)[source]¶ Parameters: - record_level (int) – log level of the record to check
- base_level (int or None) – log level to check against, will use the
object’s
dump_level
if None is passed
Returns: - True if the given log record should be shown according to the
log level
Return type:
-
-
lago.log_utils.
end_log_task
(task, logger=<module 'logging' from '/home/docs/.pyenv/versions/2.7.16/lib/python2.7/logging/__init__.pyc'>, level='info')[source]¶ Ends a log task
Parameters: - task (str) – name of the log task to end
- logger (logging.Logger) – logger to use
- level (str) – log level to use
Returns: None
-
lago.log_utils.
hide_stevedore_logs
()[source]¶ Hides the logs of stevedore, this function was added in order to support older versions of stevedore
We are using the NullHandler in order to get rid from ‘No handlers could be found for logger…’ msg
Returns: None
-
lago.log_utils.
log_always
(message)[source]¶ Wraps the given message with a tag that will make it be always logged by the task logger
Parameters: message (str) – message to wrap with the tag Returns: - tagged message that will get it shown immediately by the task
- logger
Return type: str
-
lago.log_utils.
log_task
(task, logger=<module 'logging' from '/home/docs/.pyenv/versions/2.7.16/lib/python2.7/logging/__init__.pyc'>, level='info', propagate_fail=True, uuid=None)[source]¶ Parameterized decorator to wrap a function in a log task
Example
>>> @log_task('mytask') ... def do_something(): ... pass
-
lago.log_utils.
setup_prefix_logging
(logdir)[source]¶ Sets up a file logger that will create a log in the given logdir (usually a lago prefix)
Parameters: logdir (str) – path to create the log into, will be created if it does not exist Returns: None
-
lago.log_utils.
start_log_task
(task, logger=<module 'logging' from '/home/docs/.pyenv/versions/2.7.16/lib/python2.7/logging/__init__.pyc'>, level='info')[source]¶ Starts a log task
Parameters: - task (str) – name of the log task to start
- logger (logging.Logger) – logger to use
- level (str) – log level to use
Returns: None
lago.paths module¶
lago.prefix module¶
-
exception
lago.prefix.
LagoDeployError
[source]¶ Bases:
lago.utils.LagoException
-
class
lago.prefix.
Prefix
(prefix)[source]¶ Bases:
object
A prefix is a directory that will contain all the data needed to setup the environment.
-
_paths
¶ Paths handler class
Type: lago.path.Paths
-
_virt_env
¶ Lazily loaded virtual env handler
Type: lago.virt.VirtEnv
-
VIRT_ENV_CLASS
¶ alias of
lago.virt.VirtEnv
-
_add_dns_records
(conf, mgmts)[source]¶ Add DNS records dict(‘dns_records’) to
conf
for each management network. Add DNS forwarder IP(‘dns_forward’) for each none management network.Parameters: - conf (spec) – spec
- mgmts (list) – management networks names
Returns: None
-
_add_mgmt_to_domains
(conf, mgmts)[source]¶ Add management network key(‘mgmt_net’) to each domain. Note this assumes
conf
was validated.Parameters:
-
_add_nic_to_mapping
(net, dom, nic)[source]¶ Populates the given net spec mapping entry with the nics of the given domain, by the following rules:
- If
net
is management, ‘domain_name’: nic_ip - For each interface: ‘domain_name-eth#’: nic_ip, where # is the
index of the nic in the domain definition. * For each interface: ‘domain_name-net_name-#’: nic_ip, where # is a running number of interfaces from that network. * For each interface: ‘domain_name-net_name’, which has an identical IP to ‘domain_name-net_name-0’
Parameters: Returns: None
- If
-
_allocate_ips_to_nics
(conf)[source]¶ For all the nics of all the domains in the conf that have dynamic ip, allocate one and addit to the network mapping
Parameters: conf (dict) – Configuration spec to extract the domains from Returns: None
-
_allocate_subnets
(conf)[source]¶ Allocate all the subnets needed by the given configuration spec
Parameters: conf (dict) – Configuration spec where to get the nets definitions from Returns: allocated subnets and modified conf Return type: tuple(list, dict)
-
_config_net_topology
(conf)[source]¶ Initialize and populate all the network related elements, like reserving ips and populating network specs of the given confiiguration spec
Parameters: conf (dict) – Configuration spec to initalize Returns: None
-
_copy_delpoy_scripts
(scripts)[source]¶ Copy the given deploy scripts to the scripts dir in the prefix
Parameters: scripts (list of str) – list of paths of the scripts to copy to the prefix Returns: - list with the paths to the copied scripts, with a
- prefixed with $LAGO_PREFIX_PATH so the full path is not hardcoded
Return type: list of str
-
_copy_deploy_scripts_for_hosts
(domains)[source]¶ Copy the deploy scripts for all the domains into the prefix scripts dir
Parameters: domains (dict) – spec with the domains info as when loaded from the initfile Returns: None
-
_create_disk
(name, spec, template_repo=None, template_store=None)[source]¶ Creates a disc with the given name from the given repo or store
Parameters: - name (str) – Name of the domain to create the disk for
- spec (dict) – Specification of the disk to create
- template_repo (TemplateRepository or None) – template repo instance to use
- template_store (TemplateStore or None) – template store instance to use
Returns: Path to the disk and disk metadata
Return type: Raises: RuntimeError
– If the type of the disk is not supported or failed to create the disk
-
_create_ssh_keys
()[source]¶ Generate a pair of ssh keys for this prefix
Returns: None Raises: RuntimeError
– if it fails to create the keys
-
_create_virt_env
()[source]¶ Create a new virt env from this prefix
Returns: virt env created from this prefix Return type: lago.virt.VirtEnv
-
_get_scripts
(host_metadata)[source]¶ Temporary method to retrieve the host scripts
Parameters: host_metadata (dict) – host metadata to retrieve the scripts for Returns: deploy scripts for the host, empty if none found Return type: list
-
static
_init_net_specs
(conf)[source]¶ Given a configuration specification, initializes all the net definitions in it so they can be used comfortably
Parameters: conf (dict) – Configuration specification Returns: the adapted new conf Return type: dict
-
_ova_to_spec
(filename)[source]¶ Retrieve the given ova and makes a template of it. Creates a disk from network provided ova. Calculates the needed memory from the ovf. The disk will be cached in the template repo
Parameters: filename (str) – the url to retrive the data from Returns: list with the disk specification int: VM memory, None if none defined int: Number of virtual cpus, None if none defined
Return type: list of dict
Raises: RuntimeError
– If the ova format is not supportedTypeError
– If the memory units in the ova are noot supported (currently only ‘MegaBytes’)
-
_register_preallocated_ips
(conf)[source]¶ Parse all the domains in the given conf and preallocate all their ips into the networks mappings, raising exception on duplicated ips or ips out of the allowed ranges
See also
Parameters: conf (dict) – Configuration spec to parse Returns: None Raises: RuntimeError
– if there are any duplicated ips or any ip out of the allowed range
-
_select_mgmt_networks
(conf)[source]¶ Select management networks. If no management network is found, it will mark the first network found by sorted the network lists. Also adding default DNS domain, if none is set.
Parameters: conf (spec) – spec
-
_set_mtu_to_nics
(conf)[source]¶ For all the nics of all the domains in the conf that have MTU set, save the MTU on the NIC definition.
Parameters: conf (dict) – Configuration spec to extract the domains from Returns: None
-
_set_scripts
(host_metadata, scripts)[source]¶ Temporary method to set the host scripts
Parameters: host_metadata (dict) – host metadata to set scripts in Returns: the updated metadata Return type: dict
-
_use_prototype
(spec, prototypes)[source]¶ Populates the given spec with the values of it’s declared prototype
Parameters: Returns: updated spec
Return type:
-
_validate_netconfig
(conf)[source]¶ Validate network configuration
Parameters: conf (dict) – spec
Returns: None
Raises: LagoInitException
– If a VM has more than- one management network configured, or a network which is not
- management has DNS attributes, or a VM is configured with a
- none-existence NIC, or a VM has no management network.
-
cleanup
(**kwargs)[source]¶ Stops any running entities in the prefix and uninitializes it, usually you want to do this if you are going to remove the prefix afterwards
Returns: None
-
create_snapshots
(name)[source]¶ Creates one snapshot on all the domains with the given name
Parameters: name (str) – Name of the snapshots to create Returns: None
-
export_vms
(**kwargs)[source]¶ Export vm images disks and init file. The exported images and init file can be used to recreate the environment.
Parameters: - vms_names (list of str) – Names of the vms to export, if None export all the vms in the env (default=None)
- standalone (bool) – If false, export a layered image (default=False)
- export_dir (str) – Dir to place the exported images and init file
- compress (bool) – If True compress the images with xz (default=False)
- init_file_name (str) – The name of the exported init file (default=’LagoInitfile’)
- out_format (
lago.plugins.output.OutFormatPlugin
) – The type of the exported init file (the default is yaml) - collect_only (bool) – If True, return only a mapping from vm name to the disks that will be exported. (default=False)
- with_threads (bool) – If True, run the export in parallel (default=True)
- Returns
- Unless collect_only == True, a mapping between vms’ disks.
-
fetch_url
(url)[source]¶ Retrieves the given url to the prefix
Parameters: url (str) – Url to retrieve Returns: path to the downloaded file Return type: str
-
get_nets
(**kwargs)[source]¶ Retrieve info on all the nets from all the domains
Returns: dictionary with net_name -> net list Return type: dict of str->list(str)
-
get_paths
(**kwargs)[source]¶ Get the Paths object of this prefix. The Paths object contains the paths to the internal directories and files of this prefix.
Returns: The Paths object of this prefix Return type: lago.paths.Paths
-
get_snapshots
()[source]¶ Retrieve info on all the snapshots from all the domains
Returns: list(str): dictionary with vm_name -> snapshot list Return type: dict of str
-
get_vms
(**kwargs)[source]¶ Retrieve info on all the vms
Returns: dictionary with vm_name -> vm list Return type: dict of str->list(str)
-
initialize
(**kwargs)[source]¶ Initialize this prefix, this includes creating the destination path, and creating the uuid for the prefix, for any other actions see
Prefix.virt_conf()
Will safely roll back if any of those steps fail
Returns: None Raises: RuntimeError
– If it fails to create the prefix dir
-
classmethod
is_prefix
(path)[source]¶ Check if a path is a valid prefix
Parameters: path (str) – path to be checked Returns: True if the given path is a prefix Return type: bool
-
paths
¶
-
resolve_parent
(disk_path, template_store, template_repo)[source]¶ Given a virtual disk, checks if it has a backing file, if so check if the backing file is in the store, if not download it from the provided template_repo.
After verifying that the backing-file is in the store, create a symlink to that file and locate it near the layered image.
Parameters: - disk_path (str) – path to the layered disk
- template_repo (TemplateRepository or None) – template repo instance to use
- template_store (TemplateStore or None) – template store instance to use
-
classmethod
resolve_prefix_path
(start_path=None)[source]¶ Look for an existing prefix in the given path, in a path/.lago dir, or in a .lago dir under any of it’s parent directories
Parameters: start_path (str) – path to start the search from, if None passed, it will use the current dir Returns: path to the found prefix Return type: str Raises: RuntimeError
– if no prefix was found
-
revert_snapshots
(name)[source]¶ Revert all the snapshots with the given name from all the domains
Parameters: name (str) – Name of the snapshots to revert Returns: None
-
shutdown
(**kwargs)[source]¶ Shutdown this prefix
Parameters: - vm_names (list of str) – List of the vms to shutdown
- reboot (bool) – If true, reboot the requested vms
Returns: None
-
start
(**kwargs)[source]¶ Start this prefix
Parameters: vm_names (list of str) – List of the vms to start Returns: None
-
stop
(**kwargs)[source]¶ Stop this prefix
Parameters: vm_names (list of str) – List of the vms to stop Returns: None
-
virt_conf
(conf, template_repo=None, template_store=None, do_bootstrap=True, do_build=True)[source]¶ Initializes all the virt infrastructure of the prefix, creating the domains disks, doing any network leases and creating all the virt related files and dirs inside this prefix.
Parameters: - conf (dict) – Configuration spec
- template_repo (TemplateRepository) – template repository intance
- template_store (TemplateStore) – template store instance
- do_bootstrap (bool) – If true run virt-sysprep on the images
- do_build (bool) – If true run build commands on the images, see lago.build.py for more info.
Returns: None
-
virt_conf_from_stream
(conf_fd, template_repo=None, template_store=None, do_bootstrap=True, do_build=True)[source]¶ Initializes all the virt infrastructure of the prefix, creating the domains disks, doing any network leases and creating all the virt related files and dirs inside this prefix.
Parameters: - conf_fd (File) – File like object to read the config from
- template_repo (TemplateRepository) – template repository intance
- template_store (TemplateStore) – template store instance
Returns: None
-
virt_env
¶ Getter for this instance’s virt env, creates it if needed
Returns: virt env instance used by this prefix Return type: lago.virt.VirtEnv
-
-
lago.prefix.
_create_ip
(subnet, index)[source]¶ Given a subnet or an ip and an index returns the ip with that lower index from the subnet (255.255.255.0 mask only subnets)
Parameters: Returns: The dotted decimal representation of the ip
Return type:
-
lago.prefix.
_ip_in_subnet
(subnet, ip)[source]¶ Checks if an ip is included in a subnet.
Note
only 255.255.255.0 masks allowed
Parameters: Returns: True
if ip is in subnet,False
otherwiseReturn type:
-
lago.prefix.
log_task
(task, level='info', propagate_fail=True, uuid=None)¶ Parameterized decorator to wrap a function in a log task
Example
>>> @log_task('mytask') ... def do_something(): ... pass
lago.sdk module¶
-
class
lago.sdk.
SDK
(workdir, prefix)[source]¶ Bases:
object
The SDK can be initialized in 3 ways:
- (Preferred) - by calling
sdk.init()
. - By loading an existing workdir from the disk, with
load_env()
.
- By passing already created workdir and prefix objects.
-
ansible_inventory
(keys=['vm-type', 'groups', 'vm-provider'])[source]¶ Get an Ansible inventory as a string,
keys
should be list on which to group the hosts by. You can use any key defined in LagoInitFile.Examples of possible keys:
keys=[‘disks/0/metadata/arch’], would group the hosts by the architecture.
- keys=[‘/disks/0/metadata/distro’, ‘disks/0/metadata/arch’],
- would create groups by architecture and also by distro.
- keys=[‘groups’] - would group hosts by the groups defined for
each VM in the LagoInitFile, i.e.:
domains:
- vm-01:
- … groups: web-server ..
- vm-02:
- groups: db-server
Parameters: keys (list of str) – Path to the keys that will be used to create groups. Returns: INI-like Ansible inventory Return type: str
-
ansible_inventory_temp_file
(keys=['vm-type', 'groups', 'vm-provider'])[source]¶ Context manager which returns Ansible inventory written on a tempfile. This is the same as
ansible_inventory()
, only the inventory file is written to a tempfile.Parameters: keys (list of str) – Path to the keys that will be used to create groups. Yields: tempfile.NamedTemporaryFile – Temp file containing the inventory
- (Preferred) - by calling
-
lago.sdk.
add_stream_logger
(level=10, name=None)[source]¶ Add a stream logger. This can be used for printing all SDK calls to stdout while working in an interactive session. Note this is a logger for the entire module, which will apply to all environments started in the same session. If you need a specific logger pass a
logfile
toinit()
Parameters: Returns: None
-
lago.sdk.
init
(config, workdir=None, logfile=None, loglevel=20, **kwargs)[source]¶ Initialize the Lago environment
Parameters: Returns: Initialized Lago enviornment
Return type: Raises: LagoException
– If initialization failed
-
lago.sdk.
load_env
(workdir, logfile=None, loglevel=20)[source]¶ Load an existing Lago environment
Parameters: workdir (str) – Path to the workdir directory, as created by :param
init()
or created by the CLI.: :param logfile: A Path to setup a log file. :type logfile: str :param loglevel:logging
log level. :type loglevel: intReturns: Initialized Lago environment Return type: SDK
Raises: LagoException
– If loading the environment failed.
lago.sdk_utils module¶
-
class
lago.sdk_utils.
SDKMethod
(name)[source]¶ Bases:
object
Metadata to store inside the decorated function
-
class
lago.sdk_utils.
SDKWrapper
(*args, **kwargs)[source]¶ Bases:
sphinx.ext.autodoc.importer._MockObject
A proxy object that exposes only methods which were decorated with
expose()
decorator.
-
lago.sdk_utils.
expose
(func)[source]¶ Decorator to be used with
SDKWrapper
. This decorator indicates that the wrapped method or class should be exposed in the proxied object.Parameters: func (types.FunctionType/types.MethodType) – function to decorate Returns: None
lago.service module¶
-
class
lago.service.
SysVInitService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/sbin/service'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
_request_start
()[source]¶ Low level implementation of the service start request, used by the func:start method
Returns: True if the service succeeded to start, False otherwise Return type: bool
-
_request_stop
()[source]¶ Low level implementation of the service stop request, used by the func:stop method
Returns: True if the service succeeded to stop, False otherwise Return type: bool
-
state
()[source]¶ Check the current status of the service
Returns: Which state the service is at right now Return type: ServiceState
-
-
class
lago.service.
SystemdContainerService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/usr/bin/docker'¶
-
HOST_BIN_PATH
= '/usr/bin/systemctl'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
_request_start
()[source]¶ Low level implementation of the service start request, used by the func:start method
Returns: True if the service succeeded to start, False otherwise Return type: bool
-
_request_stop
()[source]¶ Low level implementation of the service stop request, used by the func:stop method
Returns: True if the service succeeded to stop, False otherwise Return type: bool
-
state
()[source]¶ Check the current status of the service
Returns: Which state the service is at right now Return type: ServiceState
-
-
class
lago.service.
SystemdService
(vm, name)[source]¶ Bases:
lago.plugins.service.ServicePlugin
-
BIN_PATH
= '/usr/bin/systemctl'¶
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
_request_start
()[source]¶ Low level implementation of the service start request, used by the func:start method
Returns: True if the service succeeded to start, False otherwise Return type: bool
-
_request_stop
()[source]¶ Low level implementation of the service stop request, used by the func:stop method
Returns: True if the service succeeded to stop, False otherwise Return type: bool
-
state
()[source]¶ Check the current status of the service
Returns: Which state the service is at right now Return type: ServiceState
-
lago.ssh module¶
-
exception
lago.ssh.
LagoSSHTimeoutException
[source]¶ Bases:
lago.utils.LagoException
-
lago.ssh.
drain_ssh_channel
(chan, stdin=None, stdout=<open file '<stdout>', mode 'w'>, stderr=<open file '<stderr>', mode 'w'>)[source]¶
-
lago.ssh.
get_ssh_client
(ip_addr, ssh_key=None, host_name=None, ssh_tries=None, propagate_fail=True, username='root', password='123456')[source]¶ Get a connected SSH client
Parameters: - ip_addr (str) – IP address of the endpoint
- ssh_key (str or list of str) – Path to a file which contains the private key
- hotname (str) – The hostname of the endpoint
- ssh_tries (int) – The number of attempts to connect to the endpoint
- propagate_fail (bool) – If set to true, this event will be in the log and fail the outer stage. Otherwise, it will be discarded.
- username (str) – The username to authenticate with
- password (str) – Used for password authentication or for private key decryption
Raises: LagoSSHTimeoutException
– If the client failed to connect after “ssh_tries”
-
lago.ssh.
interactive_ssh
(ip_addr, command=None, host_name=None, ssh_key=None, username='root', password='123456')[source]¶
-
lago.ssh.
interactive_ssh_channel
(chan, command=None, stdin=<open file '<stdin>', mode 'r'>)[source]¶
-
lago.ssh.
log_task
(task, level='info', propagate_fail=True, uuid=None)¶ Parameterized decorator to wrap a function in a log task
Example
>>> @log_task('mytask') ... def do_something(): ... pass
-
lago.ssh.
ssh
(ip_addr, command, host_name=None, data=None, show_output=True, propagate_fail=True, tries=None, ssh_key=None, username='root', password='123456')[source]¶
lago.subnet_lease module¶
-
exception
lago.subnet_lease.
LagoSubnetLeaseException
(msg, prv_msg=None)[source]¶ Bases:
lago.utils.LagoException
-
exception
lago.subnet_lease.
LagoSubnetLeaseOutOfRangeException
(required_subnet, store_range)[source]¶
-
class
lago.subnet_lease.
Lease
(store_path, subnet)[source]¶ Bases:
object
Lease object is an abstraction of a lease file.
-
exist
¶
-
has_env
¶
-
metadata
¶
-
path
¶
-
subnet
¶
-
uuid
¶
-
uuid_path
¶
-
valid
¶
-
-
class
lago.subnet_lease.
SubnetStore
(path=None, min_third_octet=200, max_third_octet=255)[source]¶ Bases:
object
SubnetStore object represents a store of subnets used by lago for network bridges.
Note
Currently only /24 ranges are handled, and all of them under the 192.168._min_third_octet to 192.168._max_third_octet ranges.
The leases are stored under the store’s directory (which is specified with the path argument) as json files with the form:
[ "/path/to/prefix/uuid/file", "uuid_hash", ]
Where the uuid_hash is the 32 char uuid of the prefix (the contents of the uuid file at the time of doing the lease).
The helper class
Lease
is used to abstract the interaction with the lease files in the store (each file will be represented with a Lease object).Cleanup of stale leases is done in a lazy manner during a request for a lease. The store will remove at most 1 stale lease in each request (see SubnetStore._lease_valid for more info).
-
_path
¶ Path to the store, if not specified defaults to the value of lease_dir in the config
Type: str
-
_min_subnet
¶ The lowest subnet in the range of the store.
Type: netaddr.IPNetwork
-
_max_subnet
¶ The highest subnet in the range of the store.
Type: netaddr.IPNetwork
-
_acquire
(uuid_path)[source]¶ Lease a free network for the given uuid path
Parameters: uuid_path (str) – Path to the uuid file of a lago.Prefix
Returns: Which represents the selected subnet Return type: netaddr.IPNetwork Raises: LagoSubnetLeaseException
– If the store is full
-
_acquire_given_subnet
(uuid_path, subnet)[source]¶ Try to create a lease for subnet
Parameters: Returns: Which represents the selected subnet
Return type: netaddr.IPNetwork
Raises: LagoSubnetLeaseException
– If the requested subnet is not in the range of this store or its already been taken
-
_lease_owned
(lease, current_uuid_path)[source]¶ Checks if the given lease is owned by the prefix whose uuid is in the given path
Note
The prefix must be also in the same path it was when it took the lease
Parameters: Returns: True
if the given lease in owned by the prefix,False
otherwise
Return type:
-
_lease_valid
(lease)[source]¶ Check if the given lease exist and still has a prefix that owns it. If the lease exist but its prefix isn’t, remove the lease from this store.
Parameters: lease (lago.subnet_lease.Lease) – Object representation of the lease Returns: - If the lease and its prefix exists, return the path
- to the uuid of the prefix, else return None.
Return type: str or None
-
_release
(lease)[source]¶ Free the given lease
Parameters: lease (lago.subnet_lease.Lease) – The lease to free
-
_take_lease
(lease, uuid_path, safe=True)[source]¶ Persist the given lease to the store and make the prefix in uuid_path his owner
Parameters: - lease (lago.subnet_lease.Lease) – Object representation of the lease
- uuid_path (str) – Path to the prefix uuid
- safe (bool) – If true (the default), validate the the lease isn’t taken.
Raises: LagoSubnetLeaseException
– If safe == True and the lease is already taken.
-
_validate_lease_dir
()[source]¶ - Validate that the directory used by this store exist,
- otherwise create it.
-
acquire
(uuid_path, subnet=None)[source]¶ Lease a free subnet for the given uuid path. If subnet is given, try to lease that subnet, otherwise try to lease a free subnet.
Parameters: Returns: An object which represents the subnet.
Return type: netaddr.IPAddress
Raises: LagoSubnetLeaseException
– 1. If this store is full 2. If the requested subnet is already taken.LagoSubnetLeaseLockException
– If the lock to self.path can’t be acquired.
-
create_lease_object_from_idx
(idx)[source]¶ Create a lease from self._subnet_template and put idx as its third octet.
Parameters: idx (str) – The value of the third octet
Returns: Lease object which represents the requested subnet.
Return type: Raises: LagoSubnetLeaseOutOfRangeException
– If the resultant subnet is- malformed or out of the range of the store.
-
create_lease_object_from_subnet
(subnet)[source]¶ Create a lease from ip in a dotted decimal format, (for example 192.168.200.0/24). the _cidr will be added if not exist in subnet.
Parameters: subnet (str) – The value of the third octet
Returns: Lease object which represents the requested subnet.
Return type: Raises: LagoSubnetLeaseOutOfRangeException
– If the resultant subnet is- malformed or out of the range of the store.
-
get_allowed_range
()[source]¶ Returns: - The range of the store (with lowest and highest subnets as
- the bounds).
Return type: str
-
is_leasable_subnet
(subnet)[source]¶ Checks if a given subnet is inside the defined provision-able range
Parameters: subnet (str) – Ip in dotted decimal format with _cidr notation (for example 192.168.200.0/24) Returns: - True if subnet can be parsed into IPNetwork object and is
- inside the range, False otherwise
Return type: bool Raises: netaddr.AddrFormatError
– If subnet can not be parsed into an ip.
-
list_leases
(uuid=None)[source]¶ List current subnet leases
Parameters: uuid (str) – Filter the leases by uuid Returns: class:~Lease: current leases Return type: list of
-
path
¶
-
release
(subnets)[source]¶ Free the lease of the given subnets
Parameters: subnets (list of str or netaddr.IPAddress) – dotted ipv4 subnet in CIDR notation (for example
`192.168.200.0/24`
) or IPAddress object.Raises: LagoSubnetLeaseException
– If subnet is a str and can’t be parsedLagoSubnetLeaseLockException
– If the lock to self.path can’t be acquired.
-
lago.sysprep module¶
-
lago.sysprep.
sysprep
(disk, distro, loader=None, backend='direct', **kwargs)[source]¶ Run virt-sysprep on the
disk
, commands are built from the distro specific template and arguments passed inkwargs
. If no template is available it will default tosysprep-base.j2
.Parameters: Returns: None
Raises: RuntimeError
– On virt-sysprep none 0 exit code.
lago.templates module¶
-
class
lago.templates.
FileSystemTemplateProvider
(root)[source]¶ Handles file type templates, that is, getting a disk template from the host’s filesystem
-
_prefixed
(*path)[source]¶ Join all the given paths prefixed with this provider’s base root path
Parameters: *path (str) – sections of the path to join, passed as positional arguments Returns: Joined paths prepended with the provider root path Return type: str
-
download_image
(handle, dest)[source]¶ Copies over the handl to the destination
Parameters: Returns: None
-
-
class
lago.templates.
HttpTemplateProvider
(baseurl)[source]¶ This provider allows the usage of http urls for templates
-
download_image
(handle, dest)[source]¶ Downloads the image from the http server
Parameters: Returns: None
-
get_hash
(handle)[source]¶ Get the associated hash for the given handle, the hash file must exist (
handle + '.hash'
).Parameters: handle (str) – Path to the template to get the hash from Returns: Hash for the given handle Return type: str
-
get_metadata
(handle)[source]¶ Returns the associated metadata info for the given handle, the metadata file must exist (
handle + '.metadata'
). If the given handle has an.xz
extension, it will get removed when calculating the handle metadata pathParameters: handle (str) – Path to the template to get the metadata from Returns: Metadata for the given handle Return type: dict
-
open_url
(url, suffix='', dest=None)[source]¶ Opens the given url, trying the compressed version first. The compressed version url is generated adding the
.xz
extension to theurl
and adding the given suffix after that.xz
extension. If dest passed, it will download the data to that path if ableParameters: Returns: - response object to read from (lazy read), closed
if no dest passed
Return type: urllib.addinfourl
Raises: RuntimeError
– if the url gave http error when retrieving it
-
-
exception
lago.templates.
LagoMissingTemplateError
(name, path)[source]¶ Bases:
lago.utils.LagoException
-
class
lago.templates.
Template
(name, versions)[source]¶ Disk image template class
-
_versions (dict(str
TemplateVersion)): versions for this template
-
get_latest_version
()[source]¶ Retrieves the latest version for this template, the latest being the one with the newest timestamp
Returns: TemplateVersion
-
get_version
(ver_name=None)[source]¶ Get the given version for this template, or the latest
Parameters: ver_name (str or None) – Version to retieve, None for the latest Returns: - The version matching the given name or the latest
- one
Return type: TemplateVersion
-
-
class
lago.templates.
TemplateRepository
(dom, path)[source]¶ A template repository is a single source for templates, that uses different providers to actually retrieve them. That means for example that the ‘ovirt’ template repository, could support the ‘http’ and a theoretical ‘gluster’ template providers.
-
_get_provider
(spec)[source]¶ Get the provider for the given template spec
Parameters: spec (dict) – Template spec Returns: A provider instance for that spec Return type: HttpTemplateProvider or FileSystemTemplateProvider
-
classmethod
from_url
(path)[source]¶ Instantiate a
TemplateRepository
instance from the data in a file or urlParameters: path (str) – Path or url to the json file to load Returns: A new instance Return type: TemplateRepository
-
get_by_name
(name)[source]¶ Retrieve a template by it’s name
Parameters: name (str) – Name of the template to retrieve Raises: LagoMissingTemplateError
– if no template is found
-
-
class
lago.templates.
TemplateStore
(path)[source]¶ Local cache to store templates
The store uses various files to keep track of the templates cached, access and versions. An example template store looks like:
$ tree /var/lib/lago/store/ /var/lib/lago/store/ ├── in_office_repo:centos6_engine:v2.tmp ├── in_office_repo:centos7_engine:v5.tmp ├── in_office_repo:fedora22_host:v2.tmp ├── phx_repo:centos6_engine:v2 ├── phx_repo:centos6_engine:v2.hash ├── phx_repo:centos6_engine:v2.metadata ├── phx_repo:centos6_engine:v2.users ├── phx_repo:centos7_engine:v4.tmp ├── phx_repo:centos7_host:v4.tmp └── phx_repo:storage-nfs:v1.tmp
There you can see the files:
- *.tmp
- Temporary file created while downloading the template from the repository (depends on the provider)
- ${repo_name}:${template_name}:${template_version}
- This file is the actual disk image template
- *.hash
- Cached hash for the template disk image
- *.metadata
- Metadata for this template image in json format, usually this includes the distro and root-password
-
__contains__
(temp_ver)[source]¶ Checks if a given version is in this store
Parameters: temp_ver (TemplateVersion) – Version to look for Returns: True
if the version is in this storeReturn type: bool
-
_prefixed
(*path)[source]¶ Join the given paths and prepend this stores path
Parameters: *path (str) – list of paths to join, as positional arguments Returns: all the paths joined and prepended with the store path Return type: str
-
download
(temp_ver, store_metadata=True)[source]¶ Retrieve the given template version
Parameters: - temp_ver (TemplateVersion) – template version to retrieve
- store_metadata (bool) – If set to
False
, will not refresh the local metadata with the retrieved one
Returns: None
-
get_path
(temp_ver)[source]¶ Get the path of the given version in this store
Parameters: TemplateVersion (temp_ver) – version to look for Returns: The path to the template version inside the store Return type: str Raises: RuntimeError
– if the template is not in the store
-
get_stored_hash
(temp_ver)[source]¶ Retrieves the hash for the given template version from the store
Parameters: temp_ver (TemplateVersion) – template version to retrieve the hash for Returns: hash of the given template version Return type: str
-
get_stored_metadata
(temp_ver)[source]¶ Retrieves the metadata for the given template version from the store
Parameters: temp_ver (TemplateVersion) – template version to retrieve the metadata for Returns: the metadata of the given template version Return type: dict
-
class
lago.templates.
TemplateVersion
(name, source, handle, timestamp)[source]¶ Each template can have multiple versions, each of those is actually a different disk template file representation, under the same base name.
-
download
(destination)[source]¶ Retrieves this template to the destination file
Parameters: destination (str) – file path to write this template to Returns: None
-
get_hash
()[source]¶ Returns the associated hash for this template version
Returns: Hash for this version Return type: str
-
-
lago.templates.
_PROVIDERS
= {'file': <class lago.templates.FileSystemTemplateProvider>, 'http': <class lago.templates.HttpTemplateProvider>}¶ Registry for template providers
-
lago.templates.
find_repo_by_name
(name, repo_dir=None)[source]¶ Searches the given repo name inside the repo_dir (will use the config value ‘template_repos’ if no repo dir passed), will rise an exception if not found
Parameters: Returns: path to the repo
Return type: Raises: RuntimeError
– if not found
lago.utils module¶
-
class
lago.utils.
CommandStatus
[source]¶ Bases:
lago.utils.CommandStatus
-
class
lago.utils.
Flock
(path, readonly=False, blocking=True)[source]¶ Bases:
object
A wrapper class around flock
-
exception
lago.utils.
LagoInitException
[source]¶ Bases:
lago.utils.LagoException
-
exception
lago.utils.
LagoUserException
[source]¶ Bases:
lago.utils.LagoException
-
class
lago.utils.
LockFile
(path, timeout=None, lock_cls=None, **kwargs)[source]¶ Bases:
object
Context manager that creates a file based lock, with optional timeout in the acquire operation.
This context manager should be used only from the main Thread.
Parameters: - path (str) – path to the dir to lock
- timeout (int) – timeout in seconds to wait while acquiring the lock
- lock_cls (callable) – A callable which returns a Lock object that implements the acquire and release methods. The default is Flock.
- **kwargs (dict) – Any other param to pass to the lock_cls instance.
-
__enter__
()[source]¶ Start the lock with timeout if needed in the acquire operation
Raises: TimerException
– if the timeout is reached before acquiring the lock
-
class
lago.utils.
RollbackContext
(*args)[source]¶ Bases:
object
A context manager for recording and playing rollback. The first exception will be remembered and re-raised after rollback
Sample usage: > with RollbackContext() as rollback: > step1() > rollback.prependDefer(lambda: undo step1) > def undoStep2(arg): pass > step2() > rollback.prependDefer(undoStep2, arg)
More examples see tests/utilsTests.py @ vdsm code
-
class
lago.utils.
TemporaryDirectory
(ignore_errors=True)[source]¶ Bases:
object
Context manager that creates a temporary directory and provides its path as a property.
Parameters: ignore_errors (bool) – ignore errors when trying to remove directory Raises: OSError
– anything that ‘shutil.rmtree’ might raise
-
exception
lago.utils.
TimerException
[source]¶ Bases:
exceptions.Exception
Exception to throw when a timeout is reached
-
lago.utils.
_CommandStatus
¶ alias of
lago.utils.CommandStatus
-
lago.utils.
_run_command
(command, input_data=None, stdin=None, out_pipe=-1, err_pipe=-1, env=None, uuid=None, **kwargs)[source]¶ Runs a command
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- input_data (str) – If passed, will feed that data to the subprocess through stdin
- out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
- stdin (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdin
- err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- uuid (uuid) – If set the command will be logged with the given uuid converted to string, otherwise, a uuid v4 will be generated.
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
-
lago.utils.
deepcopy
(original_obj)[source]¶ Creates a deep copy of an object with no crossed referenced lists or dicts, useful when loading from yaml as anchors generate those cross-referenced dicts and lists
Parameters: original_obj (object) – Object to deep copy Returns: deep copy of the object Return type: object
-
lago.utils.
filter_spec
(spec, paths, wildcard='*', separator='/')[source]¶ Remove keys from a spec file. For example, with the following path: domains//disks//metadata all the metadata dicts from all domains disks will be removed.
Parameters: Returns: None
Raises: utils.LagoUserException
– If a malformed path was detected
-
lago.utils.
get_hash
(file_path, checksum='sha1')[source]¶ Generate a hash for the given file
Parameters: Returns: hash for that file
Return type:
-
lago.utils.
get_qemu_info
(path, backing_chain=False, fail_on_error=True)[source]¶ Get info on a given qemu disk
Parameters: - path (str) – Path to the required disk
- backing_chain (boo) – if true, include also info about
- image predecessors. (the) –
Returns: if backing_chain == True then a list of dicts else a dict
Return type:
-
lago.utils.
load_virt_stream
(virt_fd)[source]¶ Loads the given conf stream into a dict, trying different formats if needed
Parameters: virt_fd (str) – file like objcect with the virt config to load Returns: Loaded virt config Return type: dict
-
lago.utils.
qemu_rebase
(target, backing_file, safe=True, fail_on_error=True)[source]¶ changes the backing file of ‘source’ to ‘backing_file’ If backing_file is specified as “” (the empty string), then the image is rebased onto no backing file (i.e. it will exist independently of any backing file). (Taken from qemu-img man page)
Parameters:
-
lago.utils.
run_command
(command, input_data=None, out_pipe=-1, err_pipe=-1, env=None, **kwargs)[source]¶ Runs a command non-interactively
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- input_data (str) – If passed, will feed that data to the subprocess through stdin
- out_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stdout
- err_pipe (int or file) – File descriptor as passed to :ref:subprocess.Popen to use as stderr
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
-
lago.utils.
run_command_with_validation
(cmd, fail_on_error=True, msg='An error has occurred')[source]¶
-
lago.utils.
run_interactive_command
(command, env=None, **kwargs)[source]¶ Runs a command interactively, reusing the current stdin, stdout and stderr
Parameters: - command (list of str) – args of the command to execute, including the command itself as command[0] as [‘ls’, ‘-l’]
- of str (env(dict) – str): If set, will use the given dict as env for the subprocess
- **kwargs – Any other keyword args passed will be passed to the :ref:subprocess.Popen call
Returns: result of the interactive execution
Return type:
lago.validation module¶
lago.virt module¶
-
exception
lago.virt.
LagoUnknownVMTypeError
(vm_type_name, vm_types)[source]¶ Bases:
lago.utils.LagoUserException
-
class
lago.virt.
VirtEnv
(prefix, vm_specs, net_specs)[source]¶ Bases:
object
Env properties: * prefix * vms * net
-
_get_stop_shutdown_common_args
(vm_names)[source]¶ Get the common arguments for stop and shutdown commands :param vm_names: The names of the requested vms :type vm_names: list of str
- Returns
- list of plugins.vm.VMProviderPlugin:
- vms objects that should be stopped
list of virt.Network: net objects that should be stopped str: log message
Raises: utils.LagoUserException
– If a vm name doesn’t exist
-
_get_unused_nets
(vms_to_stop)[source]¶ Return a list of nets that used only by the vms in vms_to_stop :param vms_to_stop: The names of the requested vms :type vms_to_stop: list of str
- Returns
- list of virt.Network: net objects that used only by
- vms in vms_to_stop
Raises: utils.LagoUserException
– If a vm name doesn’t exist
-
export_vms
(vms_names, standalone, dst_dir, compress, init_file_name, out_format, collect_only=False, with_threads=True)[source]¶
-
generate_init
(dst, out_format, vms_to_include, filters=None)[source]¶ Generate an init file which represents this env and can be used with the images created by self.export_vms :param dst: path and name of the new init file :type dst: str :param out_format: formatter for the output (the default is yaml) :type out_format: plugins.output.OutFormatPlugin :param filters: list of paths to keys that should be removed from
the init fileParameters: (list of (vms_to_include) – class:lago.plugins.vm.VMPlugin): list of vms to include in the init file Returns: None
-
get_compat
()[source]¶ Get compatibility level for this environment - which is the Lago version used to create this environment
-
get_env_spec
(filters=None)[source]¶ Get the spec of the current env. The spec will hold the info about all the domains and networks associated with this env. :param filters: list of paths to keys that should be removed from
the init fileReturns: the spec of the current env Return type: dict
-
get_snapshots
(domains=None)[source]¶ Get the list of snapshots for each domain :param domanins: list of the domains to get the snapshots :type domanins: list of str :param for, all will be returned if none or empty list passed:
Returns: with the domain names and the list of snapshots for each Return type: dict of str -> list(str)
-
get_vms
(vm_names=None)[source]¶ Returns the vm objects associated with vm_names if vm_names is None, return all the vms in the prefix :param vm_names: The names of the requested vms :type vm_names: list of str
- Returns
- dict: Which contains the requested vm objects indexed by name
Raises: utils.LagoUserException
– If a vm name doesn’t exist
-
prefixed_name
(unprefixed_name, max_length=0)[source]¶ Returns a uuid pefixed identifier :param unprefixed_name: Name to add a prefix to :type unprefixed_name: str :param max_length: maximum length of the resultant prefixed name,
will adapt the given name and the length of the uuid ot fit itReturns: prefixed identifier for the given unprefixed name Return type: str
-
-
lago.virt.
log_task
(task, level='info', propagate_fail=True, uuid=None)¶ Parameterized decorator to wrap a function in a log task
Example
>>> @log_task('mytask') ... def do_something(): ... pass
lago.vm module¶
-
class
lago.vm.
DefaultVM
(env, spec)[source]¶ Bases:
lago.plugins.vm.VMPlugin
-
_abc_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache
= <_weakrefset.WeakSet object>¶
-
_abc_negative_cache_version
= 43¶
-
_abc_registry
= <_weakrefset.WeakSet object>¶
-
-
class
lago.vm.
SSHVMProvider
(vm)[source]¶ Bases:
lago.plugins.vm.VMProviderPlugin
-
bootstrap
(*args, **kwargs)[source]¶ Does any actions needed to get the domain ready to be used, ran on prefix init. :returns: None
-
create_snapshot
(name, *args, **kwargs)[source]¶ Take any actions needed to create a snapshot :param name: Name for the snapshot, will be used as key to retrieve
it laterReturns: None
-
revert_snapshot
(name, *args, **kwargs)[source]¶ Take any actions needed to revert/restore a snapshot :param name: Name for the snapshot, same that was set on creation :type name: str
Returns: None
-
lago.workdir module¶
-
exception
lago.workdir.
LagoPrefixAlreadyExistsError
(prefix_name, workdir_path)[source]¶ Bases:
lago.utils.LagoException
-
exception
lago.workdir.
MalformedWorkdir
[source]¶ Bases:
lago.workdir.WorkdirError
-
exception
lago.workdir.
PrefixAlreadyExists
[source]¶ Bases:
lago.workdir.WorkdirError
-
exception
lago.workdir.
PrefixNotFound
[source]¶ Bases:
lago.workdir.WorkdirError
-
class
lago.workdir.
Workdir
(path, prefix_class=<class 'lago.prefix.Prefix'>)[source]¶ Bases:
object
This class reperesents a base workdir, where you can store multiple prefixes
- Properties:
- path(str): Path to the workdir perfixes(dict of str->self.prefix_class): dict with the prefixes in the workdir, by name current(str): Name of the current prefix prefix_class(type): Class to use when creating prefixes
-
_set_current
(new_current)[source]¶ Change the current default prefix, for internal usage
Parameters: new_current (str) – Name of the new current prefix, it must already exist Returns: None Raises: PrefixNotFound
– if the given prefix name does not exist in the workdir
-
add_prefix
(*args, **kwargs)[source]¶ Adds a new prefix to the workdir.
Parameters: - name (str) – Name of the new prefix to add
- *args – args to pass along to the prefix constructor
- *kwargs – kwargs to pass along to the prefix constructor
Returns: The newly created prefix
Raises: LagoPrefixAlreadyExistsError
– if prefix name already exists in theworkdir
-
cleanup
()[source]¶ Attempt to set a new current symlink if it is broken. If no other prefixes exist and the workdir is empty, try to delete the entire workdir.
Raises: MalformedWorkdir
– if no prefixes were found, but the workdir is not empty.
-
destroy
(*args, **kwargs)[source]¶ Destroy all the given prefixes and remove any left files if no more prefixes are left
Parameters: - prefix_names (list of str) – list of prefix names to destroy, if None
- passed (default) –
-
get_prefix
(*args, **kwargs)[source]¶ Retrieve a prefix, resolving the current one if needed
Parameters: name (str) – name of the prefix to retrieve, or current to get the current one Returns: instance of the prefix with the given name Return type: self.prefix_class
-
initialize
(prefix_name='default', *args, **kwargs)[source]¶ Initializes a workdir by adding a new prefix to the workdir.
Parameters: - prefix_name (str) – Name of the new prefix to add
- *args – args to pass along to the prefix constructor
- *kwargs – kwargs to pass along to the prefix constructor
Returns: The newly created prefix
Raises: PrefixAlreadyExists
– if the prefix name already exists in the workdir
-
static
is_possible_workdir
(path)[source]¶ A quick method to suggest if the path is a possible workdir. This does not guarantee that the workdir is not malformed, only that by simple heuristics it might be one. For a full check use
is_workdir()
.Parameters: path (str) – Path Returns: True if path
might be a work dir.Return type: bool
-
classmethod
is_workdir
(path)[source]¶ Check if the given path is a workdir
Parameters: path (str) – Path to check Returns: True if the given path is a workdir Return type: bool
-
join
(*args)[source]¶ Gets a joined path prefixed with the workdir path
Parameters: *args (str) – path sections to join Returns: Joined path prefixed with the workdir path Return type: str
-
load
()[source]¶ Loads the prefixes that are available is the workdir
Returns: None Raises: MalformedWorkdir
– if the wordir is malformed
-
classmethod
resolve_workdir_path
(start_path='.')[source]¶ Look for an existing workdir in the given path, in a path/.lago dir, or in a .lago dir under any of it’s parent directories
Parameters: start_path (str) – path to start the search from, if None passed, it will use the current dir Returns: path to the found prefix Return type: str Raises: LagoUserException
– if no prefix was found
-
set_current
(*args, **kwargs)[source]¶ Change the current default prefix
Parameters: new_current (str) – Name of the new current prefix, it must already exist Returns: None Raises: PrefixNotFound
– if the given prefix name does not exist in the workdir