toxicbuild.core package

Submodules

toxicbuild.core.client module

This module implements a base client for basic tcp communication, reading and writing json data.

Usage:

host = 'somehost.net'
port = 1234
async with BaseToxicClient(host, port):
    await client.write({'hello': 'world'})
    json_response = await client.get_response()

class toxicbuild.core.client.BaseToxicClient(host, port, loop=None, use_ssl=False, validate_cert=True, **ssl_kw)

Bases: toxicbuild.core.utils.LoggerMixin

Base client for communication with toxicbuild servers.

coroutine connect()

Connects to the server.

Note

This is called by the asynchronous context manager (aka async with)

disconnect()

Disconnects from the server

coroutine get_response()

Reads data from the server and raises and exception in case of error

coroutine read()

Reads data from the server. Expects a json.

coroutine write(data)

Writes data to the server.

Parameters:data – Data to be sent to the server. Will be converted to json and enconded using utf-8.

toxicbuild.core.cmd module

class toxicbuild.core.cmd.ToxicProgram(*args, **kwargs)

Bases: mando.core.Program

Transforms a python function into a command line program. Extends mando’s Program to execute _generate_command() only when execute() is called to enable toxicbuild command hacks.

execute(*args, **kwargs)

Parse the arguments and execute the resulting command.

Parameters:args – The arguments to parse.

toxicbuild.core.conf module

class toxicbuild.core.conf.Settings(envvar, default_filename)

Bases: object

Simple interface to a settings file.

toxicbuild.core.exceptions module

exception toxicbuild.core.exceptions.BadJsonData

Bases: Exception

exception toxicbuild.core.exceptions.ConfigError

Bases: Exception

exception toxicbuild.core.exceptions.ExecCmdError

Bases: Exception

exception toxicbuild.core.exceptions.ImpossibillityError

Bases: Exception

exception toxicbuild.core.exceptions.PluginNotFound

Bases: Exception

exception toxicbuild.core.exceptions.ToxicClientException

Bases: Exception

exception toxicbuild.core.exceptions.VCSError

Bases: Exception

toxicbuild.core.plugins module

class toxicbuild.core.plugins.Plugin(*args, **kwargs)

Bases: object

This is a base plugin. Plugins may implement aditional behavior to your builds.

classmethod get_plugin(name)

Returns a Plugin subclass based on its name.

Parameters:name – Plugin’s name.

classmethod list_plugins(plugin_type=None)

Returns a list of Plugin subclasses.

Parameters:plugin_type – the plugin’s type.

name = 'BaseCorePlugin'

no_list = False

class toxicbuild.core.plugins.PluginMeta

Bases: type

toxicbuild.core.protocol module

class toxicbuild.core.protocol.BaseToxicProtocol(loop, connection_lost_cb=None)

Bases: asyncio.streams.StreamReaderProtocol, toxicbuild.core.utils.LoggerMixin

Base protocol for toxicbulid servers

check_data()

Checks if the data is valid, it means, checks if has some data, checks if it is a valid json and checks if it has a action key

client_connected()

Coroutine that handles connections. You must implement this in your sub-classes. When this method is called, self.data, containing a dictionary with the data passed in the request and self.action, a string indicating which action to take are already available.

close_connection()

Closes the connection with the client

connection_lost(exc)

Called once, when the connection is lost.

Parameters:exc – The exception, if some.

connection_made(transport)

Called once, when the client connects.

Parameters:transport – transport for asyncio.StreamReader and asyncio.StreamWriter.

get_json_data()

Returns the json sent by the client.

get_raw_data()

Returns the raw data sent by the client

send_response(code, body)

Send a response to client formated by the (unknown) toxicbuild remote build specs.

Parameters:

• code – code for this message. code == 0 is success and code > 0 is error.
• body – response body. It has to be a serializable object.

encrypted_token = None

toxicbuild.core.requests module

This module implements a simple asynchronous interface for http requests.

Usage:

from toxicbuild.core import requests
response = yield from requests.get('http://google.com/')
print(response.text)

class toxicbuild.core.requests.Response(status, text)

Bases: object

Encapsulates a response from a http request

json()

Loads the json in the response text.

toxicbuild.core.requests.delete(url, **kwargs)

Performs a http DELETE request

Parameters:

• url – Request’s url.
• kwargs – Args passed to toxicbuild.core.requests._request().

toxicbuild.core.requests.get(url, **kwargs)

Performs a http GET request

Parameters:

• url – Request’s url.
• kwargs – Args passed to toxicbuild.core.requests._request().

toxicbuild.core.requests.post(url, **kwargs)

Performs a http POST request

Parameters:

• url – Request’s url.
• kwargs – Args passed to toxicbuild.core.requests._request().

toxicbuild.core.requests.put(url, **kwargs)

Performs a http PUT request

Parameters:

• url – Request’s url.
• kwargs – Args passed to toxicbuild.core.requests._request().

toxicbuild.core.utils module

class toxicbuild.core.utils.LoggerMixin

Bases: object

A simple mixin to use log on a class.

log(msg, level='info')

Appends the class name before the log message.

classmethod log_cls(msg, level='info')

class toxicbuild.core.utils.MatchKeysDict

Bases: dict

A dictionary that returns the values matching the keys using toxicbuild.core.utils.match_string().

>>> d = MatchKeysDict()
>>> d['k*'] = 1
>>> d['key']
1
>>> k['keyboard']
1

class toxicbuild.core.utils.SettingsPatcher

Bases: mongomotor.monkey.MonkeyPatcher

Patches the settings from pyrocumulus to use the same settings as toxibuild.

patch_pyro_settings(settings)

class toxicbuild.core.utils.SourceSuffixesPatcher

Bases: mongomotor.monkey.MonkeyPatcher

We must to path SOURCE_SUFFIXES in the python importlib so we can import source code files with extension other than .py, in this case, namely .conf

patch_source_suffixes()

Patches the SOURCE_SUFFIXES constant in the module importlib._bootstrap_external.

SOURCE_SUFFIXES = ['.py', '.conf']

class toxicbuild.core.utils.changedir(path)

Bases: object

toxicbuild.core.utils.bcrypt_string(src_string, salt=None)

toxicbuild.core.utils.compare_bcrypt_string(original, encrypted)

Compares if a un-encrypted string matches an encrypted one.

Parameters:

• original – An un-encrypted string.
• encrypted – An bcrypt encrypted string.

toxicbuild.core.utils.create_random_string(length)

toxicbuild.core.utils.daemonize(call, cargs, ckwargs, stdout, stderr, workdir, pidfile)

Run a callable as a daemon

Parameters:

• call – a callable.
• cargs – args to call.
• ckwargs – kwargs to call.
• stdout – daemon’s stdout.
• stderr – daemon’s stderr.
• workdir – daemon’s workdir
• pidfile – pidfile’s path

toxicbuild.core.utils.datetime2string(dt, dtformat='%w %m %d %H:%M:%S %Y %z')

Transforms a datetime object into a formated string.

Parameters:

• dt – The datetime object.
• dtformat – The format to use.

toxicbuild.core.utils.exec_cmd(cmd, cwd, timeout=3600, out_fn=None, **envvars)

Executes a shell command. Raises with the command output if return code > 0.

Parameters:

• cmd – command to run.
• cwd – Directory to execute the command.
• timeout – How long we should wait some output. Default is 3600.
• out_fn – A coroutine that receives each line of the step output. The coroutine signature must be in the form: mycoro(line_index, line).
• envvars – Environment variables to be used in the command.

toxicbuild.core.utils.format_timedelta(td)

Format a timedelta object to a human-friendly string.

Parameters:dt – A timedelta object.

toxicbuild.core.utils.get_toxicbuildconf(directory)

Returns the toxicbuild.conf module.

Parameters:directory – Directory to look for toxicbuild.conf

toxicbuild.core.utils.get_toxicbuildconf_yaml(directory, filename='toxicbuild.yml')

Returns the python objet representing the yaml build configuration.

Parameters:

• directory – The path of the directory containing the config file.
• filename – The actual name of the config file.

toxicbuild.core.utils.inherit_docs(cls)

Inherit docstrings from base classes’ methods. Can be used as a decorator

Parameters:cls – Class that will inherit docstrings from its parents.

toxicbuild.core.utils.list_builders_from_config(config, branch=None, slave=None, config_type='py')

Lists builders from a build config

Parameters:

• config – The build configuration.
• branch – The branch for which builders are being listed.
• slave – The slave for which builders are being listed.

toxicbuild.core.utils.load_module_from_file(filename)

Load a module from a source file :param filename: full path for file to be loaded.

toxicbuild.core.utils.localtime2utc(localdatetime)

Transforms a local datetime object into a datetime object in utc time.

Parameters:localdatetime – A datetime object.

toxicbuild.core.utils.log(msg, level='info')

toxicbuild.core.utils.match_string(smatch, filters)

Checks if a string match agains a list of filters containing wildcards.

Parameters:

• smatch – String to test against the filters
• filters – Filter to match a string.

toxicbuild.core.utils.now()

Returns the localtime with timezone info.

toxicbuild.core.utils.read_file(filename)

Reads the contents of a file asynchronously.

Parameters:filename – The path of the file.

toxicbuild.core.utils.read_stream(reader)

Reads the input stream. First reads the bytes until the first “n”. These first bytes are the length of the full message.

Parameters:reader – An instance of asyncio.StreamReader

toxicbuild.core.utils.run_in_thread(fn, *args, **kwargs)

Runs a callable in a background thread.

Parameters:

• fn – A callable to be executed in a thread.
• args – Positional arguments to fn
• kwargs – Named arguments to fn

Usage

r = yield from run_in_thread(call, 1, bla='a')

toxicbuild.core.utils.set_tzinfo(dt, tzoff)

Sets a timezone info to a datetime object.

Parameters:dt – A datetime object. Para tzoff:The timezone offset from utc in seconds

toxicbuild.core.utils.string2datetime(dtstr, dtformat='%w %m %d %H:%M:%S %Y %z')

Transforns a string into a datetime object acording to dtformat.

Parameters:

• dtstr – The string containing the formated date.
• dtformat – The format of the formated date.

toxicbuild.core.utils.utc2localtime(utcdatetime)

Transforms a utc datetime object into a datetime object in local time.

Parameters:utcdatetime – A datetime object

toxicbuild.core.utils.write_stream(writer, data)

Writes data to output. Encodes data to utf-8 and prepend the lenth of the data before sending it.

Parameters:

• writer – An instance of asyncio.StreamWriter
• data – String data to be sent.

toxicbuild.core.vcs module

class toxicbuild.core.vcs.Git(workdir)

Bases: toxicbuild.core.vcs.VCS

An interface to git version control system

add_remote(remote_url, remote_name)

Adds a new remote to the repository.

Parameters:

• remote_url – The url of the remote repository.
• remote_name – The name of the remote.

branch_exists(branch_name)

Checks if a local branch exists.

Parameters:branch_name – The name of the branch to check.

checkout(named_tree)

Checkout to named_tree :param named_tree: A commit, branch, tag…

clone(url)

Clones a repository into self.workdir :param url: repository url

create_local_branch(branch_name, base_name)

Creates a branch new in the local repository

Parameters:

• branch_name – The name for the new branch
• base_name – The name of the base branch.

delete_local_branch(branch_name)

Deletes a local branch.

Parameters:branch_name – The name of the branch to be deleted.

fetch()

Fetch changes from remote repository

get_remote(remote_name='origin')

Returns the remote url used in the repo.

Parameters:remote_name – The name of the remote url to change.

get_remote_branches()

Returns a list of the remote branches available.

get_revisions(since=None, branches=None)

Returns the revisions for branches since since.

Parameters:

• since – dictionary in the format: {branch_name: since_date}. since is a datetime object.
• branches – A list of branches to look for new revisions. If branches is None all remote branches will be used. You can use wildcards in branches to filter the remote branches.

get_revisions_for_branch(branch, since=None)

Returns the revisions for branch since since. If since is None, all revisions will be returned.

Parameters:

• branch – branch name
• since – datetime

has_changes()

Informs if there are new revisions in the repository

coroutine import_external_branch(external_url, external_name, external_branch, into)

Imports a branch from an external (not the origin one) repository into a local branch.

Parameters:

• external_url – The url of the external repository.
• external_name – Name to idenfity the remote url.
• external_branch – The name of the branch in the external repo.
• into – The name of the local branch.

pull(branch_name, remote_name='origin')

Pull changes from branch_name on remote repo.

Parameters:

• branch_name – A branch name, like ‘master’.
• remote_name – The remote repository to push from.

set_remote(url, remote_name='origin')

Sets the remote url of the repository.

Parameters:

• url – The new remote url.
• remote_name – The name of the remote url to change.

try_set_remote(url, remote_name='origin')

Sets the remote url if the remote is not equal as url.

Parameters:

• url – The new url for the remote.
• remote_name – The name of the remote url to change.

update_submodule()

date_format = '%a %b %d %H:%M:%S %Y'

vcsbin = 'git'

class toxicbuild.core.vcs.VCS(workdir)

Bases: toxicbuild.core.utils.LoggerMixin

Generic inteface to a vcs (clone, fetch, get revisions etc…).

add_remote(remote_url, remote_name)

Adds a new remote to the repository.

Parameters:

• remote_url – The url of the remote repository.
• remote_name – The name of the remote.

classmethod branch_exists(branch_name)

Checks if a local branch exists.

Parameters:branch_name – The name of the branch to check.

checkout(named_tree)

Checkout to named_tree :param named_tree: A commit, branch, tag…

clone(url)

Clones a repository into self.workdir :param url: repository url

create_local_branch(branch_name, base_name)

Creates a branch new in the local repository

Parameters:

• branch_name – The name for the new branch
• base_name – The name of the base branch.

delete_local_branch(branch_name)

Deletes a local branch.

Parameters:branch_name – The name of the branch to be deleted.

exec_cmd(cmd, cwd=None)

Executes a shell command. If cwd is None self.workdir will be used.

Parameters:cwd – Directory where the command will be executed.

fetch()

Fetch changes from remote repository

get_remote(remote_name)

Returns the remote url used in the repo.

Parameters:remote_name – The name of the remote url to change.

get_remote_branches()

Returns a list of the remote branches available.

get_revisions(since=None, branches=None)

Returns the revisions for branches since since.

Parameters:

• since – dictionary in the format: {branch_name: since_date}. since is a datetime object.
• branches – A list of branches to look for new revisions. If branches is None all remote branches will be used. You can use wildcards in branches to filter the remote branches.

get_revisions_for_branch(branch, since=None)

Returns the revisions for branch since since. If since is None, all revisions will be returned.

Parameters:

• branch – branch name
• since – datetime

has_changes()

Informs if there are new revisions in the repository

import_external_branch(external_url, external_name, external_branch, into)

Imports a branch from an external (not the origin one) repository into a local branch.

Parameters:

• external_url – The url of the external repository.
• external_name – Name to idenfity the remote url.
• external_branch – The name of the branch in the external repo.
• into – The name of the local branch.

pull(branch_name, remote_name='origin')

Pull changes from branch_name on remote repo.

Parameters:

• branch_name – A branch name, like ‘master’.
• remote_name – The remote repository to push from.

set_remote(url, remote_name)

Sets the remote url of the repository.

Parameters:

• url – The new remote url.
• remote_name – The name of the remote url to change.

try_set_remote(url, remote_name)

Sets the remote url if the remote is not equal as url.

Parameters:

• url – The new url for the remote.
• remote_name – The name of the remote url to change.

workdir_exists()

Informs if the workdir for this vcs exists

vcsbin = None

toxicbuild.core.vcs.get_vcs(vcs_type)

Retuns a subclass of toxicbuild.core.vcs.VCS for vcs_type

Module contents