toxicbuild.core package

Submodules

toxicbuild.core.client module

This module implements a base client for basic tcp communication, reading and writing json data.

Usage:

host = 'somehost.net'
port = 1234
async with BaseToxicClient(host, port):
    await client.write({'hello': 'world'})
    json_response = await client.get_response()
class toxicbuild.core.client.BaseToxicClient(host, port, loop=None, use_ssl=False, validate_cert=True, **ssl_kw)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Base client for communication with toxicbuild servers.

coroutine connect()[source]

Connects to the server.

Note

This is called by the asynchronous context manager (aka async with)

disconnect()[source]

Disconnects from the server

coroutine get_response()[source]

Reads data from the server and raises and exception in case of error

coroutine read()[source]

Reads data from the server. Expects a json.

coroutine write(data)[source]

Writes data to the server.

Parameters:data – Data to be sent to the server. Will be converted to json and enconded using utf-8.

toxicbuild.core.cmd module

class toxicbuild.core.cmd.ToxicProgram(*args, **kwargs)[source]

Bases: mando.core.Program

Transforms a python function into a command line program. Extends mando’s Program to execute _generate_command() only when execute() is called to enable toxicbuild command hacks.

execute(*args, **kwargs)[source]

Parse the arguments and execute the resulting command.

Parameters:args – The arguments to parse.

toxicbuild.core.conf module

class toxicbuild.core.conf.Settings(envvar, default_filename)[source]

Bases: object

Simple interface to a settings file.

toxicbuild.core.exceptions module

exception toxicbuild.core.exceptions.BadJsonData[source]

Bases: Exception

exception toxicbuild.core.exceptions.ConfigError[source]

Bases: Exception

exception toxicbuild.core.exceptions.ExecCmdError[source]

Bases: Exception

exception toxicbuild.core.exceptions.ImpossibillityError[source]

Bases: Exception

exception toxicbuild.core.exceptions.PluginNotFound[source]

Bases: Exception

exception toxicbuild.core.exceptions.ToxicClientException[source]

Bases: Exception

exception toxicbuild.core.exceptions.VCSError[source]

Bases: Exception

toxicbuild.core.plugins module

class toxicbuild.core.plugins.Plugin(*args, **kwargs)[source]

Bases: object

This is a base plugin. Plugins may implement aditional behavior to your builds.

classmethod get_plugin(name)[source]

Returns a Plugin subclass based on its name.

Parameters:name – Plugin’s name.
classmethod list_plugins(plugin_type=None)[source]

Returns a list of Plugin subclasses.

Parameters:plugin_type – the plugin’s type.
name = 'BaseCorePlugin'
no_list = False
class toxicbuild.core.plugins.PluginMeta[source]

Bases: type

toxicbuild.core.protocol module

class toxicbuild.core.protocol.BaseToxicProtocol(loop, connection_lost_cb=None)[source]

Bases: asyncio.streams.StreamReaderProtocol, toxicbuild.core.utils.LoggerMixin

Base protocol for toxicbulid servers

check_data()[source]

Checks if the data is valid, it means, checks if has some data, checks if it is a valid json and checks if it has a action key

client_connected()[source]

Coroutine that handles connections. You must implement this in your sub-classes. When this method is called, self.data, containing a dictionary with the data passed in the request and self.action, a string indicating which action to take are already available.

close_connection()[source]

Closes the connection with the client

connection_lost(exc)[source]

Called once, when the connection is lost.

Parameters:exc – The exception, if some.
connection_made(transport)[source]

Called once, when the client connects.

Parameters:transport – transport for asyncio.StreamReader and asyncio.StreamWriter.
get_json_data()[source]

Returns the json sent by the client.

get_raw_data()[source]

Returns the raw data sent by the client

send_response(code, body)[source]

Send a response to client formated by the (unknown) toxicbuild remote build specs.

Parameters:
  • code – code for this message. code == 0 is success and code > 0 is error.
  • body – response body. It has to be a serializable object.
encrypted_token = None

toxicbuild.core.requests module

This module implements a simple asynchronous interface for http requests.

Usage:

from toxicbuild.core import requests
response = yield from requests.get('http://google.com/')
print(response.text)
class toxicbuild.core.requests.Response(status, text)[source]

Bases: object

Encapsulates a response from a http request

json()[source]

Loads the json in the response text.

toxicbuild.core.requests.delete(url, **kwargs)[source]

Performs a http DELETE request

Parameters:
  • url – Request’s url.
  • kwargs – Args passed to toxicbuild.core.requests._request().
toxicbuild.core.requests.get(url, **kwargs)[source]

Performs a http GET request

Parameters:
  • url – Request’s url.
  • kwargs – Args passed to toxicbuild.core.requests._request().
toxicbuild.core.requests.post(url, **kwargs)[source]

Performs a http POST request

Parameters:
  • url – Request’s url.
  • kwargs – Args passed to toxicbuild.core.requests._request().
toxicbuild.core.requests.put(url, **kwargs)[source]

Performs a http PUT request

Parameters:
  • url – Request’s url.
  • kwargs – Args passed to toxicbuild.core.requests._request().

toxicbuild.core.utils module

class toxicbuild.core.utils.LoggerMixin[source]

Bases: object

A simple mixin to use log on a class.

log(msg, level='info')[source]

Appends the class name before the log message.

classmethod log_cls(msg, level='info')[source]
class toxicbuild.core.utils.MatchKeysDict[source]

Bases: dict

A dictionary that returns the values matching the keys using toxicbuild.core.utils.match_string().

>>> d = MatchKeysDict()
>>> d['k*'] = 1
>>> d['key']
1
>>> k['keyboard']
1
class toxicbuild.core.utils.SettingsPatcher[source]

Bases: mongomotor.monkey.MonkeyPatcher

Patches the settings from pyrocumulus to use the same settings as toxibuild.

patch_pyro_settings(settings)[source]
class toxicbuild.core.utils.SourceSuffixesPatcher[source]

Bases: mongomotor.monkey.MonkeyPatcher

We must to path SOURCE_SUFFIXES in the python importlib so we can import source code files with extension other than .py, in this case, namely .conf

patch_source_suffixes()[source]

Patches the SOURCE_SUFFIXES constant in the module importlib._bootstrap_external.

SOURCE_SUFFIXES = ['.py', '.conf']
class toxicbuild.core.utils.changedir(path)[source]

Bases: object

toxicbuild.core.utils.bcrypt_string(src_string, salt=None)[source]
toxicbuild.core.utils.compare_bcrypt_string(original, encrypted)[source]

Compares if a un-encrypted string matches an encrypted one.

Parameters:
  • original – An un-encrypted string.
  • encrypted – An bcrypt encrypted string.
toxicbuild.core.utils.create_random_string(length)[source]
toxicbuild.core.utils.daemonize(call, cargs, ckwargs, stdout, stderr, workdir, pidfile)[source]

Run a callable as a daemon

Parameters:
  • call – a callable.
  • cargs – args to call.
  • ckwargs – kwargs to call.
  • stdout – daemon’s stdout.
  • stderr – daemon’s stderr.
  • workdir – daemon’s workdir
  • pidfile – pidfile’s path
toxicbuild.core.utils.datetime2string(dt, dtformat='%w %m %d %H:%M:%S %Y %z')[source]

Transforms a datetime object into a formated string.

Parameters:
  • dt – The datetime object.
  • dtformat – The format to use.
toxicbuild.core.utils.exec_cmd(cmd, cwd, timeout=3600, out_fn=None, **envvars)[source]

Executes a shell command. Raises with the command output if return code > 0.

Parameters:
  • cmd – command to run.
  • cwd – Directory to execute the command.
  • timeout – How long we should wait some output. Default is 3600.
  • out_fn – A coroutine that receives each line of the step output. The coroutine signature must be in the form: mycoro(line_index, line).
  • envvars – Environment variables to be used in the command.
toxicbuild.core.utils.format_timedelta(td)[source]

Format a timedelta object to a human-friendly string.

Parameters:dt – A timedelta object.
toxicbuild.core.utils.get_toxicbuildconf(directory)[source]

Returns the toxicbuild.conf module.

Parameters:directory – Directory to look for toxicbuild.conf
toxicbuild.core.utils.get_toxicbuildconf_yaml(directory, filename='toxicbuild.yml')[source]

Returns the python objet representing the yaml build configuration.

Parameters:
  • directory – The path of the directory containing the config file.
  • filename – The actual name of the config file.
toxicbuild.core.utils.inherit_docs(cls)[source]

Inherit docstrings from base classes’ methods. Can be used as a decorator

Parameters:cls – Class that will inherit docstrings from its parents.
toxicbuild.core.utils.list_builders_from_config(config, branch=None, slave=None, config_type='py')[source]

Lists builders from a build config

Parameters:
  • config – The build configuration.
  • branch – The branch for which builders are being listed.
  • slave – The slave for which builders are being listed.
toxicbuild.core.utils.load_module_from_file(filename)[source]

Load a module from a source file :param filename: full path for file to be loaded.

toxicbuild.core.utils.localtime2utc(localdatetime)[source]

Transforms a local datetime object into a datetime object in utc time.

Parameters:localdatetime – A datetime object.
toxicbuild.core.utils.log(msg, level='info')[source]
toxicbuild.core.utils.match_string(smatch, filters)[source]

Checks if a string match agains a list of filters containing wildcards.

Parameters:
  • smatch – String to test against the filters
  • filters – Filter to match a string.
toxicbuild.core.utils.now()[source]

Returns the localtime with timezone info.

toxicbuild.core.utils.read_file(filename)[source]

Reads the contents of a file asynchronously.

Parameters:filename – The path of the file.
toxicbuild.core.utils.read_stream(reader)[source]

Reads the input stream. First reads the bytes until the first “n”. These first bytes are the length of the full message.

Parameters:reader – An instance of asyncio.StreamReader
toxicbuild.core.utils.run_in_thread(fn, *args, **kwargs)[source]

Runs a callable in a background thread.

Parameters:
  • fn – A callable to be executed in a thread.
  • args – Positional arguments to fn
  • kwargs – Named arguments to fn

Usage

r = yield from run_in_thread(call, 1, bla='a')
toxicbuild.core.utils.set_tzinfo(dt, tzoff)[source]

Sets a timezone info to a datetime object.

Parameters:dt – A datetime object.
Para tzoff:The timezone offset from utc in seconds
toxicbuild.core.utils.string2datetime(dtstr, dtformat='%w %m %d %H:%M:%S %Y %z')[source]

Transforns a string into a datetime object acording to dtformat.

Parameters:
  • dtstr – The string containing the formated date.
  • dtformat – The format of the formated date.
toxicbuild.core.utils.utc2localtime(utcdatetime)[source]

Transforms a utc datetime object into a datetime object in local time.

Parameters:utcdatetime – A datetime object
toxicbuild.core.utils.write_stream(writer, data)[source]

Writes data to output. Encodes data to utf-8 and prepend the lenth of the data before sending it.

Parameters:
  • writer – An instance of asyncio.StreamWriter
  • data – String data to be sent.

toxicbuild.core.vcs module

class toxicbuild.core.vcs.Git(workdir)[source]

Bases: toxicbuild.core.vcs.VCS

An interface to git version control system

add_remote(remote_url, remote_name)[source]

Adds a new remote to the repository.

Parameters:
  • remote_url – The url of the remote repository.
  • remote_name – The name of the remote.
branch_exists(branch_name)[source]

Checks if a local branch exists.

Parameters:branch_name – The name of the branch to check.
checkout(named_tree)[source]

Checkout to named_tree :param named_tree: A commit, branch, tag…

clone(url)[source]

Clones a repository into self.workdir :param url: repository url

create_local_branch(branch_name, base_name)[source]

Creates a branch new in the local repository

Parameters:
  • branch_name – The name for the new branch
  • base_name – The name of the base branch.
delete_local_branch(branch_name)[source]

Deletes a local branch.

Parameters:branch_name – The name of the branch to be deleted.
fetch()[source]

Fetch changes from remote repository

get_remote(remote_name='origin')[source]

Returns the remote url used in the repo.

Parameters:remote_name – The name of the remote url to change.
get_remote_branches()[source]

Returns a list of the remote branches available.

get_revisions(since=None, branches=None)[source]

Returns the revisions for branches since since.

Parameters:
  • since – dictionary in the format: {branch_name: since_date}. since is a datetime object.
  • branches – A list of branches to look for new revisions. If branches is None all remote branches will be used. You can use wildcards in branches to filter the remote branches.
get_revisions_for_branch(branch, since=None)[source]

Returns the revisions for branch since since. If since is None, all revisions will be returned.

Parameters:
  • branch – branch name
  • since – datetime
has_changes()[source]

Informs if there are new revisions in the repository

coroutine import_external_branch(external_url, external_name, external_branch, into)[source]

Imports a branch from an external (not the origin one) repository into a local branch.

Parameters:
  • external_url – The url of the external repository.
  • external_name – Name to idenfity the remote url.
  • external_branch – The name of the branch in the external repo.
  • into – The name of the local branch.
pull(branch_name, remote_name='origin')[source]

Pull changes from branch_name on remote repo.

Parameters:
  • branch_name – A branch name, like ‘master’.
  • remote_name – The remote repository to push from.
set_remote(url, remote_name='origin')[source]

Sets the remote url of the repository.

Parameters:
  • url – The new remote url.
  • remote_name – The name of the remote url to change.
try_set_remote(url, remote_name='origin')[source]

Sets the remote url if the remote is not equal as url.

Parameters:
  • url – The new url for the remote.
  • remote_name – The name of the remote url to change.
update_submodule()[source]
date_format = '%a %b %d %H:%M:%S %Y'
vcsbin = 'git'
class toxicbuild.core.vcs.VCS(workdir)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Generic inteface to a vcs (clone, fetch, get revisions etc…).

add_remote(remote_url, remote_name)[source]

Adds a new remote to the repository.

Parameters:
  • remote_url – The url of the remote repository.
  • remote_name – The name of the remote.
classmethod branch_exists(branch_name)[source]

Checks if a local branch exists.

Parameters:branch_name – The name of the branch to check.
checkout(named_tree)[source]

Checkout to named_tree :param named_tree: A commit, branch, tag…

clone(url)[source]

Clones a repository into self.workdir :param url: repository url

create_local_branch(branch_name, base_name)[source]

Creates a branch new in the local repository

Parameters:
  • branch_name – The name for the new branch
  • base_name – The name of the base branch.
delete_local_branch(branch_name)[source]

Deletes a local branch.

Parameters:branch_name – The name of the branch to be deleted.
exec_cmd(cmd, cwd=None)[source]

Executes a shell command. If cwd is None self.workdir will be used.

Parameters:cwd – Directory where the command will be executed.
fetch()[source]

Fetch changes from remote repository

get_remote(remote_name)[source]

Returns the remote url used in the repo.

Parameters:remote_name – The name of the remote url to change.
get_remote_branches()[source]

Returns a list of the remote branches available.

get_revisions(since=None, branches=None)[source]

Returns the revisions for branches since since.

Parameters:
  • since – dictionary in the format: {branch_name: since_date}. since is a datetime object.
  • branches – A list of branches to look for new revisions. If branches is None all remote branches will be used. You can use wildcards in branches to filter the remote branches.
get_revisions_for_branch(branch, since=None)[source]

Returns the revisions for branch since since. If since is None, all revisions will be returned.

Parameters:
  • branch – branch name
  • since – datetime
has_changes()[source]

Informs if there are new revisions in the repository

import_external_branch(external_url, external_name, external_branch, into)[source]

Imports a branch from an external (not the origin one) repository into a local branch.

Parameters:
  • external_url – The url of the external repository.
  • external_name – Name to idenfity the remote url.
  • external_branch – The name of the branch in the external repo.
  • into – The name of the local branch.
pull(branch_name, remote_name='origin')[source]

Pull changes from branch_name on remote repo.

Parameters:
  • branch_name – A branch name, like ‘master’.
  • remote_name – The remote repository to push from.
set_remote(url, remote_name)[source]

Sets the remote url of the repository.

Parameters:
  • url – The new remote url.
  • remote_name – The name of the remote url to change.
try_set_remote(url, remote_name)[source]

Sets the remote url if the remote is not equal as url.

Parameters:
  • url – The new url for the remote.
  • remote_name – The name of the remote url to change.
workdir_exists()[source]

Informs if the workdir for this vcs exists

vcsbin = None
toxicbuild.core.vcs.get_vcs(vcs_type)[source]

Retuns a subclass of toxicbuild.core.vcs.VCS for vcs_type

Module contents