toxicbuild.master package

Submodules

toxicbuild.master.aws module

Integrations with Amazon web services. Used to start/stop instances.

class toxicbuild.master.aws.EC2Instance(instance_id, region)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Performs operations on the ec2 api

coroutine get_description()[source]
coroutine get_ip()[source]
coroutine get_status()[source]
coroutine is_running()[source]
coroutine is_stopped()[source]
coroutine start()[source]
coroutine stop()[source]
lock

toxicbuild.master.build module

class toxicbuild.master.build.Build(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument, toxicbuild.core.utils.LoggerMixin

A set of steps for a repository. This is the object that stores the build data. The build is carried by the slave.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

coroutine cancel()[source]

Cancel the build if it is not started yet.

coroutine classmethod get(uuid)[source]

Returns a build based on a uuid.

Parameters:uuid – The uuid of the build.
coroutine get_buildset()[source]

Returns the buildset that ‘owns’ this build.

coroutine is_ready2run()[source]

Checks if all trigger conditions are met. If not triggered_by the build is ready. If the return value is None it means that the build must be cancelled because trigger conditions can’t be met.

coroutine notify(event_type)[source]

Send a notification to the build_notification exchange informing about event_type

Parameters:event_type – The name of the event.
coroutine set_slave(slave)[source]

Sets the slave for this build and increments the slave queue.

Parameters:slave – An Slave instance.
to_dict(output=True, steps_output=None)[source]

Transforms the object into a dictionary.

Parameters:
  • output – Should we include the build output?
  • steps_output – Should we include the steps output? If None, the the value of output will be used.
to_json()[source]

Returns a json representation of the buld.

coroutine update()[source]

Does an atomic update in this embedded document.

CANCELLED = 'cancelled'
CONFIG_TYPES = ['py', 'yaml']
PENDING = 'pending'
PREPARING = 'preparing'
STATUSES = ['running', 'fail', 'success', 'exception', 'warning', 'pending', 'cancelled', 'preparing']
branch

The branch of the code that will be tested.

builder

A reference to an instance of Builder.

builders_from

Indicates the branch from which the builders for this build came from. This may not be the same as the build branch.

external

A reference to ExternalRevisionIinfo

finished

When the build was finished. It must be in UTC.

named_tree

A identifier of the commit, a sha, a tag name, etc…

number

A sequencial number for builds in the repository

output

The build output. It is the commands + the output of the steps.

repository

A referece to the Repository that owns the build

slave

A reference to the Slave that will execute the build.

started

When the build was started. It must be in UTC.

status

The current status of the build. May be on of the values in STATUSES.

steps

A list of BuildStep

total_time

The total time of the build execution.

triggered_by

A list of configurations for build triggers

uuid

An uuid that identifies the build

class toxicbuild.master.build.BuildExecuter(repository, builds)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Class responsible for executing builds taking care of wich builds can be executed in parallel, which ones are triggered and what are the repository’s parallel builds limit.

coroutine execute()[source]
class toxicbuild.master.build.BuildManager(repository)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Controls which builds should be executed sequentially or in parallel.

coroutine add_builds(revisions)[source]

Adds the builds for a given revision in the build queue.

Parameters:revision – A list of toxicbuild.master.RepositoryRevision instances for the build.
coroutine add_builds_for_buildset(buildset, conf, builders=None, builders_origin=None)[source]

Adds builds for for a given buildset.

Parameters:
  • buildset – An instance of toxicbuild.master.BuildSet.
  • builders – A list of toxicbuild.master.Builder. If not builders all builders for this slave and revision will be used.
coroutine cancel_build(build_uuid)[source]

Cancel a given build.

Parameters:build_uuid – The uuid that indentifies the build to be cancelled.
coroutine cancel_previous_pending(buildset)[source]

Cancels the builds previous to buildset.

Parameters:buildset – An instance of BuildSet.
coroutine get_builders(revision, conf, include=None, exclude=None)[source]

Get builders for a given revision.

Parameters:
  • revision – A toxicbuild.master.repository.RepositoryRevision.
  • conf – The build configuration.
  • include – A list of builder names to include. Builders not in this list will not be returned.
  • exclude – A list of builder names to exclude. Builders in this list will not be returned.
coroutine start_pending()[source]

Starts all pending buildsets that are not already scheduled for self.repository.

build_queues

Returns the build queues for a repository.

is_building

Indicates if has some active build for a repository.

class toxicbuild.master.build.BuildSet(*args, **kwargs)[source]

Bases: toxicbuild.master.build.SerializeMixin, toxicbuild.core.utils.LoggerMixin, mongomotor.document.Document

A list of builds associated with a revision.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine classmethod aggregate_get(**kwargs)[source]

Returns information about a buildset Uses the aggregation framework to $lookup on builds’ builder name and on repository. I does not returns steps information, only buildset and builds information.

Parameters:kwargs – Named arguments to match the buildset.
coroutine classmethod create(repository, revision)[source]

Creates a new buildset.

Parameters:
  • repository – An instance of toxicbuild.master.Repository.
  • revision – An instance of toxicbuild.master.RepositoryRevision.
  • save – Indicates if the instance should be saved to database.
coroutine get_builds_for(builder=None, branch=None)[source]

Returns the builds for a specific builder and/or branch.

Parameters:
  • builder – An instance of Builder.
  • branch – The name of the branch.
get_pending_builds()[source]

Returns the pending builds of the buildset.

get_status()[source]

Returns the status of the BuildSet

coroutine notify(event_type, status=None)[source]

Notifies an event to the build_notifications exchange.

Parameters:
  • event_type – The event type to notify about
  • status – The status of the buildset. If None, the value of status will be used.
to_dict(builds=True)[source]

Returns a dict representation of the object

Parameters:builds – Should the builds be included in the dict?
to_json()[source]

Returns a json representation of the object.

coroutine update_status(status=None)[source]

Updates the status of the buildset.

Parameters:status – Status to update the buildset. If None, self.get_status() will be used.
NO_BUILDS = 'no builds'
NO_CONFIG = 'no config'
PENDING = 'pending'
STATUSES = ['no builds', 'no config', 'running', 'fail', 'success', 'exception', 'warning', 'pending', 'cancelled', 'preparing']
author

Commit author’s name.

branch

The branch of the commit

builds

A list of Build intances.

commit

The identifier of the commit that generated the buildset.

commit_body

The body of the commit.

commit_date

The date of the commit

created

When the BuildSet was first created. It must be in UTC.

finished

When the buildset finished. It must be in UTC.

id

A field wrapper around MongoDB’s ObjectIds.

number

A sequencial number for the buildsets in the repository

objects = QuerySet
repository

A referece to the Repository that owns the buildset

revision

A reference to the RepositoryRevision that generated this buildset.

started

When the BuildSet started to run. It must be in UTC.

status

The current status of the buildset. May be on of the values in STATUSES.

title

Commit title

total_time

The total time spent in the buildset

class toxicbuild.master.build.BuildStep(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument

A step for a build. This is the object that will store the step data. Who actually execute the steps is the slave.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

coroutine classmethod get(uuid)[source]

Returns a step based on a uuid.

Parameters:uuid – The uuid of the step.
to_dict(output=True)[source]

Returns a dict representation of the BuildStep.

Parameters:output – Indicates if the output of the step should be included.
to_json()[source]

Returns a json representation of the BuildStep.

EXCEPTION = 'exception'
FAIL = 'fail'
RUNNING = 'running'
STATUSES = ['running', 'fail', 'success', 'exception', 'warning']
SUCCESS = 'success'
WARNING = 'warning'
command

The command that executes the step

finished

When the step finished. It msut be in UTC.

index

The index of the step in the build.

name

The name of the step. Will be displayed in the ui.

output

The output of the step

repository

A referece to Repository

started

When the step stated. It msut be in UTC.

status

The current status of the step. May be one of the values in STATUSES`

total_time

The total time spen in the step.

uuid

The uuid that indentifies the build step

class toxicbuild.master.build.BuildTrigger(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument

A configuration for what triggers a build.

builder_name

The name of a builder

statuses

A list of statuses that trigger a build.

class toxicbuild.master.build.Builder(*args, **kwargs)[source]

Bases: toxicbuild.master.build.SerializeMixin, mongomotor.document.Document

The entity responsible for executing the build steps.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine classmethod create(**kwargs)[source]

Creates a new Builder.

Parameters:kwargs – kwargs passed to the builder constructor
coroutine classmethod get(**kwargs)[source]

Returns a builder instance.

coroutine classmethod get_or_create(**kwargs)[source]

Returns a builder instance. If it does not exist, creates it.

Parameters:kwargs – kwargs to match the builder.
coroutine get_status()[source]

Returns the builder status.

coroutine to_dict()[source]

Returns a dictionary for this builder.

coroutine to_json()[source]

Returns a json for this builder.

id

A field wrapper around MongoDB’s ObjectIds.

name

The name of the builder.

objects = QuerySet
repository

A referece to the Repository that owns the builder

class toxicbuild.master.build.SerializeMixin[source]

Bases: object

Simple mixin to serialization relatad stuff.

coroutine async_to_json()[source]

Async version of to_json. Expects a to_dict coroutine.

to_dict(id_as_str=True)[source]

Transforms a Document into a dictionary.

Parameters:id_as_str – If true, transforms the id field into a string.

toxicbuild.master.client module

class toxicbuild.master.client.BuildClient(slave, *args, **kwargs)[source]

Bases: toxicbuild.core.client.BaseToxicClient, toxicbuild.core.utils.LoggerMixin

A client to toxicbuild.slave.server.BuildServer

coroutine build(build, envvars=None, process_coro=None)[source]

Requests a build for the build server.

Parameters:
  • build – The build that will be executed. param evvars: Environment variables to use in the build.
  • process_coro – A coroutine to process the intermediate build information sent by the build server.
coroutine healthcheck()[source]

Asks to know if the server is up and running

coroutine list_builders(repo_url, vcs_type, branch, named_tree)[source]

Asks the server for the builders available for repo_url, on branch and named_tree.

toxicbuild.master.client.get_build_client(slave, addr, port, use_ssl=True, validate_cert=True)[source]

Instanciate toxicbuild.master.client.BuildClient and connects it to a build server

toxicbuild.master.consumers module

This module implements consumers for messages published in exchanges. The consumer react (call something) in response for a incomming message.

class toxicbuild.master.consumers.BaseConsumer(exchange, msg_callback, routing_key=None, loop=None)[source]

Bases: toxicbuild.core.utils.LoggerMixin

A base class for the consumer that react to incomming messages from exchanges

coroutine run()[source]

Starts the consumer.

coroutine shutdown()[source]
stop()[source]
sync_shutdown(signum=None, frame=None)[source]
class toxicbuild.master.consumers.RepositoryMessageConsumer[source]

Bases: toxicbuild.core.utils.LoggerMixin

Waits for messages published in the repo_notifications exchange and reacts to them.

REPOSITORY_CLASS

alias of toxicbuild.master.repository.Repository

REPOSITORY_REVISION_CLASS

alias of toxicbuild.master.repository.RepositoryRevision

run()[source]

Starts the repository message consumer.

classmethod stop()[source]
build_consumer = None
removal_consumer = None
revision_consumer = None
update_consumer = None

toxicbuild.master.coordination module

class toxicbuild.master.coordination.Lock(path)[source]

Bases: toxicbuild.core.utils.LoggerMixin

coroutine acquire_read(timeout=None)[source]
coroutine acquire_write(timeout=None)[source]
class toxicbuild.master.coordination.ToxicZKClient[source]

Bases: toxicbuild.core.utils.LoggerMixin

coroutine get_lock(path)[source]

toxicbuild.master.document module

class toxicbuild.master.document.ExternalRevisionIinfo(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument

Information about code that came from an external source. Shared by RepositoryRevision and Build

to_dict()[source]

Returns a dict representations of the object

branch

The name of the branch in the external repo.

into

A name for a local branch to clone the external branch into.

name

A name to indentify the external repo.

url

The url of the external repo

class toxicbuild.master.document.OwnedDocument(*args, **kwargs)[source]

Bases: mongomotor.document.Document

An abstract document for objects that have an owner. Subclass it to use it.

Example: ´´´´´´´´

class MyOwnedDocument(OwnedDocument):

some_field = StringField()
coroutine check_perms(user)[source]
coroutine get_allowed_users()[source]

Returns a queryset of users that have read permission.

coroutine classmethod get_for_user(user, **kwargs)[source]

Returns a repository if user has permission for it. If not raises an error.

Parameters:
  • user – User who is requesting the repository.
  • kwargs – kwargs to match the repository.
coroutine classmethod list_for_user(user, **kwargs)[source]

Returns a queryset of repositories in which the user has read permission

coroutine save(*args, **kwargs)[source]

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:
  • force_insert – only try to create a new document, don’t allow updates of existing documents.
  • validate – validates the document; set to False to skip.
  • clean – call the document clean method, requires validate to be True.
  • write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
  • cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
  • cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
  • _refs – A list of processed references used in cascading saves
  • save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
  • signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

full_name

Full name of the document in the form owner-name/doc-name

name

The name of the document.

owner

The user or organization who owns the document

toxicbuild.master.exceptions module

exception toxicbuild.master.exceptions.CloneException[source]

Bases: Exception

exception toxicbuild.master.exceptions.DBError[source]

Bases: Exception

exception toxicbuild.master.exceptions.ImpossibleCancellation[source]

Bases: Exception

exception toxicbuild.master.exceptions.InvalidCredentials[source]

Bases: Exception

exception toxicbuild.master.exceptions.NotEnoughPerms[source]

Bases: Exception

exception toxicbuild.master.exceptions.OwnerDoesNotExist[source]

Bases: Exception

exception toxicbuild.master.exceptions.RepoBranchDoesNotExist[source]

Bases: Exception

exception toxicbuild.master.exceptions.UIFunctionNotFound[source]

Bases: Exception

exception toxicbuild.master.exceptions.UnknownSchedulerAction[source]

Bases: Exception

toxicbuild.master.exchanges module

toxicbuild.master.exchanges.connect_exchanges()[source]
toxicbuild.master.exchanges.disconnect_exchanges()[source]

toxicbuild.master.hole module

class toxicbuild.master.hole.HoleHandler(data, action, protocol)[source]

Bases: object

Handles the incomming connections for the UIHole. It has the following methods available to the clients:

  • repo-add
  • repo-get
  • repo-list
  • repo-remove
  • repo-update
  • repo-add-slave
  • repo-remove-slave
  • repo-add-branch
  • repo-remove-branch
  • repo-start-build
  • repo-cancel-build
  • repo-enable
  • repo-disable
  • repo-request-code-update
  • repo-add-envvars
  • repo-rm-envvars
  • repo-replace-envvars
  • repo-list-branches
  • slave-add
  • slave-get
  • slave-list
  • slave-remove
  • slave-update
  • buildset-list
  • buildset-get
  • build-get
  • builder-show
  • list-funcs
  • user-add
  • user-remove
  • user-authenticate
  • user-get
  • user-exists
coroutine build_get(build_uuid)[source]

Returns a build.

Parameters:build_uuid – The uuid of a build.
coroutine builder_list(**kwargs)[source]

List builders.

Parameters:kwargs – Arguments to filter the list.
coroutine builder_show(repo_name_or_id, builder_name, skip=0, offset=None)[source]

Returns information about one specific builder.

Parameters:
  • repo_name_or_id – The builder’s repository name.
  • builder_name – The bulider’s name.
  • skip – How many elements we should skip in the result.
  • offset – How many results we should return.
coroutine buildset_get(buildset_id)[source]

Returns information about a specific buildset.

Parameters:buildset_id – The id of the buildset.
coroutine buildset_list(repo_name_or_id=None, skip=0, offset=None, summary=False, branch=None)[source]

Lists all buildsets. If repo_name_or_id, only builders from this repository will be listed.

Parameters:
  • repo_name_or_id – Repository’s name or id.
  • skip – skip for buildset list.
  • offset – offset for buildset list.
  • summary – If true, no builds information will be included.
  • branch – List buildsets for this branch. If None list buildsets from all branches.
coroutine buildstep_get(step_uuid)[source]

Returns a buildstep.

Parameters:step_uuid – The uuid of a step.
coroutine handle()[source]
list_funcs()[source]

Lists the functions available for user interfaces.

coroutine repo_add(repo_name, repo_url, owner_id, update_seconds, vcs_type, slaves=None, parallel_builds=None, schedule_poller=True, branches=None, external_id=None, external_full_name=None, fetch_url=None, envvars=None)[source]

Adds a new repository and first_run() it.

Parameters:
  • repo_name – Repository name
  • repo_url – Repository vcs url
  • owner_id – Id of the repository’s owner.
  • update_seconds – Time to poll for changes
  • vcs_type – Type of vcs being used.
  • slaves – A list of slave names.
  • parallel_builds – How many parallel builds this repository executes. If None, there is no limit.
  • schedule_poller – Should this repository be scheduled for polling? If this repository comes from an integration (with github, gitlab, etc…) this should be False.
  • branches – A list of branches configuration that trigger builds.
  • external_id – The id of the repository in an external service.
  • external_full_name – The full name in an external service.
  • fetch_url – If the repository uses a differente url to fetch code (ie: it has an auth token url) this is the fetch_url.
  • envvars – The environment variables for this repository.
coroutine repo_add_branch(repo_name_or_id, branch_name, notify_only_latest=False)[source]

Adds a branch to the list of branches of the repository.

Parameters:
  • repo_name_or_id – Reporitory name or id.
  • branch_name – Branch’s name.
Notify_only_latest:
 

If True only the latest commit in the branch will trigger a build.

coroutine repo_add_envvars(repo_name_or_id, **envvars)[source]

Adds environment variables to a repository.

Parameters:
  • repo_name_or_id – The name or the id of the repository.
  • envvars – Environment variables in the format {var: val, …}.
coroutine repo_add_slave(repo_name_or_id, slave_name_or_id)[source]

Adds a slave to a repository.

Parameters:
  • repo_name_or_id – Repository name or id.
  • slave_name_or_id – Slave name or id.
coroutine repo_cancel_build(repo_name_or_id, build_uuid)[source]

Cancels a build if possible.

Parameters:
  • repo_name_or_id – The name or the id of the repository.
  • buid_uuid – The uuid of the build to be cancelled.
coroutine repo_disable(repo_name_or_id)[source]

Disables a repository.

Parameters:repo_name_or_id – The name or the id of the repository.
coroutine repo_enable(repo_name_or_id)[source]

Enables a repository.

Parameters:repo_name_or_id – The name or the id of the repository.
coroutine repo_get(repo_name_or_id=None, repo_url=None)[source]

Shows information about one specific repository. One of repo_name or repo_url is required.

Parameters:
  • repo_name_or_id – Repository name or ObjectId,
  • repo_url – Repository vcs url.
coroutine repo_list(**kwargs)[source]

Lists all repositories.

Kwargs:Named arguments to filter repositories.
coroutine repo_list_branches(repo_name_or_id)[source]

List the branches known by a repository.

Parameters:repo_name_or_id – The name or the id of the repository.
coroutine repo_remove(repo_name_or_id)[source]

Removes a repository from toxicubild.

Parameters:repo_name_or_id – Repository name or id.
coroutine repo_remove_branch(repo_name_or_id, branch_name)[source]

Removes a branch from the list of branches of a repository. :param repo_name_or_id: Repository name or id. :param branch_name: Branch’s name.

coroutine repo_remove_slave(repo_name_or_id, slave_name_or_id)[source]

Removes a slave from a repository.

Parameters:
  • repo_name_or_id – Repository name or id.
  • slave_name_or_id – Slave name or id.
coroutine repo_replace_envvars(repo_name_or_id, **envvars)[source]

Replaces the repository’s environment variables.

Parameters:
  • repo_name_or_id – The name or the id of the repository.
  • envvars – Environment variables in the format {var: val, …}.
coroutine repo_request_code_update(repo_name_or_id, repo_branches=None, external=None, wait_for_lock=False)[source]

Requests to a repository the update of its code.

Parameters:
coroutine repo_rm_envvars(repo_name_or_id, **envvars)[source]

Removes environment variables from a repository.

Parameters:
  • repo_name_or_id – The name or the id of the repository.
  • envvars – Environment variables in the format {var: val, …}.
coroutine repo_start_build(repo_name_or_id, branch, builder_name_or_id=None, named_tree=None, builders_origin=None)[source]

Starts a(some) build(s) in a given repository.

coroutine repo_update(repo_name_or_id, **kwargs)[source]

Updates repository information.

Parameters:
  • repo_name_or_id – Repository name or id
  • kwargs – kwargs to update the repository
coroutine slave_add(slave_name, slave_host, slave_port, slave_token, owner_id, use_ssl=True, validate_cert=True, on_demand=False, instance_type=None, instance_confs=None)[source]

Adds a new slave to toxicbuild.

Parameters:
  • slave_name – A name for the slave,
  • slave_host – Host where the slave is.
  • slave_port – Port to connect to the slave
  • slave_token – Auth token for the slave.
  • owner_id – Slave’s owner id.
  • use_ssl – Indicates if the slave uses a ssl connection.
  • validate_cert – Should the slave certificate be validated?
  • on_demand – Does this slave have an on-demand instance?
  • instance_type – Type of the on-demand instance.
  • instance_confs – Configuration parameters for the on-demand instance.
coroutine slave_get(slave_name_or_id)[source]

Returns information about one specific slave

coroutine slave_list()[source]

Lists all slaves.

coroutine slave_remove(slave_name_or_id)[source]

Removes a slave from toxicbuild.

coroutine slave_update(slave_name_or_id, **kwargs)[source]

Updates infomation of a slave.

coroutine user_add(email, password, allowed_actions, username=None)[source]

Adds a new user.

Parameters:
  • email – User email.
  • password – User password
  • allowed_actions – What the user can do.
  • username – Username for the user.
coroutine user_authenticate(username_or_email, password)[source]

Authenticates an user. Returns user.to_dict() if authenticated. Raises InvalidCredentials if a user with this credentials does not exist.

Parameters:
  • username_or_email – Username or email to use to authenticate.
  • password – Not encrypted password.
coroutine user_change_password(username_or_email, old_password, new_password)[source]

Changes a user password. First authenticates the user, if authentication is ok then changes the password.

Parameters:
  • username_or_email – Username or email to use to authenticate.
  • old_password – User’s old password. Used to authenticate.
  • new_password – The new password.
coroutine user_change_password_with_token(token, password)[source]
coroutine user_exists(**kw)[source]

Checks if a user with given information exists.

Params kw:Named arguments to check if the user exists.
coroutine user_get(**kw)[source]

Returns information about a specific user.

Params kw:Named arguments to match the user.
coroutine user_remove(**kwargs)[source]

Removes a user from the system.

coroutine user_send_reset_password_email(email, subject, message)[source]

Sends an email to a user with a link to reset his/her password.

Parameters:
  • email – The user email.
  • subject – The email subject.
  • message – The email message.
class toxicbuild.master.hole.HoleServer(addr='127.0.0.1', port=6666, loop=None, use_ssl=False, **ssl_kw)[source]

Bases: toxicbuild.core.utils.LoggerMixin

A server that uses the UIHole protocol.

get_protocol_instance()[source]
serve()[source]
coroutine shutdown()[source]
sync_shutdown(signum=None, frame=None)[source]
class toxicbuild.master.hole.UIHole(*args, **kwargs)[source]

Bases: toxicbuild.core.protocol.BaseToxicProtocol, toxicbuild.core.utils.LoggerMixin

coroutine client_connected()[source]

Coroutine that handles connections. You must implement this in your sub-classes. When this method is called, self.data, containing a dictionary with the data passed in the request and self.action, a string indicating which action to take are already available.

classmethod set_shutting_down()[source]
encrypted_token = '$2b$04$uPkH68HUDSFl5SFiAonXq.MOhW/0QGyCnRedEXi5IcqyrTvvvexUK'
class toxicbuild.master.hole.UIStreamHandler(protocol)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Handler that keeps the connection open and messages when builds and steps are stated or finished.

coroutine build_added(sender, **kw)[source]
coroutine build_cancelled(sender, **kw)[source]
coroutine build_finished(sender, **kw)[source]
coroutine build_preparing(sender, **kw)[source]
coroutine build_started(sender, **kw)[source]
coroutine buildset_added(sender, **kw)[source]
coroutine buildset_finished(sender, **kw)[source]
coroutine buildset_started(sender, **kw)[source]
coroutine check_repo_added(msg)[source]
coroutine handle()[source]
coroutine send_buildset_info(event_type, buildset)[source]
coroutine send_info(info_type, sender, build=None, step=None)[source]
coroutine send_repo_added_info(message)[source]

Sends a message about a repository’s creation.

Parameters:message – A message from the repo_added exchange.
coroutine send_repo_status_info(message)[source]

Sends a message about a repository’s new status

Parameters:message – A message from the repo_status_changed exchange.
coroutine send_response(code, body)[source]
coroutine step_finished(sender, **kw)[source]
step_output_arrived(repo, step_info)[source]

Called by the signal step_output_arrived.

Parameters:
  • repo – The repository that is building something.
  • step_info – The information about the step output.
coroutine step_started(sender, **kw)[source]

toxicbuild.master.pollers module

class toxicbuild.master.pollers.Poller(repository, vcs_type, workdir)[source]

Bases: toxicbuild.core.utils.LoggerMixin

Class to poll changes from a vcs, process them and notificate about incoming changes

coroutine external_poll(external_url, external_name, external_branch, into)[source]

Fetches the changes of a external (not the origin) repository into a local branch.

Parameters:
  • external_url – The url of the external remote repository.
  • external_name – The name to identiry the external repo.
  • external_branch – The name of the branch in the external repo.
  • into – The name of the local repository.
is_polling()[source]
log(msg, level='info')[source]

Appends the class name before the log message.

coroutine notify_change(*revisions)[source]

Notify about new revisions added to the repository.

Parameters:revisions – A list of new revisions
coroutine poll(repo_branches=None)[source]

Check for changes on repository and if there are changes, notify about it.

Parameters:repo_branches – Param to be passed to process_changes().
coroutine process_changes(repo_branches=None)[source]

Process all changes since the last revision in db

Parameters:repo_branches

The branches to look for incomming changes. If no branches, all branches in repo config will be used. It is a dictionary with the following format:

class toxicbuild.master.pollers.PollerServer(loop=None)[source]

Bases: toxicbuild.master.consumers.BaseConsumer

A server for pollers. Uses Rabbitmq to publish/consume messages from the master

coroutine handle_update_request(msg)[source]

Handle an update code request sent by the master.

toxicbuild.master.repository module

class toxicbuild.master.repository.Repository(*args, **kwargs)[source]

Bases: toxicbuild.master.document.OwnedDocument, toxicbuild.core.utils.LoggerMixin

Repository is where you store your code and where toxicbuild looks for incomming changes.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine add_builds_for_buildset(buildset, conf, builders=None, builders_origin=None)[source]

Adds a buildset to the build queue of a given slave for this repository.

Parameters:
  • buildset – An instance of toxicbuild.master.build.BuildSet.
  • conf – The build configuration for the buidset.
  • builders – The builders to use in the buids. If no builds, all builders for the revision will be used.
  • builders_origin – Indicates from which branch config the builds came. Useful for merge requests to test agains the tests on the main branch.
coroutine add_envvars(**envvars)[source]

Adds new environment variables to this repository.

Parameters:envvars – A dictionary {var: val, …}.
coroutine add_or_update_branch(branch_name, notify_only_latest=False)[source]

Adds a new branch to this repository. If the branch already exists updates it with a new value.

Parameters:
  • branch_name – The name of a branch
  • notify_only_latest – If we should build only the most recent build of this branch
coroutine add_revision(branch, commit, commit_date, author, title, body=None, external=None, builders_fallback='')[source]

Adds a revision to the repository.

Parameters:
  • commit – Commit’s sha
  • branch – branch name
  • commit_date – commit’s date (on authors time)
  • author – The author of the commit
  • title – The commit title.
  • body – The commit body.
  • external – Information about an external remote if the revision came from an external.
  • builders_fallback – If not None, builders from this branch will be used in case of the revision branch has no builders configured for it
classmethod add_running_build()[source]

Add a running build to the count of running builds among all repositories.

coroutine add_slave(slave)[source]

Adds a new slave to a repository.

Parameters:slave – A slave instance.
coroutine bootstrap()[source]

Initialise the needed stuff. Schedules updates for code, start of pending builds, connect to signals.

coroutine classmethod bootstrap_all()[source]
coroutine cancel_build(build_uuid)[source]

Cancels a build.

Parameters:build_uuid – The uuid of the build.
coroutine classmethod create(**kwargs)[source]

Creates a new repository and schedule it if needed.

Parameters:kwargs – kwargs used to create the repository.
coroutine disable()[source]
coroutine enable()[source]
coroutine classmethod get(**kwargs)[source]

Returns a repository instance and create locks if needed

Parameters:kwargs – kwargs to match the repository.
get_branch(branch_name)[source]

Returns an instance of RepositoryBranch

coroutine get_config_for(revision)[source]

Returns the build configuration for a given revision.

Parameters:revision – A :class`~toxicbuild.master.repository.RepositoryRevision` instance.
coroutine classmethod get_for_user(user, **kwargs)[source]

Returns a repository if user has permission for it. If not raises an error.

Parameters:
  • user – User who is requesting the repository.
  • kwargs – kwargs to match the repository.
coroutine get_known_branches()[source]

Returns the names for the branches that already have some revision here.

coroutine get_last_buildset()[source]
coroutine get_latest_revision_for_branch(branch)[source]

Returns the latest revision for a given branch

Parameters:branch – branch name
coroutine get_latest_revisions()[source]

Returns the latest revision for all known branches

classmethod get_running_builds()[source]

Returns the number of running builds among all the repos.

coroutine get_status()[source]

Returns the status for the repository. The status is the status of the last buildset created for this repository that is not pending.

get_url()[source]
notify_only_latest(branch_name)[source]

Indicates if a branch notifies only the latest revision.

Parameters:branch_name – The name of the branch.
coroutine remove()[source]

Removes all builds and builders and revisions related to the repository, removes the poller from the scheduler, removes the source code from the file system and then removes the repository.

coroutine remove_branch(branch_name)[source]

Removes a branch from this repository.

Parameters:branch_name – The branch name.
classmethod remove_running_build()[source]

Removes a running build from the count of running builds among all repositories.

coroutine remove_slave(slave)[source]

Removes a slave from a repository.

Parameters:slave – A slave instance.
coroutine replace_envvars(**envvars)[source]

Replaces the current environment variables of the repository.

Parameters:envvars – The environment variables that will replace the current one.
coroutine request_build(branch, builder_name=None, named_tree=None, slaves=None)[source]

Publishes a message in the repo_notifications exchange requesting a build. Uses the routing_key build-requested

coroutine request_code_update(repo_branches=None, external=None, wait_for_lock=False)[source]

Request the code update of a repository by publishing a message in the repo_notifications queue with the routing key repo-update-code-requested.

Parameters:
  • repo_branches

    A dictionary with information about the branches to be updated. If no repo_branches all branches in the repo config will be updated.

    The dictionary has the following format.

    {'branch-name': {'notify_only_latest': True}}
    
  • external – If we should update code from an external (not the origin) repository, external is the information about this remote repo.
  • wait_for_lock – Indicates if we should wait for the release of the lock or simply return if we cannot get a lock.
coroutine request_removal()[source]

Request the removal of a repository by publishing a message in the repo_notifications queue with the routing key repo-removal-requested.

coroutine rm_envvars(**envvars)[source]

Removes environment variables from this repository.

Parameters:envvars – A dictionary {var: val, …}.
coroutine save(*args, **kwargs)[source]

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:
  • force_insert – only try to create a new document, don’t allow updates of existing documents.
  • validate – validates the document; set to False to skip.
  • clean – call the document clean method, requires validate to be True.
  • write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
  • cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
  • cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
  • _refs – A list of processed references used in cascading saves
  • save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
  • signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

schedule()[source]

Schedules all needed actions for a repository. The actions are:

  • Sends an add-udpate-code to the scheduler server.
  • Starts builds that are pending using self.build_manager.start_pending.
  • Connects to build_started and build_finished signals to handle changing of status.
coroutine classmethod schedule_all()[source]

Schedule all repositories.

coroutine start_build(branch, builder_name_or_id=None, named_tree=None, builders_origin=None)[source]

Starts a (some) build(s) in the repository.

coroutine to_dict()[source]

Returns a dict representation of the object.

coroutine update_code(repo_branches=None, external=None, wait_for_lock=False)[source]

Requests a code update to a poller and waits for its response. This is done using update_code and poll_status exchanges.

Parameters:
  • repo_branches

    A dictionary with information about the branches to be updated. If no repo_branches all branches in the repo config will be updated.

    The dictionary has the following format.

    {'branch-name': notify_only_latest}
    
  • external – If we should update code from an external (not the origin) repository, external is the information about this remote repo.
  • wait_for_lock – Indicates if we should wait for the release of the lock or simply return if we cannot get a lock.
branches

A list of RepositoryBranch. These branches are the ones that trigger builds. If no branches, all branches will trigger builds.

clone_status

The status of the clone.

enabled

Indicates if this repository is enabled to run builds.

envvars

Environment variables that are used in every build in this repository. It is a dictionary {‘VAR’: ‘VAL’, …}

external_full_name

The full name of the repository in an external service

external_id

The repository id in an external service. Is a string because is not guaranteed a external id is a int.

fetch_url

A url used to actually fetch the code. If using some kind of authentication based in a url, this may change often.

id

A field wrapper around MongoDB’s ObjectIds.

objects = QuerySet
parallel_builds

Max number of builds in parallel that this repo exeutes If None, there’s no limit for parallel builds.

schedule_poller

Indicates if we should schedule periodical polls for changes in code. If the repo was imported from an external service that sends webhooks (or something else) this should be False.

slaves

A list of Slave. The slaves here are the slaves allowed to run builds for this repo.

toxicbuild_conf_lock
update_code_lock
update_seconds

If the repository added manually (not imported), indicates the inteval for polling for new revisions.

url

The url of the repository.

vcs

An instance of a subclass of VCS.

vcs_type

The type of vcs used in this repo.

workdir

The directory where the source code of this repository is cloned into

class toxicbuild.master.repository.RepositoryBranch(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument

The configuration for a branch of a repository.

to_dict()[source]

Returns a dict representation of the obj.

name

The name of the branch.

notify_only_latest

If True, only the latest revision will be notified and only the last revision will generate a buildset.

class toxicbuild.master.repository.RepositoryRevision(*args, **kwargs)[source]

Bases: mongomotor.document.Document

A commit in the code tree.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

create_builds()[source]

Checks for instructions in the commit body to know if a revision should create builds.

Known instructions:

  • ci: skip - If in the commit body there’s this instruction,

    no builds will be created for this revision. The regex for match this instruction is (^|.*\s+)ci:\s*skip(\s+|$)

coroutine classmethod get(**kwargs)[source]

Returs a RepositoryRevision object.

get_builders_conf()[source]

Returns the builder configuration - includes and excludes - for a given revison in its commit body.

Known instructions:

  • ci: include-builders builder-name,other-builder: Include only the builders listed in the configuration. The names are separated by comma.
  • ci: exclude-builders builder-name,other-builder: Exclude the builders listed in the configuration. The names are separated by comma.
coroutine to_dict()[source]

Returns a dict representation of the object.

author

The author of the commit.

body

The commit body.

branch

The name of the revison branch.

builders_fallback

A name of a branch. If not None, builders from this branch will be used if there are no builders for the branch of the revision.

commit

The identifier of the revision, a sha, a tag name, etc…

commit_date

Commit’s date.

external

A list of RepositoryRevisionExternal.

id

A field wrapper around MongoDB’s ObjectIds.

objects = QuerySet
repository

A referece to Repository.

title

The title of the commit.

toxicbuild.master.scheduler module

A very simple implementation of an in memory task scheduler using asyncio.

class toxicbuild.master.scheduler.PeriodicTask(call_or_coro, interval)[source]

Bases: object

A task that will be executed in a periodic time interval

class toxicbuild.master.scheduler.SchedulerServer(loop=None)[source]

Bases: toxicbuild.master.consumers.BaseConsumer

Simple server to add or remove something from the scheduler.

coroutine handle_add_update_code(msg)[source]
coroutine handle_request(msg)[source]
coroutine handle_rm_update_code(msg)[source]
coroutine run()[source]

Starts the consumer.

class toxicbuild.master.scheduler.TaskScheduler[source]

Bases: toxicbuild.core.utils.LoggerMixin

A simple scheduler for periodic tasks.

add(call_or_coro, interval)[source]

Adds call_or_coro to be consumed at interval.

Parameters:
  • call_or_coro – callable or coroutine to be consumed.
  • interval – time in seconds to consume call_or_coro.
consume_tasks()[source]
remove(call_or_coro)[source]

Removes call_or_coro from the scheduler :param call_or_coro: callable or coroutine to remove

remove_by_hash(cc_hash)[source]

Removes the callable or couroutine scheduled using cc_hash.

coroutine start()[source]
stop()[source]

toxicbuild.master.signals module

toxicbuild.master.slave module

class toxicbuild.master.slave.Slave(*args, **kwargs)[source]

Bases: toxicbuild.master.document.OwnedDocument, toxicbuild.core.utils.LoggerMixin

Slaves are the entities that actualy do the work of execute steps. The comunication to slaves is through the network (using toxicbuild.master.client.BuildClient). The steps are actually decided by the slave.

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine add_running_repo(repo_id)[source]

Increments the number of running builds in this slave and adds the repository id to the running repos list. Also decrements the queue count.

Parameters:repo_id – An id of a repository.
coroutine build(build, **envvars)[source]

Connects to a build server and requests a build on that server

Parameters:
coroutine classmethod create(**kwargs)[source]

Creates a new slave

coroutine decrement_queue()[source]

Decrements the queue’s count in this slave.

coroutine classmethod get(**kwargs)[source]

Returns a slave instance.

coroutine get_client()[source]

Returns a BuildClient instance already connected to the server.

coroutine healthcheck()[source]

Check if the build server is up and running

coroutine increment_queue()[source]

Increments the queue’s count in this slave.

coroutine list_builders(revision)[source]

List builder available in for a given revision

Parameters:revision – An instance of toxicbuild.master.repository.RepositoryRevision
coroutine rm_running_repo(repo_id)[source]

Decrements the number of running builds in this slave and removes the repository id from the running repos list

Parameters:repo_id – An id of a repository.
coroutine save(*args, **kwargs)[source]

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:
  • force_insert – only try to create a new document, don’t allow updates of existing documents.
  • validate – validates the document; set to False to skip.
  • clean – call the document clean method, requires validate to be True.
  • write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
  • cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
  • cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
  • _refs – A list of processed references used in cascading saves
  • save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
  • signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

coroutine start_instance()[source]

Starts an on-demand instance if needed.

coroutine stop_instance()[source]

Stops an on-demand instance

to_dict(id_as_str=False)[source]

Returns a dict representation of the object.

coroutine wait_service_start(timeout=10)[source]

Waits for the toxicslave service start in the on-demand instance.

DYNAMIC_HOST = '<DYNAMIC-HOST>'
INSTANCE_CLS = {'ec2': <class 'toxicbuild.master.aws.EC2Instance'>}
INSTANCE_TYPES = ('ec2',)
host

Slave’s host.

id

A field wrapper around MongoDB’s ObjectIds.

instance

Returns an on-demand instance wrapper.

instance_confs

Configuration paramenters for the on-demand instance.

instance_type

The type of instance used. Currently only ‘ec2’ is supported.

is_alive

Indicates if the slave is up and running.

objects = QuerySet
on_demand

If the slave is on-demand it will be started when needed and will be stopped when all the builds for this slave are completed.

parallel_builds

Max number of builds in parallel that this slave exeutes. If no parallel_builds there’s no limit.

port

Port for the slave to listen.

queue_count

How many builds are waiting to run in this repository.

running_count

How many builds are running in this slave.

running_repos

The ids of the repositories that have builds running in this slave.

token

Token for authentication.

use_ssl

Indicates if the build server in uses ssl connection.

validate_cert

Indicates if the certificate from the build server should be validated.

toxicbuild.master.users module

class toxicbuild.master.users.Organization(*args, **kwargs)[source]

Bases: mongomotor.document.Document

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine add_user(user)[source]

Adds a user to the organization

coroutine remove_user(user)[source]

Removes a user from the organization.

coroutine save(*args, **kwargs)[source]

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:
  • force_insert – only try to create a new document, don’t allow updates of existing documents.
  • validate – validates the document; set to False to skip.
  • clean – call the document clean method, requires validate to be True.
  • write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
  • cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
  • cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
  • _refs – A list of processed references used in cascading saves
  • save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
  • signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

id

A field wrapper around MongoDB’s ObjectIds.

name

A unicode string field.

objects = QuerySet
owner

A reference to a document that will be automatically dereferenced on access (lazily).

Use the reverse_delete_rule to handle what should happen if the document the field is referencing is deleted. EmbeddedDocuments, DictFields and MapFields does not support reverse_delete_rule and an InvalidDocumentError will be raised if trying to set on one of these Document / Field types.

The options are:

  • DO_NOTHING (0) - don’t do anything (default).
  • NULLIFY (1) - Updates the reference to null.
  • CASCADE (2) - Deletes the documents associated with the reference.
  • DENY (3) - Prevent the deletion of the reference object.
  • PULL (4) - Pull the reference from a ListField of references

Alternative syntax for registering delete rules (useful when implementing bi-directional delete rules)

class Bar(Document):
    content = StringField()
    foo = ReferenceField('Foo')

Foo.register_delete_rule(Bar, 'foo', NULLIFY)
teams

A ListField designed specially to hold a list of embedded documents to provide additional query helpers.

Note

The only valid list values are subclasses of EmbeddedDocument.

New in version 0.9.

users
uuid

A UUID field.

New in version 0.6.

class toxicbuild.master.users.ResetUserPasswordToken(*args, **kwargs)[source]

Bases: mongomotor.document.Document

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine classmethod create(user)[source]

Creates a new reset token for a given user.

Parameters:user – An instance of User
coroutine send_reset_email(subject, msg)[source]

Sends an email with information on how to reset a password.

Parameters:
  • recipient – The recipient email address.
  • subject – The subject of the e-mail.
  • msg – The body of the email. If you have a {token} field in the body, the expire token will be included. It usually is is inserted in the reset url.
TOKEN_LEN = 64
email

A unicode string field.

expires

Datetime field.

Uses the python-dateutil library if available alternatively use time.strptime to parse the dates. Note: python-dateutil’s parser is fully featured and when installed you can utilise it to convert varying types of date formats into valid python datetime objects.

Note: To default the field to the current datetime, use: DateTimeField(default=datetime.utcnow)

Note: Microseconds are rounded to the nearest millisecond.
Pre UTC microsecond support is effectively broken. Use ComplexDateTimeField if you need accurate microsecond support.
id

A field wrapper around MongoDB’s ObjectIds.

objects = QuerySet
token

A unicode string field.

user

A reference to a document that will be automatically dereferenced on access (lazily).

Use the reverse_delete_rule to handle what should happen if the document the field is referencing is deleted. EmbeddedDocuments, DictFields and MapFields does not support reverse_delete_rule and an InvalidDocumentError will be raised if trying to set on one of these Document / Field types.

The options are:

  • DO_NOTHING (0) - don’t do anything (default).
  • NULLIFY (1) - Updates the reference to null.
  • CASCADE (2) - Deletes the documents associated with the reference.
  • DENY (3) - Prevent the deletion of the reference object.
  • PULL (4) - Pull the reference from a ListField of references

Alternative syntax for registering delete rules (useful when implementing bi-directional delete rules)

class Bar(Document):
    content = StringField()
    foo = ReferenceField('Foo')

Foo.register_delete_rule(Bar, 'foo', NULLIFY)
valid

Boolean field type.

New in version 0.1.2.

class toxicbuild.master.users.Team(*args, **kwargs)[source]

Bases: mongomotor.document.EmbeddedDocument

members
name

A unicode string field.

uuid

A UUID field.

New in version 0.6.

class toxicbuild.master.users.User(*args, **kwargs)[source]

Bases: mongomotor.document.Document

exception DoesNotExist

Bases: mongoengine.errors.DoesNotExist

exception MultipleObjectsReturned

Bases: mongoengine.errors.MultipleObjectsReturned

coroutine classmethod authenticate(username_or_email, password)[source]

Authenticates an user. Returns an user if the user is authenticated. Raises InvalidCredentials if a user with this credentials does not exist.

Parameters:
  • username_or_email – Username or email to use to authenticate.
  • password – Not encrypted password.
coroutine delete(*args, **kwargs)[source]

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parm signal_kwargs:
 (optional) kwargs dictionary to be passed to the signal calls.
Parameters:write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
coroutine save(*args, **kwargs)[source]

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:
  • force_insert – only try to create a new document, don’t allow updates of existing documents.
  • validate – validates the document; set to False to skip.
  • clean – call the document clean method, requires validate to be True.
  • write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
  • cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
  • cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
  • _refs – A list of processed references used in cascading saves
  • save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
  • signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

set_password(password)[source]
to_dict()[source]
allowed_actions
email

A field that validates input as an email address.

New in version 0.4.

id

A field wrapper around MongoDB’s ObjectIds.

is_superuser

Boolean field type.

New in version 0.1.2.

member_of
name
objects = QuerySet
organizations
password

A unicode string field.

username

A unicode string field.

uuid

A UUID field.

New in version 0.6.

toxicbuild.master.utils module

class toxicbuild.master.utils.PrettyFieldMixin(*args, **kwargs)[source]

Bases: object

A field with a descriptive name for humans

class toxicbuild.master.utils.PrettyListField(*args, **kwargs)[source]

Bases: toxicbuild.master.utils.PrettyFieldMixin, mongomotor.fields.ListField

class toxicbuild.master.utils.PrettyStringField(*args, **kwargs)[source]

Bases: toxicbuild.master.utils.PrettyFieldMixin, mongoengine.fields.StringField

class toxicbuild.master.utils.PrettyURLField(*args, **kwargs)[source]

Bases: toxicbuild.master.utils.PrettyFieldMixin, mongoengine.fields.URLField

toxicbuild.master.utils.get_build_config_filename()[source]

Returns the build config filename.

toxicbuild.master.utils.get_build_config_type()[source]

Returns the build config type.

toxicbuild.master.utils.send_email(recipients, subject, message)[source]

Sends an email using the output’s web api

Parameters:
  • recipients – A list of email addresses.
  • subject – The email’s subject.
  • message – The email’s body.

Module contents

toxicbuild.master.add_slave(configfile, name, host, port, token, owner, use_ssl=False, validate_cert=False)[source]

Adds a new slave to the master installation.

Parameters:
  • name – The slave name
  • host – The slave host.
  • port – The slave port.
  • token – The access token to the slave
  • owner – The id of the slave owner.
  • use_ssl – Does the slave use secure connection?
  • validate_cert – Should the slave ssl certificate be validated?
toxicbuild.master.create(root_dir, notifications_token='', no_token=False)[source]

Creates a new toxicmaster environment.

Parameters:
  • --root_dir – Root directory for toxicmaster.
  • --notifications-token – The auth token for the output web api.
  • --no-token – Should we create a access token?
toxicbuild.master.create_scheduler()[source]
toxicbuild.master.create_settings_and_connect()[source]
toxicbuild.master.create_token(conffile, show_encrypted=False)[source]

Creates the access token to the master.

Parameters:
  • conffile – The path for the toxicmaster.conf
  • --show-encrypted – Show the encrypted token?
toxicbuild.master.create_user(configfile, email=None, password=None, superuser=False, _limited=False)[source]

Creates a superuser in the master.

Parameters:
  • --email – User’s email.
  • --password – Password for authentication.
  • --superuser – Indicates if the user is a super user. Defaults to False
toxicbuild.master.ensure_indexes(*classes)[source]
toxicbuild.master.poller_server_init(server)[source]

Starts a poller server.

toxicbuild.master.restart(workdir, pidfile='toxicmaster.pid', loglevel='info')[source]

Restarts toxicmaster

The instance of toxicmaster in workdir will be restarted. :param workdir: Workdir for master to be killed. :param –pidfile: Name of the file to use as pidfile. Defaults to toxicmaster.pid :param –loglevel: Level for logging messages. Defaults to info.

toxicbuild.master.restart_poller(workdir, pidfile='toxicpoller.pid', loglevel='info')[source]

Restarts toxicmaster poller. The instance of toxicmaster poller in workdir will be restarted.

Parameters:
  • workdir – Workdir for the poller to be killed.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicpoller.pid
  • --loglevel – Level for logging messages. Defaults to info.
toxicbuild.master.restart_scheduler(workdir, pidfile='toxicscheduler.pid', loglevel='info')[source]

Restarts toxicmaster scheduler. The instance of toxicmaster scheduler in workdir will be restarted.

Parameters:
  • workdir – Workdir for the scheduler to be killed.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicscheduler.pid
  • --loglevel – Level for logging messages. Defaults to info.
toxicbuild.master.run(loglevel)[source]

Runs Toxicbuild master

toxicbuild.master.run_poller(loglevel)[source]

Runs a poller server for toxicbuild master.

toxicbuild.master.run_scheduler(loglevel)[source]

Runs a scheduler server for toxicbuild master.

toxicbuild.master.scheduler_server_init(server)[source]

Starts the scheduler server

toxicbuild.master.start(workdir, daemonize=False, stdout='toxicmaster.log', stderr='toxicmaster.log', conffile=None, loglevel='info', pidfile='toxicmaster.pid', no_scheduler=False, no_poller=False)[source]

Starts toxicmaster.

Parameters:
  • workdir – Work directory for server.
  • --daemonize – Run as daemon. Defaults to False
  • --stdout – stdout path. Defaults to /dev/null
  • --stderr – stderr path. Defaults to /dev/null
  • --conffile (-c,) – path to config file. Defaults to None. If not conffile, will look for a file called toxicmaster.conf inside workdir
  • --loglevel – Level for logging messages. Defaults to info.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicmaster.pid
toxicbuild.master.start_poller(workdir, daemonize=False, stdout='toxicpoller.log', stderr='toxicpoller.log', conffile=None, loglevel='info', pidfile='toxicpoller.pid')[source]

Starts a poller server.

Parameters:
  • workdir – Work directory for the server.
  • --daemonize – Run as daemon. Defaults to False
  • --stdout – stdout path. Defaults to /dev/null
  • --stderr – stderr path. Defaults to /dev/null
  • --conffile (-c,) – path to config file. Defaults to None. If not conffile, will look for a file called toxicmaster.conf inside workdir
  • --loglevel – Level for logging messages. Defaults to info.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicslave.pid
toxicbuild.master.start_scheduler(workdir, daemonize=False, stdout='toxicscheduler.log', stderr='toxicscheduler.log', conffile=None, loglevel='info', pidfile='toxicscheduler.pid')[source]

Starts a scheduler server.

Parameters:
  • workdir – Work directory for the server.
  • --daemonize – Run as daemon. Defaults to False
  • --stdout – stdout path. Defaults to /dev/null
  • --stderr – stderr path. Defaults to /dev/null
  • --conffile (-c,) – path to config file. Defaults to None. If not conffile, will look for a file called toxicmaster.conf inside workdir
  • --loglevel – Level for logging messages. Defaults to info.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicslave.pid
toxicbuild.master.stop(workdir, pidfile='toxicmaster.pid', kill=False)[source]

Kills toxicmaster.

Parameters:
  • --workdir – Workdir for master to be killed. Looks for a file toxicmaster.pid inside workdir.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicmaster.pid
  • kill – If true, send signum 9, otherwise, 15.
toxicbuild.master.stop_poller(workdir, pidfile='toxicpoller.pid', kill=False)[source]

Kills toxicmaster poller.

Parameters:
  • --workdir – Workdir for master to be killed. Looks for a file toxicmaster.pid inside workdir.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicpoller.pid
  • kill – If true, send signum 9, otherwise, 15.
toxicbuild.master.stop_scheduler(workdir, pidfile='toxicscheduler.pid', kill=False)[source]

Kills toxicmaster scheduler.

Parameters:
  • --workdir – Workdir for master to be killed. Looks for a file toxicmaster.pid inside workdir.
  • --pidfile – Name of the file to use as pidfile. Defaults to toxicscheduler.pid.
  • kill – If true, send signum 9, otherwise, 15.
toxicbuild.master.toxicinit(server)[source]

Initialize services.