Job Handles

Handles

class ornithology.Handle(condor)[source]

A connection to a set of jobs defined by a constraint. The handle can be used to query, act on, or edit those jobs.

edit(attr, value)[source]

Edit attributes of jobs.

Warning

Many attribute edits will not affect jobs that have already matched. For example, changing RequestMemory will not affect the memory allocation of a job that is already executing. In that case, you would need to vacate (or release the job if it was held) before the edit had the desired effect.

Parameters:

attr – The attribute to edit. Case-insensitive.
value – The new value for the attribute.

Returns:

ad – An ad describing the results of the edit.

Return type:

classad.ClassAd

hold()[source]

Hold jobs.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

pause()[source]

Pause jobs. Jobs will stop running, but will hold on to their claimed resources.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

query(projection=None, options=htcondor.htcondor.QueryOpts.Default, limit=-1)[source]

Query against this set of jobs.

Parameters:

projection – The classad.ClassAd attributes to retrieve, as a list of case-insensitive strings. If None (the default), all attributes will be returned.
options –
limit – The total number of matches to return from the query. If None (the default), return all matches.

Returns:

ads – An iterator over the classad.ClassAd that match the constraint.

Return type:

Iterator[classad.ClassAd]

release()[source]

Release held jobs. They will return to the queue in the idle state.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

remove()[source]

Remove jobs from the queue.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

resume()[source]

Resume (un-pause) jobs.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

vacate()[source]

Vacate running jobs. This will force them off of their current execute resource, causing them to become idle again.

Returns:: ad – An ad describing the results of the action.
Return type:: classad.ClassAd

class ornithology.ConstraintHandle(condor, constraint)[source]

A connection to a set of jobs defined by an ConstraintHandle.constraint. The handle can be used to query, act on, or edit those jobs.

property constraint: ExprTree: The constraint that defines this ConstraintHandle, as an classad.ExprTree.

property constraint_string: str: The constraint that defines this ConstraintHandle, as a string.

class ornithology.ClusterHandle(condor, submit_result)[source]

A subclass of ConstraintHandle that targets a single cluster of jobs, as produced by Condor.submit().

Because this handle targets a single cluster of jobs, it has superpowers. If the cluster has an event log (log = <path> in the submit description, see the docs), this handle’s state attribute will be a ClusterState that provides information about the current state of the jobs in the cluster.

Warning

You shouldn’t have to construct a ClusterHandle yourself. Instead, use the ones returned by Condor.submit().

property clusterad: The cluster’s cluster ad.

property clusterid: The cluster’s cluster ID.

property event_log: The EventLog for this ClusterHandle.

property first_proc: The process ID of the first job in the cluster.

property job_ids: List[JobID]: Return the list of JobID in this ClusterHandle.

property num_procs: The number of jobs in the cluster.

property state: A ClusterState that provides information about job state for this cluster.

wait(condition=None, fail_condition=None, timeout=120, verbose=False)[source]

Waits for the condition to become True.

Parameters:

condition (Optional[Callable[[ClusterState], bool]]) – The function to wait to become True. It will be passed the ClusterState as its only argument. Because of how Python calls unbound class methods, you may directly pass ClusterState methods as conditions (e.g., handle.wait(condition = ClusterState.any_held)). The default condition is ClusterState.all_complete(), which means “wait until all the jobs in this cluster are completed”.
fail_condition (Optional[Callable[[ClusterState], bool]]) – If this function becomes True, wait will immediately return False. Use this to avoid waiting for a long time when a test is failing.
timeout (int) – After this amount of time, wait will return False and emit a warning in the log.
verbose (bool) – If True, the handle’s state counts will be logged during the wait.

Returns:

success – True if the wait finished because the condition became True; False otherwise.

Return type:

bool

State Tracking

class ornithology.EventLog(handle)[source]

This class represents the job event log for a ClusterHandle.

Warning

You shouldn’t have to construct this yourself. Instead, use ClusterHandle.event_log.

filter(condition)[source]

Return a list containing the job events that the condition is True for.

Return type:: List[JobEvent]

read_events()[source]

Yield all un-read events in the event log.

Return type:: Iterator[JobEvent]

class ornithology.ClusterState(handle)[source]

A class that manages the state of the cluster tracked by a ClusterHandle. It reads from the cluster’s event log internally and provides a variety of views of the individual job states.

Warning

ClusterState objects should not be instantiated manually. ClusterHandle will create them automatically when needed.

all_complete()[source]

Return True if all of the jobs in the cluster are complete. Note that this definition does include jobs that have left the queue, not just ones that are in the “Completed” state in the queue.

Return type:: bool

all_held()[source]

Return True if all of the jobs in the cluster are held.

Return type:: bool

all_idle()[source]

Return True if all of the jobs in the cluster are held.

Return type:: bool

all_running()[source]

Return True if all of the jobs in the cluster are running.

Return type:: bool

all_status(*statuses)[source]

Return True if all of the jobs in the cluster are in one of the statuses. Prefer one of the explicitly-named helper methods when possible, and don’t be afraid to make a new helper method!

Return type:: bool

all_terminal()[source]

Return True if all of the jobs in the cluster are completed, held, or removed.

Return type:: bool

any_complete()[source]

Return True if any of the jobs in the cluster are complete. Note that this definition does include jobs that have left the queue, not just ones that are in the “Completed” state in the queue.

Return type:: bool

any_held()[source]

Return True if any of the jobs in the cluster are held.

Return type:: bool

any_idle()[source]

Return True if any of the jobs in the cluster are idle.

Return type:: bool

any_running()[source]

Return True if any of the jobs in the cluster are running.

Return type:: bool

any_status(*statuses)[source]

Return True if any of the jobs in the cluster are in one of the statuses. Prefer one of the explicitly-named helper methods when possible, and don’t be afraid to make a new helper method!

Return type:: bool

any_terminal()[source]

Return True if any of the jobs in the cluster are completed, held, or removed.

Return type:: bool

count_status(*statuses)[source]

Return the total number of jobs in the cluster in any of the given statuses.

Return type:: int

counts()[source]: Return the number of jobs in each JobStatus, as a collections.Counter.

none_held()[source]

Return True if none of the jobs in the cluster are held.

Return type:: bool

none_idle()[source]

Return True if none of the jobs in the cluster are idle.

Return type:: bool

none_status(*statuses)[source]

Return True if none of the jobs in the cluster are in one of the statuses. Prefer one of the explicitly-named helper methods when possible, and don’t be afraid to make a new helper method!

Return type:: bool

static running_exactly(count)[source]

Returns True if count of the jobs in the cluster are running.

Return type:: bool

status_exactly(count, *statuses)[source]

Return True if exactly count of the jobs in the cluster are in one of the statuses. Prefer one of the explicitly-named helper methods when possible, and don’t be afraid to make a new helper method!

Return type:: bool