HTCondor Pool Management

class ornithology.Condor(local_dir, config=None, raw_config=None, clean_local_dir_before=True, submit_user=None, condor_user=None, use_sudo=None)[source]

A Condor is responsible for managing the lifecycle of an HTCondor pool.

Parameters:

  • local_dir (Path) – The local directory for the HTCondor pool. All HTCondor state will be stored in this directory.

  • config (Optional[Mapping[str, str]]) – HTCondor configuration parameters to inject, as a mapping of key-value pairs.

  • raw_config (Optional[str]) – Raw HTCondor configuration language to inject, as a string.

  • clean_local_dir_before (bool) – If True, any existing directory at local_dir will be removed before trying to stand up the new instance.

  • submit_user (str) – If set, the user to switch to when submitting a job.

  • condor_user (str) – If set, the user that HTCondor will run as.

  • use_sudo (Optional[bool]) – If True, start condor_master with sudo. If None (default), checks HTCONDOR_TEST_USE_SUDO environment variable. If False, never use sudo even if environment variable is set.

act(action, constraint='true')[source]

Perform a job action against the pool’s schedd.

Parameters:

  • action

  • constraint

property collector_log: DaemonLog

A DaemonLog for the pool’s collector.

property credmon_oauth_log: DaemonLog

A DaemonLog for the OAuth CredMon.

direct_status(daemon_type, ad_type, constraint='true', projection=None)[source]

Perform a direct status query against a daemon.

Parameters:

  • daemon_type

  • ad_type

  • constraint

  • projection

edit(attr, value, constraint='true')[source]

Perform a job attribute edit action against the pool’s schedd.

Parameters:

  • attr

  • value

  • constraint

get_local_collector()[source]

Return the htcondor.Collector for this pool’s collector.

get_local_schedd()[source]

Return the htcondor.Schedd for this pool’s schedd.

property job_queue_log: Path

The pathlib.Path to the pool’s job queue log.

property master_log: DaemonLog

A DaemonLog for the pool’s master.

property negotiator_log: DaemonLog

A DaemonLog for the pool’s negotiator.

query(constraint='true', projection=None, limit=-1, opts=<QueryOpt.Default: 0>)[source]

Perform a job information query against the pool’s schedd.

Parameters:

  • constraint

  • projection

  • limit

  • opts

run_command(*args, **kwargs)[source]

Execute a command with CONDOR_CONFIG set to point to this HTCondor pool. Arguments and keyword arguments are passed through to run_command().

Return type:

CompletedProcess

property schedd_log: DaemonLog

A DaemonLog for the pool’s schedd.

property shadow_log: DaemonLog

A DaemonLog for the pool’s shadows.

property startd_address

The address of the pool’s startd.

property startd_log: DaemonLog

A DaemonLog for the pool’s startd.

status(ad_type=AdType.Any, constraint='true', projection=None)[source]

Perform a status query against the pool’s collector.

Parameters:

  • ad_type

  • constraint

  • projection

submit(description, count=1, itemdata=None)[source]

Submit jobs to the pool.

Internally, this method uses the Python bindings to submit jobs and returns a rich handle object (a ClusterHandle) that can be used to inspect and manage the jobs.

Warning

If your intent is to test job submission itself, DO NOT use this method. Instead, submit jobs by calling condor_q with Condor.run_command(), using the Python bindings directly, or whatever other method you want to test.

Parameters:
Return type:

ClusterHandle

submit_dag(dagfile)[source]

Submit a DAG.

Parameters:

dagfile (Path)

Return type:

ClusterHandle

use_config()[source]

Returns a context manager that sets CONDOR_CONFIG to point to the config file for this HTCondor pool.