Class Condor
- java.lang.Object
-
- edu.isi.pegasus.planner.transfer.sls.Condor
-
-
Field Summary
Fields Modifier and Type Field Description static java.lang.String
DESCRIPTION
A short description of the transfer implementation.private boolean
mBypassStagingForInputs
A boolean indicating whether to bypass first level staging for inputsprotected LogManager
mLogger
The handle to the logging manager.private PlannerCache
mPlannerCache
A SimpleFile Replica Catalog, that tracks all the files that are being materialized as part of workflow execution.protected PlannerOptions
mPOptions
The handle to the planner options.protected PegasusProperties
mProps
The handle to the properties.protected SiteStore
mSiteStore
The handle to the site catalog.
-
Constructor Summary
Constructors Constructor Description Condor()
The default constructor.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.util.Collection<FileTransfer>
determineSLSInputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Generates a second level staging file of the input files to the worker node directory.java.util.Collection<FileTransfer>
determineSLSOutputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Generates a second level staging file of the input files to the worker node directory.boolean
doesCondorModifications()
Returns a boolean whether the SLS implementation does a condor based modification or not.java.lang.String
getSLSInputLFN(Job job)
Returns the LFN of sls input file.java.lang.String
getSLSOutputLFN(Job job)
Returns the LFN of sls output file.void
initialize(PegasusBag bag)
Initializes the SLS implementation.java.lang.String
invocationString(Job job, java.io.File slsFile)
Constructs a command line invocation for a job, with a given sls file.boolean
modifyJobForFirstLevelStaging(Job job, java.lang.String submitDir, java.lang.String slsInputLFN, java.lang.String slsOutputLFN)
Modifies a job for the first level staging to headnode.This is to add any files that needs to be staged to the head node for a job specific to the SLS implementation.boolean
modifyJobForWorkerNodeExecution(Job job, java.lang.String stagingSiteURLPrefix, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Modifies a compute job for second level staging.boolean
needsSLSInputTransfers(Job job)
Returns a boolean indicating whether it will an input file for a job to do the transfers.boolean
needsSLSOutputTransfers(Job job)
Returns a boolean indicating whether it will an output file for a job to do the transfers.
-
-
-
Field Detail
-
DESCRIPTION
public static final java.lang.String DESCRIPTION
A short description of the transfer implementation.- See Also:
- Constant Field Values
-
mSiteStore
protected SiteStore mSiteStore
The handle to the site catalog.
-
mProps
protected PegasusProperties mProps
The handle to the properties.
-
mPOptions
protected PlannerOptions mPOptions
The handle to the planner options.
-
mLogger
protected LogManager mLogger
The handle to the logging manager.
-
mBypassStagingForInputs
private boolean mBypassStagingForInputs
A boolean indicating whether to bypass first level staging for inputs
-
mPlannerCache
private PlannerCache mPlannerCache
A SimpleFile Replica Catalog, that tracks all the files that are being materialized as part of workflow execution.
-
-
Method Detail
-
initialize
public void initialize(PegasusBag bag)
Initializes the SLS implementation.- Specified by:
initialize
in interfaceSLS
- Parameters:
bag
- the bag of objects. Contains access to catalogs etc.
-
doesCondorModifications
public boolean doesCondorModifications()
Returns a boolean whether the SLS implementation does a condor based modification or not. By condor based modification we mean whether it uses condor specific classads to achieve the second level staging or not.- Specified by:
doesCondorModifications
in interfaceSLS
- Returns:
- false
-
invocationString
public java.lang.String invocationString(Job job, java.io.File slsFile)
Constructs a command line invocation for a job, with a given sls file. The SLS maybe null. In the case where SLS impl does not read from a file, it is advised to create a file in generateSLSXXX methods, and then read the file in this function and put it on the command line.- Specified by:
invocationString
in interfaceSLS
- Parameters:
job
- the job that is being sls enabledslsFile
- the slsFile can be null- Returns:
- invocation string
-
needsSLSInputTransfers
public boolean needsSLSInputTransfers(Job job)
Returns a boolean indicating whether it will an input file for a job to do the transfers. Transfer reads from stdin the file transfers that it needs to do. Always returns true, as we need to transfer the proxy always.- Specified by:
needsSLSInputTransfers
in interfaceSLS
- Parameters:
job
- the job being detected.- Returns:
- false
-
needsSLSOutputTransfers
public boolean needsSLSOutputTransfers(Job job)
Returns a boolean indicating whether it will an output file for a job to do the transfers. Transfer reads from stdin the file transfers that it needs to do.- Specified by:
needsSLSOutputTransfers
in interfaceSLS
- Parameters:
job
- the job being detected.- Returns:
- false
-
getSLSInputLFN
public java.lang.String getSLSInputLFN(Job job)
Returns the LFN of sls input file.- Specified by:
getSLSInputLFN
in interfaceSLS
- Parameters:
job
- Job- Returns:
- the name of the sls input file.
-
getSLSOutputLFN
public java.lang.String getSLSOutputLFN(Job job)
Returns the LFN of sls output file.- Specified by:
getSLSOutputLFN
in interfaceSLS
- Parameters:
job
- Job- Returns:
- the name of the sls input file.
-
determineSLSInputTransfers
public java.util.Collection<FileTransfer> determineSLSInputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Generates a second level staging file of the input files to the worker node directory.- Specified by:
determineSLSInputTransfers
in interfaceSLS
- Parameters:
job
- job for which the file is being createdfileName
- name of the file that needs to be written out.stagingSiteServer
- the file server on the staging site to be used for retrieval of files i.e the get operationstagingSiteDirectory
- directory on the head node of the compute site.workerNodeDirectory
- worker node directory- Returns:
- a Collection of FileTransfer objects listing the transfers that need to be done.
- See Also:
needsSLSInputTransfers( Job)
-
determineSLSOutputTransfers
public java.util.Collection<FileTransfer> determineSLSOutputTransfers(Job job, java.lang.String fileName, FileServer stagingSiteServer, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Generates a second level staging file of the input files to the worker node directory.- Specified by:
determineSLSOutputTransfers
in interfaceSLS
- Parameters:
job
- the job for which the file is being createdfileName
- the name of the file that needs to be written out.stagingSiteServer
- the file server on the staging site to be used for retrieval of files i.e the put operationstagingSiteDirectory
- the directory on the head node of the compute site.workerNodeDirectory
- the worker node directory- Returns:
- a Collection of FileTransfer objects listing the transfers that need to be done.
- See Also:
needsSLSOutputTransfers( Job)
-
modifyJobForFirstLevelStaging
public boolean modifyJobForFirstLevelStaging(Job job, java.lang.String submitDir, java.lang.String slsInputLFN, java.lang.String slsOutputLFN)
Modifies a job for the first level staging to headnode.This is to add any files that needs to be staged to the head node for a job specific to the SLS implementation. If any file needs to be added, aFileTransfer
object should be created and added as an input or an output file.- Parameters:
job
- the jobsubmitDir
- the submit directoryslsInputLFN
- the sls input file if required, that is used for staging in from the head node to worker node directory.slsOutputLFN
- the sls output file if required, that is used for staging in from the head node to worker node directory.- Returns:
- boolean
-
modifyJobForWorkerNodeExecution
public boolean modifyJobForWorkerNodeExecution(Job job, java.lang.String stagingSiteURLPrefix, java.lang.String stagingSiteDirectory, java.lang.String workerNodeDirectory)
Modifies a compute job for second level staging. Adds the appropriate condor classads. It assumes that all the files are being moved to and from the submit directory directly. Ignores any headnode parameters passed.- Specified by:
modifyJobForWorkerNodeExecution
in interfaceSLS
- Parameters:
job
- the job to be modified.stagingSiteURLPrefix
- the url prefix for the server on the staging sitestagingSitedirectory
- the directory on the staging site, where the nput data is read from and the output data written out.workerNodeDirectory
- the directory in the worker node tmp- Returns:
- boolean indicating whether job was successfully modified or not.
-
-