Class Condor


  • public class Condor
    extends Namespace
    This helper class helps in handling the arguments specified in the Condor namespace by the user either through dax or through profiles in pool.
    Version:
    $Revision$
    Author:
    Karan Vahi
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String ARGUMENTS_KEY
      The name of the key that denotes the arguments of the job.
      static java.lang.String BATCH_QUEUE_KEY
      The queue to be used when using batch gahp.
      static java.lang.String COLLECTOR_KEY
      The key that designates the collector associated with the job.
      static java.lang.String EXECUTABLE_KEY
      The name of the key that denotes the executable of the job.
      static java.lang.String FILE_SYSTEM_DOMAIN_KEY
      The name of the key that denotes the File System Domain.
      static java.lang.String GET_ENV_KEY
      The condor key for using the local environment
      static java.lang.String GLOBUS_UNIVERSE
      The condor universe key value for vanilla universe.
      static java.lang.String GRID_JOB_TYPE_KEY
      The name of the key that specifies the grid job type.
      static java.lang.String GRID_RESOURCE_KEY
      The condor key for designation the grid_resource.
      static java.lang.String GRID_UNIVERSE
      The condor universe key value for grid universe.
      static java.lang.String JOBMANAGER_TYPE_KEY
      The name of the key that specifies the jobmanager type.
      static java.lang.String LOCAL_UNIVERSE
      The condor universe key value for local universe.
      protected java.lang.String mNamespace
      The name of the implementing namespace.
      static java.lang.String NAMESPACE_NAME
      The name of the namespace that this class implements.
      static java.lang.String PARALLEL_UNIVERSE
      The condor universe key value for parallel universe.
      static java.lang.String PERIODIC_RELEASE_KEY
      The name of the key that specifies the peridic release
      static java.lang.String PERIODIC_REMOVE_KEY
      The name of the key that specifies the peridic remove
      static java.lang.String PRIORITY_KEY
      The name of the key that specifies the priority for the job.
      static java.lang.String REMOTE_SHOULD_TRANSFER_FILES_KEY
      The corresponding remote kye name that designates that files should be transferred via Condor File Transfer mechanism.
      static java.lang.String REMOTE_UNIVERSE_KEY
      The name of the key that denotes the remote condor universe key.
      static java.lang.String REMOTE_WHEN_TO_TRANSFER_OUTPUT_KEY
      The corresponding name of the remote key that designated when to transfer output.
      static java.lang.String REQUEST_CPUS_KEY
      The Condor Key designating the numnber of cpu's to request.
      static java.lang.String REQUEST_DISK_KEY
      The Condor Key designating the amount of disk to request.
      static java.lang.String REQUEST_MEMORY_KEY
      The Condor Key designating the amount of memory to request.
      static java.lang.String REQUIREMENTS_KEY
      The name of the key that denotes the requirements of the job.
      static java.lang.String SCHEDULER_UNIVERSE
      The condor universe key value for scheduler universe.
      static java.lang.String SHOULD_TRANSFER_FILES_KEY
      The name of the key that designates that files should be transferred via Condor File Transfer mechanism.
      static java.lang.String STANDARD_UNIVERSE
      The condor universe key value for standard universe.
      static java.lang.String STREAM_STDERR_KEY
      The name of the key that specifies whether to stream stderr or not
      static java.lang.String STREAM_STDOUT_KEY
      The name of the key that specifies whether to stream stderr or not
      static java.lang.String TRANSFER_EXECUTABLE_KEY
      The name of the key that specifies transfer of executable
      static java.lang.String TRANSFER_IP_FILES_KEY
      The name of the key that specifies transfer of input files.
      static java.lang.String TRANSFER_OP_FILES_KEY
      The name of the key that specifies transfer of input files.
      static java.lang.String UNIVERSE_KEY
      The name of the key that denotes the condor universe key.
      static java.lang.String VANILLA_UNIVERSE
      The condor universe key value for vanilla universe.
      static java.lang.String WHEN_TO_TRANSFER_OUTPUT_KEY
      The name of key that designates when to transfer output.
      static java.lang.String X509USERPROXY_KEY
      The key that overrides the default x509 proxy location.
    • Constructor Summary

      Constructors 
      Constructor Description
      Condor()
      The default constructor.
      Condor​(java.util.Map mp)
      The overloaded constructor
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addFilesForTransfer​(java.lang.String file, java.lang.String key)
      Adds an input file that is to be transferred from the submit host via the Condor File Transfer Mechanism.
      void addFilesForTransfer​(java.util.Collection<java.lang.String> files, java.lang.String key)
      Adds multiple files that are to be transferred from the submit host via the Condor File Transfer Mechanism.
      void addIPFileForTransfer​(java.lang.String file)
      Adds an input file that is to be transferred from the submit host via the Condor File Transfer Mechanism.
      void addIPFileForTransfer​(java.util.Collection<java.lang.String> files)
      Adds multiple files that are to be transferred from the submit host via the Condor File Transfer Mechanism.
      void addOPFileForTransfer​(java.lang.String file)
      Adds an output file that is to be transferred from the submit host via the Condor File Transfer Mechanism.
      void addOPFileForTransfer​(java.util.Collection<java.lang.String> files)
      Adds multiple output files that are to be transferred from the submit host via the Condor File Transfer Mechanism.
      int checkKey​(java.lang.String key, java.lang.String value)
      Additional method to handle the Condor namespace with convenience mappings.
      void checkKeyInNS​(PegasusProperties properties, java.lang.String pool)
      It puts in the namespace specific information specified in the properties file into the namespace.
      void checkKeyInNS​(java.lang.String key, java.lang.String value)
      This checks the whether a key value pair specified is valid in the current namespace or not by calling the checkKey function and then on the basis of the values returned puts them into the associated map in the class.
      java.lang.Object clone()
      Returns a copy of the current namespace object.
      void construct​(java.lang.String key, java.lang.String value)
      Constructs a new element of the format (key=value).
      boolean getBooleanValue​(java.lang.Object key)
      Returns a boolean value, that a particular key is mapped to in this namespace.
      java.lang.String getIPFilesForTransfer()
      Returns a comma separated list of files that are designated for transfer via condor file transfer mechanism for the job.
      java.lang.String getOutputFilesForTransfer()
      Returns a comma separated list of files that are designated for transfer via condor file transfer mechanism for the job.
      boolean isInteger​(java.lang.String value)
      Returns a boolean value indicating if the string passed is an integer or not
      void merge​(Namespace profiles)
      Merge the profiles in the namespace in a controlled manner.
      java.lang.String namespaceName()
      Returns the name of the namespace associated with the profile implementations.
      void removeIPFilesForTransfer()
      Remove the input files that were designated for transfer using Condor File Transfer Mechanism.
      void removeOutputFilesForTransfer()
      Remove the output files that were designated for transfer using Condor File Transfer Mechanism.
      void setExecutableForTransfer()
      Adds the executable for transfer via the condor file transfer mechanism.
      java.lang.String toCondor()
      Converts the contents of the map into the string that can be put in the Condor file for printing.
      • Methods inherited from class java.lang.Object

        equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Field Detail

      • NAMESPACE_NAME

        public static final java.lang.String NAMESPACE_NAME
        The name of the namespace that this class implements.
        See Also:
        Constant Field Values
      • ARGUMENTS_KEY

        public static final java.lang.String ARGUMENTS_KEY
        The name of the key that denotes the arguments of the job.
        See Also:
        Constant Field Values
      • BATCH_QUEUE_KEY

        public static final java.lang.String BATCH_QUEUE_KEY
        The queue to be used when using batch gahp.
        See Also:
        Constant Field Values
      • EXECUTABLE_KEY

        public static final java.lang.String EXECUTABLE_KEY
        The name of the key that denotes the executable of the job.
        See Also:
        Constant Field Values
      • REQUIREMENTS_KEY

        public static final java.lang.String REQUIREMENTS_KEY
        The name of the key that denotes the requirements of the job.
        See Also:
        Constant Field Values
      • UNIVERSE_KEY

        public static final java.lang.String UNIVERSE_KEY
        The name of the key that denotes the condor universe key.
        See Also:
        Constant Field Values
      • REMOTE_UNIVERSE_KEY

        public static final java.lang.String REMOTE_UNIVERSE_KEY
        The name of the key that denotes the remote condor universe key.
        See Also:
        Constant Field Values
      • FILE_SYSTEM_DOMAIN_KEY

        public static final java.lang.String FILE_SYSTEM_DOMAIN_KEY
        The name of the key that denotes the File System Domain. Is actually propogated to the expression for the Requirements Key.
        See Also:
        REQUIREMENTS_KEY, Constant Field Values
      • GRID_JOB_TYPE_KEY

        public static final java.lang.String GRID_JOB_TYPE_KEY
        The name of the key that specifies the grid job type.
        See Also:
        Constant Field Values
      • JOBMANAGER_TYPE_KEY

        public static final java.lang.String JOBMANAGER_TYPE_KEY
        The name of the key that specifies the jobmanager type.
        See Also:
        Constant Field Values
      • SHOULD_TRANSFER_FILES_KEY

        public static final java.lang.String SHOULD_TRANSFER_FILES_KEY
        The name of the key that designates that files should be transferred via Condor File Transfer mechanism.
        See Also:
        Constant Field Values
      • REMOTE_SHOULD_TRANSFER_FILES_KEY

        public static final java.lang.String REMOTE_SHOULD_TRANSFER_FILES_KEY
        The corresponding remote kye name that designates that files should be transferred via Condor File Transfer mechanism.
        See Also:
        Constant Field Values
      • WHEN_TO_TRANSFER_OUTPUT_KEY

        public static final java.lang.String WHEN_TO_TRANSFER_OUTPUT_KEY
        The name of key that designates when to transfer output.
        See Also:
        Constant Field Values
      • REMOTE_WHEN_TO_TRANSFER_OUTPUT_KEY

        public static final java.lang.String REMOTE_WHEN_TO_TRANSFER_OUTPUT_KEY
        The corresponding name of the remote key that designated when to transfer output.
        See Also:
        Constant Field Values
      • STREAM_STDERR_KEY

        public static final java.lang.String STREAM_STDERR_KEY
        The name of the key that specifies whether to stream stderr or not
        See Also:
        Constant Field Values
      • STREAM_STDOUT_KEY

        public static final java.lang.String STREAM_STDOUT_KEY
        The name of the key that specifies whether to stream stderr or not
        See Also:
        Constant Field Values
      • TRANSFER_IP_FILES_KEY

        public static final java.lang.String TRANSFER_IP_FILES_KEY
        The name of the key that specifies transfer of input files.
        See Also:
        Constant Field Values
      • TRANSFER_OP_FILES_KEY

        public static final java.lang.String TRANSFER_OP_FILES_KEY
        The name of the key that specifies transfer of input files.
        See Also:
        Constant Field Values
      • TRANSFER_EXECUTABLE_KEY

        public static final java.lang.String TRANSFER_EXECUTABLE_KEY
        The name of the key that specifies transfer of executable
        See Also:
        Constant Field Values
      • PRIORITY_KEY

        public static final java.lang.String PRIORITY_KEY
        The name of the key that specifies the priority for the job.
        See Also:
        Constant Field Values
      • PERIODIC_RELEASE_KEY

        public static final java.lang.String PERIODIC_RELEASE_KEY
        The name of the key that specifies the peridic release
        See Also:
        Constant Field Values
      • PERIODIC_REMOVE_KEY

        public static final java.lang.String PERIODIC_REMOVE_KEY
        The name of the key that specifies the peridic remove
        See Also:
        Constant Field Values
      • GRID_RESOURCE_KEY

        public static final java.lang.String GRID_RESOURCE_KEY
        The condor key for designation the grid_resource.
        See Also:
        Constant Field Values
      • COLLECTOR_KEY

        public static final java.lang.String COLLECTOR_KEY
        The key that designates the collector associated with the job.
        See Also:
        Constant Field Values
      • X509USERPROXY_KEY

        public static final java.lang.String X509USERPROXY_KEY
        The key that overrides the default x509 proxy location.
        See Also:
        Constant Field Values
      • REQUEST_CPUS_KEY

        public static final java.lang.String REQUEST_CPUS_KEY
        The Condor Key designating the numnber of cpu's to request.
        See Also:
        Constant Field Values
      • REQUEST_MEMORY_KEY

        public static final java.lang.String REQUEST_MEMORY_KEY
        The Condor Key designating the amount of memory to request.
        See Also:
        Constant Field Values
      • REQUEST_DISK_KEY

        public static final java.lang.String REQUEST_DISK_KEY
        The Condor Key designating the amount of disk to request.
        See Also:
        Constant Field Values
      • VANILLA_UNIVERSE

        public static final java.lang.String VANILLA_UNIVERSE
        The condor universe key value for vanilla universe.
        See Also:
        Constant Field Values
      • GRID_UNIVERSE

        public static final java.lang.String GRID_UNIVERSE
        The condor universe key value for grid universe.
        See Also:
        Constant Field Values
      • GET_ENV_KEY

        public static final java.lang.String GET_ENV_KEY
        The condor key for using the local environment
        See Also:
        Constant Field Values
      • GLOBUS_UNIVERSE

        public static final java.lang.String GLOBUS_UNIVERSE
        The condor universe key value for vanilla universe.
        See Also:
        Constant Field Values
      • SCHEDULER_UNIVERSE

        public static final java.lang.String SCHEDULER_UNIVERSE
        The condor universe key value for scheduler universe.
        See Also:
        Constant Field Values
      • STANDARD_UNIVERSE

        public static final java.lang.String STANDARD_UNIVERSE
        The condor universe key value for standard universe.
        See Also:
        Constant Field Values
      • LOCAL_UNIVERSE

        public static final java.lang.String LOCAL_UNIVERSE
        The condor universe key value for local universe.
        See Also:
        Constant Field Values
      • PARALLEL_UNIVERSE

        public static final java.lang.String PARALLEL_UNIVERSE
        The condor universe key value for parallel universe.
        See Also:
        Constant Field Values
      • mNamespace

        protected java.lang.String mNamespace
        The name of the implementing namespace. It should be one of the valid namespaces always.
        See Also:
        Namespace.isNamespaceValid(String)
    • Constructor Detail

      • Condor

        public Condor()
        The default constructor.
      • Condor

        public Condor​(java.util.Map mp)
        The overloaded constructor
        Parameters:
        mp - map containing the profile keys.
    • Method Detail

      • namespaceName

        public java.lang.String namespaceName()
        Returns the name of the namespace associated with the profile implementations.
        Specified by:
        namespaceName in class Namespace
        Returns:
        the namespace name.
        See Also:
        NAMESPACE_NAME
      • getIPFilesForTransfer

        public java.lang.String getIPFilesForTransfer()
        Returns a comma separated list of files that are designated for transfer via condor file transfer mechanism for the job.
        Returns:
        a csv file else null
      • getOutputFilesForTransfer

        public java.lang.String getOutputFilesForTransfer()
        Returns a comma separated list of files that are designated for transfer via condor file transfer mechanism for the job.
        Returns:
        a csv file else null
      • removeIPFilesForTransfer

        public void removeIPFilesForTransfer()
        Remove the input files that were designated for transfer using Condor File Transfer Mechanism.
      • removeOutputFilesForTransfer

        public void removeOutputFilesForTransfer()
        Remove the output files that were designated for transfer using Condor File Transfer Mechanism.
      • setExecutableForTransfer

        public void setExecutableForTransfer()
        Adds the executable for transfer via the condor file transfer mechanism.
      • addIPFileForTransfer

        public void addIPFileForTransfer​(java.util.Collection<java.lang.String> files)
        Adds multiple files that are to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
      • addIPFileForTransfer

        public void addIPFileForTransfer​(java.lang.String file)
        Adds an input file that is to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
      • addOPFileForTransfer

        public void addOPFileForTransfer​(java.util.Collection<java.lang.String> files)
        Adds multiple output files that are to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
      • addOPFileForTransfer

        public void addOPFileForTransfer​(java.lang.String file)
        Adds an output file that is to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
      • addFilesForTransfer

        public void addFilesForTransfer​(java.util.Collection<java.lang.String> files,
                                        java.lang.String key)
        Adds multiple files that are to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
        key - the name of the Condor key to be added
      • addFilesForTransfer

        public void addFilesForTransfer​(java.lang.String file,
                                        java.lang.String key)
        Adds an input file that is to be transferred from the submit host via the Condor File Transfer Mechanism. It also sets the associated condor keys like when_to_transfer and should_transfer_files.
        Parameters:
        file - the path to the file on the submit host.
        key - the name of the Condor key
      • checkKey

        public int checkKey​(java.lang.String key,
                            java.lang.String value)
        Additional method to handle the Condor namespace with convenience mappings. Currently the following keys are not supported keys as they clash with Pegasus internals
         arguments      - not supported, got from the arguments tag in DAX
         batch_queue      - the batch queue to be used
         copy_to_spool    - supported, limited to LCG sites at present where one needs
                            to stage in the kickstart. Pegasus sets it to false by default
                            for arch start stuff on the local pool, unless the user
                            overrides it.
         environment    - not supported, use env namespace fpr this
         executable       - not supported, this is got from the transformation catalog
         FileSystemDomain - supported, but is propogated to the classad expression
                            for requirements.
         globusscheduler  - not supported, Pegasus determines this on the basis of
                            it's planning strategy
         globusrsl        - not supported, rsl to populated through Globus namespace.
         grid_type        - OK (like gt2, gt4, condor)
         grid_resource    - supported . used for glite
         getevn           - OK
         log              - not supported, as it has to be same for the whole dag
         notification     - OK
         noop_job         - OK (used for synchronizing jobs in graph)
         noop_job_exit_signal - OK
         noop_job_exit_code - OK
         periodic_release - OK
         periodic_remove  - OK
         priority         - OK
         queue          - required thing. always added
         remote_initialdir- not allowed, the working directory is picked up from
                            pool file and properties file
         request_cpus     - number of cpu's required. New in Condor 7.8.0
         request_memory   - amount of memory required . New in Condor 7.8.0
         request_disk     - amount of disk required. New in Condor 7.8.0.
         stream_error     -  supported,however it is applicable only for globus jobs.
                           
         stream_output    -  supported, however it is applicable only for globus jobs.
                            
         transfer_executable  - supported, limited to LCG sites at present where one needs
                                to stage in the kickstart.
         transfer_input_files - supported, especially used to transfer proxies in
                                case of glide in pools.
         universe         - supported, especially used to incorporate glide in pools.
         x509userpoxy     - supported, overrides x509 default proxy and proxy transfers in
                            for glideins and vanilla jobs
         +xxxx            - supported, this is used to add extra classads with the jobs.
         
        Specified by:
        checkKey in class Namespace
        Parameters:
        key - is the key within the globus namespace, must be lowercase!
        value - is the value for the given key.
        Returns:
        MALFORMED_KEY VALID_KEY UNKNOWN_KEY NOT_PERMITTED_KEY DEPRECATED_KEY EMPTY_KEY
      • checkKeyInNS

        public void checkKeyInNS​(PegasusProperties properties,
                                 java.lang.String pool)
        It puts in the namespace specific information specified in the properties file into the namespace. The name of the pool is also passed, as many of the properties specified in the properties file are on a per pool basis. It handles the periodic_remove and periodic_release characteristics for condor jobs.
        Specified by:
        checkKeyInNS in class Namespace
        Parameters:
        properties - the PegasusProperties object containing all the properties that the user specified at various places (like .chimerarc, properties file, command line).
        pool - the pool name where the job is scheduled to run.
      • isInteger

        public boolean isInteger​(java.lang.String value)
        Returns a boolean value indicating if the string passed is an integer or not
        Parameters:
        value - the value
        Returns:
        boolean
      • checkKeyInNS

        public void checkKeyInNS​(java.lang.String key,
                                 java.lang.String value)
        This checks the whether a key value pair specified is valid in the current namespace or not by calling the checkKey function and then on the basis of the values returned puts them into the associated map in the class. In addition it transfers the FILE_SYSTEM_DOMAIN_KEY to the REQUIREMENTS_KEY.
        Overrides:
        checkKeyInNS in class Namespace
        Parameters:
        key - key that needs to be checked in the namespace for validity.
        value - value of the key
      • merge

        public void merge​(Namespace profiles)
        Merge the profiles in the namespace in a controlled manner. In case of intersection, the new profile value overrides, the existing profile value.
        Specified by:
        merge in class Namespace
        Parameters:
        profiles - the Namespace object containing the profiles.
      • construct

        public void construct​(java.lang.String key,
                              java.lang.String value)
        Constructs a new element of the format (key=value). All the keys are converted to lower case before storing.
        Overrides:
        construct in class Namespace
        Parameters:
        key - is the left-hand-side
        value - is the right hand side
      • getBooleanValue

        public boolean getBooleanValue​(java.lang.Object key)
        Returns a boolean value, that a particular key is mapped to in this namespace. If the key is mapped to a non boolean value or the key is not populated in the namespace false is returned.
        Parameters:
        key - The key whose boolean value you desire.
        Returns:
        boolean
      • toCondor

        public java.lang.String toCondor()
        Converts the contents of the map into the string that can be put in the Condor file for printing.
        Specified by:
        toCondor in class Namespace
        Returns:
        the textual description
      • clone

        public java.lang.Object clone()
        Returns a copy of the current namespace object.
        Overrides:
        clone in class Namespace
        Returns:
        the Cloned object