Class HourGlass

  • All Implemented Interfaces:
    Strategy

    public class HourGlass
    extends AbstractStrategy
    This class inserts the nodes for creating the random directories on the remote execution pools. This is done when the resources have already been selected to execute the jobs in the Dag. It adds a make directory node at the top level of the graph, and all these concat to a single dummy job before branching out to the root nodes of the original/ concrete dag so far. So we end up introducing a classic X shape at the top of the graph. Hence the name HourGlass.
    Version:
    $Revision$
    Author:
    Karan Vahi, Gaurang Mehta
    • Field Detail

      • DUMMY_CONCAT_JOB

        public static final java.lang.String DUMMY_CONCAT_JOB
        The name concatenating dummy job that ensures that Condor does not start staging in before the directories are created.
        See Also:
        Constant Field Values
      • DUMMY_CONCAT_JOB_PREFIX

        public static final java.lang.String DUMMY_CONCAT_JOB_PREFIX
        The prefix assigned to the concatenating dummy job that ensures that Condor does not start staging in before the directories are created.
        See Also:
        Constant Field Values
      • TRANSFORMATION_NAMESPACE

        public static final java.lang.String TRANSFORMATION_NAMESPACE
        The transformation namespace for the create dir jobs.
        See Also:
        Constant Field Values
      • TRANSFORMATION_NAME

        public static final java.lang.String TRANSFORMATION_NAME
        The logical name of the transformation that creates directories on the remote execution pools.
        See Also:
        Constant Field Values
      • TRANSFORMATION_VERSION

        public static final java.lang.String TRANSFORMATION_VERSION
        The version number for the derivations for create dir jobs.
      • COMPLETE_TRANSFORMATION_NAME

        public static final java.lang.String COMPLETE_TRANSFORMATION_NAME
        The complete TC name for dirmanager.
      • DERIVATION_NAMESPACE

        public static final java.lang.String DERIVATION_NAMESPACE
        The derivation namespace for the create dir jobs.
        See Also:
        Constant Field Values
      • DERIVATION_NAME

        public static final java.lang.String DERIVATION_NAME
        The logical name of the transformation that creates directories on the remote execution pools.
        See Also:
        Constant Field Values
      • DERIVATION_VERSION

        public static final java.lang.String DERIVATION_VERSION
        The version number for the derivations for create dir jobs.
        See Also:
        Constant Field Values
    • Constructor Detail

      • HourGlass

        public HourGlass()
    • Method Detail

      • addCreateDirectoryNodes

        public ADag addCreateDirectoryNodes​(ADag dag)
        It modifies the concrete dag passed in the constructor and adds the create random directory nodes to it at the root level. These directory nodes have a common child that acts as a concatenating job and ensures that Condor does not start staging in the data before the directories have been added. The root nodes in the unmodified dag are now chidren of this concatenating dummy job.
        Parameters:
        dag - the workflow to which the nodes have to be added.
        Returns:
        the added workflow
      • introduceRootDependencies

        private void introduceRootDependencies​(ADag dag,
                                               Job newRoot)
        It traverses through the root jobs of the dag and introduces a new super root node to it.
        Parameters:
        dag - the DAG
        newRoot - the concat job that is the new root of the graph.
      • makeDummyConcatJob

        public Job makeDummyConcatJob​(ADag dag)
        It creates a dummy concat job that is run at the local submit host. This job should run always provided the directories were created successfully.
        Parameters:
        dag - the workflow
        Returns:
        the dummy concat job.
      • getConcatJobname

        protected java.lang.String getConcatJobname​(ADag dag)
        Returns the name of the concat job
        Returns:
        name
      • construct

        private void construct​(Job job,
                               java.lang.String key,
                               java.lang.String value)
        Constructs a condor variable in the condor profile namespace associated with the job. Overrides any preexisting key values.
        Parameters:
        job - contains the job description.
        key - the key of the profile.
        value - the associated value.