Class ADag

  • All Implemented Interfaces:
    Graph, GraphNodeContent, java.lang.Cloneable

    public class ADag
    extends Data
    implements Graph
    This class object contains the info about a Dag. DagInfo object contains the information to create the .dax file. vJobSubInfos is a Vector containing Job objects of jobs making the Dag. Each subinfo object contains information needed to generate a submit file for that job.
    Version:
    $Revision$
    Author:
    Karan Vahi, Gaurang Mehta
    See Also:
    DagInfo, Job
    • Field Summary

      Fields 
      Modifier and Type Field Description
      private DagInfo mDAGInfo
      The DagInfo object which contains the information got from parsing the dax file.
      private Graph mGraphImplementor
      Handle to the Graph implementor.
      protected Notifications mNotifications
      All the notifications associated with the job
      protected ReplicaStore mReplicaStore
      Handle to the replica store that stores the replica catalog user specifies in the DAX
      private java.lang.String mRequestID
      The optional request ID associated with the DAX.
      protected java.lang.String mRootWorkflowUUID
      The Root Workflow UUID.
      private java.lang.String mSubmitDirectory
      The root of the submit directory hierarchy for the DAG.
      protected TransformationStore mTransformationStore
      Handle to the transformation store that stores the transformation catalog user specifies in the DAX
      protected boolean mWorkflowRefinementStarted
      Boolean indicating whether the refinement process on the workflow has started or not.
      protected java.lang.String mWorkflowUUID
      The UUID associated with the workflow.
      • Fields inherited from interface edu.isi.pegasus.planner.partitioner.graph.Graph

        VERSION
    • Constructor Summary

      Constructors 
      Constructor Description
      ADag()
      Initialises the class member variables.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void add​(Job job)
      This adds a new job to the ADAG object.
      void addEdge​(GraphNode parent, GraphNode child)
      Adds an edge between two already existing nodes in the graph.
      void addEdge​(java.lang.String parent, java.lang.String child)
      Adds an edge between two already existing nodes in the graph.
      void addEdges​(java.lang.String child, java.util.List<java.lang.String> parents)
      A convenience method that allows for bulk addition of edges between already existing nodes in the graph.
      void addNewRelation​(java.lang.String parent, java.lang.String child)
      Adds a new PCRelation pair to the Vector of PCRelation pairs.
      void addNode​(GraphNode node)
      Adds a node to the Graph.
      void addNotification​(Invoke invoke)
      Adds a Invoke object corresponding to a notification.
      void addNotifications​(Notifications invokes)
      Adds all the notifications passed to the underlying container.
      void addRoot​(GraphNode root)
      Adds a single root node to the Graph.
      java.util.Iterator<GraphNode> bottomUpIterator()
      Returns an iterator that traverses the graph bottom up from the leaves.
      void checkForCorruption()
      Checks the underlying graph structure for any corruption.
      java.lang.Object clone()
      Returns a new copy of the Object.
      private java.lang.String complain​(java.lang.String desc, GraphNode node, GraphNode linkedNode)
      Convenience method to complain for a linked node from a node that does not exist in the DAG
      private void edgeToDOT​(java.io.Writer stream, java.lang.String indent, java.lang.String parent, java.lang.String child)
      Returns the DOT description of the object.
      void generateFlowID()
      Generates the flow id for this current run.
      void generateFlowName()
      Generates the name of the flow.
      protected java.lang.String generateWorkflowUUID()
      Generates the UUID for the workflow
      java.lang.String getAbstractWorkflowName()
      Returns the workflow id
      java.lang.String getBaseSubmitDirectory()
      Returns the root of submit directory hierarchy for the workflow.
      java.lang.String getCount()
      Returns the count of the workflow, that was specified in the DAX.
      DagInfo getDAGInfo()
      Returns the DAGInfo that stores the metadata about the DAX
      java.lang.String getDAXVersion()
      Returns the dax version
      java.lang.String getExecutableWorkflowName()
      Returns the workflow id
      java.lang.String getFlowID()
      Returns the flow ID for the workflow.
      java.lang.String getFlowName()
      Returns the flow name
      java.lang.String getFlowTimestamp()
      Returns the flow timestamp for the workflow.
      java.lang.String getIndex()
      Returns the index of the workflow, that was specified in the DAX.
      java.lang.String getLabel()
      Returns the label of the workflow, that was specified in the DAX.
      java.util.List<GraphNode> getLeaves()
      Returns the leaf nodes of the Graph.
      java.lang.String getMTime()
      Returns the last modified time for the file containing the workflow description.
      GraphNode getNode​(java.lang.String identifier)
      Returns the node matching the id passed.
      int getNoOfJobs()
      Returns the number of jobs in the dag on the basis of number of elements in the dagJobs Vector.
      Notifications getNotifications()
      Returns all the notifications associated with the job.
      java.util.Collection<Invoke> getNotifications​(Invoke.WHEN when)
      Returns a collection of all the notifications that need to be done for a particular condition
      java.lang.String getReleaseVersion()
      Return the release version
      ReplicaStore getReplicaStore()
      Returns the Replica Store
      java.lang.String getRequestID()
      Returns the request id.
      java.util.List<GraphNode> getRoots()
      Returns the root nodes of the Graph.
      java.lang.String getRootWorkflowUUID()
      Returns the UUID for the Root workflow
      TransformationStore getTransformationStore()
      Returns the Transformation Store
      WorkflowMetrics getWorkflowMetrics()
      Returns the metrics about the workflow.
      java.lang.String getWorkflowUUID()
      Returns the UUID for the workflow
      boolean hasWorkflowRefinementStarted()
      Returns a boolean indicating whether the workflow refinement has started or not
      boolean isEmpty()
      Returns a boolean if there are no nodes in the graph.
      java.util.Iterator<GraphNode> iterator()
      Returns an iterator that traverses through the graph using a graph traversal algorithm.
      java.util.Iterator<GraphNode> jobIterator()
      Returns an iterator for traversing through the jobs in the workflow.
      java.util.Iterator<GraphNode> nodeIterator()
      Returns an iterator for the nodes in the Graph.
      boolean remove​(Job job)
      Removes a particular job from the workflow.
      boolean remove​(java.lang.String identifier)
      Removes a node from the Graph.
      void resetEdges()
      Resets all the dependencies in the Graph, while preserving the nodes.
      void resetStores()
      Resets the replica and transformation stores;
      void setBaseSubmitDirectory​(java.lang.String dir)
      Sets the submit directory for the workflow.
      void setCount​(java.lang.String count)
      Set the count of the workflow, that was specified in the DAX.
      void setDAXMTime​(java.io.File f)
      Sets the mtime (last modified time) for the DAX.
      void setDAXVersion​(java.lang.String version)
      Sets the dax version
      void setFlowTimestamp​(java.lang.String timestamp)
      Sets the flow timestamp for the workflow.
      void setIndex​(java.lang.String index)
      Set the index of the workflow, that was specified in the DAX.
      void setLabel​(java.lang.String label)
      Sets the label for the workflow.
      void setReleaseVersion()
      Grabs the release version from VDS.Properties file.
      void setReplicaStore​(ReplicaStore store)
      Sets the Replica Store
      void setRequestID​(java.lang.String id)
      Sets the request id.
      void setRootWorkflowUUID​(java.lang.String uuid)
      Sets the root UUID for the workflow
      void setTransformationStore​(TransformationStore store)
      Sets the Transformation Store
      void setWorkflowRefinementStarted​(boolean state)
      Sets whether the workflow refinement has started or not
      void setWorkflowUUID​(java.lang.String uuid)
      Sets the UUID for the workflow
      int size()
      Returns the number of nodes in the graph.
      java.lang.String toDOT()
      Returns the DOT description of the object.
      void toDOT​(java.io.Writer stream, java.lang.String indent)
      Returns the DOT description of the object.
      java.util.Iterator<GraphNode> topologicalSortIterator()
      Returns an iterator for the graph that traverses in topological sort order.
      java.lang.String toString()
      Returns the String description of the dag associated with this object.
      void writeDOTHeader​(java.io.Writer stream, java.lang.String indent)
      Writes out the static DOT Header.
      • Methods inherited from class java.lang.Object

        equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Field Detail

      • mDAGInfo

        private DagInfo mDAGInfo
        The DagInfo object which contains the information got from parsing the dax file.
      • mSubmitDirectory

        private java.lang.String mSubmitDirectory
        The root of the submit directory hierarchy for the DAG. This is the directory where generally the DAG related files like the log files, .dag and dagman output files reside.
      • mRequestID

        private java.lang.String mRequestID
        The optional request ID associated with the DAX.
      • mReplicaStore

        protected ReplicaStore mReplicaStore
        Handle to the replica store that stores the replica catalog user specifies in the DAX
      • mTransformationStore

        protected TransformationStore mTransformationStore
        Handle to the transformation store that stores the transformation catalog user specifies in the DAX
      • mRootWorkflowUUID

        protected java.lang.String mRootWorkflowUUID
        The Root Workflow UUID.
      • mWorkflowUUID

        protected java.lang.String mWorkflowUUID
        The UUID associated with the workflow.
      • mWorkflowRefinementStarted

        protected boolean mWorkflowRefinementStarted
        Boolean indicating whether the refinement process on the workflow has started or not.
      • mNotifications

        protected Notifications mNotifications
        All the notifications associated with the job
      • mGraphImplementor

        private Graph mGraphImplementor
        Handle to the Graph implementor.
    • Constructor Detail

      • ADag

        public ADag()
        Initialises the class member variables.
    • Method Detail

      • addNotification

        public void addNotification​(Invoke invoke)
        Adds a Invoke object corresponding to a notification.
        Parameters:
        invoke - the invoke object containing the notification
      • addNotifications

        public void addNotifications​(Notifications invokes)
        Adds all the notifications passed to the underlying container.
        Parameters:
        invokes - the notifications to be added
      • getNotifications

        public java.util.Collection<Invoke> getNotifications​(Invoke.WHEN when)
        Returns a collection of all the notifications that need to be done for a particular condition
        Parameters:
        when - the condition
        Returns:
      • getNotifications

        public Notifications getNotifications()
        Returns all the notifications associated with the job.
        Returns:
        the notifications
      • resetStores

        public void resetStores()
        Resets the replica and transformation stores;
      • clone

        public java.lang.Object clone()
        Returns a new copy of the Object.
        Overrides:
        clone in class java.lang.Object
        Returns:
        the clone of the object.
      • getRootWorkflowUUID

        public java.lang.String getRootWorkflowUUID()
        Returns the UUID for the Root workflow
        Returns:
        the UUID of the workflow
      • setRootWorkflowUUID

        public void setRootWorkflowUUID​(java.lang.String uuid)
        Sets the root UUID for the workflow
        Parameters:
        uuid - the UUID of the workflow
      • getWorkflowUUID

        public java.lang.String getWorkflowUUID()
        Returns the UUID for the workflow
        Returns:
        the UUID of the workflow
      • setWorkflowUUID

        public void setWorkflowUUID​(java.lang.String uuid)
        Sets the UUID for the workflow
        Parameters:
        uuid - the UUID of the workflow
      • generateWorkflowUUID

        protected java.lang.String generateWorkflowUUID()
        Generates the UUID for the workflow
        Returns:
        the UUID of the workflow
      • hasWorkflowRefinementStarted

        public boolean hasWorkflowRefinementStarted()
        Returns a boolean indicating whether the workflow refinement has started or not
        Returns:
        boolean
      • setWorkflowRefinementStarted

        public void setWorkflowRefinementStarted​(boolean state)
        Sets whether the workflow refinement has started or not
        Parameters:
        state - the boolean value
      • toString

        public java.lang.String toString()
        Returns the String description of the dag associated with this object.
        Specified by:
        toString in class Data
        Returns:
        textual description.
      • add

        public void add​(Job job)
        This adds a new job to the ADAG object. It ends up adding both the job name and the job description to the internal structure.
        Parameters:
        job - the new job that is to be added to the ADag.
      • remove

        public boolean remove​(Job job)
        Removes a particular job from the workflow. It however does not delete the relations the edges that refer to the job.
        Parameters:
        job - the Job object containing the job description.
        Returns:
        boolean indicating whether the removal was successful or not.
      • getNoOfJobs

        public int getNoOfJobs()
        Returns the number of jobs in the dag on the basis of number of elements in the dagJobs Vector.
        Returns:
        the number of jobs.
      • setRequestID

        public void setRequestID​(java.lang.String id)
        Sets the request id.
        Parameters:
        id - the request id.
      • getRequestID

        public java.lang.String getRequestID()
        Returns the request id.
        Returns:
        the request id.
      • getAbstractWorkflowName

        public java.lang.String getAbstractWorkflowName()
        Returns the workflow id
        Returns:
        the abstract workflow id
      • getExecutableWorkflowName

        public java.lang.String getExecutableWorkflowName()
        Returns the workflow id
        Returns:
        the executable workflow id
      • addNewRelation

        public void addNewRelation​(java.lang.String parent,
                                   java.lang.String child)
        Adds a new PCRelation pair to the Vector of PCRelation pairs. For the new relation the isDeleted parameter is set to false.
        Parameters:
        parent - The parent in the relation pair
        child - The child in the relation pair
      • setBaseSubmitDirectory

        public void setBaseSubmitDirectory​(java.lang.String dir)
        Sets the submit directory for the workflow.
        Parameters:
        dir - the submit directory.
      • getLabel

        public java.lang.String getLabel()
        Returns the label of the workflow, that was specified in the DAX.
        Returns:
        the label of the workflow.
      • setLabel

        public void setLabel​(java.lang.String label)
        Sets the label for the workflow.
        Parameters:
        label - the label to be assigned to the workflow
      • getIndex

        public java.lang.String getIndex()
        Returns the index of the workflow, that was specified in the DAX.
        Returns:
        the index of the workflow.
      • setIndex

        public void setIndex​(java.lang.String index)
        Set the index of the workflow, that was specified in the DAX.
        Parameters:
        index - the count
      • setCount

        public void setCount​(java.lang.String count)
        Set the count of the workflow, that was specified in the DAX.
        Parameters:
        count - the count
      • getCount

        public java.lang.String getCount()
        Returns the count of the workflow, that was specified in the DAX.
        Returns:
        the count
      • getDAXVersion

        public java.lang.String getDAXVersion()
        Returns the dax version
        Returns:
        teh dax version.
      • getMTime

        public java.lang.String getMTime()
        Returns the last modified time for the file containing the workflow description.
        Returns:
        the MTime
      • getBaseSubmitDirectory

        public java.lang.String getBaseSubmitDirectory()
        Returns the root of submit directory hierarchy for the workflow.
        Returns:
        the directory.
      • checkForCorruption

        public void checkForCorruption()
        Checks the underlying graph structure for any corruption. Corruption can be where a parent or a child of a node refers to an object, that is not in underlying graph node list.
        Throws:
        java.lang.RuntimeException - in case of corruption.
      • complain

        private java.lang.String complain​(java.lang.String desc,
                                          GraphNode node,
                                          GraphNode linkedNode)
        Convenience method to complain for a linked node from a node that does not exist in the DAG
        Parameters:
        desc -
        node -
        linkedNode -
      • setReplicaStore

        public void setReplicaStore​(ReplicaStore store)
        Sets the Replica Store
        Parameters:
        store - the Replica Store
      • getReplicaStore

        public ReplicaStore getReplicaStore()
        Returns the Replica Store
        Returns:
        the Replica Store
      • setTransformationStore

        public void setTransformationStore​(TransformationStore store)
        Sets the Transformation Store
        Parameters:
        store - the Transformation Store
      • getTransformationStore

        public TransformationStore getTransformationStore()
        Returns the Transformation Store
        Returns:
        the Replica Store
      • getDAGInfo

        public DagInfo getDAGInfo()
        Returns the DAGInfo that stores the metadata about the DAX
        Returns:
      • generateFlowID

        public void generateFlowID()
        Generates the flow id for this current run. It is made of the name of the dag and a timestamp. This is a simple concat of the mFlowTimestamp and the flowName. For it work correctly the function needs to be called after the flow name and timestamp have been generated.
      • getFlowID

        public java.lang.String getFlowID()
        Returns the flow ID for the workflow.
        Returns:
      • generateFlowName

        public void generateFlowName()
        Generates the name of the flow. It is same as the mNameOfADag if specified in dax generated by Chimera.
      • getFlowName

        public java.lang.String getFlowName()
        Returns the flow name
      • setDAXVersion

        public void setDAXVersion​(java.lang.String version)
        Sets the dax version
        Parameters:
        version - the version of the DAX
      • setDAXMTime

        public void setDAXMTime​(java.io.File f)
        Sets the mtime (last modified time) for the DAX. It is the time, when the DAX file was last modified. If the DAX file does not exist or an IO error occurs, the MTime is set to OL i.e . The DAX mTime is always generated in an extended format. Generating not in extended format, leads to the XML parser tripping while parsing the invocation record generated by Kickstart.
        Parameters:
        f - the file descriptor to the DAX|PDAX file.
      • getReleaseVersion

        public java.lang.String getReleaseVersion()
        Return the release version
      • setReleaseVersion

        public void setReleaseVersion()
        Grabs the release version from VDS.Properties file.
      • getFlowTimestamp

        public java.lang.String getFlowTimestamp()
        Returns the flow timestamp for the workflow.
        Returns:
        the flowtimestamp
      • setFlowTimestamp

        public void setFlowTimestamp​(java.lang.String timestamp)
        Sets the flow timestamp for the workflow.
        Parameters:
        timestamp - the flowtimestamp
      • jobIterator

        public java.util.Iterator<GraphNode> jobIterator()
        Returns an iterator for traversing through the jobs in the workflow.
        Returns:
        a bative java failsafe iterator to the underlying collection.
      • getWorkflowMetrics

        public WorkflowMetrics getWorkflowMetrics()
        Returns the metrics about the workflow.
        Returns:
        the WorkflowMetrics
      • toDOT

        public java.lang.String toDOT()
                               throws java.io.IOException
        Returns the DOT description of the object. This is used for visualizing the workflow.
        Returns:
        String containing the Partition object in XML.
        Throws:
        java.io.IOException - if something fishy happens to the stream.
      • toDOT

        public void toDOT​(java.io.Writer stream,
                          java.lang.String indent)
                   throws java.io.IOException
        Returns the DOT description of the object. This is used for visualizing the workflow.
        Parameters:
        stream - is a stream opened and ready for writing. This can also be a StringWriter for efficient output.
        indent - is a String of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.
        Throws:
        java.io.IOException - if something fishy happens to the stream.
      • edgeToDOT

        private void edgeToDOT​(java.io.Writer stream,
                               java.lang.String indent,
                               java.lang.String parent,
                               java.lang.String child)
                        throws java.io.IOException
        Returns the DOT description of the object. This is used for visualizing the workflow.
        Parameters:
        stream - is a stream opened and ready for writing. This can also be a StringWriter for efficient output.
        indent - is a String of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.
        parent - the parent
        child - the child
        Throws:
        java.io.IOException - if something fishy happens to the stream.
      • writeDOTHeader

        public void writeDOTHeader​(java.io.Writer stream,
                                   java.lang.String indent)
                            throws java.io.IOException
        Writes out the static DOT Header.
        Parameters:
        stream - is a stream opened and ready for writing. This can also be a StringWriter for efficient output.
        indent - is a String of spaces used for pretty printing. The initial amount of spaces should be an empty string. The parameter is used internally for the recursive traversal.
        Throws:
        java.io.IOException - if something fishy happens to the stream.
      • addNode

        public void addNode​(GraphNode node)
        Adds a node to the Graph. It overwrites an already existing node with the same ID.
        Specified by:
        addNode in interface Graph
        Parameters:
        node - the node to be added to the Graph.
      • addEdge

        public void addEdge​(java.lang.String parent,
                            java.lang.String child)
        Adds an edge between two already existing nodes in the graph.
        Specified by:
        addEdge in interface Graph
        Parameters:
        parent - the parent node ID.
        child - the child node ID.
      • addEdge

        public void addEdge​(GraphNode parent,
                            GraphNode child)
        Adds an edge between two already existing nodes in the graph.
        Specified by:
        addEdge in interface Graph
        Parameters:
        parent - the parent node .
        child - the child node .
      • addEdges

        public void addEdges​(java.lang.String child,
                             java.util.List<java.lang.String> parents)
        A convenience method that allows for bulk addition of edges between already existing nodes in the graph.
        Specified by:
        addEdges in interface Graph
        Parameters:
        child - the child node ID
        parents - list of parent identifiers as String.
      • getNode

        public GraphNode getNode​(java.lang.String identifier)
        Returns the node matching the id passed.
        Specified by:
        getNode in interface Graph
        Parameters:
        identifier - the id of the node.
        Returns:
        the node matching the ID else null.
      • addRoot

        public void addRoot​(GraphNode root)
        Adds a single root node to the Graph. All the exisitng roots of the Graph become children of the root.
        Specified by:
        addRoot in interface Graph
        Parameters:
        root - the GraphNode to be added as a root.
        Throws:
        java.lang.RuntimeException - if a node with the same id already exists.
      • remove

        public boolean remove​(java.lang.String identifier)
        Removes a node from the Graph.
        Specified by:
        remove in interface Graph
        Parameters:
        identifier - the id of the node to be removed.
        Returns:
        boolean indicating whether the node was removed or not.
      • resetEdges

        public void resetEdges()
        Resets all the dependencies in the Graph, while preserving the nodes. The resulting Graph is a graph of independent nodes.
        Specified by:
        resetEdges in interface Graph
      • nodeIterator

        public java.util.Iterator<GraphNode> nodeIterator()
        Returns an iterator for the nodes in the Graph. These iterators are fail safe.
        Specified by:
        nodeIterator in interface Graph
        Returns:
        Iterator
      • iterator

        public java.util.Iterator<GraphNode> iterator()
        Returns an iterator that traverses through the graph using a graph traversal algorithm.
        Specified by:
        iterator in interface Graph
        Returns:
        Iterator through the nodes of the graph.
      • bottomUpIterator

        public java.util.Iterator<GraphNode> bottomUpIterator()
        Returns an iterator that traverses the graph bottom up from the leaves. At any one time, only one iterator can iterate through the graph.
        Specified by:
        bottomUpIterator in interface Graph
        Returns:
        Iterator through the nodes of the graph.
      • topologicalSortIterator

        public java.util.Iterator<GraphNode> topologicalSortIterator()
        Returns an iterator for the graph that traverses in topological sort order.
        Specified by:
        topologicalSortIterator in interface Graph
        Returns:
        Iterator through the nodes of the graph.
      • size

        public int size()
        Returns the number of nodes in the graph.
        Specified by:
        size in interface Graph
      • getRoots

        public java.util.List<GraphNode> getRoots()
        Returns the root nodes of the Graph.
        Specified by:
        getRoots in interface Graph
        Returns:
        a list containing GraphNode corressponding to the root nodes.
      • getLeaves

        public java.util.List<GraphNode> getLeaves()
        Returns the leaf nodes of the Graph.
        Specified by:
        getLeaves in interface Graph
        Returns:
        a list containing GraphNode corressponding to the leaf nodes.
      • isEmpty

        public boolean isEmpty()
        Returns a boolean if there are no nodes in the graph.
        Specified by:
        isEmpty in interface Graph
        Returns:
        boolean