org.apache.tinkerpop.gremlin.hadoop.process.computer

Class AbstractHadoopGraphComputer

java.lang.Object

org.apache.tinkerpop.gremlin.hadoop.process.computer.AbstractHadoopGraphComputer

All Implemented Interfaces:

GraphComputer

Direct Known Subclasses:

GiraphGraphComputer, SparkGraphComputer

public abstract class AbstractHadoopGraphComputer
extends Object
implements GraphComputer

Author:

Marko A. Rodriguez (http://markorodriguez.com)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	AbstractHadoopGraphComputer.Features

Nested classes/interfaces inherited from interface org.apache.tinkerpop.gremlin.process.computer.GraphComputer

GraphComputer.Exceptions, GraphComputer.Persist, GraphComputer.ResultGraph

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	executed
protected GraphFilter	graphFilter
protected HadoopGraph	hadoopGraph
protected Logger	logger
protected Set<MapReduce>	mapReducers
protected GraphComputer.Persist	persist
protected GraphComputer.ResultGraph	resultGraph
protected VertexProgram<Object>	vertexProgram
protected int	workers

Constructor Summary

Constructors

Constructor and Description
AbstractHadoopGraphComputer(HadoopGraph hadoopGraph)

Method Summary

Modifier and Type	Method and Description
static File	copyDirectoryIfNonExistent(FileSystem fileSystem, String directory)
GraphComputer	edges(Traversal<Vertex,Edge> edgeFilter) Add a filter that will limit which edges of the vertices are loaded from the graph source.
AbstractHadoopGraphComputer.Features	features()
protected abstract void	loadJar(Configuration hadoopConfiguration, File file, Object... params)
protected void	loadJars(Configuration hadoopConfiguration, Object... params)
GraphComputer	mapReduce(MapReduce mapReduce) Add a MapReduce job to the set of MapReduce jobs to be executed by the GraphComputer.
GraphComputer	persist(GraphComputer.Persist persist) Set the GraphComputer.Persist level of the computation.
GraphComputer	program(VertexProgram vertexProgram) Set the VertexProgram to be executed by the GraphComputer.
GraphComputer	result(GraphComputer.ResultGraph resultGraph) Set the GraphComputer.ResultGraph of the computation.
String	toString()
protected void	validateStatePriorToExecution()
GraphComputer	vertices(Traversal<Vertex,Vertex> vertexFilter) Add a filter that will limit which vertices are loaded from the graph source.
GraphComputer	workers(int workers) Set the desired number of workers to execute the VertexProgram and MapReduce jobs.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.tinkerpop.gremlin.process.computer.GraphComputer

configure, submit

Field Detail

logger

protected final Logger logger

hadoopGraph

protected final HadoopGraph hadoopGraph

executed

protected boolean executed

mapReducers

protected final Set<MapReduce> mapReducers

vertexProgram

protected VertexProgram<Object> vertexProgram

workers

protected int workers

resultGraph

protected GraphComputer.ResultGraph resultGraph

persist

protected GraphComputer.Persist persist

graphFilter

protected GraphFilter graphFilter

Constructor Detail

AbstractHadoopGraphComputer

public AbstractHadoopGraphComputer(HadoopGraph hadoopGraph)

Method Detail

vertices

public GraphComputer vertices(Traversal<Vertex,Vertex> vertexFilter)

Description copied from interface: GraphComputer

Add a filter that will limit which vertices are loaded from the graph source. The provided Traversal can only check the vertex, its vertex properties, and the vertex property properties. The loaded graph will only have those vertices that pass through the provided filter.

Specified by:

vertices in interface GraphComputer

Parameters:

vertexFilter - the traversal to verify whether or not to load the current vertex

Returns:

the updated GraphComputer with newly set vertex filter

edges

public GraphComputer edges(Traversal<Vertex,Edge> edgeFilter)

Description copied from interface: GraphComputer

Add a filter that will limit which edges of the vertices are loaded from the graph source. The provided Traversal can only check the local star graph of the vertex and thus, can not access properties/labels of the adjacent vertices. The vertices of the loaded graph will only have those edges that pass through the provided filter.

Specified by:

edges in interface GraphComputer

Parameters:

edgeFilter - the traversal that determines which edges are loaded for each vertex

Returns:

the updated GraphComputer with newly set edge filter

result

public GraphComputer result(GraphComputer.ResultGraph resultGraph)

Description copied from interface: GraphComputer

Set the GraphComputer.ResultGraph of the computation. If this is not set explicitly by the user, then the VertexProgram can choose the most efficient result for its intended use. If there is no declared vertex program, then the GraphComputer defaults to GraphComputer.ResultGraph.ORIGINAL.

Specified by:

result in interface GraphComputer

Parameters:

resultGraph - the type of graph to be returned by ComputerResult.graph()

Returns:

the updated GraphComputer with newly set result graph

persist

public GraphComputer persist(GraphComputer.Persist persist)

Description copied from interface: GraphComputer

Set the GraphComputer.Persist level of the computation. If this is not set explicitly by the user, then the VertexProgram can choose the most efficient persist for the its intended use. If there is no declared vertex program, then the GraphComputer defaults to GraphComputer.Persist.NOTHING.

Specified by:

persist in interface GraphComputer

Parameters:

persist - the persistence level of the resultant computation

Returns:

the updated GraphComputer with newly set persist

program

public GraphComputer program(VertexProgram vertexProgram)

Description copied from interface: GraphComputer

Set the VertexProgram to be executed by the GraphComputer. There can only be one VertexProgram for the GraphComputer.

Specified by:

program in interface GraphComputer

Parameters:

vertexProgram - the VertexProgram to be executed

Returns:

the updated GraphComputer with newly set VertexProgram

mapReduce

public GraphComputer mapReduce(MapReduce mapReduce)

Description copied from interface: GraphComputer

Add a MapReduce job to the set of MapReduce jobs to be executed by the GraphComputer. There can be any number of MapReduce jobs.

Specified by:

mapReduce in interface GraphComputer

Parameters:

mapReduce - the MapReduce job to add to the computation

Returns:

the updated GraphComputer with newly added MapReduce job

workers

public GraphComputer workers(int workers)

Description copied from interface: GraphComputer

Set the desired number of workers to execute the VertexProgram and MapReduce jobs. This is a recommendation to the underlying GraphComputer implementation and is allowed to deviate accordingly by the implementation.

Specified by:

workers in interface GraphComputer

Parameters:

workers - the number of workers to execute the submission

Returns:

the updated GraphComputer with newly set worker count

toString

public String toString()

Overrides:

toString in class Object

validateStatePriorToExecution

protected void validateStatePriorToExecution()

loadJars

protected void loadJars(Configuration hadoopConfiguration,
                        Object... params)

loadJar

protected abstract void loadJar(Configuration hadoopConfiguration,
                                File file,
                                Object... params)
                         throws IOException

Throws:

IOException

features

public AbstractHadoopGraphComputer.Features features()

Specified by:

features in interface GraphComputer

copyDirectoryIfNonExistent

public static File copyDirectoryIfNonExistent(FileSystem fileSystem,
                                              String directory)