org.apache.tinkerpop.gremlin.spark.structure.io

Class SparkContextStorage

java.lang.Object

org.apache.tinkerpop.gremlin.spark.structure.io.SparkContextStorage

All Implemented Interfaces:

Storage

public final class SparkContextStorage
extends Object
implements Storage

Author:

Marko A. Rodriguez (http://markorodriguez.com)

Field Summary

Fields inherited from interface org.apache.tinkerpop.gremlin.structure.io.Storage

FILE_SEPARATOR, ROOT_DIRECTORY

Method Summary

Modifier and Type	Method and Description
boolean	cp(String sourceLocation, String targetLocation) Recursively copy all the data sources from the source location to the target location.
String	describe(String location)
boolean	exists(String location) Determine whether the specified location has a data source.
Iterator<Vertex>	head(String location, Class readerClass, int totalLines) Get the vertices at the specified graph location.
Iterator<String>	head(String location, int totalLines) Get a string representation of the specified number of lines at the data source location.
<K,V> Iterator<KeyValue<K,V>>	head(String location, String memoryKey, Class readerClass, int totalLines) Get the KeyValue data at the specified memory location.
List<String>	ls() List all the data sources in the root directory.
List<String>	ls(String pattern) List all the files (e.g.
static SparkContextStorage	open()
static SparkContextStorage	open(Configuration configuration)
static SparkContextStorage	open(SparkContext sparkContext)
static SparkContextStorage	open(String master)
boolean	rm(String pattern) Recursively remove the file (data source) at the specified location.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.tinkerpop.gremlin.structure.io.Storage

head, head, head, toPath, toPath

Method Detail

open

public static SparkContextStorage open()

open

public static SparkContextStorage open(String master)

open

public static SparkContextStorage open(Configuration configuration)

open

public static SparkContextStorage open(SparkContext sparkContext)

ls

public List<String> ls()

Description copied from interface: Storage

List all the data sources in the root directory.

Specified by:

ls in interface Storage

Returns:

non-null list of files (data sources) and directories in the root directory (/)

See Also:

Storage.ls(String)

ls

public List<String> ls(String pattern)

Description copied from interface: Storage

List all the files (e.g. data sources) and directories matching the location pattern.

Specified by:

ls in interface Storage

Parameters:

pattern - non-null pattern specifying a set of files and directories. Cases:

• a path to a file - specifies a single file to list
• a path to a directory - specifies all files and directories immediately nested in that directory to list
• pattern - specifies a set of files and directories to list
• / - specifies the root directory to list its contents

Returns:

non-null list of files (data sources) and directories matching the pattern.

cp

public boolean cp(String sourceLocation,
                  String targetLocation)

Description copied from interface: Storage

Recursively copy all the data sources from the source location to the target location.

Specified by:

cp in interface Storage

Parameters:

sourceLocation - non-null pattern specifying a set of files and directories. Cases:

• a path to a file - specifies a single file
• a path to a directory - specifies all files and directories nested (recursively) in that directory
• pattern - specifies a set of files and directories
• / - specifies the contents of the root directory (recursively)

targetLocation - non-null directory where to copy to

Returns:

whether data sources were copied

exists

public boolean exists(String location)

Description copied from interface: Storage

Determine whether the specified location has a data source.

Specified by:

exists in interface Storage

Parameters:

location - non-null pattern specifying a set of files and directories. Examples:

• a path to a file - specifies a single file
• a path to a directory - specifies the contents of that directory as all files and directories immediately nested in it
• pattern - specifies a set of files and directories
• / - specifies the immediate contents of the root directory

Returns:

true if the pattern specifies a non-empty set of files and directories

rm

public boolean rm(String pattern)

Description copied from interface: Storage

Recursively remove the file (data source) at the specified location. NOTE: Some implementations derive the notion of the containing directory from the presence of the file, so removing all files from a directory in those implementations removes also their directory.

Specified by:

rm in interface Storage

Parameters:

pattern - non-null pattern specifying a set of files and directories. Examples:

• a path to a file - specifies a single file
• a path to a directory - specifies that directory and all files and directories recursively nested in it
• pattern - specifies a set of files and directories
• / - specifies the root directory

Returns:

true if all specified files and directories were removed

head

public Iterator<Vertex> head(String location,
                             Class readerClass,
                             int totalLines)

Description copied from interface: Storage

Get the vertices at the specified graph location.

Specified by:

head in interface Storage

Parameters:

location - the location of the graph (or the root location and search will be made)

readerClass - the class of the parser that understands the graph format

totalLines - the total number of lines of the graph to return

Returns:

an iterator of vertices.

head

public <K,V> Iterator<KeyValue<K,V>> head(String location,
                                          String memoryKey,
                                          Class readerClass,
                                          int totalLines)

Description copied from interface: Storage

Get the KeyValue data at the specified memory location.

Specified by:

head in interface Storage

Parameters:

location - the root location of the data

memoryKey - the memory key

readerClass - the class of the parser that understands the memory format

totalLines - the total number of key-values to return

Returns:

an iterator of key-values.

head

public Iterator<String> head(String location,
                             int totalLines)

Description copied from interface: Storage

Get a string representation of the specified number of lines at the data source location.

Specified by:

head in interface Storage

Parameters:

location - the data source location

totalLines - the total number of lines to retrieve

Returns:

an iterator of lines.

describe

public String describe(String location)

toString

public String toString()

Overrides:

toString in class Object