While the concept of an open source contribution can refer to doing development work on the code base, there are many other ways outside of coding to contribute to Apache TinkerPop. Participating on the various mailing lists, offering ideas, reporting bugs, writing documentation are all welcome contributions to the project that help improve the TinkerPop. This section of the document is designed to help provide some structure for potential contributors and to give them ideas for how they could get started becoming more involved in the TinkerPop community.

Code changes can take a fair bit of effort from both the contributor doing the work, as well as the people who will be reviewing and testing that work. It is beneficial to all involved that some basic considerations be made by the contributor to ensure the best use is made of everyone’s time.

Before proceeding, contributors should evaluate if the proposed change is likely to be relevant, new and actionable:

Before considering how to contribute code, it is useful to understand how code is reviewed, and why changes may be rejected. Simply put, changes that have many or large positives, and few negative effects or risks, are much more likely to be merged, and merged quickly. Risky and less valuable changes are very unlikely to be merged, and may be rejected outright rather than receive iterations of review.

Positives

Negatives, Risks

Generally, TinkerPop uses JIRA to track logical issues, including bugs and improvements, and uses GitHub pull requests to manage the review and merge of specific code changes. That is, JIRA issues are used to describe what should be fixed or changed, and high-level approaches, and pull requests describe how to implement that change in the project’s source code.

As a first step to making a contribution, consider JIRA:

Next, make changes and prepare a pull request:

Once the pull request has been placed it will go into review:

On successful review, the pull request will be merged to the main repository and the JIRA issue will be closed.

At a minimum, development of TinkerPop requires Java 1.8.0_40+ and Maven 3.2.5+. Maven is used as the common build system, which even controls the builds of non-JVM GLVs such as gremlin-python. Java and Maven are described as a "minimum" for a development environment, because they will only build JVM portions of TinkerPop and many integration tests will not fire with this simple setup. It is possible to get a clean and successful build with this minimum, but it will not be possible to build non-JVM aspects of the project and those will go untested.

To gain the ability to execute all aspects of the TinkerPop build system, other environmental configurations must be established. Those prerequisites are defined in the following subsections.

The following commands are a mix of Maven flags and shell scripts that handle different build operations

TinkerPop provides a shell script, that can start several build tasks within a Docker container. The required Docker images will be built automatically if they don’t exist yet. Thus the first invocation of the Docker script is expected to take some time.

The script can be found under PROJECT_HOME/docker/build.sh. The following tasks are currently supported:

A list of command line options is provided by docker/build.sh --help. The container will install, configure and start all required dependencies, such as Hadoop.

Options can be passed to Docker by setting the TINKERPOP_DOCKER_OPTS environment variable. A speed boost can be gained at the expense of memory by using tmpfs and the special directory /usr/src/tinkermem.

A custom maven settings.xml can be supplied, for example, to point to a local proxy. Copy the settings.xml to the PROJECT_HOME/ directory. The Docker script will detect and copy it to the running container.

If the container is used to generate the user docs, it will start a web server and show the URL that is used to host the HTML docs.

After finishing all tasks, the script will immediately destroy the container.

This section refers specifically to setup within Intellij. TinkerPop has a module called gremlin-shaded which contains shaded dependencies for some libraries that are widely used and tend to introduce conflicts. To ensure that Intellij properly interprets this module after importing the Maven pom.xml perform the following steps:

Note that it maybe be necessary to re-execute these steps if the gremlin-shaded pom.xml is ever updated.

Developers working on the neo4j-gremlin module should enabled the include-neo4j Maven profile in Intellij. This will ensure that tests will properly execute within the IDE.

If Intellij complains about "duplicate sources" for the Groovy files when attempting to compile/run tests, then install the GMavenPlus Intellij plugin.

The gremlin-core module uses a Java annotation processor to help support DSLs. To support this capability be sure that:

TinkerPop has a user mailing list and a dev@tinkerpop.apache.org">developer mailing list. As a committer, it is a good idea to join both.

It would also be helpful to join the public TinkerPop HipChat room for developer discussion. This helps contributors to communicate in a more real-time way. Anyone can join as a guest, but for regular contributors it may be best to request that an Apache HipChat account be created.

Occasionally, online meetings via video conference are held. These meetings are schedule via the dev mailing list about a week before they are to occur to find a day and time that is available for those interested in attending. On the day of the meeting, the meeting organizer will create a Google Hangout (or similar video conferencing link) to post to the TinkerPop room in HipChat. At that point, all who are interested can attend. Meeting minutes should be taken and added to the Meetings section of this document using the pattern already established.

There is a two-pronged approach to maintaining the change log and preparing the release notes.

Given the dependence on the JIRA report for generating additions to the CHANGELOG.asciidoc, which uses the title of the issue as the line presented in the release note report, titles should be edited prior to release to be useful in that context. In other words, an issue title should be understandable as a change in the fewest words possible while still conveying the gist of the change.

Changes that break the public APIs should be marked with a "breaking" label and should be distinguished from other changes in the release notes.

TinkerPop has several release branches:

Changes to tp32 should merge to master. Please read more about this process in the Pull Requests section.

Other branches may be created for collaborating on features or for RFC’s that other developers may want to inspect. It is suggested that the JIRA issue ID be used as the prefix, since that triggers certain automation, and it provides a way to account for the branch lifecycle, i.e. "Who’s branch is this, and can I delete it?"

For branches that are NOT associated with JIRA issues, developers should utilize their Apache ID as a branch name prefix. This provides a unique namespace, and also a way to account for the branch lifecycle.

Developers should remove their own branches when they are no longer needed.

Tags are used for milestones, release candidates, and approved releases. Please refrain from creating arbitrary tags, as they produce permanent clutter.

TinkerPop uses Apache JIRA as its issue tracker. JIRA is a very robust piece of software with many options and configurations. To simplify usage and ensure consistency across issues, the following conventions should be adhered to:

Contributors should examine the current code base to determine what the code style patterns are and should match their style to what is already present. Of specific note however, TinkerPop does not use "import wildcards" - IDEs should be adjusted accordingly to not auto-wildcard the imports.

TinkerPop uses Travis for CI services. The build status can be found here. Note that the CI process does not run integration tests or include Neo4j-related tests as those tests would likely exceed the allowable times for builds on these servers.

When possible, committers should avoid direct "breaking" change (e.g. removing a method from a class) and favor deprecation. Deprecation should come with sufficient documentation and notice especially when the change involves public APIs that might be utilized by users or implemented by providers:

The JIRA issues that track removal of deprecated methods should be periodically evaluated to determine if it is prudent to schedule them into a release.

TinkerPop has a wide variety of test types that help validate its internal code as well as external provider code. There are "unit tests" and "integration tests". Unit tests execute on standard runs of mvn clean install. These tests tend to run quickly and provide a reasonable level of coverage and confidence in the code base. Integration tests are disabled by default and must be explicitly turned on with a special build property by adding -DskipIntegrationTests=false to the mvn execution. Integration tests run slower and may require external components to be running when they are executed. They are "marked" as separate from unit tests by inclusion of the suffix "IntegrateTest".

Here are some other points to consider when developing tests:

Benchmarks are a useful tool to track performance between TinkerPop versions and also as tools to aid development decision making. TinkerPop uses OpenJDK JMH for benchmark development. The JMH framework provides tools for writing robust benchmarking code that avoid many of the pitfalls inherent in benchmarking JIT compiled code on the JVM. Example JMH benchmarks can be found here.

TinkerPop benchmarks live in the gremlin-benchmark module and can either be run from within your IDE or as a standalone uber-jar. The uber-jar is the JMH recommended approach and also makes it easy to distribute artifacts to various environments to gather benchmarking numbers. Having said that, in most cases it should be sufficient to run it from within the IDE.

Benchmarks will not run by default because they are time consuming. To enable benchmarks during the test phase do -DskipBenchmarks=false. To change the number of warmup iterations, measurement iterations, and forks you can do mvn clean test -DskipBenchmarks=false -DdefaultForks=5 -DmeasureIterations=20 -DwarmupIterations=20. Benchmark results will be output by default to the benchmarks directory in JSON format.

Benchmarks may also be run from the command line using the JMH runner. Build the uber-jar and simply run java -jar gremlin-benchmark-TP-VERSION.jar. To see a list of JMH runner options, add the -h flag.

The JUnit/JMH integration was inspired by the Netty projects microbenchmarking suite. Please refer to the Netty docs for more details. Presently there are 3 abstract benchmark classes that may be used as building blocks for your benchmarks; AbstractBenchmarkBase, AbstractGraphBenchmark, and AbstractGraphMutateBenchmark.

Code modifications must go through a review-then-committ (RTC) process before being merged into a release branch. All committers should follow the pattern below, where "you" refers to the committer wanting to put code into a release branch.

The following exceptions to the RTC (review-then-commit) model presented above are itemized below. It is up to the committer to self-regulate as the itemization below is not complete and only hints at the types of commits that do not require a review.

When the committer chooses CTR, it is considered good form to include something in the commit message that explains that CTR was invoked and the reason for doing so. For example, "Invoking CTR as this change encompasses minor adjustments to text formatting."

There are many dependencies on other open source libraries in TinkerPop modules. When adding dependencies or altering the version of a dependency, developers must consider the implications that may apply to the TinkerPop LICENSE and NOTICE files. There are two implications to consider:

Understanding these implications is important for insuring that TinkerPop stays compliant with the Apache 2 license that it releases under.

Regarding the first item, refer to the Apache Legal for a list of approved licenses that are compatible with the Apache 2 license.

The second item requires a bit more effort to follow. The Apache website offers a how-to guide on the approach to maintaining appropriate LICENSE and NOTICE files, but this guide is designed to offer some more specific guidance as it pertains to TinkerPop and its distribution.

To get started, TinkerPop has both "source" and "binary" LICENSE/NOTICE files:

The documentation for TinkerPop is stored in the git repository in docs/src/ and are then split into several subdirectories, each representing a "book" (or its own publishable body of work). If a new AsciiDoc file is added to a book, then it should also be included in the index.asciidoc file for that book, otherwise the preprocessor will ignore it. Likewise, if a whole new book (subdirectory) is added, it must include an index.asciidoc file to be recognized by the AsciiDoc preprocessor.

Adding a book also requires a change to the root pom.xml file. Find the "asciidoc" Maven profile and add a new <execution> to the asciidoctor-maven-plugin configuration. For each book in docs/src/, there should be a related <execution> that generates the HTML from the AsciiDoc. Follows the patterns already established by the existing <execution> entries, paying special attention to the pathing of the '<sourceDirectory>', <outputDirectory> and <imagesdir>. Note that the <outputDirectory> represents where the book will exist when uploaded to the server and should preserve the directory structure in git as referenced in <sourceDirectory>.

Please see the Building and Testing section for more information on how to generate the documentation.

The content for the TinkerPop home page and related pages that make up the web site at tinkerpop.apache.org is stored in the git repository under /docs/site. In this way, it becomes easier for the community to provide content presented there, because the content can be accepted via the standard workflow of a pull request. To generate the site for local viewing, run bin/generate-home.sh, which will build the site in target/site/. PMC members can officially publish the site with bin/publish-home.sh <username>.

"Publishing" does not publish documentation (e.g. reference docs, javadocs, etc) and only publishes what is generated from the content in /docs/site. Publishing the site can be performed out of band with the release cycle and is no way tied to a version. The master branch should always be considered the "current" web site and publishing should only happen from that branch.

TinkerPop uses SLF4j for logging and typically leans back on Log4j as the implementation. Configuring log outputs for debugging purposes within tests can be altered by editing the log4j-test.properties file in each module’s test resources. That file gets copied to the target/test-classes on build and surefire and failsafe plugins in maven are then configured to point at that area of the file system for those configuration files. The properties files can be edited to fine tune control of the log output, but generally speaking the current configuration is likely best for everyone’s general purposes, so if changes are made please revert them prior to commit.

If this is your first time as release manager, you will need to setup keys for signing purposes per the Apache release process. Generally speaking, this will mean that you will need to generate a key-pair and then upload your public key to a public keyserver.

For a general overview of key basics, refer to this. For detailed step-by-step instructions, please follow the instructions here.

After completing the key-pair setup instructions, be sure to add yourself to the PGP signature section of bin/validate-distribution.sh.

The "pre-flight check" is a list of things performed by the release manager during the weeks leading up to a scheduled day to release. These checks will help to ensure that that release day goes smoothly by identifying problems up early and communicating with other members of the community.

A release candidate is an unofficial release that is represented by a tagged version in the Git repository. It is offered in cases where there is significant change in a particular version and the potential for upgrades and problems might be high.

This section describes the process that process that prepares a release for VOTE by the community. If there are multiple releases (as there usually are) being prepared, it may be best for downstream releases to wait for upstream releases to complete their process to assure that no last minute commits are required to get the upstream release completed. It is up to the discretion of the release managers to decide how they wish to proceed with respect to preparing releases in parallel or in a more serial fashion.

A number of administration tasks should be taken care of after release is public. Some of these items can be performed during the VOTE period at the release manager’s discretion, though it may be wise to wait until a successful VOTE is eminent before reopening development:

When a candidate is identified by a PMC member as someone who might be a good official committer to TinkerPop, the PMC member should open a DISCUSS thread on the private TinkerPop mailing list. The thread should provide some background and reasoning for why that member believes the candidate would be a good committer. Given enough time for feedback on the candidate and presuming there is still positive interest in doing so, a VOTE thread on the private TinkerPop mailing list is started to get the official stance. As per usual, the VOTE will be made open for no less than 72 hours.

If the VOTE closes with a successful positive vote to make the candidate a committer, then send the following email to the candidate and copy the private TinkerPop mailing list:

Assuming the iCLA included items from step 3, the secretary will handle account creation, otherwise it will be up to the PMC chair to handle such things. Once the account is established the PMC chair will then need to:

Finally, new committers should be sent an email that covers some of the administrative elements of their new role:

The process for bringing on new PMC members is not so different from the one for new committers. The process begins with a DISCUSS thread to the private mailing list for building consensus followed by a VOTE thread to confirm. Presuming the new PMC member is a committer already (which is mostly likely for TinkerPop), there should be no need for any additional paperwork. On successful vote however, a NOTICE should be sent to the board@apache.org (copying private@tinkerpop.apache.org):

If the candidate does not (yet) have an Apache account, then please note that fact in the notification email. Provide the board 72 hours to respond with objection before extending the invite to the new PMC member. Once that time has elapsed, the invite may be sent to the new PMC member. If they accept, then the PMC chair should refer to the How to Add a New PMC Member section in the Apache docs for how to complete the process. Use Whimsy to update PMC membership.

The PMC Chair is responsible for submitting a report to the board on a quarterly basis. TinkerPop reports on the following months: January, April, July, October. The report has the following format:

A draft of the report should be sent to the TinkerPop developer mailing list for review at least three days prior to submitting to the board. The final report should be sent in plain-text format to board@apache.org and should be committed to the appropriate meeting agenda in SVN at:

The meeting was scheduled for 12:00pm EST, started at 12:05pm when sufficient attendance to constitute a quorum was reached and was held via Google Hangout hosted by Marko Rodriguez. Meeting adjourned at 2:35pm EST.

Committers/PPMC Members

Guests

None

Minutes

The following tickets were assigned to 3.1.1-incubating:

The following items were identified for 3.2.0-incubating:

The following issues were simply closed during review - the reasons for closing can be found in the comments of the issues themselves:

The meeting was scheduled for 1:00pm EST, started on time and was held via Google Hangout hosted by Stephen Mallette. Meeting adjourned at 1:45pm EST.

Committers/PPMC Members

Guests

None

Minutes

Upon review of the tickets in JIRA assigned to 3.1.0-incubating, the following were removed from the 3.1.0-incubating roadmap: