Skip to content

Commit 32f468c

Browse files
sjudengsjudeng
authored
Merge pull request #141 from amcp/docker
Dockerize gremlin-server.sh with JanusGraph
2 parents 9f88523 + 4c8c414 commit 32f468c

12 files changed

Lines changed: 186 additions & 6 deletions

File tree

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ target/
1414
/output/
1515
/scripts/
1616
/elasticsearch/
17+
/docker/*.zip
1718
# When executing tests in alphabetical order, Maven generates temporary
1819
# files with names like this:
1920
#

‎AUTHORS.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111

1212
# Please keep the list sorted.
1313

14+
Amazon
1415
Aurelius
1516
DataStax
1617
Dylan Bethune-Waddell <dylan.bethune.waddell@mail.utoronto.ca>

‎BUILDING.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,37 @@ To build with only the TinkerPop tests\*:
3030
mvn clean install -Dtest.skip.tp=false -DskipTests=true
3131
```
3232

33+
## Building Docker Image for JanusGraph Gremlin Server
34+
35+
To build and run Docker images with JanusGraph and Gremlin Server, configured
36+
to run the BerkeleyJE backend and Elasticsearch (requires [Docker Compose](https://docs.docker.com/compose/)):
37+
38+
```bash
39+
mvn clean install -Pjanusgraph-release -Dgpg.skip=true -DskipTests=true && mvn docker:build -Pjanusgraph-docker -pl janusgraph-dist
40+
docker-compose -f janusgraph-dist/janusgraph-dist-hadoop-2/docker-compose.yml up
41+
```
42+
43+
Note the above `docker-compose` call launches containers in the foreground and is convenient for monitoring logs but add "-d" to instead run in the background.
44+
45+
To connect to the server in the same container on the console:
46+
47+
```bash
48+
docker exec -i -t janusgraph /var/janusgraph/bin/gremlin.sh
49+
```
50+
51+
Then you can interact with the graph on the console through the `:remote` interface:
52+
53+
```groovy
54+
gremlin> :remote connect tinkerpop.server conf/remote.yaml
55+
==>Configured localhost/127.0.0.1:8182
56+
gremlin> :remote console
57+
==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182] - type ':remote console' to return to local mode
58+
gremlin> GraphOfTheGodsFactory.load(graph)
59+
==>null
60+
gremlin> g = graph.traversal()
61+
==>graphtraversalsource[standardjanusgraph[berkeleyje:db/berkeley], standard]
62+
```
63+
3364
## Building on Eclipse IDE
3465
Note that this has only been tested on Eclipse Neon.2 Release (4.6.2) with m2e (1.7.0.20160603-1933) and m2e-wtp (1.3.1.20160831-1005) plugin.
3566

‎README.md‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
![JanusGraph logo](janusgraph.png)
22

3-
JanusGraph is a highly scalable [graph database](http://en.wikipedia.org/wiki/Graph_database) optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster. JanusGraph is a transactional database that can support thousands of concurrent users, complex traversals, and analytic graph queries.
3+
JanusGraph is a highly scalable [graph database](http://en.wikipedia.org/wiki/Graph_database)
4+
optimized for storing and querying large graphs with billions of vertices and edges
5+
distributed across a multi-machine cluster. JanusGraph is a transactional database that
6+
can support thousands of concurrent users, complex traversals, and analytic graph queries.
47

58
[![Build Status][travis-shield]][travis-link]
69
[![Maven][maven-shield]][maven-link]
@@ -21,7 +24,8 @@ JanusGraph is a highly scalable [graph database](http://en.wikipedia.org/wiki/Gr
2124

2225
## Learn More
2326

24-
The [project homepage](http://janusgraph.org) contains more information on JanusGraph and provides links to documentation, getting-started guides and release downloads.
27+
The [project homepage](http://janusgraph.org) contains more information on JanusGraph and
28+
provides links to documentation, getting-started guides and release downloads.
2529

2630
## Community
2731

‎janusgraph-dist/janusgraph-dist-hadoop-2/Dockerfile‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
FROM openjdk:8
2+
3+
ARG server_zip
4+
ADD ${server_zip} /var
5+
6+
RUN apt-get update -y && apt-get install -y zip && \
7+
server_base=`basename ${server_zip} .zip` && \
8+
unzip -q /var/${server_base}.zip -d /var && \
9+
ln -s /var/${server_base} /var/janusgraph
10+
11+
WORKDIR /var/janusgraph

‎janusgraph-dist/janusgraph-dist-hadoop-2/docker-compose.yml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
version: '2'
2+
services:
3+
janusgraph:
4+
image: janusgraph/server:latest
5+
container_name: janusgraph
6+
ports:
7+
- "8182:8182"
8+
volumes:
9+
- ./es/wait-for-es.sh:/usr/bin/wait-for-es.sh
10+
depends_on:
11+
- "elasticsearch"
12+
command: ["wait-for-es.sh", "elasticsearch", "./bin/gremlin-server.sh", "./conf/gremlin-server/gremlin-server-berkeleyje.yaml"]
13+
14+
elasticsearch:
15+
image: docker.elastic.co/elasticsearch/elasticsearch:5.3.2
16+
environment:
17+
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
18+
- "http.host=0.0.0.0"
19+
- "transport.host=127.0.0.1"
20+
ports:
21+
- "9200"
22+
volumes:
23+
- ./es/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
24+

‎janusgraph-dist/janusgraph-dist-hadoop-2/es/elasticsearch.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
cluster.name: "docker-cluster"
2+
network.host: 0.0.0.0
3+
4+
# minimum_master_nodes need to be explicitly set when bound on a public IP
5+
# set to 1 to allow single node clusters
6+
# Details: https://github.com/elastic/elasticsearch/pull/17288
7+
discovery.zen.minimum_master_nodes: 1
8+
9+
xpack.security.enabled : false
10+

‎janusgraph-dist/janusgraph-dist-hadoop-2/es/wait-for-es.sh‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
#!/bin/bash
2+
3+
set -e
4+
5+
host="$1"
6+
7+
shift
8+
cmd="$@"
9+
10+
until $(curl --output /dev/null --silent --head --fail http://$host:9200); do
11+
printf '.'
12+
sleep 5
13+
done
14+
15+
>&2 echo "Elasticsearch is up"
16+
17+
exec $cmd
18+

‎janusgraph-dist/pom.xml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
<pkgtools.dir>${project.basedir}/..</pkgtools.dir>
4040

4141
<skipDefaultDistroIT>${it.skip}</skipDefaultDistroIT>
42+
<docker.maven.version>0.4.13</docker.maven.version>
4243
</properties>
4344

4445
<modules>
@@ -622,6 +623,32 @@
622623
</build>
623624
</profile>
624625

626+
<profile>
627+
<id>janusgraph-docker</id>
628+
<build>
629+
<pluginManagement>
630+
<plugins>
631+
<plugin>
632+
<groupId>com.spotify</groupId>
633+
<artifactId>docker-maven-plugin</artifactId>
634+
<version>${docker.maven.version}</version>
635+
<configuration>
636+
<dockerDirectory>${project.basedir}/janusgraph-dist-hadoop-2</dockerDirectory>
637+
<buildArgs>
638+
<server_zip>target/janusgraph-${project.version}-hadoop2.zip</server_zip>
639+
</buildArgs>
640+
<forceTags>true</forceTags>
641+
<imageName>janusgraph/server</imageName>
642+
<imageTags>
643+
<imageTag>${project.version}</imageTag>
644+
</imageTags>
645+
</configuration>
646+
</plugin>
647+
</plugins>
648+
</pluginManagement>
649+
</build>
650+
</profile>
651+
625652
<profile>
626653
<id>janusgraph-release</id>
627654

‎janusgraph-dist/src/assembly/static/conf/gremlin-server/gremlin-server-berkeleyje.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
host: localhost
2+
port: 8182
3+
scriptEvaluationTimeout: 30000
4+
channelizer: org.apache.tinkerpop.gremlin.server.channel.WebSocketChannelizer
5+
graphs: {
6+
graph: conf/gremlin-server/janusgraph-berkeleyje-es-server.properties}
7+
plugins:
8+
- janusgraph.imports
9+
scriptEngines: {
10+
gremlin-groovy: {
11+
imports: [java.lang.Math],
12+
staticImports: [java.lang.Math.PI],
13+
scripts: [scripts/empty-sample.groovy]}}
14+
serializers:
15+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
16+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoLiteMessageSerializerV1d0, config: {ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
17+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GryoMessageSerializerV1d0, config: { serializeResultToString: true }}
18+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
19+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerGremlinV2d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
20+
- { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
21+
processors:
22+
- { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }}
23+
- { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor, config: { cacheExpirationTime: 600000, cacheMaxSize: 1000 }}
24+
metrics: {
25+
consoleReporter: {enabled: true, interval: 180000},
26+
csvReporter: {enabled: true, interval: 180000, fileName: /tmp/gremlin-server-metrics.csv},
27+
jmxReporter: {enabled: true},
28+
slf4jReporter: {enabled: true, interval: 180000},
29+
gangliaReporter: {enabled: false, interval: 180000, addressingMode: MULTICAST},
30+
graphiteReporter: {enabled: false, interval: 180000}}
31+
maxInitialLineLength: 4096
32+
maxHeaderSize: 8192
33+
maxChunkSize: 8192
34+
maxContentLength: 65536
35+
maxAccumulationBufferComponents: 1024
36+
resultIterationBatchSize: 64
37+
writeBufferLowWaterMark: 32768
38+
writeBufferHighWaterMark: 65536
39+

0 commit comments

Comments
 (0)