Skip to content

[SPARK-2963] [SQL] There no documentation about building to use HiveServer and CLI for SparkSQL #1885

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sarutak wants to merge 4 commits into from
Closed

[SPARK-2963] [SQL] There no documentation about building to use HiveServer and CLI for SparkSQL #1885

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c88fa93
Added a description about building to use HiveServer and CLI for Spar…
sarutak Aug 11, 2014
6e6645a
Merge branch 'master' of git://git.apache.org/spark into SPARK-2963
sarutak Aug 11, 2014
07c59fc
Added a description about how to build to use HiveServer and CLI for …
sarutak Aug 11, 2014
ed53329
Modified description and notaton of proper noun
sarutak Aug 11, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,15 @@ If your project is built with Maven, add this to your POM file's `<dependencies>
</dependency>


## A Note About HiveServer and CLI for SparkSQL
Copy link
Contributor

@liancheng liancheng Aug 11, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be "Hive Thrift server", which is compatible with HiveServer2, rather than HiveServer. And usually we use "Spark SQL" instead of "SparkSQL".

Copy link
Member Author

@sarutak sarutak Aug 11, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In sql-programming-guide.md, it's called "Thrift JDBC server".
Should I use this notation?
The latest PR uses "Thrift JDBC server".


SparkSQL supports HiveServer and CLI.
See README.md in sql project for more information about those features.
Copy link
Contributor

@liancheng liancheng Aug 11, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, actually sql/README.md doesn't provide much useful information about Hive Thrift server and CLI... Maybe pointing readers to the corresponding sections of the SQL programming guide can be more helpful?

Copy link
Member Author

@sarutak sarutak Aug 11, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it's wrong for programming guide.

You can use those features by setting `-Phive-thriftserver` when building spark as follows.
Copy link
Contributor

@liancheng liancheng Aug 11, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please capitalize "Spark" :)


$ sbt/sbt -Phive-thriftserver assembly


## Configuration

Please refer to the [Configuration guide](http://spark.apache.org/docs/latest/configuration.html)
Expand Down
9 changes: 9 additions & 0 deletions docs/building-with-maven.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,15 @@ mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package
mvn -Pyarn-alpha -Phadoop-2.3 -Dhadoop.version=2.3.0 -Dyarn.version=0.23.7 -DskipTests clean package
{% endhighlight %}

# Building HiveServer and CLI for SparkSQL

SparkSQL supports HiveServer and CLI.
See README.md in sql project for more information about those features.
You can use those features by setting `-Phive-thriftserver` when building spark as follows.
{% highlight bash %}
mvn -Phive-thriftserver assembly
{% endhighlight %}

# Spark Tests in Maven

Tests are run by default via the [ScalaTest Maven plugin](http://www.scalatest.org/user_guide/using_the_scalatest_maven_plugin).
Expand Down