Skip to content

SPARK-1349: spark-shell gets its own command history#267

Closed
aarondav wants to merge 1 commit intoapache:masterfrom
aarondav:repl
Closed

SPARK-1349: spark-shell gets its own command history#267
aarondav wants to merge 1 commit intoapache:masterfrom
aarondav:repl

Conversation

@aarondav
Copy link
Copy Markdown
Contributor

Currently, spark-shell shares its command history with scala repl.

This fix is simply a modification of the default FileBackedHistory file setting:
https://github.com/scala/scala/blob/master/src/repl/scala/tools/nsc/interpreter/session/FileBackedHistory.scala#L77

@AmplabJenkins
Copy link
Copy Markdown

Merged build triggered. Build is starting -or- tests failed to complete.

@AmplabJenkins
Copy link
Copy Markdown

Merged build started. Build is starting -or- tests failed to complete.

@AmplabJenkins
Copy link
Copy Markdown

Merged build finished. Build is starting -or- tests failed to complete.

@AmplabJenkins
Copy link
Copy Markdown

Build is starting -or- tests failed to complete.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13563/

@aarondav
Copy link
Copy Markdown
Contributor Author

Jenkins, retest this please.

@AmplabJenkins
Copy link
Copy Markdown

Merged build triggered. Build is starting -or- tests failed to complete.

@AmplabJenkins
Copy link
Copy Markdown

Merged build started. Build is starting -or- tests failed to complete.

@AmplabJenkins
Copy link
Copy Markdown

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link
Copy Markdown

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13564/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wondering - does this handle systems with different path separators correctly?

Does historyFile need to be scala.reflect.io.File? Or can it be a java.io.File? It seems a little heavyweight to pull in the reflection library to create a file path, but it makes sense if this is the needed type of historyFile.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I did this was to fully mimic the code from the Scala interpreter. I think this will make it easier to compare in case the Scala version is updated some time in the future, but you're right we could use a lighter weight construct, we just risk having different behavior than the standard Scala shell history.

Not a big deal to me, but I'm inclined to stick with cloning the Scala shell behavior. Let me know if you still prefer the java.io.File route, I'd be happy to change to that.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sounds good - just wondering why it was there.

@pwendell
Copy link
Copy Markdown
Contributor

pwendell commented Apr 7, 2014

Thanks, merged.

@asfgit asfgit closed this in 7ce52c4 Apr 7, 2014
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
Currently, spark-shell shares its command history with scala repl.

This fix is simply a modification of the default FileBackedHistory file setting:
https://github.com/scala/scala/blob/master/src/repl/scala/tools/nsc/interpreter/session/FileBackedHistory.scala#L77

Author: Aaron Davidson <aaron@databricks.com>

Closes apache#267 from aarondav/repl and squashes the following commits:

f9c62d2 [Aaron Davidson] SPARK-1349: spark-shell gets its own command history separate from scala repl
mccheah pushed a commit to mccheah/spark that referenced this pull request Oct 12, 2017
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
…oud and orange clouds (apache#267)

* Add support for running ansible fuctional tests against opentelekomcloud and orange clouds
arjunshroff pushed a commit to arjunshroff/spark that referenced this pull request Nov 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants