Skip to content

[Spark-4041][SQL]attributes names in table scan should converted to lowercase when compare with relation attributes #2884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
scwf wants to merge 4 commits into from
Closed

[Spark-4041][SQL]attributes names in table scan should converted to lowercase when compare with relation attributes #2884

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
294fcb7
attributes names in table scan should convert lowercase in neededColu…
scwf Oct 21, 2014
3ff3a80
more safer change
scwf Oct 22, 2014
dc74a24
use lowerName and add a test case for this issue
scwf Oct 25, 2014
6174046
use AttributeMap for this issue
scwf Oct 27, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ case class HiveTableScan(
// Specifies needed column IDs for those non-partitioning columns.
val neededColumnIDs =
attributes.map(a =>
relation.attributes.indexWhere(_.name == a.name): Integer).filter(index => index >= 0)
relation.attributes.indexWhere(
_.name.toLowerCase == a.name): Integer).filter(index => index >= 0)
Copy link
Contributor

@liancheng liancheng Oct 22, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: it would be safer if you use _.name.toLowerCase == a.name.toLowerCase.


ColumnProjectionUtils.appendReadColumnIDs(hiveConf, neededColumnIDs)
ColumnProjectionUtils.appendReadColumnNames(hiveConf, attributes.map(_.name))
Expand Down