Skip to content

[SPARK-28390][SQL][PYTHON][TESTS] [FOLLOW-UP] Update the TODO with actual blocking JIRA IDs #25415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shivusondur wants to merge 1 commit into from
Closed

[SPARK-28390][SQL][PYTHON][TESTS] [FOLLOW-UP] Update the TODO with actual blocking JIRA IDs #25415

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sql/core/src/test/resources/sql-tests/inputs/udf/pgSQL/udf-select_having.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
-- https://github.com/postgres/postgres/blob/REL_12_BETA2/src/test/regress/sql/select_having.sql
--
-- This test file was converted from inputs/pgSQL/select_having.sql
-- TODO: We should add UDFs in GROUP BY clause when [SPARK-28445] is resolved.
-- TODO: We should add UDFs in GROUP BY clause when [SPARK-28386] and [SPARK-26741] is resolved.
Copy link
Member

@dongjoon-hyun dongjoon-hyun Aug 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SPARK-28445 was wrong from the beginning, @shivusondur ?

Copy link
Contributor Author

@shivusondur shivusondur Aug 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dongjoon-hyun
After resolving the SPARK-28445 also, test were failing and found [SPARK-28386] and [SPARK-26741] are blocking it.

for furher details follow #25215 (comment)

Copy link
Member

@HyukjinKwon HyukjinKwon Aug 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit lost about this or I forget something.
Can't we add UDF in group-by clause (resolved in SPARK-28445)?

Copy link
Contributor Author

@shivusondur shivusondur Aug 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@HyukjinKwon
From this
#25215 (comment)
I thought I need to update todo with blocking jira numbers

Copy link
Member

@HyukjinKwon HyukjinKwon Aug 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, right. I forgot. Can we enable all other tests with UDF in group-by and comment out the test?

-- !query 11
SELECT udf(b), udf(c) FROM test_having
GROUP BY udf(b), udf(c) HAVING udf(count(*)) = 1 ORDER BY udf(b), udf(c)
-- !query 11 schema
struct<>
-- !query 11 output
org.apache.spark.sql.AnalysisException
cannot resolve 'b' given input columns: [CAST(udf(cast(b as string)) AS INT), CAST(udf(cast(c as string)) AS STRING)]; line 2 pos 63

I guess we can still add some more tests?


-- load test data
CREATE TABLE test_having (a int, b int, c string, d string) USING parquet;
Expand Down