Esri JSON SerDe with Hive Date type column

[randallwhitman]

CREATE TABLE qd5(eqdate date);
insert OVERWRITE TABLE qd5

select to_date(eqdate) from qd0;

-- ok

CREATE TABLE qd3(eqdate date, shape binary)
ROW FORMAT SERDE 'com.esri.hadoop.hive.serde.EsriJsonSerDe'              
STORED AS INPUTFORMAT 'com.esri.json.hadoop.EnclosedJsonInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat';

insert OVERWRITE TABLE qd3
select to_date(eqdate), st_point(longitude,latitude) from qd0;

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"eqdate":"1898-06-29T18:36:00.00,52.0,172.0,0.0,7.6,ML,0,,,,AK,","latitude":null,"longitude":null,"depth":null,"magnitude":null,"magtype":null,"mbstations":null,"gap":null,"distance":null,"rms":null,"source":null,"eventid":null}
        at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:176)
        ... 8 more
Caused by: java.lang.IllegalStateException: No ObjectCodec defined for the generator, can only serialize simple wrapper types (type passed java.sql.Date)
        at org.codehaus.jackson.impl.JsonGeneratorBase._writeSimpleObject(JsonGeneratorBase.java:556)
        at org.codehaus.jackson.impl.JsonGeneratorBase.writeObject(JsonGeneratorBase.java:317)
        at org.codehaus.jackson.JsonGenerator.writeObjectField(JsonGenerator.java:1090)
        at com.esri.hadoop.hive.serde.BaseJsonSerDe.generateJsonFromWritable(BaseJsonSerDe.java:319)
        at com.esri.hadoop.hive.serde.BaseJsonSerDe.generateJsonFromValue(BaseJsonSerDe.java:305)
        at com.esri.hadoop.hive.serde.BaseJsonSerDe.serialize(BaseJsonSerDe.java:245)

        jsonGen.writeObjectField(label, poi.getPrimitiveJavaObject(value));

See Esri/gis-tools-for-hadoop#45

[randallwhitman]

@climbage What is your perspective on bug versus enhancement - date column with Esri Json SerDe ?

[randallwhitman]

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types

• TIMESTAMP - "Timestamps were introduced in Hive 0.8.0 (HIVE-2272)."
• DATE - "Dates were introduced in Hive 0.12.0 (HIVE-4055)."

[randallwhitman]

As type DATE was introduced in Hive-0.12, case DATE: (for setRowFieldFromParser) will not compile with prior versions of the Hive API. Options:

• Support TIMESTAMP but not DATE;
• Require Hive v0.12+ for compilation (should execute OK on 0.10 & 0.11 which will have no DATE columns);
• Shims ? ? (like elasticsearch-hadoop)