MultipleOutputs would be hugely beneficial in conjunction with emr-s3-io. Unfortunately they appear to be incompatible on the current EMR hadoop version.
Perhaps this is hard to solve... But EMR runs on 0.20.205. This project uses the "new" mapreduce API rather than the mapred API. Unfortunately, MultipleOutputs in 0.20.205 only works on the "old" mapred API:
http://stackoverflow.com/questions/9876456/hadoop-multipleoutputs-addnamedoutput-throws-cannot-find-symbol
(Later solved in MAPREDUCE-370)
This creates a incompatibility between emr-s3-io and MultipleOutputs.
Backport emr-s3-io to the old mapred API? Or possibly do you have a fix for MultipleOutputs that would be compatible?
MultipleOutputs would be hugely beneficial in conjunction with emr-s3-io. Unfortunately they appear to be incompatible on the current EMR hadoop version.
Perhaps this is hard to solve... But EMR runs on 0.20.205. This project uses the "new" mapreduce API rather than the mapred API. Unfortunately, MultipleOutputs in 0.20.205 only works on the "old" mapred API:
http://stackoverflow.com/questions/9876456/hadoop-multipleoutputs-addnamedoutput-throws-cannot-find-symbol
(Later solved in MAPREDUCE-370)
This creates a incompatibility between emr-s3-io and MultipleOutputs.
Backport emr-s3-io to the old mapred API? Or possibly do you have a fix for MultipleOutputs that would be compatible?