Handling bad records during sqoop import or export

流过昼夜 提交于 2020-01-02 08:51:08

问题


I looked at the options provided by sqoop export operation but could not find any options to handle bad records. For example once in a while it is possible that a character is present where a number is expected in a huge set of records. Is there a way to handle these scenarios in sqoop without failing the job and providing the bad records in a file.


回答1:


Sqoop currently expects that the data to export is clean and do not provide facilities to handle corrupted data. You can use MR/Pig/Hive job to clean your data up, prior using Sqoop to export them.



来源:https://stackoverflow.com/questions/18446911/handling-bad-records-during-sqoop-import-or-export

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!