FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask、Could not find status of job:job_1637104181115_796482:28:27’
终端报错日志见上,信息较少,需要在yarn上查看更多日志信息
INFO [Thread-68] org.apache.hadoop.service.AbstractService: Service JobHistoryEventHandler failed in state STOPPED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.FSLimitException$PathComponentTooLongException): The maximum path component name limit of job
xx.jhist_tmp in directory /tmp/hadoop-yarn/staging/history/done_intermediate/ide is exceeded: limit=255 length=258
其中xx.jhist_tmp超长, 这部分是根据hive的jobname来决定的,默认是从hql中的开头和结尾截取一部分,如果sql开头或结尾有中文注释的话,会被截取进来,并进行url编码,导致超长。hive在historyserver无法获得这个job的状态,报开头的错误。
同事在用hive临时表时遇到了这个错,就是因为 with tab as太短,把后面的中文注释截取进来了,去掉中文注释后问题解决。
with user_info as ( --注册量: 注册信息
还有种解决办法,增加job_name长度
set hive.jobname.length=10