hive 查询hudi 报错，
已将hudi 对应包hudi-hadoop-mr-bundle-0.9.0.jar放入lib目录下并重启hive
select count(1) from table ;
ERROR : Vertex failed, vertexName=Map 1, vertexId=vertex_1642524380391_3302_1_00, diagnostics=[Vertex vertex_1642524380391_3302_1_00 [Map 1] killed/failed due to:INIT_FAILURE, Fail to create InputInitializerManager, org.apache.tez.dag.api.TezReflectionException: Unable to instantiate class with 1 arguments: org.apache.hadoop.hive.ql.exec.tez.HiveSplitGeneratorat org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:71)at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:89)at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:152)at org.apache.tez.dag.app.dag.RootInputInitializerManager$1.run(RootInputInitializerManager.java:148)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:422)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)at org.apache.tez.dag.app.dag.RootInputInitializerManager.createInitializer(RootInputInitializerManager.java:148)at org.apache.tez.dag.app.dag.RootInputInitializerManager.runInputInitializers(RootInputInitializerManager.java:121)at org.apache.tez.dag.app.dag.impl.VertexImpl.setupInputInitializerManager(VertexImpl.java:4123)at org.apache.tez.dag.app.dag.impl.VertexImpl.access$3100(VertexImpl.java:208)at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.handleInitEvent(VertexImpl.java:2933)at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2880)at org.apache.tez.dag.app.dag.impl.VertexImpl$InitTransition.transition(VertexImpl.java:2862)at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)at org.apache.hadoop.yarn.state.StateMachineFactory.access$500(StateMachineFactory.java:46)at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:487)at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:59)at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1958)at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:207)at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2317)at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:2303)at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:180)at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:115)at java.lang.Thread.run(Thread.java:748)Caused by: java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.apache.tez.common.ReflectionUtils.getNewInstance(ReflectionUtils.java:68)..、25 moreCaused by: java.lang.RuntimeException: Failed to load plan: hdfs://xxx/tmp/hive/root/8344ec71-67c8-4733-8dd1-41123f1e1729/hive_2022-02-08_16-54-15_006_1554035332667755211-922/root/_tez_scratch_dir/fcdb3e29-0d57-40cc-8bfe-21cfa9490cd6/map.xmlat org.apache.hadoop.hive.ql.exec.Utilities.getbaseWork(Utilities.java:509)at org.apache.hadoop.hive.ql.exec.Utilities.getMapWork(Utilities.java:342)at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.
解决办法:
hive lib目录同级目录下创建auxlib，并将包放入此目录后 重启Hive
必须要auxlib目录
select count(1) from document_ro; INFO : Compiling command(queryId=hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644): select count(1) from document_roINFO : Semantic Analysis Completed (retrial = false)INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)INFO : Completed compiling command(queryId=hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644); Time taken: 1.619 secondsINFO : Executing command(queryId=hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644): select count(1) from document_roINFO : Query ID = hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644INFO : Total jobs = 1INFO : Launching Job 1 out of 1INFO : Starting task [Stage-1:MAPRED] in serial modeINFO : Subscribed to counters: [] for queryId: hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644INFO : Tez session hasn't been created yet、Opening sessionINFO : Dag name: select count(1) from document_ro (Stage-1)INFO : Status: Running (Executing on YARN cluster with App id application_1644382793459_0002)---------------------------------------------------------------------------------------------- VERTICES MODE STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED ----------------------------------------------------------------------------------------------Map 1 .........、container SUCCEEDED 3 3 0 0 0 0 Reducer 2 .....、container SUCCEEDED 1 1 0 0 0 0 ----------------------------------------------------------------------------------------------VERTICES: 02/02 [==========================>>] 100% ELAPSED TIME: 3.55 s ----------------------------------------------------------------------------------------------INFO : Status: DAG finished successfully in 3.45 secondsINFO : INFO : Query Execution SummaryINFO : ----------------------------------------------------------------------------------------------INFO : OPERATION DURATIONINFO : ----------------------------------------------------------------------------------------------INFO : Compile Query 1.62sINFO : Prepare Plan 4.15sINFO : Get Query Coordinator (AM) 0.01sINFO : Submit Plan 0.23sINFO : Start DAG 0.89sINFO : Run DAG 3.45sINFO : ----------------------------------------------------------------------------------------------INFO : INFO : Task Execution SummaryINFO : ----------------------------------------------------------------------------------------------INFO : VERTICES DURATION(ms) CPU_TIME(ms) GC_TIME(ms) INPUT_RECORDS OUTPUT_RECORDSINFO : ----------------------------------------------------------------------------------------------INFO : Map 1 1925.00 17,270 126 31,623 3INFO : Reducer 2 1.00 600 0 3 0INFO : ----------------------------------------------------------------------------------------------INFO : INFO : org.apache.tez.common.counters.DAGCounter:INFO : NUM_SUCCEEDED_TASKS: 4INFO : TOTAL_LAUNCHED_TASKS: 4INFO : DATA_LOCAL_TASKS: 1INFO : RACK_LOCAL_TASKS: 2INFO : AM_CPU_MILLISECONDS: 3110INFO : AM_GC_TIME_MILLIS: 0INFO : File System Counters:INFO : FILE_BYTES_READ: 57INFO : FILE_BYTES_WRITTEN: 174INFO : HDFS_BYTES_READ: 48409293INFO : HDFS_BYTES_WRITTEN: 105INFO : HDFS_READ_OPS: 303INFO : HDFS_WRITE_OPS: 2INFO : HDFS_OP_CREATE: 1INFO : HDFS_OP_GET_FILE_STATUS: 153INFO : HDFS_OP_OPEN: 150INFO : HDFS_OP_RENAME: 1INFO : org.apache.tez.common.counters.TaskCounter:INFO : SPILLED_RECORDS: 0INFO : NUM_SHUFFLED_INPUTS: 3INFO : NUM_FAILED_SHUFFLE_INPUTS: 0INFO : GC_TIME_MILLIS: 126INFO : TASK_DURATION_MILLIS: 5607INFO : CPU_MILLISECONDS: 17870INFO : PHYSICAL_MEMORY_BYTES: 4219469824INFO : VIRTUAL_MEMORY_BYTES: 36327419904INFO : COMMITTED_HEAP_BYTES: 4219469824INFO : INPUT_RECORDS_PROCESSED: 82INFO : INPUT_SPLIT_LENGTH_BYTES: 26237450INFO : OUTPUT_RECORDS: 3INFO : OUTPUT_LARGE_RECORDS: 0INFO : OUTPUT_BYTES: 12INFO : OUTPUT_BYTES_WITH_OVERHEAD: 36INFO : OUTPUT_BYTES_PHYSICAL: 150INFO : ADDITIONAL_SPILLS_BYTES_WRITTEN: 0INFO : ADDITIONAL_SPILLS_BYTES_READ: 0INFO : ADDITIONAL_SPILL_COUNT: 0INFO : SHUFFLE_BYTES: 78INFO : SHUFFLE_BYTES_DECOMPRESSED: 36INFO : SHUFFLE_BYTES_TO_MEM: 53INFO : SHUFFLE_BYTES_TO_DISK: 0INFO : SHUFFLE_BYTES_DISK_DIRECT: 25INFO : SHUFFLE_PHASE_TIME: 193INFO : FIRST_EVENT_RECEIVED: 91INFO : LAST_EVENT_RECEIVED: 190INFO : HIVE:INFO : CREATED_FILES: 1INFO : DESERIALIZE_ERRORS: 0INFO : RECORDS_IN_Map_1: 31623INFO : RECORDS_OUT_0: 1INFO : RECORDS_OUT_INTERMEDIATE_Map_1: 3INFO : RECORDS_OUT_INTERMEDIATE_Reducer_2: 0INFO : RECORDS_OUT_OPERATOR_FS_11: 1INFO : RECORDS_OUT_OPERATOR_GBY_10: 1INFO : RECORDS_OUT_OPERATOR_GBY_8: 3INFO : RECORDS_OUT_OPERATOR_MAP_0: 0INFO : RECORDS_OUT_OPERATOR_RS_9: 3INFO : RECORDS_OUT_OPERATOR_SEL_7: 31623INFO : RECORDS_OUT_OPERATOR_TS_0: 31623INFO : TaskCounter_Map_1_INPUT_document_ro:INFO : INPUT_RECORDS_PROCESSED: 79INFO : INPUT_SPLIT_LENGTH_BYTES: 26237450INFO : TaskCounter_Map_1_OUTPUT_Reducer_2:INFO : ADDITIONAL_SPILLS_BYTES_READ: 0INFO : ADDITIONAL_SPILLS_BYTES_WRITTEN: 0INFO : ADDITIONAL_SPILL_COUNT: 0INFO : OUTPUT_BYTES: 12INFO : OUTPUT_BYTES_PHYSICAL: 150INFO : OUTPUT_BYTES_WITH_OVERHEAD: 36INFO : OUTPUT_LARGE_RECORDS: 0INFO : OUTPUT_RECORDS: 3INFO : SPILLED_RECORDS: 0INFO : TaskCounter_Reducer_2_INPUT_Map_1:INFO : FIRST_EVENT_RECEIVED: 91INFO : INPUT_RECORDS_PROCESSED: 3INFO : LAST_EVENT_RECEIVED: 190INFO : NUM_FAILED_SHUFFLE_INPUTS: 0INFO : NUM_SHUFFLED_INPUTS: 3INFO : SHUFFLE_BYTES: 78INFO : SHUFFLE_BYTES_DECOMPRESSED: 36INFO : SHUFFLE_BYTES_DISK_DIRECT: 25INFO : SHUFFLE_BYTES_TO_DISK: 0INFO : SHUFFLE_BYTES_TO_MEM: 53INFO : SHUFFLE_PHASE_TIME: 193INFO : TaskCounter_Reducer_2_OUTPUT_out_Reducer_2:INFO : OUTPUT_RECORDS: 0INFO : org.apache.hadoop.hive.ql.exec.tez.HiveInputCounters:INFO : GROUPED_INPUT_SPLITS_Map_1: 3INFO : INPUT_DIRECTORIES_Map_1: 11INFO : INPUT_FILES_Map_1: 50INFO : RAW_INPUT_SPLITS_Map_1: 50INFO : Completed executing command(queryId=hive_20220209132712_713df3f8-91ab-46a2-8e1a-17bda938a644); Time taken: 8.802 secondsINFO : OK+--------+| _c0 |+--------+| 31623 |+--------+1 row selected (10.553 seconds)