欢迎您访问365答案网,请分享给你的朋友!
生活常识 学习资料

Hive—进阶—MSCK

时间:2023-04-21
MSCK

MSCK REPAIR TABLE命令主要是用来解决通过hdfs dfs -put或者hdfs api写入hive分区表的数据在hive中无法被查询到的问题。

我们知道hive有个服务叫metastore,这个服务主要是存储一些元数据信息,比如数据库名,表名或者表的分区等等信息。如果不是通过hive的insert等插入语句,很多分区信息在metastore中是没有的。

MSCK命令

Hive stores a list of partitions for each table in its metastore、If, however, new partitions are directly added to HDFS (say by using hadoop fs -put command) or removed from HDFS, the metastore (and hence Hive) will not be aware of these changes to partition information unless the user runs ALTER TABLE table_name ADD/DROP PARTITION commands on each of the newly added or removed partitions, respectively.However, users can run a metastore check command with the repair table option:MSCK [REPAIR] TABLE table_name [ADD/DROP/SYNC PARTITIONS];which will update metadata about partitions to the Hive metastore for partitions for which such metadata doesn't already exis

Copyright © 2016-2020 www.365daan.com All Rights Reserved. 365答案网 版权所有 备案号:

部分内容来自互联网,版权归原作者所有,如有冒犯请联系我们,我们将在三个工作时内妥善处理。