HDFS丢失块:不良 : 群集中有 1 个 丢失块 问题处理
Cloudera Manager管理的集群中hdfs报错
不良 : 群集中有 1 个 丢失块 块。群集中共有 6,956 个块。百分比 丢失块: 0.01%。 临界阈值:任意。
看来是有文件损坏了,解决方法:
1、检查缺失块:
[root@devnode01 ~]# hdfs fsck /
Connecting to namenode via http://devnode02:9870/fsck?ugi=root&path=%2F
FSCK started by root (auth:SIMPLE) from /10.23.74.196 for path / at Thu Jun 06 13:11:54 CST 2024
Status: CORRUPT
Number of data-nodes: 3
Number of racks: 1
Total dirs: 3969
Total symlinks: 0
Replicated Blocks:
Total size: 8730303943 B
Total files: 6953
Total blocks (validated): 6962 (avg. block size 1253993 B)
********************************
UNDER MIN REPL'D BLOCKS: 1 (0.014363688 %)
dfs.namenode.replication.min: 1
CORRUPT FILES: 1
CORRUPT BLOCKS: 1
CORRUPT SIZE: 35557852 B
********************************
Minimally replicated blocks: 6961 (99.985634 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 6 (0.08618213 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 2.9987073
Missing blocks: 0
Corrupt blocks: 1
Missing replicas: 6 (0.028727377 %)
Blocks queued for replication: 0
Erasure Coded Block Groups:
Total size: 0 B
Total files: 0
Total block groups (validated): 0
Minimally erasure-coded block groups: 0
Over-erasure-coded block groups: 0
Under-erasure-coded block groups: 0
Unsatisfactory placement block groups: 0
Average block group size: 0.0
Missing block groups: 0
Corrupt block groups: 0
Missing internal blocks: 0
Blocks queued for replication: 0
FSCK ended at Thu Jun 06 13:11:54 CST 2024 in 137 milliseconds
The filesystem under path '/' is CORRUPT
2、查看缺失块的详情
[root@devnode01 ~]# hdfs fsck / | egrep -v '^\.+$' | grep -v eplica
Connecting to namenode via http://devnode02:9870/fsck?ugi=root&path=%2F
FSCK started by root (auth:SIMPLE) from /10.23.74.196 for path / at Thu Jun 06 13:14:50 CST 2024
/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar: CORRUPT blockpool BP-977273909-10.23.74.199-1715236315519 block blk_1073745302
/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar: CORRUPT 1 blocks of total size 35557852 B.
Status: CORRUPT
Number of data-nodes: 3
Number of racks: 1
Total dirs: 3969
Total symlinks: 0
Total size: 8730303943 B
Total files: 6953
Total blocks (validated): 6962 (avg. block size 1253993 B)
********************************
UNDER MIN REPL'D BLOCKS: 1 (0.014363688 %)
CORRUPT FILES: 1
CORRUPT BLOCKS: 1
CORRUPT SIZE: 35557852 B
********************************
Missing blocks: 0
Corrupt blocks: 1
Erasure Coded Block Groups:
Total size: 0 B
Total files: 0
Total block groups (validated): 0
Minimally erasure-coded block groups: 0
Over-erasure-coded block groups: 0
Under-erasure-coded block groups: 0
Unsatisfactory placement block groups: 0
Average block group size: 0.0
Missing block groups: 0
Corrupt block groups: 0
Missing internal blocks: 0
FSCK ended at Thu Jun 06 13:14:50 CST 2024 in 104 milliseconds
The filesystem under path '/' is CORRUPT
在hdfs的web管理界面上也会直接显示:
/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar文件损坏了.
3、删除损坏的文件
[root@devnode01 ~]# hdfs fsck -delete /sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar
Connecting to namenode via http://devnode02:9870/fsck?ugi=root&delete=1&path=%2Fsparkforyarn%2Fjars%2Fhive-exec-2.1.1-cdh6.1.1.jar
FSCK started by root (auth:SIMPLE) from /10.23.74.196 for path /sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar at Thu Jun 06 13:20:13 CST 2024
/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar: CORRUPT blockpool BP-977273909-10.23.74.199-1715236315519 block blk_1073745302
/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar: CORRUPT 1 blocks of total size 35557852 B.
Status: CORRUPT
Number of data-nodes: 3
Number of racks: 1
Total dirs: 0
Total symlinks: 0
Replicated Blocks:
Total size: 35557852 B
Total files: 1
Total blocks (validated): 1 (avg. block size 35557852 B)
********************************
UNDER MIN REPL'D BLOCKS: 1 (100.0 %)
dfs.namenode.replication.min: 1
CORRUPT FILES: 1
CORRUPT BLOCKS: 1
CORRUPT SIZE: 35557852 B
********************************
Minimally replicated blocks: 0 (0.0 %)
Over-replicated blocks: 0 (0.0 %)
Under-replicated blocks: 0 (0.0 %)
Mis-replicated blocks: 0 (0.0 %)
Default replication factor: 3
Average block replication: 0.0
Missing blocks: 0
Corrupt blocks: 1
Missing replicas: 0
Blocks queued for replication: 0
Erasure Coded Block Groups:
Total size: 0 B
Total files: 0
Total block groups (validated): 0
Minimally erasure-coded block groups: 0
Over-erasure-coded block groups: 0
Under-erasure-coded block groups: 0
Unsatisfactory placement block groups: 0
Average block group size: 0.0
Missing block groups: 0
Corrupt block groups: 0
Missing internal blocks: 0
Blocks queued for replication: 0
FSCK ended at Thu Jun 06 13:20:13 CST 2024 in 12 milliseconds
The filesystem under path '/sparkforyarn/jars/hive-exec-2.1.1-cdh6.1.1.jar' is CORRUPT
不带文件路径可自动删除所有损坏的文件块。
最后复制hive-exec-2.1.1-cdh6.1.1.jar文件到对应/sparkforyarn/jars/路径下即可,如果不能删除,需要从上面命令中找到发生在哪台机器上,查看日志,在具体分析。
Tag标签:「Cloudera hdfs」更新时间:「2024-06-06 13:59:45」阅读次数:「975」