主页 / unix / 问题 / 784297
Accepted
Grackleflint
Asked: 2024-09-30 16:41:51 +0800 CST

mdadm - 由于“没有有效的 v1.2 超级块，无法导入”错误，无法添加回替换的驱动器

  • 772

我有一个现有的 raid 6 阵列，其中一个驱动器出现故障，但是，当我尝试使用mdadm --manage /dev/md1 --add /dev/sde3mdadm 错误替换它时，我在内核日志中收到以下消息：

md: sde3 does not have a valid v1.2 superblock, not importing!

请注意，我已将替换驱动器归零，并使用将超级块反复归零mdadm --zero-superblock -e 1.2 --force /dev/sde3。添加似乎将新的超级块写入驱动器，将驱动器标记为备用，但实际上并未将其添加到阵列中，这可能是相关的。

我认为我犯的错误是，mdadm --manage md1 --remove /dev/sde3在驱动器发生故障之后，在尝试添加替换驱动器之前，我移除了发生故障的驱动器。

我尝试了将替换驱动器的超级块归零并组装带有或不带有替换驱动器的阵列的各种组合，但它们都因相同的错误而失败。

此时，我认为我唯一的选择是使用重新创建数组--assume-clean并将失败的插槽标记为missing。

以下是该阵列的信息：

/dev/md1:
           Version : 1.2
     Creation Time : Sun May 12 20:28:14 2013
        Raid Level : raid6
        Array Size : 22686329856 (21.13 TiB 23.23 TB)
     Used Dev Size : 3781054976 (3.52 TiB 3.87 TB)
      Raid Devices : 8
     Total Devices : 7
       Persistence : Superblock is persistent

       Update Time : Mon Sep 30 17:28:13 2024
             State : clean, degraded 
    Active Devices : 7
   Working Devices : 7
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : resync

              Name : playroom:raid  (local to host playroom)
              UUID : 34484e2b:99dc0604:10b071d5:c8012127
            Events : 176577

    Number   Major   Minor   RaidDevice State
      13       8       99        0      active sync   /dev/sdg3
      10       8      115        1      active sync   /dev/sdh3
      11       8       19        2      active sync   /dev/sdb3
       -       0        0        3      removed
       9       8       83        4      active sync   /dev/sdf3
       8       8       51        5      active sync   /dev/sdd3
      15       8       35        6      active sync   /dev/sdc3
      14       8        3        7      active sync   /dev/sda3

/dev/sda3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=655 sectors
          State : clean
    Device UUID : b3b21814:7f42ad3f:cb7e8d5b:5dfd0d22

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 16 sectors - bad blocks present.
       Checksum : ef2d77c6 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 7
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdb3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=655 sectors
          State : clean
    Device UUID : 77d07ff9:e5a670c0:cfb45916:717b98e8

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 16 sectors - bad blocks present.
       Checksum : d6aeae07 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdc3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=655 sectors
          State : clean
    Device UUID : 67b4abd6:f1e5b87f:0851dd21:9200e1b6

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 16 sectors - bad blocks present.
       Checksum : 4ceee719 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 6
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdd3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=655 sectors
          State : clean
    Device UUID : fe728bca:23485bc8:a5e5ac21:73c1a89a

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 16 sectors - bad blocks present.
       Checksum : 6d085d59 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 5
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sde3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=261864 sectors, after=655 sectors
          State : clean
    Device UUID : 16c61ab0:553c3ce5:d9b300b8:54b4fe14

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 264 sectors - bad blocks present.
       Checksum : 801fb4f3 - correct
         Events : 0

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : spare
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdf3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=261864 sectors, after=655 sectors
          State : clean
    Device UUID : a8368569:56a9356f:e158fc12:9a75fcf4

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 264 sectors - bad blocks present.
       Checksum : fe7faa91 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 4
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdg3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=655 sectors
          State : clean
    Device UUID : 4b25f2f9:65988664:86916d97:f0e3ac2a

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 16 sectors - bad blocks present.
       Checksum : 3e5f2e4b - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

/dev/sdh3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x8
     Array UUID : 34484e2b:99dc0604:10b071d5:c8012127
           Name : playroom:raid  (local to host playroom)
  Creation Time : Sun May 12 20:28:14 2013
     Raid Level : raid6
   Raid Devices : 8

 Avail Dev Size : 7562110607 sectors (3.52 TiB 3.87 TB)
     Array Size : 22686329856 KiB (21.13 TiB 23.23 TB)
  Used Dev Size : 7562109952 sectors (3.52 TiB 3.87 TB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=261864 sectors, after=655 sectors
          State : clean
    Device UUID : 948d0c31:03c19927:22f18cdd:0d84ffb2

    Update Time : Mon Sep 30 17:28:13 2024
  Bad Block Log : 512 entries available at offset 264 sectors - bad blocks present.
       Checksum : 6fec028 - correct
         Events : 176577

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAA.AAAA ('A' == active, '.' == missing, 'R' == replacing)

cat /proc/mdstat生成：

md1 : active raid6 sdg3[13] sda3[14] sdc3[15] sdd3[8] sdf3[9] sdb3[11] sdh3[10]
      22686329856 blocks super 1.2 level 6, 512k chunk, algorithm 2 [8/7] [UUU_UUUU]

unused devices: <none>

感谢 frostschutz 发现坏块不一致问题。mdadm --examine-badblocks转储如下：

Bad-blocks on /dev/sda3:
             3834808 for 512 sectors
             3835320 for 368 sectors
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sdb3:
             3834808 for 512 sectors
             3835320 for 368 sectors
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sdc3:
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sdd3:
             3834808 for 512 sectors
             3835320 for 368 sectors
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sde3:
                   0 for 0 sectors
                   --- repeats for 512 lines ---
Bad-blocks on /dev/sdf3:
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sdg3:
          1787061064 for 16 sectors
          1788696152 for 8 sectors
Bad-blocks on /dev/sdh3:
          1787061064 for 16 sectors
          1788696152 for 8 sectors
mdadm

1 个回答

  1. Best Answer

    好的，所以这应该是内核错误。由于某种原因，内核阻止在此处添加设备。

    解决方法是删除坏块日志：

    mdadm --zero-superblock /dev/sdx3 # your missing drive (sde3? check!)
mdadm --stop /dev/mdX
mdadm --assemble /dev/mdX --update=force-no-bbl /dev/sd…
mdadm /dev/mdX --add /dev/sdx3

    无论如何，您都必须删除坏块日志，因为否则这些坏块不会消失（当多个冗余驱动器共享相同的坏块时，情况确实如此）。坏块日志是一种半损坏的功能……

    这些扇区中的数据（如图所示--examine-badblocks）可能不同步，甚至可能根本不包含正确的数据，因此要当心那里潜在的数据损坏。

    另外，请检查您的日志，看看是否能找出触发这些坏块的原因，并对您的驱动器进行 SMART 检查，以防万一，坏块实际上是物理的。

    删除坏块日志后，添加设备应该可以再次工作，或者至少在我本地测试时是这样。

    如果仍然不起作用，您必须与 linux-raid 邮件列表上的开发人员讨论此问题（首先尝试使用当前内核）。

    解决驱动器问题后，运行 mdadm--action=check以检测阵列中的不匹配情况。如果有，除非您想恢复 raid 阵列中的不匹配扇区（老问题 — 答案不太好），否则请继续执行mdadm --action=repair。

    如果 raid456 阵列中的不匹配问题得不到纠正，从长远来看可能会导致数据损坏，因为它们不一定会通过新的写入得到纠正。

    • 0

相关问题