Over the years, I’ve had to replaced failed raid1 drives or change drives so that I could use larger disks in an existing raid1 array. Since it seems I have to google the process every time, I figured I’d take a moment to jot down the process.
In my example I have two hard drives in software RAID1 (mdadm), /dev/sda and /dev/sdb. The partitions are /dev/sda1, /dev/sda3, /dev/sdb1 and /dev/sdb3. They look like this
/dev/sda1 and /dev/sdb1 are RAID1 array /dev/md0 /dev/sda3 and /dev/sdb3 are RAID1 array /dev/md1/dev/md0 is my /boot partition /dev/md1 is my / partition
This example will cover replacing a single failed drive, and also replacing both drives with larger disks, while maintaining the RAID1 array.
For now, I’ll pretend that /dev/sdb has failed and we will replace it.
Note: If you are replacing /dev/sda as you follow along, you’ll want to be sure that you have Grub installed on /dev/sdb first. See further down for details on how to do that.
Shut down your server
shutdown -h now
Remove the /dev/sdb hard drive from the server and replace it with a new drive. Be sure the drive is the same size or greater. If you add a drive that is the same size as the original, it is recommended that it’s the same model as /dev/sda or you may run into complications (not all same-size drives contain the same amount of sectors)
Once the server has booted back up, you’ll want to copy the exact partition structure from /dev/sda to your new drive at /dev/sdb
* If your server did not boot and got stuck, it’s likely that grub was not installed on the remaining drive.
** If the replacement drive had an OS and this OS is booting instead of the one you want, go into your BIOS and swap which is the primary boot drive
Now we’ll replicate the disk partition structure between by copying it from the raid1 disk to the new disk you’ve just installed.
sfdisk -d /dev/sda | sfdisk /dev/sdb
Make sure you get the order of these correct or you’ll destroy your data!
To check that both drives have an identical partition structure, do:
If the are different then you’re likely using two disks that are the same size but different models. The new drive needs to contain at least the same amount of sectors as the old drive – or greater.
Now we’ll add /dev/sdb into the raid array so that it can begin synchronizing.
mdadm --manage /dev/md0 --add /dev/sdb1
Do the same for the / partition
mdadm --manage /dev/md1 --add /dev/sdb3
Both arrays should now be syncing, though your md0 may already be complete if it’s a small partition.
This will you show you the current status of the synchronization process.
Personalities : [raid1] md0 : active raid1 sda1 sdb1 104320 blocks [2/2] [UU] md1 : active raid1 sda3 sdb3 73850688 blocks [2/1] [_U] [=>...................] recovery = 5.6% (4180352/73850688) finish=281.6min speed=4120K/sec unused devices: <none>
When that is complete, you should see [UU] in all arrays, like in md0 example above.
Now that the synch is done, we should be sure to install the Grub bootloader to the new drive as a failover. If your primary drive fails you want to be able to boot off the secondary, right?
# grub grub> find /grub/stage1
You’ll likely see:
This means both /dev/sda1 and /dev/sdb1 contain grub files, but grub is really only installed on /dev/sda1 right now.
grub> device (hd0) /dev/sdb grub> root (hd0,0) grub> setup (hd0) grub> quit
This is like telling grub that (hd0) is refer instead to sdb and then proceed to set it up on /dev/sdb1
If you’re only replacing a failed drive, you can stop here. You should be done. However if you’re replacing both drives and installing larger ones, continue on, but first be sure that your raid synchronization process is complete…
Shut down the server so that you can remove the second disk.
shutdown -h now
Pull out the /dev/sda drive and replace it with your new larger drive. You may need to go into your BIOS and set the secondary drive as the primary boot drive. Since we’ve already complete synching to /dev/sdb in the process above, it’s now the drive with the data we want.
If your server still doesn’t boot up after that, it’s likely grub wasn’t installed correctly on /dev/sdb. Plug back in /dev/sda, boot it up, and be sure to follow the grub install mentioned above.
Ok, so your server is back online now. We’ll need to match the partition structure from /dev/sdb onto /dev/sda
sfdisk -d /dev/sdb | sfdisk /dev/sda
Verify they’re identical using
Now we’ll add /dev/sda1 to /dev/md0 and /dev/sda3 to /dev/md1
mdadm --manage /dev/md0 --add /dev/sda1 mdadm --manage /dev/md1 --add /dev/sda3
To see the progress…
cat /proc/mdstatPersonalities : [raid1] md0 : active raid1 sda1 sdb1 104320 blocks [2/2] [UU] md1 : active raid1 sda3 sdb3 73850688 blocks [2/1] [_U] [=>...................] recovery = 5.6% (4180352/73850688) finish=281.6min speed=4120K/sec unused devices: <none>
Wait for the synchronization process to complete by checking /proc/mdstat every now and then.
Installing Grub onto /dev/sda (you may not need to do this, depending on how you got here). It generally won’t hurt to do it if you’re not sure.
# grub grub> find /grub/stage1
Again, this is because both disks contain grub, but grub is currently only installed on /dev/sdb (hd1,0)
grub> root (hd0,0) grub> setup (hd0) grub> quit
You should now have two new drives in your raid1 array, both with the original data from your old drives. In addition, both drives have grub installed, so should the primary disk fail, the secondary will still be bootable.
Today, I started getting the following error while trying to update some packages using the yum command.
error: rpmts_HdrFromFdno: Header V3 RSA/SHA1 Signature, key ID c105b9de: BAD
It seems that the signatures have become corrupt somehow. Here’s what I did to fix it.
First I went into the root directory
# cd /root
I then downloaded the NSS Softokn Freebl rpm (link below is recent as of January 27 2015)
# wget http://mirror.centos.org/centos-6/6.6/updates/x86_64/Packages/nss-softokn-3.14.3-19.el6_6.x86_64.rpm
# rpm2cpio nss-softokn-freebl-3.14.3-19.el6_6.x86_64.rpm | cpio -idmv
Then I copied the files to the correct location
# cp ./lib64/libfreeblpriv3.* /lib64
Lastly, I ran through the update again
# yum update
That took care of it for me.
It looks like Research in Motion has done it! Break the 100k app mark for BB10, that is.
Last week, Research in Motion reported that during the two days the Blackberry 10 port-a-thon ran that there were 15,000 submissions. Today, Alec Saunders, VP Developer Relations at Research In Motion announced that, over the last day and a half, developers have submitted yet another 19,000 apps (19,071 to be exact) for BB10’s pre-launch. That brings the total to 34,000 from the port-a-thons alone, meaning there could be as many as 104,000 apps available by the January 30th launch date of the Blackberry 10 operating system.
A date for the new Blackberry Z10 phone has not been officially released yet, but dates as early as the end of February have been reported.
While I’m not a huge user of apps myself, this bodes well for RIM as they attempt to compete with Microsoft’s Windows Phone (by app count, BB is obliterating MS) for third position in the market behind iPhone and Android. RIM has mostly focused on the business user, which they have done a very good job of, but they’ve lost a lot of ground over the last few years as their products have become stale. From everything I’ve seen, BB10 is shaping up to be not only the greatest business phone operating system ever, especially for BYOD businesses, but with BB10’s Balance (basically two completely sand-boxed yet simultaneously-accessible BB10 instances on a single phone – 1 business and 1 personal), the consumer market is sure to take up the devices.
For the past few years many businesses, analysts, and consumers have written Blackberry off. Especially with very little new being produced over the last year and a half. It seems that all RIM’s time and effort has been put into BB10, which has been delayed quite some time; but for the better in my opinion. Releasing a half-assed phone/OS would have certainly meant the demise of the Blackberry. All the extra time and effort that they’ve put in to make things just right is really shaping up to have been the right move.
Personally I’ll be waiting for the QWERTY edition phone as I love having the physical feedback that a physical keyboard provides, but from all the videos I’ve seen of the touchscreen version, BB10’s on-screen keyboard looks like it’ll allow for faster typing than any other keyboard ever created.
In other news, I have surpassed the 1-year mark of developing my video game, Super Space Trooper. Be sure to check that out. I’m hoping to have it completed in five months and launched on the PlayStation Network another three months after that. There’s no word yet on whether or not Unity3D’s gaming engine will be ported to BB10, but if they do port it you can be sure I’ll be releasing Super Space Trooper to that platform as well.