Category Archives: Linux & Technical

Replace a Bad Drive, or Replace With Larger Disks – Software RAID1 (mdadm)

Replace a Bad Drive in Software RAID1, or Replace Drives With Larger Drives in Software RAID1

Over the years, I’ve had to replaced failed raid1 drives or change drives so that I could use larger disks in an existing raid1 array.  Since it seems I have to google the process every time, I figured I’d take a moment to jot down the process.

1. My Configuration

In my example I have two hard drives in software RAID1 (mdadm), /dev/sda and /dev/sdb.  The partitions are /dev/sda1, /dev/sda3, /dev/sdb1 and /dev/sdb3.  They look like this

/dev/sda1 and /dev/sdb1 are RAID1 array /dev/md0
/dev/sda3 and /dev/sdb3 are RAID1 array /dev/md1
/dev/md0 is my /boot partition
/dev/md1 is my / partition

 

This example will cover replacing a single failed drive, and also replacing both drives with larger disks, while maintaining the RAID1 array.

For now, I’ll pretend that /dev/sdb has failed and we will replace it.

Note: If you are replacing /dev/sda as you follow along, you’ll want to be sure that you have Grub installed on /dev/sdb first.  See further down for details on how to do that.

 

2. Removing the Drive

Shut down your server

shutdown -h now

Remove the /dev/sdb hard drive from the server and replace it with a new drive.  Be sure the drive is the same size or greater.  If you add a drive that is the same size as the original, it is recommended that it’s the same model as /dev/sda or you may run into complications (not all same-size drives contain the same amount of sectors)

Once the server has booted back up, you’ll want to copy the exact partition structure from /dev/sda to your new drive at /dev/sdb

* If your server did not boot and got stuck, it’s likely that grub was not installed on the remaining drive.

** If the replacement drive had an OS and this OS is booting instead of the one you want, go into your BIOS and swap which is the primary boot drive

 

3. Replicating Partition Structure

Now we’ll replicate the disk partition structure between by copying it from the raid1 disk to the new disk you’ve just installed.

sfdisk -d /dev/sda | sfdisk /dev/sdb

Make sure you get the order of these correct or you’ll destroy your data!

To check that both drives have an identical partition structure, do:

fdisk -l

If the are different then you’re likely using two disks that are the same size but different models.  The new drive needs to contain at least the same amount of sectors as the old drive – or greater.

 

4. Adding the Disk to the RAID1 Array

Now we’ll add /dev/sdb into the raid array so that it can begin synchronizing.

mdadm --manage /dev/md0 --add /dev/sdb1

Do the same for the / partition

mdadm --manage /dev/md1 --add /dev/sdb3

Both arrays should now be syncing, though your md0 may already be complete if it’s a small partition.

cat /proc/mdstat

This will you show you the current status of the synchronization process.

Personalities : [raid1] 
md0 : active raid1 sda1[0] sdb1[1]
      104320 blocks [2/2] [UU]
      
md1 : active raid1 sda3[2] sdb3[1]
      73850688 blocks [2/1] [_U]
      [=>...................]  recovery =  5.6% (4180352/73850688) finish=281.6min speed=4120K/sec
      
unused devices: <none>

When that is complete, you should see [UU] in all arrays, like in md0 example above.

 

5. Install Grub on Secondary Drive

Now that the synch is done, we should be sure to install the Grub bootloader to the new drive as a failover.  If your primary drive fails you want to be able to boot off the secondary, right?

# grub
grub> find /grub/stage1

You’ll likely see:

(hd0,0)
(hd1,0)

This means both /dev/sda1 and /dev/sdb1 contain grub files, but grub is really only installed on /dev/sda1 right now.

grub> device (hd0) /dev/sdb
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

This is like telling grub that (hd0) is refer instead to sdb and then proceed to set it up on /dev/sdb1

You May be Done Already!  See below.

If you’re only replacing a failed drive, you can stop here.  You should be done.  However if you’re replacing both drives and installing larger ones, continue on, but first be sure that your raid synchronization process is complete…

 

6. Removing the Second Old RAID1 Disk

Shut down the server so that you can remove the second disk.

shutdown -h now

Pull out the /dev/sda drive and replace it with your new larger drive.  You may need to go into your BIOS and set the secondary drive as the primary boot drive.  Since we’ve already complete synching to /dev/sdb in the process above, it’s now the drive with the data we want.

If your server still doesn’t boot up after that, it’s likely grub wasn’t installed correctly on /dev/sdb.  Plug back in /dev/sda, boot it up, and be sure to follow the grub install mentioned above.

 

7. Replicate the Partition Structure

Ok, so your server is back online now.  We’ll need to match the partition structure from /dev/sdb onto /dev/sda

sfdisk -d /dev/sdb | sfdisk /dev/sda

Verify they’re identical using

fdisk -l

8. Add the Disk to the RAID1 Array

Now we’ll add /dev/sda1 to /dev/md0 and /dev/sda3 to /dev/md1

mdadm --manage /dev/md0 --add /dev/sda1
mdadm --manage /dev/md1 --add /dev/sda3

To see the progress…

cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sda1[0] sdb1[1]
      104320 blocks [2/2] [UU]
      
md1 : active raid1 sda3[2] sdb3[1]
      73850688 blocks [2/1] [_U]
      [=>...................]  recovery =  5.6% (4180352/73850688) finish=281.6min speed=4120K/sec
      
unused devices: <none>

Wait for the synchronization process to complete by checking /proc/mdstat every now and then.

 

9. Installing Grub on the New Primary Disk

Installing Grub onto /dev/sda (you may not need to do this, depending on how you got here).  It generally won’t hurt to do it if you’re not sure.

# grub
grub> find /grub/stage1

will display:

(hd0,0)
(hd1,0)

Again, this is because both disks contain grub, but grub is currently only installed on /dev/sdb (hd1,0)

grub> root (hd0,0)
grub> setup (hd0)
grub> quit

 

We’re done!

You should now have two new drives in your raid1 array, both with the original data from your old drives.  In addition, both drives have grub installed, so should the primary disk fail, the secondary will still be bootable.

mysql_real_escape_string() Problems, Null, Explained

Just a quick note about PHP’s mysql_real_escape_string() function.  Basically the function will remove possibility of visitors throwing binary data or other exploits into your forms etc on your website.  I recently had a problem figuring out why my mysql_real_escape_string($variable) was null.  Silly me, I didn’t even think about the fact that the MySQL database connection must be made first, as in before the mysql_real_escape_string() function can be used.

So if your mysql_real_escape_string() is not receiving any data, make sure you’re connecting to the MySQL database first and then using the function.

Rogers High Speed 50Mbps Cable Internet Connections are FAST, But Misleading

Here in Toronto, Rogers is the dominant provider of high speed Internet, cable and digital cable services.  Rogers delivers a wide variety of residential Internet connections speeds.

I’m going to briefly go over a few of their packages and try to explain to the average person what exactly it is that they’re getting when they subscribe to a Rogers Lite 3Mbps, Rogers Express 10Mbps, Rogers Extreme 15Mbps, Rogers Extreme Plus 25Mbps and Rogers Ultimate 50Mbps.

Lets start with their fastest plan first, the Rogers Ultimate 50Mbps high speed Internet.

50Mbps is fast!  Really fast, but keep in mind that the plan offers “up to” 50Mbps so you’re certainly not guaranteed that speed.  I would guess you’ll receive somewhere between 5Mbps and 50Mbps at any given time.  That’s because in order to truly receive such high speeds, the website or file that you’re downloading on the other end of the Internet connection must also have enough speed to make it fast.  So for example if the website is on a 10Mbps and you’re using a 50Mbps connection, you cannot receive anything faster than the 10Mbps the website is supporing.  This is of course true for any Rogers high speed Internet plan you choose.

Ok, that said, here’s what you’re really getting with the 50Mbps.  You receive 175GB of monthly data transfer, after which you begin to pay on a per GB basis.  Here is a calculation so you can see what that really means.

On the Rogers website it claims that 175GB lets you “Download 88 HD Movies”, “537 hours of YouTube HD”, or “35840 songs”.  Lets analyse this claim.

1. Download 88 HD Movies: Well, sure, if they’re 45-60 minutes each.  Most HD movies are about 4GB so it would actually be closer to 43 HD movies per month.

2. 537 hours of YouTube HD: It’s difficult to truly estimate how much bandwidth HD YouTube videos use, but I could see 537 hours being about right

3. 3580 songs: Yup, they’re right on this one

Here’s what needs to be explained about the Rogers Ultimate 50Mbps high speed Internet though.  You’re subscribing to a 50Mbps service.  Here’s a calculation that shows how much transfer you can do on a 50Mbps Internet connection over a month: 50 (Mb per second) = 15.6744947 TB per month.  That’s equal to over 16,050 gigabytes.  WAIT A MINUTE.  Why does the Rogers Ultimate 50Mbps high speed Internet connection come with only 175GB of bandwidth per month when a 50Mbps connection can actually do over 16,000GB per month, or over 91 times as much?  Well, because they’re overselling by a HUGE margin.  A margin of 91:1 actually.  Except that 95% of customers won’t fully utilize their monthly allotment, so its more like overselling by 1820:1.

If you were to look at this in reverse and ignore the 50Mbps speed for now, you’re subscribing to a 175GB/month service.  Here’s the calculation: 175 (GB per month) = 0.545148156 Mb per second.  Yikes, that doesn’t sound nearly as good does it?  If you were to sustain 0.54Mbps for an entire month you’d already use up your 175GB allotment and have to start paying more after.

Now, imagine if you use your Rogers Ultimate 50Mbps high speed Internet connection at 50Mbps sustained for a month and downloaded over 16,000GB of data on your 175GB plan.  You will have to pay extra for the 15,825GB that you went over your bandwidth cap.  Most providers charge about $1 per GB for overage, but lets say Rogers were to charge just a penny ($0.01), which is NOT the case – it’s much more.  You would be stuck paying an extra $158.25 because you fully utilized the 50Mbps connection you’re subscribed to.  Something is certainly not right here.

What about Rogers Extreme Plus 25Mbps high speed Internet?  Here’s the same calculations: 25 (Mb per second) = 8,025 GB per month.  The plan is 125GB per month, so in speed that would be: 125 (GB per month) = 0.39 Mb per second if sustained over a month.

And for Rogers Extreme 15Mbps high speed Internet, the same calculations: 15 (Mb per second) = 4,815 GB per month.  In speed, the provided 80GB per month: 80 (GB per month) = 0.25 Mb per second if sustained over a month.

Rogers Express 10Mbps high speed Internet: 10 (Mb per second) = 3,210 GB per month.  You get 60GB monthly so in speed: 60 (GB per month) = 0.19 Mb per second sustained over a month

And again with the Rogers Lite high speed Internet: 3 (Mb per second) = 963 GB per month.  You get 15GB per month so if we again reverse this for speed: 15 (GB per month) = 0.047 megabits per second if sustained for a month.  That’s slower than dialup.

So what does this tell us?  Well, Rogers is way overselling the high speed Internet plans they offer because the monthly bandwidth allotments they’re providing are waaaaay under the amount of monthly transfer that can be done on their high speed Internet connections.  Just to be clear, Rogers is certainly not the only provider doing this.  They all do it.  So what’s the point of this article then?  Just to inform those consumers that don’t fully understand what it is they’re buying.

Question, comments?  Submit a comment below.