[SOLVED] Understanding RAID 5 and Disk Failures

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
Hello everyone! I'm in the process of deploying a new server (running Windows Server 2019) that I plan to host multiple VMs on. I have a RAID and/or Intel RST related question I was hoping someone could help me with.

Before I get too far, let me break down the key storage specs of the server I'll be deploying:

HP Z640
  • HDDs: 4x 4TB WD Re Enterprise drives (WD4000FYYZ)
  • Intel RST C600+/C220+ RAID controller
    • Note: not compatible with Intel's RST application (GUI), just has drivers for the controller.
Alright, so Windows Server will be installed on a 500GB SSD, and I have all 4 HDDs in a RAID 5 array. To simulate a failure, I took out a drive, and swapped it with another one that was identical. Upon booting into Windows Server, I saw my D: drive (that's what the RAID 5 array is mapped to) was still showing up (I suppose as expected, since RAID 5's redundancy). That is the part that is concerning to me though, and is my main question.. Why did Windows Server not notify me that the array was in a 'degraded' state?

Now, to confirm that the array was indeed degraded (how could it not be after swapping a drive..) I chose recovery mode and had the machine boot to the UEFI settings. I chose the RST 3rd party option ROM and sure enough, it listed the RAID 5 array as degraded. I then followed the prompts to rebuild the array with the drive that I had swapped in and it's currently in the rebuilding state.

Other questions:
  • Is there a way to query the controller from within Windows Server to see the status of an array (Powershell or CMD, etc.)?
    • if so, can this be automated?
  • Will the array continue to rebuild itself if I leave the RST option ROM and boot into the Windows Server?
Thanks so much for your time and expertise!


EDIT, SOLUTION: 10/11/2021 - 9:44am
So, I'm guessing that some early intel RST controllers or specific ones are unsupported when it comes to the intel RST application. Instead they rely on files within the actual driver (the ones that would be extracted when they go to be installed on a Windows machine). In my case, I found something under the "RSTe_4.700.2072_CLI\x64" directory of the extracted driver. the application is called rstcli64, the 'cli' being a refernce to command line interface. If you open CMD or powershell in windows, you can pass it some arguments to get the status and information about your arrays. Below is a quick guide on how to do so:
  1. Navigate to the location of the rstcli64 application in file explorer. SHIFT + Right click and select 'open powershell window here'
  2. type in the following command: .\rstcli64 --information --volume
  3. See the status of the volume and drives in the system response.
I may automate this as a scheduled task, but we'll see. You may be able to actually add cool functionality where you can have it send you an email if status is 'degraded'. Hope this helps someone with a similar issue!

EDIT, CODE-BASED SOLUTION: 10/11/2021 - 12:06pm
Hey everyone, here's a snippet of code that can be used to create a .ps1 file to check the status of the array if you have the intel RST CLI application. The result will get emailed to yourself. You can also add support for text notifications as well! My example in the $EmailTo field sends to an email address, and a phone number (separated by a comma). Hope this helps!


Code:
cd "PATH\TO\CLI_APPLICATION     (will look similar to)     D:\HP DRIVER RST\RSTe_4.7.0.2072_CLI\x64"
$degraded = .\rstcli64 --information --volume |Select-String  -Pattern "Degraded"

if ($degraded -eq $null)
{
    $EmailFrom = "YOUR_EMAIL@MAIL.COM"
    $EmailTo = "YOUR_EMAIL@MAIL.COM,1234567890@cellular.com"
    $Subject = "RAID 5 is healthy!"
    $Body = "RAID 5 is healthy!"
    $SMTPServer = "smtp.gmail.com"
    $SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 587)
    $SMTPClient.EnableSsl = $true
    $SMTPClient.Credentials = New-Object System.Net.NetworkCredential("YOUR_USERNAME", "YOUR_PASSWORD");
    $SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)
}
else
{
    $EmailFrom = "YOUR_EMAIL@MAIL.COM"
    $EmailTo = "YOUR_EMAIL@MAIL.COM,1234567890@cellular.com"
    $Subject = "RAID 5 is Degraded!"
    $Body = "RAID 5 is Degraded!"
    $SMTPServer = "smtp.gmail.com"
    $SMTPClient = New-Object Net.Mail.SmtpClient($SmtpServer, 587)
    $SMTPClient.EnableSsl = $true
    $SMTPClient.Credentials = New-Object System.Net.NetworkCredential("YOUR_USERNAME", "YOUR_PASSWORD");
    $SMTPClient.Send($EmailFrom, $EmailTo, $Subject, $Body)
}
 
Last edited:

USAFRet

Titan
Moderator
Mar 16, 2013
147,429
9,348
175,390
23,025
Why did it not notify you?
Completely unknown.

A RAID 5 will survive the death (or removal) of a single drive. All data still accessible.
Upon replacing with a new blank drive, the array will/should rebuild.
Depending on how much data is in the array, figure 1.5+ hours per GB to rebuild.
 

USAFRet

Titan
Moderator
Mar 16, 2013
147,429
9,348
175,390
23,025
Why did it not notify you?
Completely unknown.

A RAID 5 will survive the death (or removal) of a single drive. All data still accessible.
Upon replacing with a new blank drive, the array will/should rebuild.
Depending on how much data is in the array, figure 1.5+ hours per GB to rebuild.
 

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
Why did it not notify you?
Completely unknown.

A RAID 5 will survive the death (or removal) of a single drive. All data still accessible.
Upon replacing with a new blank drive, the array will/should rebuild.
Depending on how much data is in the array, figure 1.5+ hours per GB to rebuild.
@USAFRet, Thanks for your reply. Yes, I'm aware that RAID 5 allows for one drive loss 😁. That's why I only removed one to simulate a failure so I know how to deal with it in the future. I appreciate your explanation!

So, onto the issue, I would have expected Windows server to tell me of the issue with the array. I have the RST drivers installed to the machine, so it makes no sense that there were no alerts. Is there any way that you know of to check if there are any issues with an array at any given moment? I'm thinking like a batch script or something to get the status of the array?

Thanks again for your time and I'd love to hear if others have a suggestion!
 

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
Hey everyone.

I have an update on this post actually and will mark this as the best asnwer. I'm guessing that some early intel RST controllers or specific ones are unsupported when it comes to the intel RST application. Instead they rely on files within the actual driver (the ones that would be extracted when they go to be installed on a Windows machine). In my case, I found something under the "RSTe_4.700.2072_CLI\x64" directory of the extracted driver. the application is called rstcli64, the 'cli' being a refernce to command line interface. If you open CMD or powershell in windows, you can pass it some arguments to get the status and information about your arrays. Below is a quick guide on how to do so:
  1. Navigate to the location of the rstcli64 application in file explorer. SHIFT + Right click and select 'open powershell window here'
  2. type in the following command: .\rstcli64 --information --volume
  3. See the status of the volume and drives in the system response.
I may automate this as a scheduled task, but we'll see. You may be able to actually add cool functionality where you can have it send you an email if status is 'degraded'. Hope this helps someone with a similar issue!
 

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
Thank you for clarifying. I'll remove my best answer award, and make an edit to my original post as well so it's easier for people to find the answer. 👍
 

DSzymborski

Titan
Moderator
Thank you for clarifying. I'll make an edit to my original post as well so it's easier for people to find. 👍
Thanks! We've had problems in the past with people doing this to game themselves Best Answer trophies. I have no concern that you're in this group, but we had to ban the practice across the board or we end up in a situation in which we're arbitrarily deciding whether we allow the original poster to award themselves a trophy or not.
 

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
Thanks! We've had problems in the past with people doing this to game themselves Best Answer trophies. I have no concern that you're in this group, but we had to ban the practice across the board or we end up in a situation in which we're arbitrarily deciding whether we allow the original poster to award themselves a trophy or not.
Makes sense, I totally understand! My apologies again for the inconvenience. Figured having the green border arond the solution would help draw attention, but having an edit section in the original post would be just as good! Thanks again!
 

USAFRet

Titan
Moderator
Mar 16, 2013
147,429
9,348
175,390
23,025
Makes sense, I totally understand! My apologies again for the inconvenience. Figured having the green border arond the solution would help draw attention, but having an edit section in the original post would be just as good! Thanks again!
We've had people try to do it repeatedly, on the same thread, after being asked not to.
"Well, the system lets me, so I'm gonna do it!"

They are no longer with us...:)
 

SparkyTech934

Prominent
Jan 22, 2020
78
10
565
24
We've had people try to do it repeatedly, on the same thread, after being asked not to.
"Well, the system lets me, so I'm gonna do it!"

They are no longer with us...:)
Ha! Directions are fairly easy to follow, people should try it more often, eh?

I added a nice code snippet to the original post. I careted a little powershell script can be set as a scheduled task to automatically send an email if your RAID array is degraded. 👍
 

ASK THE COMMUNITY