Skip to content
ahouston edited this page Jun 18, 2018 · 4 revisions

Formatting EMC locked drives with 520 byte sectors back to 512 bytes

I'm guessing if you found your way here that you've come by some disks that were until recently in some kind of storage array (mine were ex-EMC) and you've found that you can't actually use them as regular SAS disks, even though your RAID controller detects them.

I initially thought that a firmware flash was required on the disks themselves, but even after the Dell firmware I found allegedly flashed my disks successfully (after much hacking through the code on the Centos version to get around the Dell detection), it still didn't actually help.

What DID work was to format the disks back to 512 byte sectors, as EMC array disks (and others apparently) use a 520 byte sector, which as you might guess isn't supported by anything but the array.

I found some pretty good resources which helped me, listed below:

https://forums.servethehome.com/index.php?threads/how-to-reformat-hdd-ssd-to-512b-sector-size.4968/ http://pissedoffadmins.com/general/unsupported-sector-size-520.html

The general idea is to simply install sg3-utils and format the disks with the sg_format command like so:

sg_format --format --size=512 /dev/sg8

The problem I encountered (apparently not just me reading into the posts above), was something this:

root@ubuntu:~# sg_format --format --size=512 -v /dev/sg2
    HITACHI   HUS72303CLAR3000  C442   peripheral_type: disk [0x0]
      PROTECT=1
      << supports protection information>>
      Unit serial number: YXGMGGTK        
      LU name: 5000cca046236594
    mode sense (10) cdb: 5a 00 01 00 00 00 00 00 fc 00 
Mode Sense (block descriptor) data, prior to changes:
Mode sense number of blocks maxed out, set longlba
    mode sense (10) cdb: 5a 10 01 00 00 00 00 00 fc 00 
Mode Sense (block descriptor) data, prior to changes:
  <<< longlba flag set (64 bit lba) >>>
  Number of blocks=5770371118 [0x157f0e02e]
  Block size=520 [0x208]
    mode select (10) cdb: 55 11 00 00 00 00 00 00 22 00 
mode select (10):
Descriptor format, current; Sense key: Illegal Request
Additional sense: Parameter list length error
  Descriptor type: Information:    >> descriptor too short
    00 00 00 00 00 00 00 00 00 00 
  Descriptor type: Sense key specific: Field pointer:
        Error in Command: byte 7
  Descriptor type: Field replaceable unit code: 0x0
  Descriptor type: Block commands: Incorrect Length Indicator (ILI) clear
  Descriptor type: Vendor specific [0x80]
    f8 20 
  Descriptor type: Vendor specific [0x81]
    00 0MODE SELECT command: Illegal request sense key, apart from Invalid opcode

...OK, more reading required.

Having found the pissedoffadmins.com post, I decided to give the setblocksize code from here a whirl. Aha! Now we're getting somewhere, the format start, I high five myself and crack a celebratory beer. Two hours later (the default) the format times out, because it takes up to 4.5 hours per TB of disk you have.

The maximum in the code was set too low, so I fixed it, put it on GitHub and here we are, you can now set the maximum timeout with a -t to up to 1800 which should cater for disks up to 6TB I guess. If not, just edit the code ;)

Instructions

Make sure you have a HBA card in "IT" mode

I bought a "IBM 46M0861 Serveraid M1015 8 Chan PCI-E X8 SAS/SATA Raid Controller (M192246)" off eBay for about $40. It needs to be cross flashed into IT mode, there are plenty of guides on how to do this, such as https://www.servethehome.com/ibm-serveraid-m1015-part-4/

Get an Ubuntu LiveCD image

I used the Ubuntu 18.04 desktop for the below, but any recent version of Ubuntu should work, as should CentOS I guess, if you like really have to.

I downloaded ubuntu-18.04-desktop-amd64.iso from https://www.ubuntu.com/download/desktop, and flashed it to a USB drive with Rufus 3.0

Boot up and check you're seeing the disks in passthrough mode

Besides hearing your disk(s) spinning up and grinding a bit now, you should also see something like this in your dmesg output:

[   49.724403] sd 7:0:0:0: [sdb] Unsupported sector size 520.
[   49.724682] sd 7:0:0:0: [sdb] 0 512-byte logical blocks: (0 B/0 B)
[   49.724684] sd 7:0:0:0: [sdb] 520-byte physical blocks
[   49.724919] sd 7:0:0:0: [sdb] Write Protect is off
[   49.724921] sd 7:0:0:0: [sdb] Mode Sense: cf 00 10 08
[   49.725244] sd 7:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[   49.728266] sd 7:0:0:0: [sdb] Unsupported sector size 520.
[   49.731554] sd 7:0:0:0: [sdb] Attached SCSI disk

Install SSH

apt-get install ssh

Set a password for the 'ubuntu' user

passwd ubuntu

At this point I would usually SSH to the server to complete the remainder

Lets be cheeky and sudo bash to become root for the remainder

Last login: Mon Jun 18 19:37:48 2018 from 192.168.0.12
ubuntu@ubuntu:~$ sudo bash
root@ubuntu:~# 

Install the build tools and sg3-utils

apt-get install build-essential autoconf vim git sg3-utils screen

Clone the repo

cd /root
git clone https://github.com/ahouston/setblocksize.git
cd setblocksize

Compile

git clone https://github.com/ahouston/setblocksize.git
cd setblocksize
make all

There are plenty of warnings, but it compiles fine:

root@ubuntu:~# git clone https://github.com/ahouston/setblocksize.git
Cloning into 'setblocksize'...
remote: Counting objects: 18, done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 18 (delta 2), reused 10 (delta 0), pack-reused 0
Unpacking objects: 100% (18/18), done.
root@ubuntu:~# cd setblocksize
root@ubuntu:~/setblocksize# make all

Creating main object file ...
gcc -I ./include -D_REENTRANT -Wall -pipe -c -o setblocksize.o setblocksize.c
setblocksize.c: In function ‘main’:
setblocksize.c:130:11: warning: format not a string literal and no format arguments [-Wformat-security]
    printf(sbuf);
           ^~~~
setblocksize.c:182:23: warning: format not a string literal and no format arguments [-Wformat-security]
       fprintf(stderr, sbuf);
                       ^~~~
setblocksize.c:302:18: warning: pointer targets in passing argument 2 of ‘strncpy’ differ in signedness [-Wpointer-sign]
    strncpy(sbuf, scsi_buf + sizeof(struct sg_header) + 0x08, 0x08);
                  ^~~~~~~~
In file included from setblocksize.c:69:0:
/usr/include/string.h:124:14: note: expected ‘const char * restrict’ but argument is of type ‘unsigned char *’
 extern char *strncpy (char *__restrict __dest,
              ^~~~~~~
setblocksize.c:305:18: warning: pointer targets in passing argument 2 of ‘strncpy’ differ in signedness [-Wpointer-sign]
    strncpy(sbuf, scsi_buf + sizeof(struct sg_header) + 0x10, 0x10);
                  ^~~~~~~~
In file included from setblocksize.c:69:0:
/usr/include/string.h:124:14: note: expected ‘const char * restrict’ but argument is of type ‘unsigned char *’
 extern char *strncpy (char *__restrict __dest,
              ^~~~~~~

Creating error handling object file ...
gcc -I ./include -D_REENTRANT -Wall -pipe -c -o sg_err.o sg_err.c

Creating binary ...
gcc -o setblocksize setblocksize.o sg_err.o

Finished.

Check your SCSI disk device names

root@ubuntu:~/setblocksize# sg_scan -i
/dev/sg0: scsi5 channel=0 id=0 lun=0 [em]
    HL-DT-ST  DVDRAM GH24NSD0   LH00 [rmb=1 cmdq=0 pqual=0 pdev=0x5] 
/dev/sg1: scsi6 channel=0 id=0 lun=0 [em]
    SanDisk   Cruzer Fit        1.27 [rmb=1 cmdq=0 pqual=0 pdev=0x0] 
/dev/sg2: scsi7 channel=0 id=0 lun=0
    HITACHI   HUS72303CLAR3000  C442 [rmb=0 cmdq=1 pqual=0 pdev=0x0]
/dev/sg3: scsi7 channel=0 id=1 lun=0
    ATA       KINGSTON SUV400S  96R9 [rmb=0 cmdq=1 pqual=0 pdev=0x0] 
/dev/sg4: scsi7 channel=0 id=2 lun=0
    HITACHI   HUS72303CLAR3000  C442 [rmb=0 cmdq=1 pqual=0 pdev=0x0] 

Start up a screen - trust me you don't want the format to stop when you're 8 hours in...

root@ubuntu:~/setblocksize# screen

GNU Screen version 4.06.02 (GNU) 23-Oct-17

Copyright (c) 2015-2017 Juergen Weigert, Alexander Naumov, Amadeusz Slawinski
Copyright (c) 2010-2014 Juergen Weigert, Sadrul Habib Chowdhury
Copyright (c) 2008-2009 Juergen Weigert, Michael Schroeder, Micah Cowan, Sadrul Habib Chowdhury
Copyright (c) 1993-2007 Juergen Weigert, Michael Schroeder
Copyright (c) 1987 Oliver Laumann

Lets do the format, for interest sake start with 'time' so you can see how long it takes. My 3TB drives took between 756-780 minutes!

time ./setblocksize -b512 -t1200 /dev/sg2

The output is shown below:

root@ubuntu:~/setblocksize# time ./setblocksize -b512 -t1200 /dev/sg2

setblocksize V0.2

Checking parameters ...
   Blocksize specified.
   Timeout specified.
   Done.
New blocksize: 512 Bytes
Format timeout: 1200 minutes
Open device file ...
   Done.
Prepare command ...
   Done.
Send INQUIRY command ...
   Done.
Check status ...
   Command successful.
Check for LUN ...
   LUN present.

===============================================================================
SCSI ID     : 0
LUN         : 0
Connected to: Host7 / Channel0
Manufacturer: HITACHI 
Model       : HUS72303CLAR3000
Device type : Disk
===============================================================================
Do you really want to reformat this device [y/n]? y

Prepare command ...
   Done.
Send MODE SELECT command ...
   Done.
Check status ...
   Command successful.
Prepare command ...
   Done.
Send FORMAT UNIT command ...
   *** Please wait - Do not manually interrupt or power down! ***
   Check status ... 
   Command successful.
   Close device file ...
 Done.
 
 Exiting ...
 
 real    780m50.983s
 user    0m0.000s
 sys     0m0.002s

If you remount / hotswap your disks they should now work. My disks were detected but marked as "unsupported" by my LSI MegaRAID before, worked perfectly after :)