1

Considering going to ubuntu 14.04 I tried to boot the live dvd and see how it sees my hard disk setup. On my normal 10.04 install, the hard disk (which I configured with gparted) works fine:

$ parted /dev/sda
GNU Parted 2.2
Using /dev/sda
(parted) print                                                            
Model: ATA TOSHIBA DT01ACA3 (scsi)
Disk /dev/sda: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt

Number  Start   End     Size    File system  Name  Flags
 1      17.4kB  37.7GB  37.7GB  ext3               boot
 3      37.7GB  37.7GB  32.9MB                     bios_grub
 2      37.7GB  3001GB  2963GB                     raid

(parted) quit       

when I boot to 14.04 gparted complains that it can not query the drive, prompting me to retry ignore or cancel, and then shows that the hard disk is not configured (contains no partitions)

gdisk under 14.04 says that the partition table is corrupted. gdisk under 10.04 says that the partition table is fine.

How is it possible that the same tools give different results on the same hardware? I even tried to boot with the live dvd from ubuntu 10.04 to see if it is caused by the booting from cd, but it is not. The disk seems fine from 10.04 even if it has booted from dvd.

some results from 14.04:

parted /dev/sda
GNU Parted 3.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Error: Invalid argument during seek for read on /dev/sda
Retry/Ignore/Cancel? c
Model: ATA TOSHIBA DT01ACA3 (scsi)
Disk /dev/sda: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: unknown
Disk Flags: 
(parted) print                                                            
Error: Invalid argument during seek for read on /dev/sda
Retry/Ignore/Cancel? i                                                    
Error: The backup GPT table is corrupt, but the primary appears OK, so that will
be used.
OK/Cancel? o                                                              
Model: ATA TOSHIBA DT01ACA3 (scsi)
Disk /dev/sda: 3001GB
Sector size (logical/physical): 512B/4096B
Partition Table: unknown
Disk Flags: 
(parted) q


gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.8

Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! One or more CRCs don't match. You should repair the disk!

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************

Command (? for help): q

` and on 10.04:

fdisk -l /dev/sda

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sda: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      267350  2147483647+  ee  GPT
Partition 1 does not start on physical sector boundary.
root@user-desktop:~# gdisk /dev/sda
GPT fdisk (gdisk) version 0.5.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): q
7
  • What's the result of sudo fdisk -l on 14.04?
    – madneon
    Mar 28, 2015 at 15:28
  • fdisk wont work because the disk is over 2G but here: fdisk -l /dev/sda Disk /dev/sda: 2.7 TiB, 3000591900160 bytes, 5860531055 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 4096 bytes / 4096 bytes Disklabel type: dos Disk identifier: 0x00000000 Device Boot Start End Sectors Size Id Type /dev/sda1 1 4294967295 4294967295 2T ee GPT Partition 2 does not start on physical sector boundary.
    – Andy
    Mar 28, 2015 at 15:50
  • You mean 2TB?
    – madneon
    Mar 28, 2015 at 15:55
  • yes, by the way, how do you add a newline here in the comments? no, its not two spaces
    – Andy
    Mar 28, 2015 at 16:12
  • 1
    Well.. Shift-Enter adds a new line in the edit box. But it's stripped on submit. you should integrate larger info into your initial question instead of a comment.
    – JPT
    Mar 28, 2015 at 19:46

4 Answers 4

2

This is a wild guess, but it could be that your disk(s) use the Host Protected Area (HPA) feature, which "hides away" some sectors of the disk. Unfortunately, some older Linux kernels ignored HPA. If you partitioned the disk with such a kernel, a newer kernel (which honors HPA settings) would show the disk as being suddenly smaller than the older kernel showed, and you'd get exactly the symptoms you report.

To check this, use hdparm and its -N option, as in:

$ sudo hdparm -N /dev/sda

/dev/sda:
 max sectors   = 488397168/488397168, HPA is disabled

In this case, HPA is disabled; but if it's enabled, the output will reveal that fact. If this is what you find, you can set it correctly with the same command, but you must add a p and the correct size, as in:

$ sudo hdparm -N p488397168

You'll have to adjust the value passed based on the actual size of your disk.

7
  • Thanks. This is interesting. I didn't know about HPA. But I run hdparm -N and (on 10.04) it reports that HPA is disabled. (/dev/sda: max sectors = 5860533168/5860533168, HPA is disabled)
    – Andy
    Mar 29, 2015 at 8:04
  • I booted to the live 14.04 and tried hdpart -N. Here the result is 5860531055/5860533168 and HPA is enabled. So 14.04 is missing 2113 sectors from the size of the drive. After a google search for HPA I'm really starting to panic about the health of my RAID array
    – Andy
    Mar 29, 2015 at 12:32
  • It seems that you are absolutely right about HPA. However now there is a new problem. I can only disable HPA from 14.04 for sdb and sdc but not sda. Even worse, after changing sata cables to temporary 'name' the primary master disk sdc, and successfully disabling HPA for that disk, whichever disk 'is' sda when 14.04 boots becomes HPA enabled again, with the same sector value (5860531055)
    – Andy
    Mar 29, 2015 at 19:50
  • It sounds like your firmware or an early boot program may be enabling HPA. Check your firmware settings and try to verify that you don't have any odd boot programs running, like third-party boot loaders. Note that an option isn't likely to be marked "HPA control here"; it's more likely to be enabled implicitly by something like a RAID setup. (Note that if you're using Linux's software RAID, you do not need the motherboard's software RAID.) There's even a chance that this is the doing of a boot virus, so doing a virus scan scan from a CD-R prepared on another computer may be worthwhile.
    – Rod Smith
    Mar 29, 2015 at 21:06
  • I checked the BIOS but didnt find anything that looked HPA related. I only use grub. The computer had a smaller hard disk with windows 7 preinstalled, but I have discarded that drive a long time ago. Since the only OS running on that machine with these drives is ubuntu 10.04, which seems HPA unaware, I guess that even a linux virus would not be able to modify HPA, since the kernel doesnt support it. The motherboard doesnt have RAID support. The other OS that boots is 14.04 from a dvd, so that should be virus free too.
    – Andy
    Mar 30, 2015 at 8:46
0

My guess is: 10.4 is so old that the gpt partition tools contained bugs.

from http://en.wikipedia.org/wiki/History_of_hard_disk_drives

2010 – First 3.0 terabyte hard drive[23][24] (Seagate, Western Digital)

2010 – First Hard Drive Manufactured by using the Advanced Format of 4,096 bytes a block ("4K") instead of 512 bytes a block[25] 2

Solution:

  • get a new disk,
  • create new table and partitions with up to date tools,
  • move your data to the new disk.

sorry for not having a better idea.

1
  • Well, that's not really an option, this disk is a part of a 3-disk raid5 array (oh and the other two disks seem fine on both 10.04 and 14.04 by the way). But I will not start the array on 14.04 with only 2 drives just to test if that will destroy all my data.
    – Andy
    Mar 28, 2015 at 14:50
0

Other solution:

If the bug is known, maybe Ubuntu fixes the partition table during installation.

if not, at least server 10.4 has not reached it's end of life yet https://wiki.ubuntu.com/Releases

Search for or file a bug at https://bugs.launchpad.net/ubuntu/

But I fear they won't find a solution before end of life is reached (April 30, 2015)

in any case you will need space to create a backup of your system. So you might need a new disk regardless which solution you choose.

1
  • Thanks, I tried to file a bug report but it seems chaotic. I am not even sure where the bug is, the utilities? the old kernel? the new kernel? Anyway, I cant get any more disks, because I have no more hardware ports to connect them to, 3 sata ports for the drives and 1 for the dvd, all 4 ports are filled.
    – Andy
    Mar 28, 2015 at 15:02
0

Ok. Thanks to everyone who tried to help. It seems like this had not much to do with ubuntu, and was basically a BIOS bug.

Some Gigabyte motherboards write a copy of their BIOS to hard disks immediately when the disks are connected. So, simply by connecting a new HD on the motherboard it is likely that it will be written to, and HPA will be activated reducing the size of the disk. Ubuntu 10.04 by default ignores HPA (libata.ignore_hpa=1 kernel parameter) and so treated the whole disk as if it had not HPA activated, using all the sectors from the disk. Ubuntu 14.04 by default honors HPA (libata.ignore_hpa=0 kernel parameter) and so treated the disk as if it was smaller than it actually is. The disks were originally partitioned form 10.04 for the total sum of sectors. Changing the cabling resulted in finally activating HPA on all the disks of the RAID. I was very lucky because the original partitioning did not use all the available space, but had left enough space at the end of the drive for the sectors that HPA was stealing. I dont know why, perhaps it was aligning to the cylinder previous to last. The workaround I concluded to, was to repartition the drives as if they were smaller than they actually are, as big as they are made to be by HPA. I boot to 14.04 and fixed the partitions with gedit.

THE PROCESS WAS VERY DANGEROUS AND COULD POTENTIALLY DESTROY ALL MY DATA. DO NOT REPEAT IT IF YOU CARE ABOUT THE CONTENTS OF THE DRIVES.

After I fixed all three drives both OS's could use the raid, so I actually installed 14.04 on hard disk. When I booted at 10.04 the partitioning utilities complained that I was not using all of the disk, but would ask before changing anything so no harm done. To fix this little inconvenience I added libata.ignore_hpa=0 to the kernel parameters of my /etc/default/grub.conf for 10.04 and updated grub.

For the sake of history only, here is the DANGEROUS process I used to convert the drives partition scheme for /dev/sda (it was the same for all drives):

root@ubuntu:/cdrom/HPA_recovery# gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.8

Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.

Warning! One or more CRCs don't match. You should repair the disk!

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: damaged

****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
Command (? for help): ?
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

Command (? for help): r
Recovery/transformation command (? for help): ?
b       use backup GPT header (rebuilding main)
c       load backup partition table from disk (rebuilding main)
d       use main GPT header (rebuilding backup)
e       load main partition table from disk (rebuilding backup)
f       load MBR and build fresh GPT from it
g       convert GPT into MBR and exit
h       make hybrid MBR
i       show detailed information on a partition
l       load partition data from a backup file
m       return to main menu
o       print protective MBR data
p       print the partition table
q       quit without saving changes
t       transform BSD disklabel partition
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

Recovery/transformation command (? for help): d

Recovery/transformation command (? for help): p
Disk /dev/sda: 5860531055 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 2D50CD70-85F5-46FE-85B3-35A027E3C828
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 8-sector boundaries
Total free space is 5070 sectors (2.5 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34        73658024   35.1 GiB    EF00
   2        73722285      5860528064   2.7 TiB     FD00
   3        73658025        73722284   31.4 MiB    EF02

Recovery/transformation command (? for help): x
Expert command (? for help): ?
a       set attributes
c       change partition GUID
d       display the sector alignment value
e       relocate backup data structures to the end of the disk
g       change disk GUID
h       recompute CHS values in protective/hybrid MBR
i       show detailed information on a partition
l       set the sector alignment value
m       return to main menu
n       create a new protective MBR
o       print protective MBR data
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       resize partition table
t       transpose two partition table entries
u       Replicate partition table on new device
v       verify disk
w       write table to disk and exit
z       zap (destroy) GPT data structures and exit
?       print this menu

Expert command (? for help): e
Relocating backup data structures to the end of the disk

Expert command (? for help): ?
a       set attributes
c       change partition GUID
d       display the sector alignment value
e       relocate backup data structures to the end of the disk
g       change disk GUID
h       recompute CHS values in protective/hybrid MBR
i       show detailed information on a partition
l       set the sector alignment value
m       return to main menu
n       create a new protective MBR
o       print protective MBR data
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       resize partition table
t       transpose two partition table entries
u       Replicate partition table on new device
v       verify disk
w       write table to disk and exit
z       zap (destroy) GPT data structures and exit
?       print this menu

Expert command (? for help): v

Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.

Caution: Partition 1 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Caution: Partition 2 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Caution: Partition 3 doesn't begin on a 8-sector boundary. This may
result in degraded performance on some modern (2009 and later) hard disks.

Consult http://www.ibm.com/developerworks/linux/library/l-4kb-sector-disks/
for information on disk alignment.

Identified 1 problems!

Expert command (? for help): x
a       set attributes
c       change partition GUID
d       display the sector alignment value
e       relocate backup data structures to the end of the disk
g       change disk GUID
h       recompute CHS values in protective/hybrid MBR
i       show detailed information on a partition
l       set the sector alignment value
m       return to main menu
n       create a new protective MBR
o       print protective MBR data
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       resize partition table
t       transpose two partition table entries
u       Replicate partition table on new device
v       verify disk
w       write table to disk and exit
z       zap (destroy) GPT data structures and exit
?       print this menu

Expert command (? for help): p
Disk /dev/sda: 5860531055 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 2D50CD70-85F5-46FE-85B3-35A027E3C828
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860531021
Partitions will be aligned on 8-sector boundaries
Total free space is 2957 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              34        73658024   35.1 GiB    EF00
   2        73722285      5860528064   2.7 TiB     FD00
   3        73658025        73722284   31.4 MiB    EF02

Expert command (? for help): w
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.
root@ubuntu:/cdrom/HPA_recovery# gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.8

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): q

Again, thanks to everyone for all the help.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .