Bad sector remapping
When a hard disk is manufactured, there are areas on the platter that will have bad sectors. Considering that on a 3 TB hard disk there are 6 billion sectors, then a few thousand bad sectors is only a small proportion of the total number of sectors on the drive. When the number exceeds or comes close to the 'threshold' value, the drive may start acting strangely, often as very slow to respond, or not responding at all hanging the computer. If this is the case with your hard drive, you may need to look at hard drive recovery to regain access to the data stored on the drive.
During the test phases of a hard disk at the factory, the platters are scanned and all the bad sectors are mapped out - into a table or list usually called 'Primary Defect List'. The primary defect list is stored in within the firmware zone, or in some cases the ROM of a hard disk. Either way they are not usually accessible unless factory commands are used to read the tables.
When a user buys a brand new hard disk or computer, they will most likely be completely unaware that the drive contains bad sectors because they are 'mapped out' using the 'translator' algorithms.
Modern hard disks use Logical Block Addressing or LBA, this describes the sector numbering system on the hard disk, and goes in sequence 0,1,2,3,4,5,.....n-1,n (where n is the last sector on the drive.
Spare sector pools
All modern hard disk drives have a spare sector pool. This is used when bad sectors develop during the normal life of the hard disk and any new bad sectors are 'replaced' with good ones from the spare sector pool. This process is invisible to the user and will probably never know that anything has changed.
How bad sector mapping works.
There are at least two methods of bad sector re-mapping or translation these are named...
- The P-list contains defects found during the final stages of the manufacturing process and are known as Primary Defects
- The G-List contains defects that develop during use of the drive and are known as Grown Defects
There are also some other defect lists found in modern drives but the principles are similar. For example, you may find a T-List also known as the Track defect list, and there are a few others that describe errors in the system area of the drive.
Some manufacturers name these lists differently but for the sake of simplicity we will stick to Primary (P-List) and Grown (G-List).
Lets get into how these defect lists actually work, so lets say we have a very small hard disk with just 100 sectors and a 10 sector spares pool.
When bad sectors are found at the factory, shift-points are entered into the P-List, if we take the following LBA sequence 0,1,2,3,4,5,6,7,8,9,10 ...99 Lets say that Sectors 3, 6 and 9 are found to be bad. When the first bad sector is found, the first part of the re-mapping process will look like this
0,1, 2, B, 4, 5, 6, 7, 8, 9, 10 ..
What happens is the bad sector at position 3 is recorded in the P-List and is marked as Bad. The good sectors are then 'translated' and the new map now looks like this;
0,1, 2, P, 3, 4, 5, 6, 7, 8, 9 .. You can see now that 3 is where 4 was. This is known as 'translation'
The next bad sector at LBA 6 is now found 0,1, 2, B, 4, 5, B, 7, 8, 9 .. and is again mapped out giving
0,1, 2, P, 3, 4, 5, P, 6, 7, 8 when the whole sequence is complete, after LBA 9 is mapped out our final map looks like this.0,1, 2, P, 3, 4, P, 5, 6, 7,P, 8, 9,10 note that at each stage, the last LBA decreases by one at each step, this is because the bad sectors reduce the storage space. How this is altered to give the true Max LBA value is described later. I made the final sequence above back to ten. If you coult the original values 0-10 = 11, and the final value 0-10 contains 14 LBA vaues, the increase of 3 is the count of the bad sectors.
Because these sectors are mapped out and the LBA values are translated so that the bad ones are not included in the LBA map, the user will never be aware that they ever existed. If you want to look at sector 6, the drive will translate that to physical sector 8. It takes the 6 and adds the shift points to it, +1 for the bad sector at LBA3 and +1 for the bad sector at LBA 6
When the testing gets to the end of the drive, in order that it is of the correct size of 100 sectors, it allocates the sectors from the spare sector pool completely concealing the fact that there are bad sectors on the media. To all intents and purposes the drive looks just like the original as 0,1,2,3,4,5,6,7,8,9,10. However, our spare pool has reduced in size and there are now 7 sectors remaining in the spares pool.
After using the drive for a while some bad sectors may develop, so the drive has a method of taking care of these using a grown defect list or G-List.
The G-List contains the location of bad sector defects found during normal operation of the hard disk drive. When a bad sector develops during normal use of the computer, a similar process to the P-List generation occurs - resulting with the bad sectors being mapped out. The process for G-List mapping is slightly different though. Lets say our hard disk develops a bad sector at the current LBA 6. What happens in this case is first the bad sector is mapped out. Giving; 0,1,2,3,4,5,G,7,8,9,10 .. A sector from the spare pool is allocated in the bad sectors place. We used 3 of these sectors in factory testing, so the next available bad sector is 104 this now becomes mapped to LBA 6 so our true LBA sequence would look like this; 0,1,2,3,4,5,104,7,8,9,10 - the new bad sector is added to the translator and 'translated' out. Again, this process is completely invisible to the user and will still look like the original sequence of 0,1,2,3,4,5,6,7,8,9,10
It should be mentioned that once the spare sector pool is used up, the SMART technology in the drive will usually set a flag indicating that the drive will fail very shortly and a if SMART monitoring is turned on in the computer, a message will appear pointing this out and that a backup and replacement is recommended..
You might ask, 'why don't the new defects get added to the P-List?' the answer is that LBA is only a pointer value, the computer will ask for a file stored at a specific LBA value, the drive will translate that value and return the data, however if the LBA value is now in the plist then it will return the data from the wrong LBA resulting with a corrupted file.
If you are not sure, look again at the methodology behind the P-List and G-List it will help understand what happens.
If there was data stored in the new bad-sector then to the user the data is lost. This may appear as a file that not longer opens, a program that doesn't run any more or some other errant behaviour. This type of failure will not become apparent until the next time the file is attempted to be opened.
It may also be that it is such a long time since the file was opened that a backup plan could mean there are no backups of the working version. So bear this in mind when developing your backup plan.
It is possible to recover data directly from sectors listed in the G-list in some cases. This is one of the reasons, government regulations changed and hard disks that store sensitive data have to be physically destroyed once they have failed or when computers are upgraded.
Defect mapping in a live system
When a hard disk is powered up, the P-list and G-list are usually combined into a translator either dynamically or as a system-area module and then loaded into RAM on the drive's circuit board.
As requests for data come through, the location where the data is required from is passed to the translator, which makes the calculations necessary so as to determine which sectors to read in order to get to the requested data.
In our example above, if we wanted the data from LBA 6 the translator would first run through the P-list and add 2 sectors to the count for the two bad sectors found at the factory, it then checks this value in the G-list and finds it has been re-allocated to sector 104. It then reads sector 104 and presents you with the data.
In the good old days when you got change from a quid after buying a pint of beer and a packet of crisps, bad sector mapping was done manually with the user having to map out the bad sectors from a list in the documentation that came with the drive or was printed on the drive label. Each time the drive was formatted the manual mapping needed to be performed again. It was done by marking the cluster containing the bad sector as BAD in the FAT table or other indexing system depending on the operating system.