Check real size of USB thumb drive

Question

I recently read a lot about fake usb thumb drives that claim to have a lot of space (often carried too far, like 128 GB), while physically offering only 0.5 to 4 GB. I recently bought a SanDisk USB drive (128 GB claimed) and want to test it's size. It's not bought via ebay or something, but I really want to test the real size before using it productively.

I could just copy stuff on it, copy it back and look if the files are okay. I could also automate it with Hashes and stuff. But I hoped there is a more accurate solution. I read that for Windows, H2testw does the trick. Is there an easy way to test this on Ubuntu/Linux? A specialized, well working tool maybe?

Update: Just to be clear, the idea is to verify that the size the linux system gets told by the controller is correct (so no data will be lost). It's not like I want to see if I get 128 GB instead of 127.3 GB. I want to test if all data I write will be readable again. Unfortunately I can only find a few information about this on english tec sites. There are good german sources, though. I'm actually searching for an application like those, but for Ubuntu/Linux: https://www.raymond.cc/blog/test-and-detect-fake-or-counterfeit-usb-flash-drives-bought-from-ebay-with-h2testw/

Answer 1

There is only one alternative I found, but I think this is even a better one than the original h2testw tool for MS Windows. Fortunately, it is really easy to use, even from command line. There are GUIs available, though. There is also a lot of information about the implementation and the problem with fake drives on the tools website.

F3 - "Fight Flash Fraud" or "Fight Fake Flash"

Source: http://oss.digirati.com.br/f3/
QT GUI: https://github.com/zwpwjwtz/f3-qt
OSX GUI: https://github.com/insidegui/F3X

The h2testw method

F3 is a collection of tools that deal with fake flash drives. Two of them together implement the h2testw-Method:

f3write [--start-at=NUM] [--end-at=NUM] <PATH>
f3read  [--start-at=NUM] [--end-at=NUM] <PATH>

f3write will ask for the devices claimed size and fill it with generated files with a size of 1gb each. f3read will read all those files and see of they are complete and not broken. As an example the commands I used to test my ~128gb thumb drive:

~> f3write /media/username/1EB8021AB801F0D7/
Free space: 117.94 GB
Creating file 1.h2w ... OK!                           
...
Creating file 118.h2w ... OK!                         
Free space: 0.00 Byte
Average writing speed: 11.67 MB/s

Now to test whether the files are correctly stored:

~> f3read /media/username/1EB8021AB801F0D7/
                  SECTORS      ok/corrupted/changed/overwritten
Validating file 1.h2w ... 2097152/        0/      0/      0
...
Validating file 118.h2w ... 1979488/        0/      0/      0

  Data OK: 117.94 GB (247346272 sectors)
Data LOST: 0.00 Byte (0 sectors)
           Corrupted: 0.00 Byte (0 sectors)
    Slightly changed: 0.00 Byte (0 sectors)
         Overwritten: 0.00 Byte (0 sectors)
Average reading speed: 32.38 MB/s

The test for a drive of this size took about three hours with this method and sometimes caused a heavy disk load on my computer, but it's said to me the most accurate.

The f3probe method

f3probe is another way to test the drives, not as accurate but faster since it does not write on the whole drive. You can read more about it on the tools website. If you want to be 100% sure, better use the h2testw method. As the developer describes on the website:

f3probe is the fastest way to identify fake drives and their real sizes.

and

Finally, thanks to f3probe being free software, and once f3probe is battle proven, f3probe could be embedded on smartphones, cameras, MP3 players, and other devices to stop once and for all the proliferation of fake flash.

There is also a usage example on the website:

$ sudo ./f3probe --destructive --time-ops /dev/sdb
[sudo] password for michel: 
F3 probe 6.0
Copyright (C) 2010 Digirati Internet LTDA.
This is free software; see the source for copying conditions.

WARNING: Probing normally takes from a few seconds to 15 minutes, but
         it can take longer. Please be patient.

Bad news: The device `/dev/sdb' is a counterfeit of type limbo

You can "fix" this device using the following command:
f3fix --last-sec=16477878 /dev/sdb

Device geometry:
             *Usable* size: 7.86 GB (16477879 blocks)
            Announced size: 15.33 GB (32155648 blocks)
                    Module: 16.00 GB (2^34 Bytes)
    Approximate cache size: 0.00 Byte (0 blocks), need-reset=yes
       Physical block size: 512.00 Byte (2^9 Bytes)

Probe time: 1'13"
 Operation: total time / count = avg time
      Read: 472.1ms / 4198 = 112us
     Write: 55.48s / 2158 = 25.7ms
     Reset: 17.88s / 14 = 1.27s

Note that it also returns a command that enables you to use the drive with it's real size, using f3fix.

The f3fix tool

f3fix enables users to use the real capacity of fake drives without losing data.

Install in Ubuntu

The described tools are part of the f3 package, which is at least available on Ubuntu 15.10. According to the website, there are some more tools that are available. To get them take a look at the website. To install the package just type into a terminal:

sudo apt-get install f3

The package comes with short but useful manpages, though I think they miss some information from the website about the difference of f3read/write and f3probe for example, which is why this answer got a little longer.

Answer 2

I have written a simple tool for just that, it's called CapacityTester (screenshot) and it has a GUI as well as a CLI.

There's a precompiled binary for Debian 7 available for download, which is very likely to work out of the box on a modern Ubuntu system.

I've written it for my own personal use because I couldn't find a graphical tool for this purpose. You just need to mount your empty USB flash drive first, select it and start the test. It's a very dumb tool because all it does is fill the drive with files and then verify that the data on the drive is correct. It will abort the test on the first error (writing or reading/verifying). It will report the offset of the chunk that could not be written or verified successfully, but this is a logical offset so this information may be useless because it depends on the filesystem where the files are located on the drive. However, when the drive has been filled with data and everything could be read and verified, it should be safe to assume that the drive's reported capacity is correct. As a side note, the test files are automatically deleted (this may not work if the drive is bad).

Again, it's very simple as it only works with files on top of an existing filesystem. So there are some KB (+ 1M buffer) that cannot be tested. And it's very slow because it really fills the whole filesystem. F3 is certainly much more sophisticated and also faster, but it has no GUI. The only reason CapacityTester exists is because it has a GUI so that it can be used by users who are not familiar with the command line or who simply prefer a GUI.

Feedback is appreciated.

Answer 3

Essentially, there isn't any real point to that because you could see the size with df. At least I haven't seen a USB that were able to masquerade its real size from software. However, if you insist to test by writing , the simple way to do it is with dd, just overwrite the device with zeros ( and of course remember to save your files before you do that ).

$> sudo dd if=/dev/zero of=/dev/sdb1 iflag=nocache oflag=direct bs=4096                        
[sudo] password for xieerqi: 
dd: error writing ‘/dev/sdb1’: No space left on device
1905097+0 records in
1905096+0 records out
7803273216 bytes (7.8 GB) copied, 5980.96 s, 1.3 MB/s

As you can see here , we've got an USB drive claiming to be 8GB, and it ran out of spate at 7.8 GB. Fairly simple, but time consuming process - took me about 2 - 3 hours.