4

I am copying a file to a USB3.0 stick with an USB3.0 port using Ubuntu 16.04 with i3wm.

/dev/sdc1 * 8064 30277631 30269568 14.4G c W95 FAT32 (LBA)

To avoid problems related with the GUI I did the test using gcp

The copy of a 400MB file goes super fast until 99% (613.68 MB/s), then it hangs for like 30 seconds, and then it completes with an overall average of 10.22 MB/s (Time: 0:00:33)

Similar thing happens using nautilus or thunar. And seems to happen with any stick and any kind of file.

I expect the total transfer time to be way lower (e.g. 4 seconds)

Any ideas ?

stilllife
  • 537

2 Answers2

1

As of this writing in mid-2024, this is fairly common behavior, though a bit unexpected. More expected would be this answer here: https://askubuntu.com/a/1019069/926756

In short, when a program copies a file from one location to another it doesn't actually do any copying itself. Instead, the program asks the operating system to read some bytes from one file, and then tells the operating system to write those bytes into another file. The operating system then lies.

When the program tells the OS to write bytes, the OS - generally - reads those bytes into a buffer and tells the program that they have been written. Unless the copy program is copying using the O_SYNC flag, this is a lie. The OS merely promises that it will try to write the bytes sometime in the future if the power doesn't go out or the disk fails or a stray cosmic particle flips a bit in the RAM.

To ensure that the bytes are actually on disk and not just in the OS buffers, the program can call the sync() functions. In essence, this is a function in the OS that does nothing until the file write buffers are empty and then returns. It sounds like your copying program is telling you how many bytes it told the OS to write, minus 1 so it can only get to 99.9999...%, and then calls sync() to make sure the write is actually on disk. Unfortunately, once it does this, it has no option now but to wait.

In the link I posted above, a similar but different problem is described. It's the same scenario except the copy command does not appear to call sync() and exits when the OS has accepted all the data into its buffers. I believe this is far more typical because the program can consider its job done - the OS knows where the file bytes are and will return them correctly to whatever program wants to read the new data back. In that example, the umount command needs to have the bytes on disk and the OS forces it to wait for the buffers to empty, so instead of the copy command hanging the umount command does.

I don't believe the basic cp command offers synchronous copies, but one could use dd with oflag=sync,dsync to do a synchronous copy, and with status=progress for realtime updates. In this manner, every block (bs=4k is "safe" but larger block sizes are faster) is written to disk syncronously, and so the % update is a more accurate representation of what bytes are actually on the final device and not just what was saved into the OS file buffers.

rand'Chris
  • 211
  • 3
  • 6
0

The realistic speeds of USB 3.0 hardly ever match the theoretical speeds and in some cases, the performance of USB 3.0 may fall below the performance of USB 2.0.

Since you have tried by both terminal and Nautilus, I feel there is some bug which is showing 613MB/s in both the cases. Nautilus messing up speeds, progress and expected time in some cases is known but I'm unaware of gcp's bugs.

Try testing your drive on another system and if you still get such results, please ask your manufacturer to replace the stick.

Anonymint
  • 164