Copy to USB memory stick really slow?

Question

When I copy files to the USB device, it takes much longer than in windows (same usb device, same port) it's faster than USB 1.0 speeds (1MB/s) but much slower than USB 2.0 speeds (12MB/s). To copy 1.8GB takes me over 10 minutes (it should be < 3 min.) I have two identical SanDisk Cruzer 8GB sticks, and I have the same problem with both. I have a super talent 32GB USB SSD in the neighboring port and it works at expected speeds.

The problem I seem to see in the GUI is that the progress bar goes to 90% almost instantly, completes to 100% a little slower and then hangs there for 10 minutes. Interrupting the copy at this point seems to result in corruption at the tail end of the file. If I wait for it to complete the copy is successful.

Any ideas? dmesg output below:

[64059.432309] usb 2-1.2: new high-speed USB device number 5 using ehci_hcd
[64059.526419] scsi8 : usb-storage 2-1.2:1.0
[64060.529071] scsi 8:0:0:0: Direct-Access     SanDisk  Cruzer           1.14 PQ: 0 ANSI: 2
[64060.530834] sd 8:0:0:0: Attached scsi generic sg4 type 0
[64060.531925] sd 8:0:0:0: [sdd] 15633408 512-byte logical blocks: (8.00 GB/7.45 GiB)
[64060.533419] sd 8:0:0:0: [sdd] Write Protect is off
[64060.533428] sd 8:0:0:0: [sdd] Mode Sense: 03 00 00 00
[64060.534319] sd 8:0:0:0: [sdd] No Caching mode page present
[64060.534327] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[64060.537988] sd 8:0:0:0: [sdd] No Caching mode page present
[64060.537995] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[64060.541290]  sdd: sdd1
[64060.544617] sd 8:0:0:0: [sdd] No Caching mode page present
[64060.544619] sd 8:0:0:0: [sdd] Assuming drive cache: write through
[64060.544621] sd 8:0:0:0: [sdd] Attached SCSI removable disk

Answer 1

Why is copying to my USB drive so slow in Linux (and faster in Windows)?

Reason 1. File caching can make writes appear slower or faster

The problem I seem to see in the GUI is that the progress bar goes to 90% almost instantly, completes to 100% a little slower and then hangs there for 10 minutes.

One thing you need to understand is file caching. Linux (and Windows) will use otherwise "empty" RAM to cache read/write operations and make them faster on subsequent accesses. Caching copy operations to slow devices results in the behavior you see -- the "fast completion" is actually writing to the cache, and then it slows and stops because the actual flushing of the data in the cache (sync) to the slow device is taking very long. If you abort at that point, the data is corrupted (as you noted) since the sync never finished.

Such copying in Windows may seem faster (including the reported MB/sec speeds) because sometimes Windows will not wait for the sync, and declare the job completed as soon as the data is written to cache.

Reason 2. Writing lots of files, especially small ones, is slow

To copy 1.8GB

Because of the way flash memory and filesystems work, the fastest throughput (speed) is achieved when writing very large files. Writing lots of small files, or even mixed data containing a number of small files can slow the process down a lot. This affects hard drives too, but to a somewhat lesser extent.

Reason 3. Write speeds of a USB stick and an SSD cannot be compared

I have a super talent 32GB USB SSD in the neighboring port and it works at expected speeds.

• A garden variety USB stick usually consists of flash memory chips that are written to serially (sequentially), and does not have any cache of its own.

• An SSD, on the other hand, contains a controller which writes to the flash memory chips parallel, increasing the throughput by a factor of 2 or more over the USB stick.

  • If your 32GB SSD had 4x 8GB chips, it would still be 4x faster than the USB stick at any write operation.
  • The SSD also contains RAM cache (like hard disks), so it can quickly store incoming data in the cache and tell the OS that it's done, while it still has to actually write that data to the flash memory.

• So, with one large file, your 32GB GB with the 4x structure we assumed, would be 4x as fast; with many small files, it would be 10x or more faster because it could intelligently store them in its cache.

---

To sum up, these are the reasons why file copying to USB sticks may appear slower in Linux. Is it actually slower because of a hardware/driver issue or whatever....

Doing a proper comparison of write speeds between Linux and Windows

• First of all, forget about the SSD because of reason 3. It's like oranges and apples.
• To negate the effects of reason 1 (caching) and reason 2 (small files), you need to test with a single large file, larger than the amount of RAM on the test system.
• In Linux you can create it with dd if=/dev/urandom of=largetest bs=1M count=7500, which gives you a 7500 MB test file. Assuming your system has less than 4GB or so of RAM, it's good enough. Copy that to a freshly formatted Sandisk 8GB stick, and time it.
• Reboot in Windows, and copy largetest from the USB stick to your hard disk. Reboot again (to remove it from the cache). Then format the USB stick (same vfat/FAT32!), and copy largetest from the hard disk to the stick.
• How do the times compare?

Answer 2

It's 2019 and I'm still having this same issue. So I figured I search the internet for a solution. I found the following page that suggests one: https://gist.github.com/2E0PGS/f63544f8abe69acc5caaa54f56efe52f

It says:

Execute the following commands in a console to see if it fixes the problem for you. You might need to sudo su first to have the required permission.

echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes
echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes

If it works you could make this change persistent across reboots by pasting the two lines at the end of your /etc/rc.local file.

For me it had the following effect:

Prior copying large files to an USB drive would start out really fast (like 60 MB/s) and become slower and slower (< 10 MB/s) until it looked like it would never finish.

Now it starts out slower, but gets faster and faster and finishes sooner than before. So it does seem to "solve" the problem or at least have a positive effect.

Answer 3

Found the fix all i did was unmount, remove drive, and run sudo modprobe ehci_hcd in the Terminal. Insert drive and agian sudo modprobe ehci_hcd when I put the drive in and wow 20/mbs thought i would share. Hope I dont have to do it every time... but it's not to hard...

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/177235 says they fixed the bug.

Answer 4

I think the chances are very low that it is a port issue. It is more likely a LINUX (or linux configuration) issue - googgle around and you will find thousands of issue reports about slow USB in linux/ubuntu. For me it is almost a showstopper for linux - I now have an Ubuntu 12.04 LTS and still have this issue (so I rather use the Win7 setup - mainly/only because of this). This issue (or something with similar symptoms) is there for several years now, apparently no fix. And during this time I tried several physical PCs with several different ubuntu versions (default config) and 2-3 different USB sticks....

Answer 5

Just umount the device if it is automounted already, and manually mount it to /mnt/foldername.

In my case,

umount /media/usb0
mount /dev/sdb1 /mnt/sam

After that it is coping very fast.

Answer 6

If you switch to a USB 3.0 , you will go from 1mb/s to a woping 5-8mb/s. I switch to a 3.0 USB pci and external HD and haven't looked back.

Answer 7

When you look in /etc/mtab, do you see that the device has been mounted with the "flush" option?

If so, this could be the cause of the problem (it was for me). Just unmount the device and remount it, it shouldn't be set by default.

Answer 8

This is an old question. Also I had the same problem for a long time. Recently I found the solution here. What I did is:

Reformatting USB from FAT to NTFS.

Answer 9

Try the NTFS-3g driver

Switching to the NTFS-3g driver increased the write speed from 1 MB/s to 231 MB/s for my Patriot Supersonic Rage 2 512 GB USB flash drive on Ubuntu 23.10.

This is how the drive gets auto-mounted (which gives mostly just 1 MB/s write speed):

$ mount | grep /dev/sd
/dev/sdb1 on /media/user/Rage2 type ntfs3
  (rw,nosuid,nodev,relatime,uid=1000,gid=1000,iocharset=utf8,uhelper=udisks2)

Re-mounting with the NTFS-3g driver:

$ umount /media/user/Rage2
$ sudo mount -t ntfs-3g -o nosuid,nodev,relatime,uid="$(id -u)",gid="$(id -g)",iocharset=utf8 /dev/sdXX /mnt
$ mount | grep /dev/sd
/dev/sdb1 on /mnt type fuseblk
  (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,
  allow_other,blksize=4096)

Testing the performance:

$ time cp -ai 360GB_of_files /mnt; time sync
real    24m36.252s
user    0m0.587s
sys     4m18.616s
real    1m24.520s
user    0m0.000s
sys     0m0.206s

One-minute average transfer speeds (from iostat -dm 60 /dev/sdb):

• From 83 MB/s to 356 MB/s

Total average transfer speed, including the sync at the end:

• 231 MB/s

Maybe the default NTFS driver writes to the flash drive in a way that is very suboptimal for this particular drive. From https://www.storagereview.com/review/patriot-supersonic-rage-2-usb-flash-drive-review:

When switching to random large-block transfers, the Supersonic Rage 2 posted 242.96MB/s read and 0.995MB/s write. [...] Our last flash drive benchmark measures 4K random transfer performance in IOPS. In this benchmark, the Supersonic Rage 2 posted pretty low numbers; 1,546.4 IOPS read and 0.57 IOPS write.

Answer 10

I had some problems also with transfer rate on a WD external disk, after opening it in a windows SO, i always used LINUX, after that the transfer rate was like 1.5mb/s than i unmount the external hard drive runed dmesg there it was saying that sdb1 it was unporperly unmounted, runed a fsck, that made a few repairs and after that 20mb/s of transfer rate again when copiyng from sda to external disk. fsck is always a risk if you have data, but it worked for me, with no data loss.

Answer 11

I also had this problem, but I use the cp command and you update your usb stick in seconds;

cp -r -u /home/user/Muziek/ /media/user/Audiousbsti
cp -r -u /home/user/Muziek/ /media/user/4F49-4A65/

I think it is a very late answer but it is still open.

Answer 12

Okay, I had the same problem for three days and how I managed to backup my 1TB hard drive was by using rsync, I know that it is used for backing up but it got the job done, even when transferring large files I use it to do that job. If you would like to use it with a GUI I suggest installing Grsync which is a graphical version of rsync since rsync runs on the terminal.

Hope this helped