Why does this dip occur? copying from internal HDD to SD Card

>>62008277
all files are about the same size. Flacs.

File: 1483491931997.png (22KB, 452x321px) Image search: [Google]

22KB, 452x321px

same HDD to same SD card.

1 file of 10gb

now the dip is at the start

File: 1502764815621.png (18KB, 441x241px) Image search: [Google]

18KB, 441x241px

>>62008386
dip is back

Buffers generally.

>>62008464
HDD buffer?

sliding windows
there is some redundancy error or drop of data, the system backs off and uses smaller pieces of data till everything is okay, then starts using larger pieces of data again

>>62008277
Buffers and shitty filesystems, coping same amount of small files takes longer than one big file.

>>62008490
I don't entirely understand but can that be avoided somehow?

>>62008492
>coping same amount of small files takes longer than one big file.
I know. but see: >>62008282
and >>62008386

>>62008510
It's a HDD, you have a spinner disk, it's fragmented by design.

>>62008501
>can that be avoided somehow
you wouldn't really want to avoid it, it's a data integrity thing
it might be cool to see a solid bar but when that solid bar has randomly dropped data because of buffers overflowing, you're not getting all your data

>>62008533
so the solution is just HDD's with bigger buffers?

>>62008510
Do you really know?
Seems like you dont

>>62008607
look again. the dip occurs while copying 1 file of 10gb

it also occurs when copying hundreds of files that are all the same fucking size.

>>62008543
no

Three possible reasons come to mind.

- for small files, their file data has to be written to the file system. As in, the OS has to tell the file system to create file x at location y with attributes {z}, etc etc. Then the filesystem calls back the OS that it is created, you can proceed giving data, etc. Because of this, even a 1 byte file will require a minimum amount of time to create that is longer than the read speed of the drive / size of the file. For example 1000x one kbyte files will take way longer to copy than one 1000kbyte file.

- Due to fragmentation, the HDD cannot transfer at its maximum speed. This is exacerbated by the fact that on HDDs, data speed is different depending on where the data is physically located (inner part of the platter or outer part). Combine the two, and you got speed drops.

- SSDs are affected by the above but in a different way, the given flash cell may be partially written and needs to be read, cleared, rewritten, which takes longer than writing to an empty cell. The OS handles all of this transparently, you'll only see some speed drops from it. Due to fragmentation, this can occur more often over time. Consequently, defragging can indeed improve speeds on an SSD (not even trolling here - I tried it before). However I still don't recommend defragging, since if you are on Windows you'll lose shadow volume copies that way which are extremely useful for accidentally deleted/damaged files.

So for those reasons, you'll see dips in file copy speeds, you won't get max speed constantly all the time.

>>62008758
oh wait, you are using an SD card.

Then you also have to include more possible reasons, such as

- whatever filesystem the card uses (for my usb drives, FAT32 seems insane slow for some reason, ntfs and exfat are orders of magnitude faster, but not all OS support them, I keep the drive in fat32 for self booting).

- whatever SD card reader you use, has a connection that might sometimes drop, assuming it is a USB one.

- the controller for the card reader might be low quality (usb3 controllers sometimes can be a real bitch)

- the sd card might just suck, or be low quality trash advertised as super hq usb3 giga speed.

additionally, sometimes antivirus will chew on the files for a little bit when copying too. Not much, but when you are processing hundreds of files, it can add up fast.

>>62008758
>>62008813
also if you use a sd card that is already slow as shit, then the flash cell rewrites I mentioned above might be orders of magnitude slower.

>>62008758
>>62008813
>>62008849
thanks anon. very informative.

I'm using the SD card reader in my high end Dell monitor. It uses USB 2 to connect to my pc. Using MTP USB3.0 to my USB-C phone had an ETA that was very long so I canceled and put the micro-SD card in an adapter and in my SD card reader.

>>62008892
Yeah, I tend to remove the sd card from my camera and connect it directly to a card reader too, because it is five times faster that way, than if I connected the camera to usb.

If you want to make sure it is a fragmentation problem, make a backup of everything on the drive, format it (full format, not quick), then copy everything back. At least that one large MKV should have a flat speed curve, unless the hdd you copy from is already shot to shit.

>>62008277
Buffer/cache fill and clean.
Your SD is shit/worn so speeds are unstable.