[gentoo-user] rsync options after backup restore. Transfer speed again.

[Dale]

Howdy,

As most know, I had to restore from backups recently.  I also reworked
my NAS box.  I'm doing my first backup given that I have more files that
need to be added to the backups.  When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed.  Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.  Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp?  Is there a way
to just update the time stamps on the NAS box?  Is there a option I
haven't thought of to work around this? 

This is the old command I was using to create the backups.

time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

I tried these to try to get around it.

time rsync -ivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/
time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/
time rsync -a --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this? 


Second problem.  The transfer speed is back to the old slower speed. 
I'm pretty sure I am using the same old options on both ends.  Still,
it's back to being slow again.  Some info:


nas / # mount | grep backup
/dev/mapper/backup on /mnt/backup type ext4 (rw,relatime)
nas / # cat /etc/exports
# /etc/exports: NFS file systems being exported.  See exports(5).
/mnt/backup/  *(rw,async,no_subtree_check)
nas / #

***@fireball / # mount | grep backup
10.0.0.5:/mnt/backup on /mnt/Backup type nfs4
(rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,nocto,proto=tcp,nconnect=4,timeo=600,retrans=2,sec=sys,clientaddr=10.0.0.2,local_lock=none,addr=10.0.0.5)
***@fireball / #

Did I miss something?  Typo maybe?  I'm pretty sure I used copy and
paste but still. 

Thanks.

Dale

:-)  :-) 

[Daniel Pielmeier]

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this?

Then you didn't read properly, the man page is really exhaustive. You can try "--size-only".
It is also possible to negate the options by prefixing them with no like e.g. "--no-times --no-owner --no-group etc." I also had this issue when transferring between different file systems which didn't have same concept of file properties.
Anyway using "-a" as you tried doesn't make sense as it tries to preserve almost everything.

Post by Dale
Second problem.  The transfer speed is back to the old slower speed. 
I'm pretty sure I am using the same old options on both ends.  Still,
it's back to being slow again.

But the new disk is different compared to the old one? Then it needs to be treated differently. There multiple factors which can affect performance.
Had the same issue with a new SSD drive. Unfortunately I currently don't have access to the machine and I don't remember what needed changing but it could well be possible that you need different mount options, format the drive using other options, using different schedulers, etc.

[Dale]

Post by Daniel Pielmeier

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this?

Then you didn't read properly, the man page is really exhaustive. You can try "--size-only".
It is also possible to negate the options by prefixing them with no like e.g. "--no-times --no-owner --no-group etc." I also had this issue when transferring between different file systems which didn't have same concept of file properties.
Anyway using "-a" as you tried doesn't make sense as it tries to preserve almost everything.

Actually, I was looking at the man page again and that option appeared
to me just a little bit ago.  I tried it and it did several updates. 
Question tho.  It finished that update just a minute ago.  I tried again
with my usual command, it wants to update everything again.  I seem to
recall that rsync goes by file size and modify time stamps.  Is there a
way to make the NAS box files newer or something so that it will not try
to copy the same files over again?  I'd like to use the default
detection options of rsync if I can just in case something changes. 

Post by Daniel Pielmeier

Post by Dale
Second problem.  The transfer speed is back to the old slower speed. 
I'm pretty sure I am using the same old options on both ends.  Still,
it's back to being slow again.

But the new disk is different compared to the old one? Then it needs to be treated differently. There multiple factors which can affect performance.
Had the same issue with a new SSD drive. Unfortunately I currently don't have access to the machine and I don't remember what needed changing but it could well be possible that you need different mount options, format the drive using other options, using different schedulers, etc.

Exact same drives.  While adding a new drive to LVM, somehow the file
system got messed up.  I couldn't get it fixed so I just started over
again.  I might add, the restore from backups was faster.  That was when
the extra options were posted in another thread.  I'd think it would go
the same speed pretty much either way. 

Dale

:-)  :-) 

[Daniel Pielmeier]

Post by Dale
Actually, I was looking at the man page again and that option appeared
to me just a little bit ago.  I tried it and it did several updates. 
Question tho.  It finished that update just a minute ago.  I tried again
with my usual command, it wants to update everything again.  I seem to
recall that rsync goes by file size and modify time stamps.  Is there a
way to make the NAS box files newer or something so that it will not try
to copy the same files over again?  I'd like to use the default
detection options of rsync if I can just in case something changes. 

We are creatures of habit eh :-)

Of course you have to specify this option every time as the timestamps remain the same in this case.
Why not? First of all it's just video files. What are the odds that the contents of change and the file size will be exactly the same?

Using the defaults is fine but you should use the options which serve the purpose.

If you want you can use find on the backup drive in combination with exec to run the touch command on each file in order to update the timestamps. Then the -u option you use with rsync wont update the file on the destination because it is newer then the source. However in the end the result is the same as if using "--size-only".
Don't nail me on this but the command could look like the following: find pathtobackup -type f -exec touch '{}' +

[Dale]

Post by Daniel Pielmeier

Post by Dale
Actually, I was looking at the man page again and that option appeared
to me just a little bit ago.  I tried it and it did several updates. 
Question tho.  It finished that update just a minute ago.  I tried again
with my usual command, it wants to update everything again.  I seem to
recall that rsync goes by file size and modify time stamps.  Is there a
way to make the NAS box files newer or something so that it will not try
to copy the same files over again?  I'd like to use the default
detection options of rsync if I can just in case something changes. 

We are creatures of habit eh :-)
Of course you have to specify this option every time as the timestamps remain the same in this case.
Why not? First of all it's just video files. What are the odds that the contents of change and the file size will be exactly the same?
Using the defaults is fine but you should use the options which serve the purpose.
If you want you can use find on the backup drive in combination with exec to run the touch command on each file in order to update the timestamps. Then the -u option you use with rsync wont update the file on the destination because it is newer then the source. However in the end the result is the same as if using "--size-only".
Don't nail me on this but the command could look like the following: find pathtobackup -type f -exec touch '{}' +

That helped.  I ended up running this command, with path being longer so
I shortened it a bit. 


find /path/to/files/* -type f -exec touch '{}' +


In case anyone runs up on this, I added the wild card on the end.  I ran
it first without but it didn't work as expected.  After I added it, it
updated the modify time to the current time.  I did the first run in a
test directory. 

I'm currently rechecking the kernel config on both puter, fireball and
nas.  Trying to sort out the speed problem and I think the problem may
be a missing driver on fireball actually.  Then again, nas may have a
missing driver as well.  I'm working on it. 

Thanks for the help.  Got one problem sorted at least.  ;-) 

Dale

:-)  :-) 

[Michael]

Post by Dale
Howdy,
As most know, I had to restore from backups recently. I also reworked
my NAS box. I'm doing my first backup given that I have more files that
need to be added to the backups. When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed. Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.

Did you figure this by guessing, or by actually comparing timestamps and even
running a checksum on a couple of files?

Post by Dale
Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp? Is there a way
to just update the time stamps on the NAS box? Is there a option I
haven't thought of to work around this?

Before you ask rsync to ignore (some) differences, I suggest you investigate
to establish what these differences are and why they could have occurred. I
assume you'd rather fix the problem, as opposed to work around it.

Options you could consider is --modify-window=1 and perhaps --size-only, but
only if you are not interested in identifying and conclusively addressing the
cause of the problem.

Post by Dale
This is the old command I was using to create the backups.
time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/
I tried these to try to get around it.
time rsync -ivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/
time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/
time rsync -a --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

The -a option on its own includes -r, as the man page explains:

--archive, -a archive mode is -rlptgoD (no -A,-X,-U,-N,-H)

and

--archive, -a
This is equivalent to -rlptgoD. It is a quick way of saying you want re‐
cursion and want to preserve almost everything. Be aware that it does
not include preserving ACLs (-A), xattrs (-X), atimes (-U), crtimes (-N),
nor the finding and preserving of hardlinks (-H).

The only exception to the above equivalence is when --files-from is spec‐
ified, in which case -r is not implied.

Post by Dale
Second problem. The transfer speed is back to the old slower speed.
I'm pretty sure I am using the same old options on both ends. Still,
it's back to being slow again.

Did you fix the your kernel on your PC to use hardware acceleration for
crypto?

Did you test the maximum network raw transfer speed to eliminate any network
problem?

[John Blinka]

Not (unfortunately) speaking from my own experience, just from Googling:

https://serverfault.com/questions/211005/rsync-difference-between-checksum-and-ignore-times-options

discusses a —ignore-times option that may be useful. The upside is that it
uses the minimum amount of network bandwidth but at the cost of
checksumming everything on both sides. May take a while if you have a big
stash of video files. However, there’s a —checksum-choice option that, with
some testing of choices, might give you a quick-and-dirty checksum that’s
good enough for your purposes. Your backup files , hopefully, are not
corrupted.

I was just wondering why your new backup seemed to want to transfer
everything all over again. Just speculating, but maybe timestamps got
changed in the NAS box work you did? I saw a post (that I can’t find now)
that discussed the —modify-window option. This, I gather, is used when
rsyncing between 2 different filesystems with different ideas about how
precise to make the timestamps. A high precision timestamp file system’s
timestamp will be a little different from a low precision timestamp file
system’s timestamp, and will appear to be different timestamps to rsync,
even though ls -l reports the same times. The stat command will give you
full precision times, and maybe that will show a discrepancy. The
—modify-window option lets you fudge subtle time differences. This is just
speculation, and maybe an exotic and unlikely one, and definitely something
I’ve never done myself.

Another idea (again something I Googled but have never done) is to use the
mtree utility. Apparently you can use this to write out the characteristics
of a directory hierarchy - things like owner, permissions, checksum, size,
timestamps, etc. on every file under that directory. Save this output to a
file, then run mtree with this file as input on another directory
hierarchy, say the backup on your NAS. Looks like you can use it to just
check for differences, or you can use it to correct differences. If your
timestamps were different on your video machine and your backup NAS, this
looks like a way to make them match without transferring files.

Hope a little of this was useful. Good luck!

John Blinka

[Frank Steinmetzger]

Post by Dale
Howdy,
As most know, I had to restore from backups recently.  I also reworked
my NAS box.  I'm doing my first backup given that I have more files that
need to be added to the backups.  When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed.  Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.  Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp?  Is there a way
to just update the time stamps on the NAS box?  Is there a option I
haven't thought of to work around this? 
This is the old command I was using to create the backups.
time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

This didn’t preserve timestamps. Hence there is one type of information lost
from which rsync knows whether two files may be identical. So now your
restore has more recent timestamps the the backup. If you use -u, Rsync
should skip all files.

My perfectionist self doesn’t like discarding timestamp information, because
then my system can’t tell me how old some file is, and how old (or young) I
was when I created it and so on. I once didn’t pay enough attention when
restoring a backup back when I was still on Windows, which is why I don’t
have many files left that are dated before April 2007, even though they are
from 2000 ± x.

BTW: -i and -v are redundant. -v will only print the file path, whereas -i
does the same and adds the reasons colum at the beginning.

Post by Dale
I tried these to try to get around it.
time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series /mnt/TV_Backup/

-a and -r are also redundant, as -a includes -r.

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this? 

My muscle memory uses `rsync -ai` for almost everything. And when I do full
root file systems or stuff where I know I will need them, I use -axAHX
instead. Since this preserves all the usual data, I’ve never really had
rsync wanting to do everything all over.

[Dale]

Post by Dale
Howdy,
As most know, I had to restore from backups recently.  I also reworked
my NAS box.  I'm doing my first backup given that I have more files that
need to be added to the backups.  When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed.  Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.  Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp?  Is there a way
to just update the time stamps on the NAS box?  Is there a option I
haven't thought of to work around this? 
This is the old command I was using to create the backups.
time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

This didn’t preserve timestamps. Hence there is one type of information lost
from which rsync knows whether two files may be identical. So now your
restore has more recent timestamps the the backup. If you use -u, Rsync
should skip all files.
My perfectionist self doesn’t like discarding timestamp information, because
then my system can’t tell me how old some file is, and how old (or young) I
was when I created it and so on. I once didn’t pay enough attention when
restoring a backup back when I was still on Windows, which is why I don’t
have many files left that are dated before April 2007, even though they are
from 2000 ± x.
BTW: -i and -v are redundant. -v will only print the file path, whereas -i
does the same and adds the reasons colum at the beginning.

Post by Dale
I tried these to try to get around it.
time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series /mnt/TV_Backup/

-a and -r are also redundant, as -a includes -r.

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this? 

My muscle memory uses `rsync -ai` for almost everything. And when I do full
root file systems or stuff where I know I will need them, I use -axAHX
instead. Since this preserves all the usual data, I’ve never really had
rsync wanting to do everything all over.

Well, I can't turn back the clocks so it is what it is now.  These files
tho, I really don't worry to much about the timestamps.  If I were
backing up my OS tho, that could become a problem. 

So my command should be more like:

rsync -ai --progress --delete /path/to/source/ path/to/target

If I want to preserve all the Linux file data, then I should use this: 

rsync -axAHX --progress --delete /path/to/source/ path/to/target

Dale

:-)  :-) 

P. S.  Working on new kernel for fireball.  Added some options for
encryption stuff.  I really need to update to a newer kernel.  I got a
newer one that boots but no GUI.  That's not very helpful. 

[William Kenworthy]

Post by Dale

Post by Dale
Howdy,
As most know, I had to restore from backups recently.  I also reworked
my NAS box.  I'm doing my first backup given that I have more files that
need to be added to the backups.  When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed.  Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.  Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp?  Is there a way
to just update the time stamps on the NAS box?  Is there a option I
haven't thought of to work around this?
This is the old command I was using to create the backups.
time rsync -uivr --progress --delete /home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

This didn’t preserve timestamps. Hence there is one type of information lost
from which rsync knows whether two files may be identical. So now your
restore has more recent timestamps the the backup. If you use -u, Rsync
should skip all files.
My perfectionist self doesn’t like discarding timestamp information, because
then my system can’t tell me how old some file is, and how old (or young) I
was when I created it and so on. I once didn’t pay enough attention when
restoring a backup back when I was still on Windows, which is why I don’t
have many files left that are dated before April 2007, even though they are
from 2000 ± x.
BTW: -i and -v are redundant. -v will only print the file path, whereas -i
does the same and adds the reasons colum at the beginning.

Post by Dale
I tried these to try to get around it.
time rsync -ar --progress --delete /home/dale/Desktop/Crypt/TV_Series /mnt/TV_Backup/

-a and -r are also redundant, as -a includes -r.

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this?

My muscle memory uses `rsync -ai` for almost everything. And when I do full
root file systems or stuff where I know I will need them, I use -axAHX
instead. Since this preserves all the usual data, I’ve never really had
rsync wanting to do everything all over.

Well, I can't turn back the clocks so it is what it is now.  These files
tho, I really don't worry to much about the timestamps.  If I were
backing up my OS tho, that could become a problem.
rsync -ai --progress --delete /path/to/source/ path/to/target
rsync -axAHX --progress --delete /path/to/source/ path/to/target
Dale
:-)  :-)
P. S.  Working on new kernel for fireball.  Added some options for
encryption stuff.  I really need to update to a newer kernel.  I got a
newer one that boots but no GUI.  That's not very helpful.
Hi Dale, I might have missed it in the thread but are you aware that
rsync is focussed on remote filefile transfer and if its a local
transfer it does a full copy (no delta) of the file without
optimisations as its usually faster than all the extra operations a
local delta requires.

You are using NFS mounts so rsync is looking at it as a local copy - it
does not know it is a remote system.

My recent use of moosefs (another network file system) had similar
problems using rsync - it also turns out some of the data rsync uses for
detecting changes may not be stable across a network mount - moosefs
certainly has problems with this, NFS likely to have it too.

The workaround is to check file-size, mtimes and ctimes to figure out
which is able to be used.

see:

http://unix.stackexchange.com/questions/450537/ddg#450666

google rsync over nfs

BillK

[Dale]

Post by William Kenworthy

Post by Dale

Post by Dale
Howdy,
As most know, I had to restore from backups recently.  I also reworked
my NAS box.  I'm doing my first backup given that I have more files that
need to be added to the backups.  When I started the rsync, it's
starting from the first file and updating each file as it goes as if all
of them changed.  Given that likely 95% of the files hasn't changed, I
figure this is being done because of a time stamp or something.  Is
there a way to tell rsync to ignore the time stamp or something or if
the files are the same size, just update the time stamp?  Is there a way
to just update the time stamps on the NAS box?  Is there a option I
haven't thought of to work around this?
This is the old command I was using to create the backups.
time rsync -uivr --progress --delete
/home/dale/Desktop/Crypt/TV_Series
/mnt/TV_Backup/

This didn’t preserve timestamps. Hence there is one type of information lost
from which rsync knows whether two files may be identical. So now your
restore has more recent timestamps the the backup. If you use -u, Rsync
should skip all files.
My perfectionist self doesn’t like discarding timestamp information, because
then my system can’t tell me how old some file is, and how old (or young) I
was when I created it and so on. I once didn’t pay enough attention when
restoring a backup back when I was still on Windows, which is why I don’t
have many files left that are dated before April 2007, even though they are
from 2000 ± x.
BTW: -i and -v are redundant. -v will only print the file path, whereas -i
does the same and adds the reasons colum at the beginning.

Post by Dale
I tried these to try to get around it.
time rsync -ar --progress --delete
/home/dale/Desktop/Crypt/TV_Series /mnt/TV_Backup/

-a and -r are also redundant, as -a includes -r.

Post by Dale
I looked at the man page and the options there.  I don't see anything
that I think will help.  Is there a way around this?

My muscle memory uses `rsync -ai` for almost everything. And when I do full
root file systems or stuff where I know I will need them, I use -axAHX
instead. Since this preserves all the usual data, I’ve never really had
rsync wanting to do everything all over.

Well, I can't turn back the clocks so it is what it is now.  These files
tho, I really don't worry to much about the timestamps.  If I were
backing up my OS tho, that could become a problem.
rsync -ai --progress --delete /path/to/source/ path/to/target
rsync -axAHX --progress --delete /path/to/source/ path/to/target
Dale
:-)  :-)
P. S.  Working on new kernel for fireball.  Added some options for
encryption stuff.  I really need to update to a newer kernel.  I got a
newer one that boots but no GUI.  That's not very helpful.
Hi Dale, I might have missed it in the thread but are you aware that
rsync is focussed on remote filefile transfer and if its a local
transfer it does a full copy (no delta) of the file without
optimisations as its usually faster than all the extra operations a
local delta requires.

You are using NFS mounts so rsync is looking at it as a local copy -
it does not know it is a remote system.
My recent use of moosefs (another network file system) had similar
problems using rsync - it also turns out some of the data rsync uses
for detecting changes may not be stable across a network mount -
moosefs certainly has problems with this, NFS likely to have it too.
The workaround is to check file-size, mtimes and ctimes to figure out
which is able to be used.
http://unix.stackexchange.com/questions/450537/ddg#450666
google rsync over nfs
BillK

I've read about using rsync as a daemon.  It is supposed to be faster. 
For some reason, I just never got around to trying it.  It would likely
be a good idea to work on that.  It would likely be really handy when
moving large amounts of data.  I may try to read a howto on the Gentoo
wiki or something.  See what all it involves. 

Interesting link.  I find some good info on that site at times. 

Dale

:-)  :-) 

[Wol]

Post by Dale
I've read about using rsync as a daemon.  It is supposed to be faster.
For some reason, I just never got around to trying it.  It would likely
be a good idea to work on that.  It would likely be really handy when
moving large amounts of data.  I may try to read a howto on the Gentoo
wiki or something.  See what all it involves.

Run the daemon on the NAS. For each file where the metadata differs, the
NAS will send a bunch of block checksums to your rsync command (or the
other way round).

If all the checksums agree, then the files are identical, without having
to send the file over the network.

If only a few checksums differ, only a few blocks have to be sent.

In other words, using a daemon will massively reduce the amount of data
going across the network, at the expense of the NAS having to scan and
checksum every "file of interest". A pretty cheap trade-off, imho.

Cheers,
Wol

[Dale]

Post by Dale
Second problem.  The transfer speed is back to the old slower speed. 
I'm pretty sure I am using the same old options on both ends.  Still,
<<< SNIP >>>
Did I miss something?  Typo maybe?  I'm pretty sure I used copy and
paste but still. 
Thanks.
Dale
:-)  :-)

I been working on the speed problem again.  I rebuilt the kernel on
fireball and I think some changes made a huge change.  This is the
results from fireball now:


***@fireball / # cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       931239 iterations per second for 256-bit key
PBKDF2-sha256    1356501 iterations per second for 256-bit key
PBKDF2-sha512     972705 iterations per second for 256-bit key
PBKDF2-ripemd160  648871 iterations per second for 256-bit key
PBKDF2-whirlpool  362077 iterations per second for 256-bit key
argon2i       5 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id      4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
#     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b       570.8 MiB/s      2045.6 MiB/s
    serpent-cbc        128b        91.1 MiB/s       310.0 MiB/s
    twofish-cbc        128b       198.7 MiB/s       218.9 MiB/s
        aes-cbc        256b       428.8 MiB/s      1670.4 MiB/s
    serpent-cbc        256b        91.6 MiB/s       309.5 MiB/s
    twofish-cbc        256b       199.8 MiB/s       219.2 MiB/s
        aes-xts        256b      1821.2 MiB/s      1767.1 MiB/s
    serpent-xts        256b       265.9 MiB/s       270.2 MiB/s
    twofish-xts        256b       201.0 MiB/s       204.2 MiB/s
        aes-xts        512b      1440.0 MiB/s      1445.9 MiB/s
    serpent-xts        512b       265.0 MiB/s       257.2 MiB/s
    twofish-xts        512b       198.2 MiB/s       201.6 MiB/s
***@fireball / #


As you can see, aes-cbc is fast now and I think that is what cryptsetup
uses.  It used to be really slow I think. 

Now on to the nas box.  I've recompiled the kernel with some added
options.  Still, it refuses to speed up.  I kinda think it is the CPU
lacking support for encryption.  I'm asking others just in case I'm
missing something.  Also, fireball uses a older kernel, 5.14 or so.  The
nas box uses 6.1 or so.  The menus are different and that is why it is
hard to get them to match up.  I may have missed something.  This is the
bench mark from nas box. 


nas ~ # cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       700919 iterations per second for 256-bit key
PBKDF2-sha256     924670 iterations per second for 256-bit key
PBKDF2-sha512     729190 iterations per second for 256-bit key
PBKDF2-ripemd160  517559 iterations per second for 256-bit key
PBKDF2-whirlpool  359593 iterations per second for 256-bit key
argon2i       4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id      4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
#     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b        63.6 MiB/s        41.6 MiB/s
    serpent-cbc        128b        81.0 MiB/s       212.4 MiB/s
    twofish-cbc        128b       192.5 MiB/s       222.1 MiB/s
        aes-cbc        256b        47.5 MiB/s        30.0 MiB/s
    serpent-cbc        256b        81.2 MiB/s       212.7 MiB/s
    twofish-cbc        256b       192.3 MiB/s       221.9 MiB/s
        aes-xts        256b        65.9 MiB/s        41.6 MiB/s
    serpent-xts        256b       201.7 MiB/s       205.7 MiB/s
    twofish-xts        256b       216.2 MiB/s       214.5 MiB/s
        aes-xts        512b        48.8 MiB/s        30.0 MiB/s
    serpent-xts        512b       202.7 MiB/s       205.6 MiB/s
    twofish-xts        512b       216.4 MiB/s       214.0 MiB/s
nas ~ #


I seem to recall it being said that the old CPU in the nas box lacks the
aes instruction set.  This is a list of the CPU flags from the nas box.


nas ~ # cpuid2cpuflags
CPU_FLAGS_X86: 3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a
nas ~ #


The aes shows up on fireball.  It does not on the nas box.  Is the speed
above as good as I can expect with this older CPU?  I can include the
kernel config if needed.  If you know what driver you are looking for,
let me know what to grep for.  If not sure, I can attach the config file. 

Is this it?  Is this as fast as this old CPU can get? 

Dale

:-)  :-) 

[Michael]

I been working on the speed problem again. I rebuilt the kernel on
fireball and I think some changes made a huge change. This is the
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1 931239 iterations per second for 256-bit key
PBKDF2-sha256 1356501 iterations per second for 256-bit key
PBKDF2-sha512 972705 iterations per second for 256-bit key
PBKDF2-ripemd160 648871 iterations per second for 256-bit key
PBKDF2-whirlpool 362077 iterations per second for 256-bit key
argon2i 5 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id 4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 570.8 MiB/s 2045.6 MiB/s
serpent-cbc 128b 91.1 MiB/s 310.0 MiB/s
twofish-cbc 128b 198.7 MiB/s 218.9 MiB/s
aes-cbc 256b 428.8 MiB/s 1670.4 MiB/s
serpent-cbc 256b 91.6 MiB/s 309.5 MiB/s
twofish-cbc 256b 199.8 MiB/s 219.2 MiB/s
aes-xts 256b 1821.2 MiB/s 1767.1 MiB/s
serpent-xts 256b 265.9 MiB/s 270.2 MiB/s
twofish-xts 256b 201.0 MiB/s 204.2 MiB/s
aes-xts 512b 1440.0 MiB/s 1445.9 MiB/s
serpent-xts 512b 265.0 MiB/s 257.2 MiB/s
twofish-xts 512b 198.2 MiB/s 201.6 MiB/s
As you can see, aes-cbc is fast now and I think that is what cryptsetup
uses. It used to be really slow I think.

Yep, it was many times slower with your previous kernel. Now it will saturate
your 1Gbps network link, as long as the other box can take it.

Now on to the nas box. I've recompiled the kernel with some added
options. Still, it refuses to speed up. I kinda think it is the CPU
lacking support for encryption. I'm asking others just in case I'm
missing something.

No, you're not missing anything, although in a previous message you had posted
a benchmark result for the Phenom performance being around 4 times higher than
what you show below. As I mentioned then, the Phenom CPU does not have AES-NI
crypto hardware acceleration. it is a rather old CPU. Assuming the MoBo has
the latest OEM firmware and you have also added microcode in your kernel
(CONFIG_MICROCODE_AMD=y), then probably that's all it can do. Small
optimisations may show up between kernel releases, yet again patches to
address CPU vulnerabilities (retpoline?) could make things a bit worse.

Also, fireball uses a older kernel, 5.14 or so. The
nas box uses 6.1 or so. The menus are different and that is why it is
hard to get them to match up. I may have missed something. This is the
bench mark from nas box.

You can diff previous and current kernel config files to see what has changed.
For the crypto settings you can grep for AES and for CRYPT to make sure you
have not left out what you'd need for disk encryption.

nas ~ # cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1 700919 iterations per second for 256-bit key
PBKDF2-sha256 924670 iterations per second for 256-bit key
PBKDF2-sha512 729190 iterations per second for 256-bit key
PBKDF2-ripemd160 517559 iterations per second for 256-bit key
PBKDF2-whirlpool 359593 iterations per second for 256-bit key
argon2i 4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id 4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b 63.6 MiB/s 41.6 MiB/s
serpent-cbc 128b 81.0 MiB/s 212.4 MiB/s
twofish-cbc 128b 192.5 MiB/s 222.1 MiB/s
aes-cbc 256b 47.5 MiB/s 30.0 MiB/s
serpent-cbc 256b 81.2 MiB/s 212.7 MiB/s
twofish-cbc 256b 192.3 MiB/s 221.9 MiB/s
aes-xts 256b 65.9 MiB/s 41.6 MiB/s
serpent-xts 256b 201.7 MiB/s 205.7 MiB/s
twofish-xts 256b 216.2 MiB/s 214.5 MiB/s
aes-xts 512b 48.8 MiB/s 30.0 MiB/s
serpent-xts 512b 202.7 MiB/s 205.6 MiB/s
twofish-xts 512b 216.4 MiB/s 214.0 MiB/s
nas ~ #

How does your aes-xts 30.0 MiB/s shown above compare with your previous
benchmark result? I'm sure it was quite higher than this.

I seem to recall it being said that the old CPU in the nas box lacks the
aes instruction set. This is a list of the CPU flags from the nas box.
nas ~ # cpuid2cpuflags
CPU_FLAGS_X86: 3dnow 3dnowext mmx mmxext popcnt sse sse2 sse3 sse4a
nas ~ #
The aes shows up on fireball. It does not on the nas box. Is the speed
above as good as I can expect with this older CPU? I can include the
kernel config if needed. If you know what driver you are looking for,
let me know what to grep for. If not sure, I can attach the config file.
Is this it? Is this as fast as this old CPU can get?

More or less. You could try serpent-xts, or twofish-xts to improve
performance, but since you're about to change your main PC you can retire the
Phenom and use fireball in its place.

[Frank Steinmetzger]

Second problem.  The transfer speed is back to the old slower speed. 
I'm pretty sure I am using the same old options on both ends.  Still,
<<< SNIP >>>
Did I miss something?  Typo maybe?  I'm pretty sure I used copy and
paste but still. 
Thanks.
Dale
:-)  :-)

I been working on the speed problem again.  I rebuilt the kernel on
fireball and I think some changes made a huge change.  This is the
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       931239 iterations per second for 256-bit key
PBKDF2-sha256    1356501 iterations per second for 256-bit key
PBKDF2-sha512     972705 iterations per second for 256-bit key
PBKDF2-ripemd160  648871 iterations per second for 256-bit key
PBKDF2-whirlpool  362077 iterations per second for 256-bit key
argon2i       5 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id      4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
#     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b       570.8 MiB/s      2045.6 MiB/s
    serpent-cbc        128b        91.1 MiB/s       310.0 MiB/s
    twofish-cbc        128b       198.7 MiB/s       218.9 MiB/s
        aes-cbc        256b       428.8 MiB/s      1670.4 MiB/s
    serpent-cbc        256b        91.6 MiB/s       309.5 MiB/s
    twofish-cbc        256b       199.8 MiB/s       219.2 MiB/s
        aes-xts        256b      1821.2 MiB/s      1767.1 MiB/s
    serpent-xts        256b       265.9 MiB/s       270.2 MiB/s
    twofish-xts        256b       201.0 MiB/s       204.2 MiB/s
        aes-xts        512b      1440.0 MiB/s      1445.9 MiB/s
    serpent-xts        512b       265.0 MiB/s       257.2 MiB/s
    twofish-xts        512b       198.2 MiB/s       201.6 MiB/s

There you go. Told ya. :)

As you can see, aes-cbc is fast now and I think that is what cryptsetup
uses.  It used to be really slow I think. 

Cryptsetup uses aes-xts these days, I think it’s been mentioned in this
thread somewhere.

Now on to the nas box.  I've recompiled the kernel with some added
options.  Still, it refuses to speed up.  I kinda think it is the CPU
lacking support for encryption.  I'm asking others just in case I'm
missing something.  Also, fireball uses a older kernel, 5.14 or so.  The
nas box uses 6.1 or so.  The menus are different and that is why it is
hard to get them to match up.  I may have missed something.

Everything you need for that is in the crypto menu at the bottom.

This is the bench mark from nas box. 
nas ~ # cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       700919 iterations per second for 256-bit key
PBKDF2-sha256     924670 iterations per second for 256-bit key
PBKDF2-sha512     729190 iterations per second for 256-bit key
PBKDF2-ripemd160  517559 iterations per second for 256-bit key
PBKDF2-whirlpool  359593 iterations per second for 256-bit key
argon2i       4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
argon2id      4 iterations, 1048576 memory, 4 parallel threads (CPUs)
for 256-bit key (requested 2000 ms time)
#     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b        63.6 MiB/s        41.6 MiB/s
    serpent-cbc        128b        81.0 MiB/s       212.4 MiB/s
    twofish-cbc        128b       192.5 MiB/s       222.1 MiB/s
        aes-cbc        256b        47.5 MiB/s        30.0 MiB/s
    serpent-cbc        256b        81.2 MiB/s       212.7 MiB/s
    twofish-cbc        256b       192.3 MiB/s       221.9 MiB/s
        aes-xts        256b        65.9 MiB/s        41.6 MiB/s
    serpent-xts        256b       201.7 MiB/s       205.7 MiB/s
    twofish-xts        256b       216.2 MiB/s       214.5 MiB/s
        aes-xts        512b        48.8 MiB/s        30.0 MiB/s
    serpent-xts        512b       202.7 MiB/s       205.6 MiB/s
    twofish-xts        512b       216.4 MiB/s       214.0 MiB/s
nas ~ #
[
]
The aes shows up on fireball.  It does not on the nas box.  Is the speed
above as good as I can expect with this older CPU?

If not done yet, you can check whether you enabled the 64 bit versions of
the crypto modules. They could push performance by a few more percent.