[gentoo-user] Certain packages refuse to use binary from build save, the -k thing.

[Dale]

Howdy,

A couple of my video players are not playing videos correctly.  I've
rebuilt a few things but no change.  Some work fine, some give a error
about a bad index or something, some just don't even try at all.  The
videos come from various sources and are of different file extensions. 
I can't find a rhyme or reason to the failure.  So, my plan, do a emerge
-ek world to reinstall all packages instead of just the obvious ones
that isn't fixing the issue.  I figure that should catch whatever is out
of sync and fix the problem.  If not, then try to figure out why it is
breaking the hard way.  This is basically a emerge -e world, just faster
on my main system.

What I did before this.  I ran emerge -eak world on my main install. 
Any packages that showed ebuild, which means it doesn't have a package
binary, I emerged in my chroot.  Once it was done installing, I copied
the binary package over to my main install.  However, when I run emerge
-eak world again on my main install, it still shows ebuild for these
packages. 


[ebuild   R    ] dev-cpp/glibmm-2.66.6:2::gentoo  USE="-debug -gtk-doc
-test" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild   R    ] dev-cpp/pangomm-2.46.3:1.4::gentoo  USE="-gtk-doc"
ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild   R    ] app-crypt/gcr-3.41.1-r2:0/1::gentoo  USE="gtk
introspection vala -gtk-doc -systemd -test" 0 KiB
[ebuild   R    ] dev-qt/qtdeclarative-5.15.10-r2:5/5.15::gentoo 
USE="jit vulkan widgets -debug -gles2-only -localstorage -test" 0 KiB
[ebuild   R    ] sys-devel/llvm-15.0.7-r3:15::gentoo 
USE="binutils-plugin libffi ncurses xml -debug -doc -exegesis -libedit
-test -verify-sig -xar -z3 -zstd" ABI_X86="32 (64) (-x32)"
LLVM_TARGETS="(AArch64) (AMDGPU) (ARM) (AVR) (BPF) (Hexagon) (Lanai)
(MSP430) (Mips) (NVPTX) (PowerPC) (RISCV) (Sparc) (SystemZ) (VE)
(WebAssembly) (X86) (XCore) (-ARC) (-CSKY) (-DirectX) (-LoongArch)
(-M68k) (-SPIRV)" 0 KiB
[ebuild   R    ] sys-libs/compiler-rt-15.0.7:15.0.7::gentoo  USE="clang
-debug -test -verify-sig" ABI_X86="32 (64)" 0 KiB
[ebuild   R    ] sys-libs/compiler-rt-sanitizers-15.0.7:15.0.7::gentoo 
USE="asan cfi clang dfsan gwp-asan hwasan libfuzzer lsan memprof msan
orc profile safestack scudo tsan ubsan xray -debug (-shadowcallstack)
-test -verify-sig" ABI_X86="32 (64)" 0 KiB
[ebuild   R    ] sys-devel/clang-15.0.7-r3:15/15g1::gentoo  USE="extra
(pie) static-analyzer xml -debug -doc (-ieee-long-double) -test
-verify-sig" ABI_X86="(64) -32 (-x32)" LLVM_TARGETS="(AArch64) (AMDGPU)
(ARM) (AVR) (BPF) (Hexagon) (Lanai) (MSP430) (Mips) (NVPTX) (PowerPC)
(RISCV) (Sparc) (SystemZ) (VE) (WebAssembly) (X86) (XCore) (-ARC)
(-CSKY) (-DirectX) (-LoongArch) (-M68k) (-SPIRV)"
PYTHON_SINGLE_TARGET="python3_11 -python3_10" 0 KiB
[ebuild   R    ] sci-electronics/klayout-0.28.9::gentoo 
PYTHON_SINGLE_TARGET="python3_11 -python3_10 -python3_12"
RUBY_TARGETS="ruby31" 0 KiB



Most package are the way they should be, showing a binary.  Like this:


[binary   R    ] x11-libs/libX11-1.8.6::gentoo  USE="-doc -test"
ABI_X86="32 (64) (-x32)" 0 KiB


Does anyone know if there is some reason these packages won't use a
binary for some reason?  I've recompiled some of these twice now with it
refusing to show it using a binary.  I expect memtest and a couple
others to fail but never ran into this before.

I do this so that my main system updates faster.  A couple of those
packages has a bit longer compile time.  I might add, I've done this
countless times doing updates with no problems, most likely even with
these packages.  Today it just refuses to cooperate. 

Oh, I posted about handbrake and opencascade blocking some updates in
another thread a while back, I unmerged those and Kicad which I think
used opencascade.  It did update some video stuff the other day.  I feel
some package is out of sync with some other package and hope a reinstall
will fix it.  Otherwise, could be a bug in some package version. 

Any ideas? 

Dale

:-)  :-) 

P. S.  Fresh backups are still working on the new NAS system.  Also, the
speed is about the same as it was on TrueNAS.  It seems encryption on
both ends does slow things down.  I suspect it is the slower machine. 
Still, it works.  Even if I switch to another OS, it can still see LVM
drives and I won't have to redo them again.  I hope.  :-D  At 7.6TB of
about 26TBs at the moment. 

[Paul Colquhoun]

Post by Dale
Howdy,
A couple of my video players are not playing videos correctly. I've
rebuilt a few things but no change. Some work fine, some give a error
about a bad index or something, some just don't even try at all. The
videos come from various sources and are of different file extensions.
I can't find a rhyme or reason to the failure. So, my plan, do a emerge
-ek world to reinstall all packages instead of just the obvious ones
that isn't fixing the issue. I figure that should catch whatever is out
of sync and fix the problem. If not, then try to figure out why it is
breaking the hard way. This is basically a emerge -e world, just faster
on my main system.
What I did before this. I ran emerge -eak world on my main install.
Any packages that showed ebuild, which means it doesn't have a package
binary, I emerged in my chroot. Once it was done installing, I copied
the binary package over to my main install. However, when I run emerge
-eak world again on my main install, it still shows ebuild for these
packages.
[binary R ] x11-libs/libX11-1.8.6::gentoo USE="-doc -test"
ABI_X86="32 (64) (-x32)" 0 KiB
Does anyone know if there is some reason these packages won't use a
binary for some reason? I've recompiled some of these twice now with it
refusing to show it using a binary. I expect memtest and a couple
others to fail but never ran into this before.
I do this so that my main system updates faster. A couple of those
packages has a bit longer compile time. I might add, I've done this
countless times doing updates with no problems, most likely even with
these packages. Today it just refuses to cooperate.
Oh, I posted about handbrake and opencascade blocking some updates in
another thread a while back, I unmerged those and Kicad which I think
used opencascade. It did update some video stuff the other day. I feel
some package is out of sync with some other package and hope a reinstall
will fix it. Otherwise, could be a bug in some package version.
Any ideas?

Does your compile server have exactly the same USE flags (and other flags) set
as the target system?

[Dale]

Post by Paul Colquhoun

Post by Dale
Howdy,
A couple of my video players are not playing videos correctly. I've
rebuilt a few things but no change. Some work fine, some give a error
about a bad index or something, some just don't even try at all. The
videos come from various sources and are of different file extensions.
I can't find a rhyme or reason to the failure. So, my plan, do a emerge
-ek world to reinstall all packages instead of just the obvious ones
that isn't fixing the issue. I figure that should catch whatever is out
of sync and fix the problem. If not, then try to figure out why it is
breaking the hard way. This is basically a emerge -e world, just faster
on my main system.
What I did before this. I ran emerge -eak world on my main install.
Any packages that showed ebuild, which means it doesn't have a package
binary, I emerged in my chroot. Once it was done installing, I copied
the binary package over to my main install. However, when I run emerge
-eak world again on my main install, it still shows ebuild for these
packages.
[binary R ] x11-libs/libX11-1.8.6::gentoo USE="-doc -test"
ABI_X86="32 (64) (-x32)" 0 KiB
Does anyone know if there is some reason these packages won't use a
binary for some reason? I've recompiled some of these twice now with it
refusing to show it using a binary. I expect memtest and a couple
others to fail but never ran into this before.
I do this so that my main system updates faster. A couple of those
packages has a bit longer compile time. I might add, I've done this
countless times doing updates with no problems, most likely even with
these packages. Today it just refuses to cooperate.
Oh, I posted about handbrake and opencascade blocking some updates in
another thread a while back, I unmerged those and Kicad which I think
used opencascade. It did update some video stuff the other day. I feel
some package is out of sync with some other package and hope a reinstall
will fix it. Otherwise, could be a bug in some package version.
Any ideas?

Does your compile server have exactly the same USE flags (and other flags) set
as the target system?

Yes, exactly the same.  The way I do this, I basically do it as a stage
4 except I don't tar it up.  I found a script online somewhere that
copies over everything needed to do this, even boot the thing if
needed.  It copies /etc, /var, /usr and everything else except for
things like /proc, /home etc.  It just does the OS part itself.  So,
everything is identical before I start.  It's also on the same machine,
just a spare 750GB hard drive is all.  I started doing this due to the
longer compile times. 

I got to thinking, maybe it was a bug so I synced the tree and am doing
regular updates.  I didn't see anything directly related to video
players but I think the problem is deeper than the players themselves. 
One reason, several video players are having the same problem and on the
same videos.  If one video works, it works on all players.  If it fails,
it fails on all players.  Thing is, the same videos used to work a week
or so ago.  I suspect they depend on a codec package that has either got
a bug or the players can't load certain codecs due to a broken link or
something.  I was hoping a re-emerge would fix it, from binary builds of
course.

It's rare now but I used to do emerge -e world every few months when
there is a huge number of packages being updated.  It fixes things like
broken links and such really good.  Weird little bugs were gone
afterwards.  Thing is, emerge is so darn good, I don't do it often
anymore.  That preserved rebuild thing usually catches them. 

I don't recall emerge ever refusing to use binaries like this before,
not even for these packages.  Usually, memtest is the only one that
fails.  I don't mount /boot in the chroot. 

Open to ideas.  Running updates in chroot as I type.  Maybe that will
fix something. 

Dale

:-)  :-)

[Dale]

Post by Dale
Yes, exactly the same.  The way I do this, I basically do it as a stage
4 except I don't tar it up.  I found a script online somewhere that
copies over everything needed to do this, even boot the thing if
needed.  It copies /etc, /var, /usr and everything else except for
things like /proc, /home etc.  It just does the OS part itself.  So,
everything is identical before I start.  It's also on the same machine,
just a spare 750GB hard drive is all.  I started doing this due to the
longer compile times. 
I got to thinking, maybe it was a bug so I synced the tree and am doing
regular updates.  I didn't see anything directly related to video
players but I think the problem is deeper than the players themselves. 
One reason, several video players are having the same problem and on the
same videos.  If one video works, it works on all players.  If it fails,
it fails on all players.  Thing is, the same videos used to work a week
or so ago.  I suspect they depend on a codec package that has either got
a bug or the players can't load certain codecs due to a broken link or
something.  I was hoping a re-emerge would fix it, from binary builds of
course.
It's rare now but I used to do emerge -e world every few months when
there is a huge number of packages being updated.  It fixes things like
broken links and such really good.  Weird little bugs were gone
afterwards.  Thing is, emerge is so darn good, I don't do it often
anymore.  That preserved rebuild thing usually catches them. 
I don't recall emerge ever refusing to use binaries like this before,
not even for these packages.  Usually, memtest is the only one that
fails.  I don't mount /boot in the chroot. 
Open to ideas.  Running updates in chroot as I type.  Maybe that will
fix something. 
Dale
:-)  :-)

Update.  I finished my regular updates in my chroot.  I did the updates
on my main install and videos still not working right.  There is one
change tho.  I wanted to see if I could get emerge -ek world to show
only binary packages, this time it did.  I have no idea why it didn't
work before but it works now.  I'm not sure what changed tho. 

Total: *1837* packages (1837 reinstalls, *1837 binaries*), Size of
downloads: 0 KiB

Thanks to those who have been pondering this.  No idea the original
cause tho. 

Dale

:-)  :-)

[Frank Steinmetzger]

Post by Dale
Howdy,

Hi

instead of going berserk mode and wasting kWh on rebuilding “just in case it
might help”, why not try and dig a little deeper.

Post by Dale
A couple of my video players are not playing videos correctly.

Whicch players?

Post by Dale
I've
rebuilt a few things but no change.  Some work fine, some give a error
about a bad index or something, some just don't even try at all.

What is the exact error message?
Can you remux the file with ffmpeg?
`ffmpeg -i Inputfile.ext -c copy -map 0 Outputfile.ext`
This will take all streams of the input file and put them into a new file
without re-encoding. Does ffmpeg give any errors about bad data? Can you
play the produced file?

Post by Dale
The videos come from various sources and are of different file extensions. 

Can we have a look at some of those videos? The least I can try is to see
whether they work here or show any sign of corruption.

[Dale]

Post by Dale
Howdy,

Hi
instead of going berserk mode and wasting kWh on rebuilding “just in case it
might help”, why not try and dig a little deeper.

Post by Dale
A couple of my video players are not playing videos correctly.

Whicch players?

Post by Dale
I've
rebuilt a few things but no change.  Some work fine, some give a error
about a bad index or something, some just don't even try at all.

What is the exact error message?
Can you remux the file with ffmpeg?
`ffmpeg -i Inputfile.ext -c copy -map 0 Outputfile.ext`
This will take all streams of the input file and put them into a new file
without re-encoding. Does ffmpeg give any errors about bad data? Can you
play the produced file?

Post by Dale
The videos come from various sources and are of different file extensions. 

Can we have a look at some of those videos? The least I can try is to see
whether they work here or show any sign of corruption.

I think one of the videos had something bad that was causing other
issues somehow.  I ended up deleting any video that didn't have a
thumbnail and rebooting, to make sure all processes were killed.  This
started with I believe a bad file.  Then it affected other videos I
tried to watch.  Once I rebooted, everything was back to normal.  While
I think it was a bad video, I'm not sure it was that either.  It is odd
that rebooting fixed it tho. 

I only had to rebuild a few packages since I have it set to save
binaries of everything I build.  It's just that those few packages
wouldn't work for some reason.  I think it may have had something to do
with video file problem.  After the reboot, everything worked as
expected, including installing from binaries.  So, it's all good now. 

I was actually in the process of starting a fresh thread on the video
problem.  I was collecting info when I started seeing some odd things. 
I decided to reboot to see what that did.  I wasn't expecting it to fix
the whole problem, this isn't windoze after all, but it did.  It's been
running fine ever since. 

Now I just wish I knew exactly what the cause was.  Was it the video,
something else, something I'd never think of?  Lots of questions.  No
real way to answer them tho.  Hard to diagnose something that is
working.  lol

Thanks for offering to look into it tho.  I was working on it when the
reboot fixed it. 

Now to read and think over replies to thread about a new case.  Sort of
skimmed over it.  Sounds like ideas for better mobos, which may be a
good idea. 

Dale

:-)  :-)