aptpkg.origin.uri is None with Elbe 15.7 (and maybe others)

[opensource-mmo]

We have run into a problem with Elbe 15.x (15.7 to be specific but the problem might be the same with earlier releases) where Elbe quits relatively early in dump_initvmpkgs.

We are trying to build an armhf rootfs on an x86_64 machine. This used to work just fine, as far as I can remember.

Here's the error we get (server names removed):

[CMD] /usr/sbin/chroot /var/cache/elbe/019aca71-6922-70c3-83db-b97e89e079bf/chroot dpkg --configure -a
[CMD] /usr/sbin/chroot /var/cache/elbe/019aca71-6922-70c3-83db-b97e89e079bf/chroot debconf-set-selections /var/cache/elbe/preseed.txt
[INFO]Keys imported
[CMD] reprepro --basedir /var/cache/elbe/019aca71-6922-70c3-83db-b97e89e079bf/repo export bookworm
Get:1 http://xxxxxxxx/xxxx stable InRelease [7582 B]
Hit http://xxxxxxxx/xxxx bookworm InRelease
Get:2 http://xxxxxxxx/xxxx stable InRelease [5946 B]
Get:3 http://xxxxxxxx/xxxx stable/main all Packages [9239 B]
Get:4 http://xxxxxxxx/xxxx stable/main armhf Packages [820 kB]
Get:5 http://xxxxxxxx/xxxx bookworm/main Sources [9494 kB]
Get:6 http://xxxxxxxx/xxxx bookworm/main Translation-en [6109 kB]
Get:7 http://xxxxxxxx/xxxx stable/main armhf Packages [16.3 kB]
[ERROR]Build failed
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/elbepack/asyncworker.py", line 164, in execute
    self.project.build(skip_pkglist=False,
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
                       build_bin=self.build_bin,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^
                       build_sources=self.build_src,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       skip_pbuild=self.skip_pbuilder)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/elbeproject.py", line 594, in build
    self.install_packages(self.buildenv)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/elbeproject.py", line 1013, in install_packages
    dump_initvmpkgs(self.xml)
    ~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/dump.py", line 60, in dump_initvmpkgs
    xml.append_initvm_pkg(p)
    ~~~~~~~~~~~~~~~~~~~~~^^^
  File "/usr/lib/python3/dist-packages/elbepack/elbexml.py", line 461, in append_initvm_pkg
    self.append_pkg(aptpkg, 'initvmpkgs')
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/elbexml.py", line 435, in append_pkg
    pak.et.set('uri', aptpkg.origin.uri)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "src/lxml/etree.pyx", line 851, in lxml.etree._Element.set
  File "src/lxml/apihelpers.pxi", line 595, in lxml.etree._setAttributeValue
  File "src/lxml/apihelpers.pxi", line 1528, in lxml.etree._utf8
TypeError: Argument must be bytes or unicode, got 'NoneType'

elbe control wait_busy Failed

The project will not be deleted from the initvm.
The files, that have been built, can be downloaded using:
elbe control get_files --output "/work/build/rootfs/elbe-build-20251128-123021" "/var/cache/elbe/019aca71-6922-70c3-83db-b97e89e079bf"

The project can then be removed using:
elbe control del_project "/var/cache/elbe/019aca71-6922-70c3-83db-b97e89e079bf" 
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/elbepack/main.py", line 43, in main
    cmdmod.run_command(cmd_argv)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/commands/initvm.py", line 51, in run_command
    args.func(args)
    ~~~~~~~~~^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/initvmaction.py", line 396, in _submit
    _submit_with_repodir_and_dl_result(initvm.control, xmlfile, cdrom, args)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/initvmaction.py", line 98, in _submit_with_repodir_and_dl_result
    _submit_and_dl_result(control, preprocess_xmlfile, cdrom, args)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/initvmaction.py", line 128, in _submit_and_dl_result
    raise with_cli_details(e, 133, textwrap.dedent(f"""
    ...<7 lines>...
        {prog} control del_project "{prjdir}" """))
  File "/usr/lib/python3/dist-packages/elbepack/initvmaction.py", line 125, in _submit_and_dl_result
    for msg in control.wait_busy(prjdir):
               ~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/usr/lib/python3/dist-packages/elbepack/soapclient.py", line 153, in wait_busy
    raise CliError(191, f'Project build was not successful, current status: {prj.status}')
elbepack.cli.CliError: Project build was not successful, current status: build_failed
Build failed!

The problem seems to be that Elbe updates the initvm's APT cache with armhf sources, and after that apt can no longer find the source URIs corresponding to packages installed in the initvm. Elbe wants to iterate over these, however, while the aptkg.origin.uri field for every single one of them is now "None". The first affected package is adduser.

Indeed, if we look at the adduser package (apt show adduser) in the initvm right after the build failure, we see:

Package: adduser
Version: 3.152
[...]
APT-Manual-Installed: yes
APT-Sources: /var/lib/dpkg/status

If we then run "apt update" in the initvm, apt show gives us a different output:

Package: adduser
Version: 3.152
[...]
APT-Manual-Installed: yes
APT-Sources: http://xxxxxxxx/xxxx trixie/main amd64 Packages

Similarly, apt list shows all packages as "[installed, local]" right after the build error but as "[installed]" or "[installed, automatic]" after re-running apt update.

As far as I can see, our elbe xml file looks innocent enough, so I really don't know what's wrong.

How can we solve this problem? Right now, we're unable to build a root file system.

Thanks and regards
Markus

[t-8ch]

Hi Markus,

as a workaround, can you upgrade the packages in the initvm?
For the real solution I am not sure yet.

[opensource-mmo]

The problem isn't upgrading the packages. The problem is that elbe first modifies the apt cache in a way that apt no longer finds the source uris for the installed packages but then in the very next step asks for those exact source uris, never expecting them to be 'None'. It doesn't matter what I do in the initvm, whenever I run elbe initvm submit myrootfs.xml (or even elbe initvm create myrootfs.xml) it shows the same error. I just started a build with 15.2, that seems to work with the same xml file.

[t-8ch]

The problem is that elbe first modifies the apt cache...

This should not be happening. Each target system gets its own apt cache.

Package: adduser
Version: 3.152

This seems to come from a trixie initvm, which is non-standard. Is this intentional?

So far I was not able to reproduce this specific issue. Can you reproduce it with a standard initvm and one of the XMLs from example/?

[t-8ch]

Can you apply this patch to the initvm?
These fields are mostly relevant for SBOM generation and we can validate there that they are present.
And of course the rootcause needs to be fixed.

diff --git a/elbepack/elbexml.py b/elbepack/elbexml.py
index ede48a48fdb1..41d667082d64 100644
--- a/elbepack/elbexml.py
+++ b/elbepack/elbexml.py
@@ -430,9 +430,12 @@ class ElbeXML:
         for k, v in hashes.items():
             pak.et.set(k, v)
 
-        pak.et.set('release-origin', aptpkg.origin.origin)
-        pak.et.set('release-name', aptpkg.origin.codename)
-        pak.et.set('uri', aptpkg.origin.uri)
+        if aptpkg.origin.origin:
+            pak.et.set('release-origin', aptpkg.origin.origin)
+        if aptpkg.origin.codename:
+            pak.et.set('release-name', aptpkg.origin.codename)
+        if aptpkg.origin.uri:
+            pak.et.set('uri', aptpkg.origin.uri)
 
         if aptpkg.source is not None:
             pak.et.set('source-name', aptpkg.source.name)

[opensource-mmo]

I'll try to reproduce the problem with one of the examples. I think I ran into the same problem with a bookworm initvm but I'm not sure. I wasn't aware that trixie initvms are non-standard, either, as the documentation explicitly mentions it:

suite <<suite-initvm,rfs:suite-initvm>> :: name of the debian suite that should be used to generate the initvm. E.g. “buster”, “bullseye”, “bookworm”, “trixie”, “sid”, …

[t-8ch]

They are supported. But not the default.
For the investigation I'd like to keep the moving parts to a minimum.

[tobiasjakobi-lr]

I think I remember this problem. AFAIR it happened to me when I updated my already existing initvm to trixie (from bookworm). The dist-upgrade resulted in several packages becoming local, i.e. they were still installed locally but didn't have any corresponding package in the trixie suite. I.e. leftovers from the original bookworm install.

Local packages don't have an URI, hence the exception. I fixed the problem on my system by just removing these packages (they didn't serve any purpose anyway).