2

First of all I am not sure that is about a bug or misunderstanding of a concept.

I am trying to make a custom ISO w/ autoinstall to automatically install Ubuntu Server 24.04.1 LTS onto our machines. We of course need to install some extra apt packages as part of the customization. The problem is, this installation needs to be done entirely offline (mostly with rj45 unplugged), so I can't just add them to the apt package list and have subiquity magic take care of everything.

Briefly:

  • Extract iso. I'll call the extracted dir 'iso';
  • Add the packages to iso/pool/universe/path/to/package.deb; added also deps (apt-get --print-uris --yes install "$package" | grep ^\' | cut -d\' -f2);
  • Then to add them to the Packages and Packages.gz file that lists all the available packages by calling;
  • Update the the Release file;
  • Create a gpg key and sign the Release file;
  • Add the public key (gpg) to trusted keys in ubuntu-server-minimal.ubuntu-server.squashfs.
  • re sum md5 (cd image && find . -type f -print0 | xargs -0 md5sum | grep -v "\./md5sum.txt" > md5sum.txt)

For testing purpose I downloaded traceroute. user-data looks like:

#cloud-config
autoinstall:
  version: 1
  source:
    id: ubuntu-server
  shutdown: poweroff
  packages:
    - traceroute
  identity:
    realname: 'UntouchedWagons'
    username: ubuntu
    password: 'secret'
    hostname: local-host

locale: en_US.UTF-8
  keyboard:
    layout: us
  timezone: "Europe/Athena"
  storage:
    layout:
      name: lvm
      sizing-policy: all
  ssh:
    allow-pw: true
    install-server: true
    authorized-keys:
      - some-keys

I've also tried variations and combinations of the above where I've also specified a late-commands: or network: section, detailed late. It "appears" to me that subiquity crash while downloading/install traceroute:

Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11210]: It is, so unshare will use --mount-proc=/target/proc
 Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11210]: Running command ['unshare', '--fork', '--pid', '--mount-proc=/target/proc', '--', 'chroot', '/target', 'apt-get', '--quiet', '--option=Acquire::Languages=none', 'update'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:27 ubuntu-server systemctl[11227]: Running in chroot, ignoring command 'start'
 Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11221]: Get:1 file:/cdrom noble InRelease
 Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11221]: Ign:1 file:/cdrom noble InRelease
 Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11221]: Get:2 file:/cdrom noble Release [1296 B]
 Sep 10 21:25:27 ubuntu-server subiquity_log.1716[11221]: Get:2 file:/cdrom noble Release [1296 B]
 Sep 10 21:25:27 ubuntu-server subiquity_event.1716[1716]:    subiquity/Install/install/postinstall/install_traceroute/cmd-system-install: curtin command system-install
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11221]: Reading package lists...

... exacly what I was hoping/needing

Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['unshare', '--fork', '--pid', '--mount-proc=/target/proc', '--', 'chroot', '/target', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', '--download-only', 'traceroute'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: Reading package lists...
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: Building dependency tree...
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: Reading state information...
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: Package traceroute is not available, but is referred to by another package.
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: This may mean that the package is missing, has been obsoleted, or
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: is only available from another source
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11333]: E: Package 'traceroute' has no installation candidate
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['udevadm', 'settle'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: TIMED subp(['udevadm', 'settle']): 0.011
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/usr/bin/ischroot'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/usr/bin/ischroot'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/sys/firmware/efi/efivars'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/sys/firmware/efi/efivars'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/sys'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/sys'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/run'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/run'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/proc'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/proc'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['mount', '--make-private', '/target/dev'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Running command ['umount', '/target/dev'] with allowed return codes [0] (capture=False)
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: system install failed for ['traceroute']: Unexpected error while running command.
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Command: ['unshare', '--fork', '--pid', '--mount-proc=/target/proc', '--', 'chroot', '/target', 'apt-get', '--quiet', '--assume-yes', '--option=Dpkg::options::=--force-unsafe-io', '--option=Dpkg::Options::=--force-confold', 'install', '--download-only', 'traceroute']
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Exit code: 100
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Reason: -
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Stdout: ''
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: Stderr: ''
 Sep 10 21:25:28 ubuntu-server subiquity_log.1716[11210]: finish: cmd-system-install: FAIL: curtin command system-install
 Sep 10 21:25:28 ubuntu-server curtin_event.1716.9[11210]: finish: cmd-system-install: FAIL: curtin command system-install

I searched for "has_network" in var/crash/1725992736.960986614.install_fail.crash and here are the results:

2024-09-10 18:22:47,079 DEBUG subiquitycore.models.network:469 has_network False

... also as I was expecting

When the installer crash's while still in the Live session, if I pop into the terminal and check the following:

apt-get install --download-only traceroute

  • works fine

apt-get install traceroute

  • works fine

I have used such a method to install our systems since ubuntu 12.04 with seed files and d-i pkgsel/include string worked like a charm. After a lot a research I found that somehow this is normal behavior:

cat /cdrom/etc/apt/sources.list

"deb cdrom:[Ubuntu-Server 24.04.1 LTS Noble Numbat - Release amd64 (20240827)]/ noble main restricted universe"

And correctly read components from /cdrom/dists/noble/Release -> Components: main restricted universe

But in chrooted env that install run:

cat /target/cdrom/etc/apt/sources.list

"deb [check-date=no] file:///cdrom noble main restricted"

the "universe" component is missing because apt.py:L245 says:

    write_file(
        self.install_tree.p('etc/apt/sources.list'),
        f'deb [check-date=no] file:///cdrom {codename} main restricted\n')

that looks like a fail-safe method *) what if I need also multiverse?

I have some workarounds for my issue like copying all my debs in a tmp /target directory and run dpkg -i *, also copying in /target/var/cache/apt/archives but not sure if it is enough for apt-get to install it, or even more go to Debian. Anyway, none of those looks for me an elegant one.

My question is that if am I doing something wrong because autoinstall/subiquity/curtin was designed for different purpose, or there is any solution?

user1929518
  • 21

1 Answers1

1

I had the exact same problem.

When I am using following guide Create an Ubuntu 24.04 Desktop Autoinstall ISO Image It works fine with vanilla ubuntu 24.04.1 Desktop images.

But when I am migrating from preseed framework from 20.04 it fails. After I analysed the vanilla one it has followings

/etc/apt/sources.list

"deb cdrom:[Ubuntu-Server 24.04.1 LTS Noble Numbat - Release amd64 (20240827)]/ noble main restricted universe"

But the migration from preseed gets lot of unnecessary entries which are failing.

So following solution works perfectly for me. It might help you or someone who is in this situation.

early-commands:
  - sudo echo "deb cdrom:[Ubuntu 24.04.1 LTS _Noble Numbat_ - Release amd64 (20240827.1)]/ noble main restricted" > /etc/apt/sources.list
apt:
  fallback: offline-install
  preserve_sources_list: true

Also found that whoever worked in the project before, changed the file permissions, that's why autoinstall failed for me additionally. 'apt' user was not getting access to /etc/apt dir. So, I prefer not to change file permissions. But found another solution in this link. In summery

    My xorriso command now looks like this:

xorriso -as mkisofs 

        -V "$(echo "${output_iso##*/}" | cut -c -32)" 

        -o "${output_iso}" 

        --grub2-mbr "${work_dir}/boot_hybrid.img" 

        --protective-msdos-label 

        -partition_offset 16 

        --mbr-force-bootable 

        -append_partition 2 28732ac11ff8d211ba4b00a0c93ec93b "${efi_partition_img}" 

        -appended_part_as_gpt 

        -iso_mbr_part_type a2a0d0ebe5b9334487c068b6b72699c7 

        -c "/boot.catalog" 

        -b "/boot/grub/i386-pc/eltorito.img" 

        -no-emul-boot 

        -boot-load-size 4 

        -boot-info-table 

        --grub2-boot-info 

        -eltorito-alt-boot 

        -e '--interval:appended_partition_2:::' 

        -no-emul-boot -J 

        -dir-mode 0755 

        "${work_dir}" ; then


The key line is -dir-mode 0755 - that fixed the problem entirely.
Mohammad Rezwan
  • 11