Moving poudriere from the host into a jail

This post is all about moving poudriere from the host into a jail, but you could probably use it for creating a new jail and running poudriere in it.

This also assumes use of iocage, but use whatever jail tools you want.

If you encounter problems with this, please post it in the comments and I’ll see how I can amend this post.

In this post:

  1. FreeBSD 12.0
  2. ZFS
  3. poudriere
  4. iocage

I will assume you are complete familiar with the above topics.

I am basing my work on Poudriere in a jail. ezjail is used in those instructions.

NOTE: Those instructions let the jail inherit all the IP4 and IPv6 network details. I don’t want that. It wasn’t clear that I needed to use vnet, but that one bit did allow me to progress.

Create the jail

I prefer thick jails, and recommend their use for anything which will persist more than a few days.

EDIT: 2021-09-27 – I have since moved away from iocage and use /etc/jail.conf. The /etc/jail.conf entry for my poudriere jail can be found at the bottom of this post as a comment.

[dan@r710-01:~] $ sudo iocage create -r 12.0-RELEASE --thickjail --name pkg01  
pkg01 successfully created!

This is where I usually set the proper hostname:

$ sudo iocage set host_hostname=pkg01.int.unixathome.org pkg01
host_hostname: pkg01 -> pkg01.int.unixathome.org

Create the poudriere dataset

Assuming you are configuring poudriere for the first time, you’ll need a dataset. Creating it is this this:

sudo zfs create tank_fast/poudriere

Associating the poudriere dataset to the jail

This dataset will be mounted in a jail. Therefore it needs this setting:

sudo zfs set jailed=on tank_fast/poudriere

It also need to be applied to all descendant datasets. There is no -r option on zfs set. I wrote this script:

$ cat ~/bin/zfs-set-jailed-on-poudriere
#!/bin/sh

FILESYSTEMS=`zfs list -Hr -o name tank_fast/poudriere`
for fs in $FILESYSTEMS
do
  sudo zfs set jailed=on $fs
done

Running that script set it all up. If you don’t do it to all the descendants, you’ll get something like this:

cannot mount 'tank_fast/poudriere/cache': permission denied

Setting jailed=on also does a umount of the zfs dataset, so don’t be surprised when they are no longer mounted.

This statement makes sure we can manipulate the poudriere dataset from within the jail.

PLEASE NOTE: You really should delay this step until after you have the jail up and running and all packages are installed. Keep the old poudriere instance running an available first.

[dan@r710-01:~] $ sudo iocage set jail_zfs_dataset=poudriere pkg01
jail_zfs_dataset: iocage/jails/pkg01/data -> poudriere

PLEASE NOTE: this is not the full name of the dataset. As mentioned in man iocage: “Takes the ZFS filesystem name without pool name”.

I have no idea why a just-created jail has that particular original value. There is no ZFS dataset by that name but that seems to be the default value according to the man page: iocage/jails/UUID/root/data.

Prepare the jail for poudriere usage

Here are all the settings in one command:

$ sudo iocage set children_max=100 \
            allow_mount=true \
            allow_mount_tmpfs=true \
            allow_mount_devfs=true \
            allow_mount_procfs=true \
            mount_linprocfs=true \
            allow_mount_zfs=true \
            allow_mount_nullfs=true \
            allow_raw_sockets=true \
            allow_socket_af=true \
            allow_chflags=true \
            enforce_statfs=1 \
            jail_zfs=1 \
            vnet=on \
            exec_poststart="jail -m allow.mount.linprocfs=1 name=ioc-pkg01" \
            pkg01
children_max: 0 -> 100
allow_mount: 0 -> 1
allow_mount_tmpfs: 0 -> 1
allow_mount_devfs: 0 -> 1
allow_mount_procfs: 0 -> 1
mount_linprocfs: 0 -> 1
allow_mount_zfs: 0 -> 1
allow_mount_nullfs: 0 -> 1
allow_raw_sockets: 0 -> 1
allow_socket_af: 0 -> 1
allow_chflags: 0 -> 1
enforce_statfs: 2 -> 1
jail_zfs: 0 -> 1
vnet: 0 -> 1
exec_poststart: /usr/bin/true -> jail -m allow.mount.linprocfs=1 name=ioc-pkg01

NOTE: in the original version of this post, I set allow_sysvipc=1. That property is deprecated. Instead, use sysvmsg, sysvsem, and sysvshm. The documentation says they default to disable. The code says they default to new. I raised an issue.

I will explain the exec_poststart later.

Configuring the vnet

NOTE: if you are running multiple vnets, you may not want vnet0. In which case, you should know what you do here.

If this is your first vnet, it will be venet0 and you can proceed.

The official documentation helped but it was my first iocage vnet experience getting a dhcpd jail running again which helped the most.

I deviated from the references notes and did not set this:

ip6=inherit
ip4=inherit

Instead, I set this:

$ sudo iocage set ip4_addr="vnet0|10.55.0.29/24" pkg01
ip4_addr: none -> 10.55.0.23

$ sudo iocage set ip6_addr="vnet0|[redacted]:23/64" pkg01
ip6_addr: none -> [redacted]:23

$ sudo iocage set defaultrouter=10.55.0.1 pkg01
defaultrouter: auto -> 10.55.0.1

The netmasks are important here. Without that, you will get errors; sorry, I did not record them.

These are the /etc/rc.conf settings I added:

cloned_interfaces="bridge0"
ifconfig_bridge0="addm ix0 up"

Where ix0 is your main network card in this server.

Multiple vnets?

If you are running more than one vnet on your host, you might need to do something like this:

$ sudo iocage set interfaces=vnet1:bridge0 pkg01
interfaces: vnet0:bridge0 -> vnet1:bridge0

As you can see, the default value for interfaces assumes you are using vnet0, but if you are using multiple vnets, you might need to change vnet0 and / or bridge0.

When I was using multiple vnets and I did not make this change, I encountered this error:

$ sudo iocage start pkg01
* Starting pkg01
  + Started OK
  + Using devfs_ruleset: 11
  + Configuring VNET FAILED
  route: writing to routing socket: Network is unreachable
add net default: gateway 10.55.0.1 fib 0: Network is unreachable

Stopped pkg01 due to VNET failure

Kernel modules

This is the first time I’ve encountered the kld_list setting for /etc/rc.conf. I did not select everything listed in the how-to, because I think not all of them are required for poudriere.

I issued this command to update /etc/rc.conf:

$ sudo sysrc kld_list="tmpfs aesni procfs" 
kld_list:  -> tmpfs aesni procfs

I manually loaded those modules on the host:

$ sudo kldload tmpfs
kldload: can't load tmpfs: module already loaded or in kernel
$ sudo kldload aesni
$ sudo kldload procfs
kldload: can't load procfs: module already loaded or in kernel
$ 

Oh, so maybe I don’t need them all in my configuration settings.

I now think iocage loaded tmpfs for me and that I don’t need procfs as it’s in the kernel.

I changed my setting to:

$ sudo sysrc kld_list="tmpfs aesni" 
kld_list: tmpfs aesni procfs -> tmpfs aesni

Time to start the jail

It is time to start the jail:

[dan@r710-01:~] $ sudo iocage do iocage start pkg01
* Starting pkg01
  + Started OK
  + Using devfs_ruleset: 12
  + Configuring VNET OK
  + Using IP options: vnet
  + Starting services OK
  + Executing poststart OK

Next, I’ll go onto the console and start installing what I need.

Setting up the jail

I want to set up this jail using ansible. By that, I mean install the things a poudriere jail needs. Things like enabled ssh, installing all the packges the jail needs, etc. I will not cover that here.

At this point, nothing is installed.

I got started with this:

[dan@r710-01:~] $ sudo iocage console pkg01
FreeBSD 12.0-RELEASE-p10 GENERIC 

Welcome to FreeBSD!

Release Notes, Errata: https://www.FreeBSD.org/releases/
Security Advisories:   https://www.FreeBSD.org/security/
FreeBSD Handbook:      https://www.FreeBSD.org/handbook/
FreeBSD FAQ:           https://www.FreeBSD.org/faq/
Questions List: https://lists.FreeBSD.org/mailman/listinfo/freebsd-questions/
FreeBSD Forums:        https://forums.FreeBSD.org/

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@pkg01:~ # 

From here, you can do everything you need to get the host online, including installing packages.

Here is the list of top level packages I installed:

[dan@pkg01:~] $ pkg query -e '%#r = 0' %o
sysutils/anvil
sysutils/beadm
security/logcheck
sysutils/ncdu
www/nginx
net-mgmt/nrpe3
security/pam_ssh_agent_auth
ports-mgmt/pkg
mail/postfix
ports-mgmt/poudriere
lang/python
security/sudo
sysutils/tmux
misc/xtail

Once everything is installed, you can start copying configuration and then assigning poudriere datasets to the jail.

Why copy configuration after installing? The installation process will create the directories into which you will copy the configuration.

Copying the configuration

I agree. All these things should be a in repo somewhere. They aren’t. Yet.

I copied these items from the old poudriere to the new poudriere:

First, poduriere.conf:

$ cd /usr/local/etc
$ sudo cp poudriere.conf /iocage/jails/pkg01/root/usr/local/etc/

Then, the poudriere.d directory:

$ sudo cp -r poudriere.d/ /iocage/jails/pkg01/root/usr/local/etc/poudriere.d/

Don’t forget the repo signing key or you’ll get these errors:

[00:00:04] Error: PKG_REPO_SIGNING_KEY defined but the file is missing.

Where is that key? Checking on the existing poudriere system:

$ grep PKG_REPO_SIGNING_KEY /usr/local/etc/poudriere.conf
PKG_REPO_SIGNING_KEY=/usr/local/etc/ssl/pkg.key

$ ls -l /usr/local/etc/ssl/pkg.* 
-rw-------  1 root  wheel   800 Nov 26  2017 /usr/local/etc/ssl/pkg.cert
-rw-------  1 root  wheel  3243 Nov 26  2017 /usr/local/etc/ssl/pkg.key

$ sudo cp -ia /usr/local/etc/ssl/pkg.* /iocage/jails/pkg01/root//usr/local/etc/ssl/

$ ls -l /iocage/jails/pkg01/root//usr/local/etc/ssl/pkg.*
-rw-------  1 root  wheel   800 Nov 26  2017 /iocage/jails/pkg01/root//usr/local/etc/ssl/pkg.cert
-rw-------  1 root  wheel  3243 Nov 26  2017 /iocage/jails/pkg01/root//usr/local/etc/ssl/pkg.key

Don’t forget your website configuration

poudriere has a fantastic web interface. It is also the mechanism used to deliver packages over the network. I use nginx, so I copied the configuration file over:

[dan@r710-01:~] $ sudo cp -i /usr/local/etc/nginx/nginx.conf /iocage/jails/pkg01/root/usr/local/etc/nginx/
overwrite /iocage/jails/pkg01/root/usr/local/etc/nginx/nginx.conf? (y/n [n]) y
[dan@r710-01:~] $ 

From within the jail, you can enable nginx with this command:

[dan@pkg01:~] $ sudo sysrc nginx_enable="YES"
nginx_enable:  -> YES

You might also want to adjust nginx.conf in the new jail to be appropriate for that jail (IP addresess/hostname, etc.

Ready for the poudriere datasets now

In this section, we will associate the poudriere dataset with this new jail.

First step, stop the jail:

$ sudo iocage stop pkg01
* Stopping pkg01
  + Executing prestop OK
  + Stopping services OK
  + Tearing down VNET OK
  + Removing devfs_ruleset: 12 OK
  + Removing jail process OK
  + Executing poststop OK

If you don’t stop the jail, you’ll get into a catch-22 situation where iocage tries to zfs umount stuff which is not mounted, it fails, crashes, and you’re stuck fixing it up manually.

This command associates the poudriere datasets with your jail.

[dan@r710-01:~] $ sudo iocage set jail_zfs_dataset=poudriere pkg01
jail_zfs_dataset: iocage/jails/pkg01/data -> poudriere

Start the jail with the poudriere datasets

[dan@r710-01:~] $ sudo iocage start pkg01
* Starting pkg01
  + Started OK
  + Using devfs_ruleset: 9
  + Configuring VNET OK
  + Using IP options: vnet
  + Starting services OK
  + Executing poststart OK
[dan@r710-01:~] $ 

Now I will ssh in, not console, because I set up sshd during my configuration described, but not detailed, in a previous section.

Once in there, I see:

[dan@pkg01:~] $ zfs list
NAME                                     USED  AVAIL  REFER  MOUNTPOINT
tank_fast                                442G   457G    88K  /tank_fast
tank_fast/poudriere                     52.8G   457G    88K  /usr/local/poudriere
tank_fast/poudriere/cache               4.94G   457G    88K  /var/cache.me.not
tank_fast/poudriere/cache/ccache        1.74G   457G  1.63G  /var/cache.me.not/ccache
tank_fast/poudriere/cache/ccache.amd64  2.44G   457G  2.23G  /var/cache.me.not/ccache.amd64
tank_fast/poudriere/cache/ccache.i386    778M   457G   730M  /var/cache.me.not/ccache.i386
tank_fast/poudriere/data                23.9G   457G  3.76G  /usr/local/poudriere/data
tank_fast/poudriere/data/cache           113M   457G   105M  /usr/local/poudriere/data/cache
tank_fast/poudriere/data/cronjob-logs   1.37M   457G  1.21M  /usr/local/poudriere/data/cronjob-logs
tank_fast/poudriere/data/packages       20.0G   457G  9.92G  /usr/local/poudriere/data/packages
tank_fast/poudriere/distfiles           6.22G   457G  6.21G  /usr/ports/distfiles
tank_fast/poudriere/jails               2.54G   457G    88K  /usr/local/poudriere/jails
tank_fast/poudriere/jails/120amd64      1.38G   457G  1.38G  /usr/local/poudriere/jails/120amd64
tank_fast/poudriere/jails/120i386       1.16G   457G  1.16G  /usr/local/poudriere/jails/120i386
tank_fast/poudriere/ports               15.2G   457G    88K  /usr/local/poudriere/ports
tank_fast/poudriere/ports/clean          865M   457G   865M  /usr/local/poudriere/ports/clean
tank_fast/poudriere/ports/default       4.39G   457G  1.57G  /usr/local/poudriere/ports/default
tank_fast/poudriere/ports/main          1.18G   457G  1.18G  /usr/local/poudriere/ports/main
tank_fast/poudriere/ports/testing       8.79G   457G  5.12G  /usr/local/poudriere/ports/testing
[dan@pkg01:~] $ 

I see tank_fast listed. I’d prefer not to see that, but from my testing, you can’t do anything with it:

[dan@pkg01:~] $ sudo zfs create tank_fast/dan
cannot create 'tank_fast/dan': permission denied

That looks good.

After that, I was able to get a list of ports trees in the new jail:

[dan@pkg01:~] $ poudriere ports -l
PORTSTREE METHOD   TIMESTAMP           PATH
2019Q3    svn      2018-02-11 04:18:50 /usr/home/dan/ports/2019Q3
clean     portsnap 2017-08-18 23:52:53 /usr/local/poudriere/ports/clean
default   svn      2019-10-23 04:18:00 /usr/local/poudriere/ports/default
testing   svn      2019-10-23 04:18:00 /usr/local/poudriere/ports/testing
working   svn      2018-02-11 04:18:50 /usr/home/dan/ports/head
[dan@pkg01:~] $ 

Changes to poudriere.conf

I made this change to /usr/local/etc/poudriere.conf in the new jail:

USE_TMPFS="all"

Otherwise, I would see this error:

[dan@pkg01:~] $ sudo poudriere bulk -p default -j 120amd64 sysutils/anvil
[00:00:00] Error: kern.securelevel >= 1. Poudriere requires no securelevel to be able to handle schg flags. USE_TMPFS=all can override this.

Try it out!

That’s it, you should be done. Try it.

You can skip down the page to the last section now. What follows is mostly for debugging.

No loopback address

PLEASE NOTE: You can ignore this section. It is recorded only because it was an interesting situation. This was before I started using vnet

[dan@pkg01:/usr/local/etc/poudriere.d] $ poudriere jail -l
[00:00:00] Warning: No loopback address defined, consider setting LOIP6/LOIP4 or assigning a loopback address to the jail.
JAILNAME VERSION          ARCH  METHOD TIMESTAMP           PATH
120amd64 12.0-RELEASE-p10 amd64 ftp    2019-08-27 00:03:28 /usr/local/poudriere/jails/120amd64
120i386  12.0-RELEASE-p10 i386  ftp    2019-08-27 00:04:03 /usr/local/poudriere/jails/120i386
[dan@pkg01:/usr/local/etc/poudriere.d] $ 

I added 127.0.0.1, what could possibly go wrong? I stopped the jail, and assigned in the IP address:

$ sudo iocage set ip4_addr="127.0.0.1,ix0|10.55.0.29" pkg01
ip4_addr: ix0|10.55.0.29 -> 127.0.0.1,ix0|10.55.0.29

I started the jail, and tried a different command:

[dan@pkg01:~] $ sudo poudriere ports -u -p default
[00:00:00] Updating portstree "default" with svn... done
[dan@pkg01:~] $ 

Success!

Missing distfiles!

I ran across this error before I tried vnet, and while it is not vnet related, it is something I had to fix.

This might not apply to you.

Let’s try a build!

[dan@pkg01:~] $ sudo poudriere bulk -p default -j 120amd64 -z master-list -f /usr/local/etc/poudriere.d/buildlists/master-list
[00:00:00] Error: DISTFILES_CACHE directory does not exist. (cf.  poudriere.conf)

Eh?

[dan@pkg01:~] $ grep DISTFILES_CACHE /usr/local/etc/poudriere.conf
DISTFILES_CACHE=/usr/ports/distfiles
[dan@pkg01:~] $ ls -l /usr/ports/distfiles
ls: /usr/ports/distfiles: No such file or directory
[dan@pkg01:~] $ 

Ahh!

I found it:

[dan@r710-01:~] $ zfs list tank_fast/distfiles
NAME                  USED  AVAIL  REFER  MOUNTPOINT
tank_fast/distfiles  6.16G   461G  6.15G  /usr/ports/distfiles

Let’s rename it so it gets included into the jail because it is then in the correct fileset.

But first, we have to stop the jail, or we’ll hit this error:

RuntimeError: cannot unmount 'tank_fast/poudriere/distfiles': not currently mounted

Then you have to manually kill the jail:

sudo jail -r SomeNumberYouGetFrom_jls

And manually umount all the poudriere stuff which didn’t get umounted.

AFTER you have stopped the jail, rename the filesystem:

[dan@r710-01:~] $ sudo zfs rename tank_fast/distfiles tank_fast/poudriere/distfiles

When the jail is started, poudriere will find the distfiles.

[dan@pkg01:~] $ sudo poudriere bulk -p default -j 120amd64 -z master-list -f /usr/local/etc/poudriere.d/buildlists/master-list
[00:00:00] Error: kern.securelevel >= 1. Poudriere requires no securelevel to be able to handle schg flags. USE_TMPFS=all can override this.

The above is something I already wrote about in the previous section. That part is now fixed.

Installing packages without a package server

This section can also be skipped unless you need to get packages install and your new poudriere jail isn’t up and running yet.

In case you need to install packages into your new package server while the old package server is offline, here is how I did it.

Let’s try USE_TMPFS.

[dan@pkg01:/usr/local/etc] $ sudo joe poudriere.conf
sudo: joe: command not found

Oh, let’s install that.

$ sudo pkg install joe
Updating local repository catalogue...
pkg: http://fedex.int.unixathome.org/packages/120amd64-default-master-list//meta.txz: Not Found
repository local has no meta file, using default settings
pkg: http://fedex.int.unixathome.org/packages/120amd64-default-master-list//packagesite.txz: Not Found
Unable to update repository local
Error updating repositories!

What?

Oh, yes, fedex.int.unixathome.org can’t serve those files because they are mounted in the jail and the web server is on the host. Time to move the web server into the jail.

But how, if the package server is offline?

Via scp.

On the host where nginx is installed, visit /var/cache/pkg and copy over the required packages.

[dan@r710-01:/var/cache/pkg] $ cp nginx-1.16.1_4,2.txz pcre-8.43_2.txz /iocage/jails/pkg01/root/tmp 

Then in the jail:

[dan@pkg01:/tmp] $ sudo pkg add -f ./nginx-1.16.1_4,2.txz 

I’ll skip over the website configuration.

I installed my editor, modified /usr/local/etc/poudriere.conf to set USE_TMPFS=”all” and tried again.

linprocfs issues

You can skip this section unless you have encountering issues with linprocfs.

I tried again:

$ sudo poudriere bulk -p default -j 120amd64 -z master-list -f /usr/local/etc/poudriere.d/buildlists/master-list
[00:00:00] Creating the reference jail... done
[00:00:03] Mounting system devices for 120amd64-default-master-list
mount: linprocfs: Operation not permitted
[00:00:03] Cleaning up
[00:00:03] Unmounting file systems

Checking the jail from the host I found:

[dan@r710-01:~] $ iocage get all pkg01 | grep procfs
allow_mount_procfs:1
mount_linprocfs:0
mount_procfs:0

Let’s set that, first stopping the jail, setting the fail, and starting the jail:

[dan@r710-01:~] $ sudo iocage set mount_linprocfs=1 pkg01
mount_linprocfs: 0 -> 1

That isn’t enough. The permissions granted to a jail can be viewed with this command issued on the host:

$ jls -n -j 9
devfs_ruleset=9 nodying enforce_statfs=1 host=new ip4=inherit ip6=inherit jid=9 linux=new name=ioc-pkg01 osreldate=1200086 
osrelease=12.0-RELEASE-p10 parent=0 path=/iocage/jails/pkg01/root persist securelevel=2 sysvmsg=new sysvsem=new sysvshm=new 
vnet=new allow.chflags allow.nomlock allow.mount allow.mount.devfs allow.mount.nofdescfs allow.mount.nolinprocfs 
allow.mount.nullfs allow.mount.procfs allow.mount.tmpfs allow.mount.zfs allow.noquotas allow.raw_sockets allow.noread_msgbuf 
allow.reserved_ports allow.set_hostname allow.socket_af allow.nosysvipc children.cur=0 children.max=100 cpuset.id=7 
host.domainname=none host.hostid=0 host.hostname=pkg01.int.unixathome.org host.hostuuid=pkg01 ip4.addr= ip4.saddrsel 
ip6.addr= ip6.saddrsel linux.osname=Linux linux.osrelease=2.6.32 linux.oss_version=198144

…. where 9 is the jail id of that jail.

In the original version of this post, allow.nosysvipc was not present; instead, allow.sysvipc was seen. This was before I changed the configuration. See Prepare the jail for poudriere usage for more information.

Look at the end of line 4: allow.mount.nolinprocfs

That is a negative form of the permission, as the Great Mr Lucas pointed out to me. linprocfs is disabled.

I could not find a way to enable this via an iocage jail configuration item. Instead, I took this approach:

$ sudo iocage set exec_poststart="jail -m allow.mount.linprocfs=1 name=ioc-pkg01" pkg01
exec_poststart: /usr/bin/true -> jail -m allow.mount.linprocfs=1 name=ioc-pkg01

This tells iocage to issue a command after the jail has started. That command modifies the jail and sets allow.mount.linprocfs=1

You will notice that the formal name of the jail must be specified in that command, specifically ioc-pkg01.

I once knew how to find the formal name, but it escapes me now.

Success!

Now it builds!

[dan@pkg01:~] $ sudo poudriere bulk -p default -j 120amd64 -C sysutils/anvil
[00:00:00] Creating the reference jail... done
[00:00:02] Mounting system devices for 120amd64-default
[00:00:02] Mounting ports/packages/distfiles
[00:00:02] Stashing existing package repository
[00:00:02] Mounting packages from: /usr/local/poudriere/data/packages/120amd64-default
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf
[00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/default-make.conf
/etc/resolv.conf -> /usr/local/poudriere/data/.m/120amd64-default/ref/etc/resolv.conf
[00:00:02] Starting jail 120amd64-default
[00:00:03] Logs: /usr/local/poudriere/data/logs/bulk/120amd64-default/2019-10-24_00h33m58s
[00:00:03] WWW: https://services.unixathome.org/poudriere/build.html?mastername=120amd64-default&build=2019-10-24_00h33m58s
[00:00:03] Loading MOVED for /usr/local/poudriere/data/.m/120amd64-default/ref/usr/ports
[00:00:04] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:04] Gathering ports metadata
[00:00:04] Calculating ports order and dependencies
[00:00:04] (-C) Cleaning specified packages to build
[00:00:04] (-C) Will delete existing package: anvil-0.0.17.txz
[00:00:04] (-C) Flushing package deletions
[00:00:04] Sanity checking the repository
[00:00:04] Checking packages for incremental rebuild needs
[00:00:04] Deleting stale symlinks... done
[00:00:04] Deleting empty directories... done
[00:00:05] Cleaning the build queue
[00:00:05] Sanity checking build queue
[00:00:05] Processing PRIORITY_BOOST
[00:00:05] Balancing pool
[00:00:05] Recording filesystem state for prepkg... done
[00:00:05] Building 1 packages using 1 builders
[00:00:05] Starting/Cloning builders
[00:00:05] Hit CTRL+t at any time to see build progress and stats
[00:00:05] [01] [00:00:00] Building sysutils/anvil | anvil-0.0.17
[00:00:07] [01] [00:00:02] Finished sysutils/anvil | anvil-0.0.17: Success
[00:00:07] Stopping 1 builders
[00:00:07] Creating pkg repository
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:00:08] Committing packages to repository: /usr/local/poudriere/data/packages/120amd64-default/.real_1571877246 via .latest symlink
[00:00:08] Removing old packages
[00:00:08] Built ports: sysutils/anvil
[120amd64-default] [2019-10-24_00h33m58s] [committing:] Queued: 1  Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 0   Time: 00:00:05
[00:00:08] Logs: /usr/local/poudriere/data/logs/bulk/120amd64-default/2019-10-24_00h33m58s
[00:00:08] WWW: https://services.unixathome.org/poudriere/build.html?mastername=120amd64-default&build=2019-10-24_00h33m58s
[00:00:08] Cleaning up
[00:00:08] Unmounting file systems
[dan@pkg01:~] $ 

poudriere in a jail!

Running poudriere in a jail has long been a goal, but I abandoned it some time ago. I am grateful to Thomas Maack Nielsen for asking why I wasn’t. That led me down a rabbit hole and I am happy with the results.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

1 thought on “Moving poudriere from the host into a jail”

  1. This is the jail.conf entry for the above jail after moving from iocage to plain jails:

    pkg01 {
        allow.chflags;
    
        allow.mount.devfs;
        allow.mount.fdescfs;
        allow.mount.linprocfs;
        allow.mount.nullfs;
        allow.mount.procfs;
        allow.mount.tmpfs;
        allow.mount.zfs=true;
        allow.mount=true;
    
        allow.raw_sockets;
        allow.socket_af;
    
        children.max=200;
    
        enforce_statfs=1;
    
        exec.created+="zfs jail $name    tank_fast/poudriere";
        exec.created+="zfs set jailed=on tank_fast/poudriere";
    
        exec.poststart  += "jail -m allow.mount.linprocfs=1 name=$name";
    
        exec.poststop   += "/usr/local/sbin/jib destroy $name";
    
        exec.prestart   += "/usr/local/sbin/jib addm    $name ix0";
    
        host.domainname=none;
    
        sysvmsg=new;
        sysvsem=new;
        sysvshm=new;
    
        vnet.interface   = "e0b_$name";
        vnet;
    }
    

Leave a Comment

Scroll to Top