pkgsrc (3/6)

Posted May 8, 2022

By Rancune

8 min read

This post is an automatic translation from French. You can read the original version here.

Today, Imil starts with a little “evangelism” introduction about the world of open source, and the important role of packagers in this ecosystem. It’s beautiful, it’s pure poetry… And if you haven’t seen it, start by going to get the good word from the source because it’s well said and deserves to be heard. Go ahead, I’ll wait here. :)

Done? Great! Let’s begin with a little flashback.

Quick summary of previous episodes

This arc is centered around package creation, and the pkgsrc tool which is based on bmake.

To illustrate our point, we decided to create a package for the truefalse application, then at version 1.0. We then used “url2pkg”, a tool that lets you go directly from a URL to a package template.

As we mentioned above, in pkgsrc, bmake is at work. The centerpiece of our package is therefore a Makefile that we filled in. After this first version, version 1.1 of truefalse was released, requiring an update of this work.

The Makefile we had at that point was the following:

# $NetBSD$

VERS=           v1.1
DISTNAME=       truefalse-${VERS}
PKGNAME=        ${DISTNAME:S,-v,-,}
CATEGORIES=     misc
MASTER_SITES=   https://gitlab.com/iMil/truefalse/-/archive/${VERS}/

MAINTAINER=     truefalse@rancune.org
HOMEPAGE=       https://gitlab.com/iMil/truefalse/-/archive/${VERS}/
COMMENT=        First attempt at making a package
LICENSE=        original-bsd

INSTALLATION_DIRS=      bin

do-install:
	${INSTALL_PROGRAM} ${WRKSRC}/${PROGNAME} ${DESTDIR}/${PREFIX}/bin/${PROGNAME}

.include "../../mk/bsd.pkg.mk"

A few important reminders:

The base configuration of pkgsrc is located in the etc/mk.conf file. All variables recognized by bmake can be overridden in this file globally. We also added the following line, which gives us access to some additional information:

PKG_DEVELOPER=     yes

The do-install target, present in our Makefile, comes from the fact that the truefalse package’s Makefile does not have an installation target.

Speaking of installation, I cannot do this summary without reminding you of the wonderful target:

bmake stage-install

Which allows you to “simulate” a package installation in the “work” directory. This is a use of the ${DESTDIR}/${PREFIX} path.

We also saw, between versions V1.0 and V1.1, how to create a patch with the pkgvi and mkpatches commands.

truefalse 2.0 is out: Trouble ahead!

All of this brings us to a new version of truefalse, in which upstream has introduced changes that are far from pleasing to us. The first is a switch to gmake (GNU make) instead of bmake. The second is the introduction of a dependency on glib.h:

truefalse 2.0:

#include <stdlib.h>
#include <glib.h>

int
main(int argc, char *argv[])
{
	if (argc < 2) {
		g_assert_true(argc < 2);

		return EXIT_SUCCESS;
	}

	g_assert_false(argc < 2);

	return EXIT_FAILURE;
}

CFLAGS=  $(shell pkg-config --cflags glib-2.0)
LDFLAGS= $(shell pkg-config --libs glib-2.0)

truefalse: truefalse.c
	$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

Alright, no panic.

We start by changing the version number in our package’s Makefile, and we attempt to download the new version:

# $NetBSD$

VERS=           v2.0
DISTNAME=       truefalse-${VERS}
PKGNAME=        ${DISTNAME:S,-v,-,}
CATEGORIES=     misc
MASTER_SITES=   https://gitlab.com/iMil/truefalse/-/archive/${VERS}/

MAINTAINER=     truefalse@rancune.org
HOMEPAGE=       https://gitlab.com/iMil/truefalse/-/archive/${VERS}/
COMMENT=        First attempt at making a package
LICENSE=        original-bsd

INSTALLATION_DIRS=      bin

do-install:
	${INSTALL_PROGRAM} ${WRKSRC}/${PROGNAME} ${DESTDIR}/${PREFIX}/bin/${PROGNAME}

.include "../../mk/bsd.pkg.mk"

$ bmake distclean
===> Cleaning for truefalse-2.0
===> Dist cleaning for truefalse-2.0

$ bmake fetch
=> Bootstrap dependency digest>=20211023: found digest-20220214
=> Fetching truefalse-v2.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   571  100   571    0     0   1349      0 --:--:-- --:--:-- --:--:--  1636

Since the files have changed, we also need to update their hashes, stored in the distinfo file:

$ bmake makesum
=> Bootstrap dependency digest>=20211023: found digest-20220214

Let’s see what happens if we naively launch a compilation:

$ bmake
=> Bootstrap dependency digest>=20211023: found digest-20220214
===> Checking for vulnerabilities in truefalse-2.0
===> Patching for truefalse-2.0
===> Creating toolchain wrappers for truefalse-2.0
===> Configuring for truefalse-2.0
===> Building for truefalse-2.0
bmake: don't know how to make all. Stop

bmake: stopped in /home/rancune/pkgsrc/wip/truefalse/work/truefalse-v2.0
*** Error code 2

Stop.
bmake[1]: stopped in /home/rancune/pkgsrc/wip/truefalse
*** Error code 1

Stop.
bmake: stopped in /home/rancune/pkgsrc/wip/truefalse

Unsurprisingly, it fails. But the error message is interesting:

bmake: don't know how to make all. Stop

The failure is caused by upstream’s switch from bmake to gmake, the GNU version of Make. We handle this by modifying our Makefile so that gmake is used for this compilation through the USE_TOOLS variable:

USE_TOOLS=      gmake

We can also notice that there is no “all” target in the project’s Makefile. We will fix that with the line:

BUILD_TARGET=   truefalse

Our Makefile now looks like this:

# $NetBSD$

VERS=           v2.0
DISTNAME=       truefalse-${VERS}
PKGNAME=        ${DISTNAME:S,-v,-,}
CATEGORIES=     misc
MASTER_SITES=   https://gitlab.com/iMil/truefalse/-/archive/${VERS}/

MAINTAINER=     truefalse@rancune.org
HOMEPAGE=       https://gitlab.com/iMil/truefalse/-/archive/${VERS}/
COMMENT=        First attempt at making a package
LICENSE=        original-bsd

INSTALLATION_DIRS=      bin

USE_TOOLS=      gmake
BUILD_TARGET=   truefalse

do-install:
	${INSTALL_PROGRAM} ${WRKSRC}/${PROGNAME} ${DESTDIR}/${PREFIX}/bin/${PROGNAME}

.include "../../mk/bsd.pkg.mk"

$ bmake
=> Bootstrap dependency digest>=20211023: found digest-20220214
===> Checking for vulnerabilities in truefalse-2.0
===> Building for truefalse-2.0
cc  truefalse.c -o truefalse
In file included from /usr/include/bits/libc-header-start.h:33,
                 from /usr/include/stdlib.h:25,
                 from truefalse.c:1:
/usr/include/features.h:412:4: warning: #warning _FORTIFY_SOURCE requires compiling
with optimization (-O) [-Wcpp]
  412 | #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
      |    ^~~~~~~
truefalse.c:2:10: fatal error: glib.h: No such file or directory
    2 | #include <glib.h>
      |          ^~~~~~~~
compilation terminated.
*** Error code 1

Stop.
bmake: stopped in /home/rancune/pkgsrc/wip/truefalse/work/truefalse-v2.0
*** Error code 1

Stop.
bmake[1]: stopped in /home/rancune/pkgsrc/wip/truefalse
*** Error code 1

Stop.
bmake: stopped in /home/rancune/pkgsrc/wip/truefalse

A new error message: we’ve made progress!!!!! \o/

Of course, it still doesn’t work. But the error is different: it’s no longer a Makefile problem, but a C problem! The software now includes a dependency on glib.h that we need to integrate into our package.

All of this seems quite complicated, but unfortunately we are in the fairly standard process of porting software whose authors did not pay too much attention to other platforms. Fortunately, buildlink3 will help us.

In the glib2 package directory, there is a file named buildlink3.mk. If we want to compile and link our project with glib2, we just need to include this file in our Makefile with the following line:

.include "../../devel/glib2/buildlink3.mk"

We could do the same for any required dependency. It must be said that this drastically simplifies our life!

Our new version of the Makefile is therefore:

# $NetBSD$

VERS=           v2.0
DISTNAME=       truefalse-${VERS}
PKGNAME=        ${DISTNAME:S,-v,-,}
CATEGORIES=     misc
MASTER_SITES=   https://gitlab.com/iMil/truefalse/-/archive/${VERS}/

MAINTAINER=     truefalse@rancune.org
HOMEPAGE=       https://gitlab.com/iMil/truefalse/-/archive/${VERS}/
COMMENT=        First attempt at making a package
LICENSE=        original-bsd

INSTALLATION_DIRS=      bin

USE_TOOLS=      gmake
BUILD_TARGET=   truefalse

do-install:
	${INSTALL_PROGRAM} ${WRKSRC}/${PROGNAME} ${DESTDIR}/${PREFIX}/bin/${PROGNAME}

.include "../../devel/glib2/buildlink3.mk"
.include "../../mk/bsd.pkg.mk"

We can now recompile everything: it works!!!! \o/

$bmake clean
$bmake
$bmake deinstall
$bmake install

Moving to V2.1

New chapter, new version of truefalse. Here is version 2.1!

Few changes this time, except for the addition of an installation target in the Makefile:

PROGNAME= truefalse
BASE= /usr/local

CFLAGS=  $(shell pkg-config --cflags glib-2.0)
LDFLAGS= $(shell pkg-config --libs glib-2.0)

$(PROGNAME): $(PROGNAME).c
	$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

install: $(PROGNAME)
	install $(PROGNAME) $(BASE)/bin

clean:
	rm -f $(PROGNAME)

This is already a bit better than before! To look on the bright side, an install target has been added: we can remove ours. However, this still lacks portability.

The first problem is that the installation goes to /usr/local. We will need to replace this with ${DESTDIR}/${PREFIX}.

The second problem is that this Makefile assumes the “install” tool is both available and in the current PATH.

We could – once again – fix this with a patch and submit it upstream. We already did so successfully with version 1.0. This time, we will take a different approach: we will see how to solve the problem by making automated modifications to the project’s Makefile with pkgsrc.

We start by removing the now-unnecessary do-install target, and we use SUBST:

SUBST_CLASSES+= cleanup
SUBST_STAGE.cleanup= pre-configure
SUBST_FILES.cleanup= Makefile
SUBST_SED.cleanup+=  -e 's,/usr/local,${DESTDIR}/${PREFIX},g'
SUBST_SED.cleanup+=  -e 's,intall\ ,${INSTALL}\ ,g'

SUBST (Substitute for those in the know) is based on sed. You will, no doubt, have recognized the regular expressions performing two replacements:

“/usr/local” with “${DESTDIR}/${PREFIX}”
“install “ with “${INSTALL} “

( Note the space in “install “, which prevents matching the string “install:”! )

The SUBST tool can be called multiple times in our file, which is why we define classes. Here, our class will be called “cleanup” (it’s a name like any other, don’t judge!) and will take place just before the configure step (SUBST_STAGE). It will be applied to the Makefile (SUBST_FILES) and consists of two sed steps (SUBST_SED).

The whole idea of our approach in this chapter is to abstract our build as much as possible from the platform.

The final version of our package’s Makefile now looks like this:

# $NetBSD$

VERS=           v2.1
DISTNAME=       truefalse-${VERS}
PKGNAME=        ${DISTNAME:S,-v,-,}
CATEGORIES=     misc
MASTER_SITES=   https://gitlab.com/iMil/truefalse/-/archive/${VERS}/

MAINTAINER=     truefalse@rancune.org
HOMEPAGE=       https://gitlab.com/iMil/truefalse/-/archive/${VERS}/
COMMENT=        First attempt at making a package
LICENSE=        original-bsd

INSTALLATION_DIRS=      bin

USE_TOOLS=      gmake
BUILD_TARGET=   truefalse

SUBST_CLASSES+= 		cleanup
SUBST_STAGE.cleanup= 	pre-configure
SUBST_FILES.cleanup= 	Makefile
SUBST_SED.cleanup+=  	-e 's,/usr/local,${DESTDIR}/${PREFIX},g'
SUBST_SED.cleanup+=  	-e 's,intall\ ,${INSTALL}\ ,g'

.include "../../devel/glib2/buildlink3.mk"
.include "../../mk/bsd.pkg.mk"

We can verify that everything works as expected:

$bmake clean
$bmake distclean
$bmake ftech
$bmake makesum
$bmake
$sudo bmake deinstall
$sudo bmake install

The compilation goes smoothly and the local Makefile of truefalse is properly modified according to our needs:

$ cat work/truefalse-v2.1/Makefile
PROGNAME= truefalse
BASE= /home/rancune/pkgsrc/wip/truefalse/work/.destdir//home/rancune/pkg

CFLAGS=  $(shell pkg-config --cflags glib-2.0)
LDFLAGS= $(shell pkg-config --libs glib-2.0)

$(PROGNAME): $(PROGNAME).c
        $(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

install: $(PROGNAME)
        /usr/bin/install $(PROGNAME) $(BASE)/bin

clean:
        rm -f $(PROGNAME)

This chapter concludes our adventures with truefalse. Next week, Imil suggests we discover how to package a simple Python application: I don’t know about you, but I can’t wait! I’m liking pkgsrc more and more!

See you soon,

Rancune.

sysadmin, tools

pkgsrc imil

This post is licensed under CC BY 4.0 by the author.

Quick summary of previous episodes

truefalse 2.0 is out: Trouble ahead!

Moving to V2.1

Trending Tags