I am trying to build a binary package on freebsd. All the guides which i am finding are saying basically to build binary package from already installed software using ports. What i am searching for a way to get that done without installing ports.
Is this possible?
My code is in golang. So source contains one binary compiled out of go code and configuration files.
The Documentation of the Packageformat can be found here: https://wiki.freebsd.org/pkgng#Package_format
You Don't have to install the software to create a Package. during make package in a port, it is installed into a staging directory and then tar'ed up.
The package format documentation (since pkgng has become pkg) is now at https://github.com/freebsd/pkg#pkgfmt.
A .pkg file is basically a tar archive, optionally compressed with one out of a few standard tools and a few extra constraints.
Firstly, the first files in the archive are metadata. Two files are specified, named +MANIFEST and +COMPACT_MANIFEST, without a path and without a leading /. These are followed by the files to be installed, each with its full destination path (starting with /).
+MANIFEST, as per the specs, is a file in UCL format, which is somewhat of a mix between YAML and JSON. Pure JSON will work (when installing a package with pkg add blah.pkg) and is what I found when examining a package from the FreeBSD repo.
Manifest values largely correspond to makefile variables as documented in https://docs.freebsd.org/en/books/porters-handbook. Specifically:
name is the name you have chosen for the package.
version is the version, following some conventions to determine which of two given versions is newer.
origin if the physical location of the package in the repo, of the form category/name, where name is identical to the value of name. Additional categories can be specified in categories.
comment is a one-line description of the package.
arch takes a form like freebsd:13:x86:64. Wildcard values such as freebsd:* have worked on install for me, for packages which do not depend on a particular architecture and/or OS version.
www and maintainer are is the web site URL and maintainer e-mail address for the project, respectively.
prefix is usually /usr/local but doesn’t seem to have any effect at install time, even if the package installs to /opt and /etc.
licenselogic: single if there is only one license, or if there is a choice between multiple ones.
licenses is a list of the licenses, referred to by handles such as GPLv3+, GPLv2, BSD. See Porter’s Handbook for details.
flatsize seems to be the combined size of all files installed. Not 100% sure, as the values were slightly off in the package I examined, but a package I built with this assumption installed just fine with pkg add.
users, groups seem to be users and groups to be created when installing the package (haven’t tried, so I cannot tell if these entries actually trigger creation of the users and groups mentioned there).
options: not sure, my package installed fine without.
desc is a longer (more than one line) description of the package.
categories is a list of additional categories the package should be listed in. The category from origin seems to be repeated here.
deps are dependencies, i.e. other packages that are needed for this package to work. They typically take the form:
name: {origin: category/name, version: 1.2.3} (requiring a minimum version) or justname: {origin: category/name} (for any version)directories seem to be directories created by the package. Entries have the form /usr/local/share/foo-1.0: y; not sure what the value does – whether or not the directory is to be removed upon uninstall? However, a package will install fine (and create directories as needed) without this entry.
files: Files with their SHA256 checksum. Not sure what happens when a file in the archive does not have an entry here – does it not get installed at all, or does it get installed but without SHA256 verification?
scripts are scripts to run before, during or after install or deinstall.
+COMPACT_MANIFEST is just +MANIFEST with some values omitted. The former is intended for listing the package in repositories, the latter is used for doing the actual installation. Installing a package with just a +MANIFEST but no +COMPACT_MANIFEST seems to work, though there may be side effects when trying to get a package into an official repo.
I have managed to cobble together a package that will install on FreeBSD with just a simple shell script. This has the nice side effect that I don’t even need BSD to build it, which comes in handy if there is no native code.
It is easiest to build a skeleton manifest in YAML, generate files and flatsize on the fly and then convert everything to JSON.
Sample shell script:
FLATSIZE=0
create_files_entry() {
# TODO if the file is a link, just insert '-'
sha256sum $1 | sed -E 's,([a-fA-F0-9]*) *(.*), \2: \1,' | sed $2
}
add_file() {
[ -d $1 ] && return
create_files_entry $1 $2 >> $(dirname $0)/cache/files.yaml
tar -Pr -f $(dirname $0)/cache/data.tar --transform=$2 $1
case `uname` in
Linux)
FLATSIZE=$(( FLATSIZE + $(stat -c %s $1) ))
;;
FreeBSD)
FLATSIZE=$(( FLATSIZE + $(stat -f %d $1) ))
;;
*)
echo "ERROR: unsupported build platform: `uname`"
exit 1
;;
esac
}
echo "files:" > cache/files.yaml
# the second parameter is a transformation to change the file path
add_file path/to/file 's,\./path/to/,/usr/local/foo/,'
# add more files in this manner as needed
echo "flatsize: $FLATSIZE" >> files.yaml
cat $manifest.yaml $cache/files.yaml | python3 -c 'import sys, yaml, json; print(json.dumps(yaml.safe_load(sys.stdin.read())))' > cache/+MANIFEST
cat cache/+MANIFEST | python3 -c 'import sys, json; manifest = json.loads(sys.stdin.read()); manifest.pop("files", None); manifest.pop("directories", None); manifest.pop("scripts", None); print(json.dumps(manifest))' > cache/+COMPACT_MANIFEST
# create new archive with +MANIFEST and +COMPACT_MANIFEST (without leading /)
tar -c -f cache/full.tar --transform='s,\./cache/,,' cache/+MANIFEST cache/+COMPACT_MANIFEST
tar -A cache/data.tar -f cache/full.tar
# compress and move to destination
xz cache/full.tar
mv cache/full.tar.xz out/$PKGNAME-$PKGVER.pkg
The resulting pkg file can now be installed with:
pkg add /path/to/foo-1.2.3.pkg
