cpkg is now live on Hackage. It is a good deal less polished than I'd originally wanted, but it already gives impressive results (among them cross-compiling XMonad).

Among the more interesting things going on under the hood, it uses Dhall to configure packages as functions.

Successes

Cross-Compiling XMonad

First, the exciting bit: we can cross-compile a Haskell project with nontrivial C dependencies! Moreover, despite cpkg's deficiencies, using cpkg is in fact the easiest way to cross-compile e.g. libXrandr. Simply run

cpkg install libX11 --target=arm-linux-gnueabihf
cpkg install libXext --target=arm-linux-gnueabihf
cpkg install libXrandr --target=arm-linux-gnueabihf
cpkg install libXinerama --target=arm-linux-gnueabihf
cpkg install libXScrnSaver --target=arm-linux-gnueabihf

and you can then install a cross-compiled XMonad with

cabal new-install xmonad --with-ghc arm-linux-gnueabihf-ghc --with-ghc-pkg arm-linux-gnueabihf-ghc-pkg $(cpkg dump-cabal libX11 libXext libXrandr libXinerama libXScrnSaver --target=arm-linux-gnueabihf)

This is quite exciting. With 1687 lines of Haskell, we get something that surpasses all available solutions!

Dhall

Configuring a package manager with Dhall is eye-opening. It is now obvious to me that the correct way to configure packages is not with a record but rather with a function.

What Dhall does that is really cool (and which I would hardly have attempted to implement myself prior to seeing it) is that it allows such functions to be lifted to Haskell generically.

As an example, if you have a Dhall file containing

let coolFunction =
  λ(x : Text) →
    "${x}-gcc"
in
coolFunction

You can lift this to a function Text -> Text in Haskell as follows:

liftedCoolFunction :: IO (T.Text -> T.Text)
liftedCoolFunction = inputFile auto "coolFile.dhall"

As an example package, consider

{- cpkg prelude imports -}
let types = https://raw.githubusercontent.com/vmchale/cpkg/master/dhall/cpkg-types.dhall
in
let prelude = https://raw.githubusercontent.com/vmchale/cpkg/master/dhall/cpkg-prelude.dhall
in
let lua =
  λ(v : List Natural) →
    let printLuaOS =
      λ(os : types.OS) →
        merge
          { FreeBSD   = λ(_ : {}) → "freebsd"
          , OpenBSD   = λ(_ : {}) → "bsd"
          , NetBSD    = λ(_ : {}) → "bsd"
          , Solaris   = λ(_ : {}) → "solaris"
          , Dragonfly = λ(_ : {}) → "bsd"
          , Linux     = λ(_ : {}) → "linux"
          , Darwin    = λ(_ : {}) → "macosx"
          , Windows   = λ(_ : {}) → "mingw"
          , Redox     = λ(_ : {}) → "generic"
          , Haiku     = λ(_ : {}) → "generic"
          , IOS       = λ(_ : {}) → "generic"
          , AIX       = λ(_ : {}) → "generic"
          , Hurd      = λ(_ : {}) → "generic"
          , Android   = λ(_ : {}) → "generic"
          , NoOs      = λ(_ : {}) → "c89"
          }
          os
    in
let luaBuild =
  λ(cfg : types.BuildVars) →
    let cc = prelude.mkCCArg cfg
    in

    let ldflags =
      (prelude.mkLDFlags cfg.linkDirs).value
    in

    let cflags =
      (prelude.mkCFlags cfg.includeDirs).value
    in

    let os =
      prelude.osCfg cfg
    in

    [ prelude.call (prelude.defaultCall ⫽ { program = "make"
                                          , arguments = cc # [ printLuaOS os, "MYLDFLAGS=${ldflags}", "MYCFLAGS=${cflags}", "MYLIBS=-lncurses", "-j${Natural/show cfg.cpus}" ]
                                          })
    ]
in

let luaInstall =
  λ(cfg : types.BuildVars) →
    [ prelude.call (prelude.defaultCall ⫽ { program = "make"
                                          , arguments = [ "install", "INSTALL_TOP=${cfg.installDir}" ]
                                          })
    ]
      # prelude.symlinkBinaries [ "bin/lua", "bin/luac" ]
in

prelude.simplePackage { name = "lua", version = v } ⫽
  { pkgUrl = "http://www.lua.org/ftp/lua-${prelude.showVersion v}.tar.gz"
  , configureCommand = prelude.doNothing
  , buildCommand = luaBuild
  , installCommand = luaInstall
  , pkgDeps = [ prelude.unbounded "readline"
              , prelude.unbounded "ncurses"
              ]
  }
in
…

It's all a bit difficult to grasp at first, but this style of configuration - as a function rather than a record - is leagues ahead of anything else out there.

Finally, Turing completeness turns out to be nice. Maybe it's too strong, but a limited language is clearly good: you can easily see what belongs in the Dhall library and what should be implemented on the Haskell side.

Pain

Packaging C

Packaging for C is pretty appalling for those of us used to cabal new-build and Hackage. Most packages are distributed via tarballs, but not all are distributed via HTTPS. alsa is only available via FTP.

There are three build systems commonly used (autotools, cmake, and meson) and each needs to be supported in the project's Dhall library. meson is written in Python and it is particularly hard to use as a build dependency without polluting any global installations.

Some projects choose to forgo these and just use a plain Makefile, which generally needs to be written in a project-specific way - the Dhall configuration for Lua is 64 lines. 21 lines are dedicated to printLuaOs,

let printLuaOS =
  λ(os : types.OS) →
    merge
      { FreeBSD   = λ(_ : {}) → "freebsd"
      , OpenBSD   = λ(_ : {}) → "bsd"
      , NetBSD    = λ(_ : {}) → "bsd"
      , Solaris   = λ(_ : {}) → "solaris"
      , Dragonfly = λ(_ : {}) → "bsd"
      , Linux     = λ(_ : {}) → "linux"
      , Darwin    = λ(_ : {}) → "macosx"
      , Windows   = λ(_ : {}) → "mingw"
      , Redox     = λ(_ : {}) → "generic"
      , Haiku     = λ(_ : {}) → "generic"
      , IOS       = λ(_ : {}) → "generic"
      , AIX       = λ(_ : {}) → "generic"
      , Hurd      = λ(_ : {}) → "generic"
      , Android   = λ(_ : {}) → "generic"
      , NoOs      = λ(_ : {}) → "c89"
      }
      os
in
luaOs

which serves no interesting function - it just translates from cpkg's OS type to something suitable for Lua's Makefile.

It is clear that many build systems for prominent C projects were not designed to be automated.

In addition, many prominent libraries contain bugs. m4 fails to build with the latest glibc. npm builds will hang indefinitely, but only when you don't look at the terminal output.

Such things are papered over by Linux distributions, which generally offer tested binary packages, but it's an unfortunate situation - I suspect that with more automated build tools like cpkg, library releases would contain fewer bugs!

Dhall's Performance

Dhall's performance is poor. Some of this is fixable - it uses Text when it should intern identifiers, for instance - but imports via HTTP are not. Most of the marketing for Dhall touts it as a replacement for YAML/JSON, so it's not too surprising that it's not perfectly suited for package configuration.

What I would really like to see here is something that takes the good parts of Dhall (types, Turing incompleteness, functions, fluent Haskell integration) and makes it into a full-blown package configuration language. I'm inclined to think something with its own package manager and "compiled" libraries is the solution, but really anything that doesn't make HTTP requests every time a file is interpreted would be an improvement.

I think it would be worth adding a real module system and better type inference in the process, but that I am less sure of.

Tar Package

The Haskell tar package is quirky, to say the least. Among other things, it chokes on the gcc and llvm tarballs - even if it's doing the right thing here (based on standards), there are enough prominent projects to make things pretty frustrating.

Failures

Package Database

The global package database is not actually implemented as a database, but rather a single file that contains a serialized S.Set BuildCfg. It would be better to store this in a real database, for the sake of performance and resiliency.

The package database also doesn't contain enough information to do nix-style garbage collection, which would be required for cpkg to be considered a full-fledged package manager in the vein of apt or nix.

Dependency Resolution

I didn't bother to implement any sort of dependency resolution. The current layout of the code shouldn't prevent anyone from doing so, but, it is still a deficiency.

With dependency resolution, we'd get something like cabal new-build for C/C++, which would be really exciting. Many C libraries use weird preprocessor hacks to support multiple versions of some library - none of this is necessary if you don't require that only one version of a particular library be installed at a time.