DotNet dependencies with Paket

Author: Florian Graf | Published: 2018-12-31

Photo by Samuel Zeller on Unsplash

In every project without a doubt dependency management is an important part. You don’t build everything from scratch but instead depend on the work others have done. But you usually want to depend on work that is being maintained and thus want to receive updates. But not all updates will work so you want to restrict which updates you want to receive. In the world of DotNet the de facto standard for dependency management and distribution is NuGet. In this article we take a look at the program Paket for managing our NuGet dependencies.

Follow along

To help understand I’ve created a Git repository with two C# projects that you can use to follow along. You can find the repository at GitLab: https://gitlab.com/fmg-software/tutorials/learnpaket

The Git repository has two branches. The master branch is at the start of this tutorial so you can immediately follow along. A second branch named solution is available. Every commit in the solution branch represents a step in the tutorial.

One project is a simple console program named LearnPaket. The second project is a library that is being used by the console program. That library just outputs the current references of the executing assembly. Initially this is done line by line and is later in this tutorial converted to a JSON output with Newtonsoft.Json.1

Installation

Paket is an F#2 project and thus can run on the DotNet platform. It works well with the Microsoft DotNet Framework, the DotNet Core Framework and the Mono runtime.

The easiest way to install Paket is to create a .paket folder in your main project directory. In this folder you download the paket.bootstrapper.exe and rename it to paket.exe. Running paket.exe will then download the latest version into the temporary directory of your system and execute it. You can get paket.bootstrapper.exe from the official GitHub page.

Afterwards your main project directory should look similar to this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/...
├── LICENSE
└── README.md

Consider committing the paket.exe into your version control system. Since it is only the paket.bootstrapper.exe it is quite small (60-70 KB). It also rarely changes. This will give people checking out your project immediate access to the Paket dependency management system. Also on your continuous integration server no additional setup is necessary to get it up and running.

Consuming packages

The most common use case is usually that you want to consume packages from third parties. Paket lets you consume from several different sources. Paket doesn’t only support NuGet repositories but can also download with Git3, from GitHub4 and by HTTP5. We will only take a look at consuming packages from a NuGet repository.

To start create a file named paket.dependencies in the main project directory. As the name suggest in this file you will define the dependencies of your project. It also includes information about which frameworks you want to use and from where the dependencies should be searched and downloaded.

At this point your main project directory should look similar to this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/...
├── LICENSE
├── paket.dependencies
└── README.md

At the top of the file the sources should be specified from where the dependencies will be gotten. Typically you would like to add at least a source to the official NuGet repository. You do this with the keyword source followed by the URL for the repository. In case of the official NuGet repository version 3 the URL is: https://api.nuget.org/v3/index.json.

I recommend to specify with which DotNet Framework you’re developing. Otherwise Paket will download all possible combination of dependencies and clutter your computer with unnecessary data. It is also possible to specify multiple frameworks. In the accompanying project I’m using the frameworks DotNet Standard 2.0 and DotNet Framework 4.7.1. You specify them with the keyword framework followed by a colon and then identifiers for those frameworks separated by commas. You can find the different identifiers for the frameworks on the documentation page of Paket.6

So far your paket.dependencies file should look like this:

source https://api.nuget.org/v3/index.json
framework: netstandard2.0, net471

Now let us add the popular Newtonsoft.Json library as a dependency to our project. You can find the libraries, their names and provided versions by searching on NuGet.org. It is also possible to search through all provided sources with paket. This can be done with the find-packages7 command.

Adding a library from a NuGet library is simple. You just use the keyword nuget followed by the exact name of the library. In our case this would be nuget Newtonsoft.Json. This will download the latest version of the library and all transitive dependencies. Transitive dependencies are the dependencies of the libraries you depend on and their dependencies and so on. They are inherited dependencies that you don’t depend directly on.

I highly encourage you to add version constraints to your dependencies. Especially if the dependency is following semantic versioning8 it will save you from unexpected surprises. Specifying a version is really simple in Paket. Paket allows several different ways to specify which version of a dependency you want to use. We are going to look at the simplest of them all which is the “twiddle-wakka” operator9.

This operator assumes your dependency follows at least in some way the semantic versioning. It will allow upgrades on the last part of the specified version. Thus if you have typical MAJOR.MINOR.PATCH format of versions, specifying MAJOR.MINOR will allow upgrades on minor versions. Specifying MAJOR.MINOR.PATCH will allow upgrades only on patch versions.

Let us now add the Newtonsoft.Json library and allow any minor upgrades.

source https://api.nuget.org/v3/index.json
framework: netstandard2.0, net471

nuget Newtonsoft.Json ~> 12.0

So this will allow the versions 12.0.z, 12.1.z, 12.2.z, etc. but nothing starting with major version 13 or above and it will also exclude anything below major version 12.

You can now go ahead and install the dependencies by running the paket.exe from your project directory with the argument install: .paket/paket.exe install. Your dependencies should now be downloaded and installed in the folder packages. Also a folder paket-files is created that can be ignored. Last but not least a paket.lock10 file is created. In this file the exact versions of all your dependencies and transitive dependencies are stored. This can be used to later restore the exact same versions of the dependencies for example on your continuous integration server.

After running the install command your project directory should look similar to this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/...
├── packages/
│   └── Newtonsoft.Json/...
├── paket-files/...
├── LICENSE
├── paket.dependencies
├── paket.lock
└── README.md

If there is more than one folder under the packages folder make sure you specified the framework constraint correctly. If the framework isn’t provided it will assume all frameworks and download a lot of additional transitive packages.

You should only add the paket.dependencies and the packet.lock to your version control system (VCS). Let your VCS ignore the packages and paket-files folder as they can be restored.

Automatic Referencing

You might have noticed that it downloaded the library but hasn’t referenced anything in your C# project. Thus at this point you would still need to manually add the reference to the library in your project. This can cause unexpected errors and it would be more convenient if it happened automatically.

And you probably already guessed it that Paket allows to automatically reference the dependent libraries. It does so with an additional file named paket.references that is put next to the project file. Thus in our example this would be the LearnPaketLib.csproj.

This will change your project directory to something like this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/
│   ├── ...
│   ├── LearnPaketLib.csproj
│   └── paket.references
├── packages/
│   └── Newtonsoft.Json/...
├── paket-files/...
├── LICENSE
├── paket.dependencies
├── paket.lock
└── README.md

Just add the libraries you want to reference into this file. One library per line. Use the exact name for the assembly that you already used in paket.dependencies. Do not include the version number of the library. There is no need to define this again here. There are some additional options possible to include how the library is added to your project but the default settings are usually good.11

Your paket.references file should now include one line with the name Newtonsoft.Json. If you now execute .paket/paket.exe install it will automatically add a reference to the library in your project. If you specified a framework version in your paket.dependencies it will include it with a condition for this framework version only. Thus make sure that your project uses the same framework as you specified in your dependencies otherwise the reference might not show up.

You should now be able to implement some code to print the references as JSON. I did it like this:

public static void PrintReferences(TextWriter writer)
{
    var json = JsonConvert.SerializeObject(
        new {
            References = Assembly.GetExecutingAssembly()
                .GetReferencedAssemblies()
                .Select(
                    a => new {
                        a.Name,
                        a.Version
                    })
        });
    
    writer.WriteLine(json);
}

Don’t forget to add the paket.references to your VCS.

Authentication

The public official NuGet repository doesn’t need any authentication to download libraries from. But if you use your own repository you might have protected it with username and password. In this case it might need HTTP basic authentication to download the libraries. You can add credentials to your global Paket configuration. It will choose which credentials to use by the URL.

To add some credentials for an URL enter the following command into a terminal:

.paket/paket.exe config add-credentials URL

Replace URL with the actual URL you want to authenticate at. Paket will then ask for your username and password and save it into the configuration file. The password is saved obfuscated but not encrypted!

Paket also supports NuGet credentials providers.12 They can provide a more secure way to store your credentials to a repository.

Creating packages

So far we looked at using libraries as dependencies provided by others. But Paket also allows us to publish our own library. It is possible to publish it to the official NuGet repository or our own private repository. In both cases we need an account and the corresponding API key for publishing.

To start you have to define what exactly you want to publish. NuGet is quite open in this regard and allows a lot of different combination. It is possible to publish an executable with several libraries as a single package. If you’re coming from for example the Java ecosystem this might seem strange as it is more common to publish single JAR (thus libraries) in Java.

In this example we only want to publish the LearnPaketLib. It is advisable to only publish the library and have transitives dependencies. This reduces duplication if other libraries will use the same transitives dependencies we have and also reduce conflicts, since automatically a suitable version of the transitive dependency is selected. If no suitable version can be found Paket will immediately show you an error.

To define what will be published create a paket.template next to your project file. The project directory should then look similar to this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/
│   ├── ...
│   ├── LearnPaketLib.csproj
│   ├── paket.references
│   └── paket.template
├── packages/
│   └── Newtonsoft.Json/...
├── paket-files/...
├── LICENSE
├── paket.dependencies
├── paket.lock
└── README.md

On the first line you have to define how Paket should package your library. Use the keyword type followed by a space and then either file or project. Using project is the easiest way. In this mode Paket will take all information from the project files and generated assemblies.

Project for our library is enough. Use file to gain fine-grained control over what should be included or excluded. Be aware that a lot of the automatic discoveries will be disabled and need to be done manually. Refer to the documentation for how to do it. This is out of scope for this article.13

It is recommended to also add an URL to the license you are distributing your package with. Currently only an URL to a license is supported. This restriction is part of the NuGet specification. They are currently working on changing this to support license files.14

Your template file should now look like this:

type project
licenseUrl https://opensource.org/licenses/MIT

We know have to create the nupkg file that we then can upload. To pack this file you simply issue the command .paket/paket.exe pack ./nupkg-output. This will create the file FmgSoftware.LearnPaketLib.1.0.0.0.nupkg in the folder nupkg-output. Your project directory should now look like this:

LearnPaket/
├── .git/...
├── .paket
│   └── paket.exe
├── LearnPaket/...
├── LearnPaketLib/
│   ├── ...
│   ├── LearnPaketLib.csproj
│   ├── paket.references
│   └── paket.template
├── nupkg-output/
│   └── FmgSoftware.LearnPaketLib.1.0.0.nupkg
├── packages/
│   └── Newtonsoft.Json/...
├── paket-files/...
├── LICENSE
├── paket.dependencies
├── paket.lock
└── README.md

All that is left is to push this file to your repository. You do this with the paket push command. Specify which file you want to push and by default Paket will push to the official NuGet repository. It is also possible to push to your private NuGet repository by specifying the URL and optionally endpoint.15

Whenever you push a package to a NuGet repository you need to specify your API key. Take a look at your repository on how to get your API key. There are different ways to specify the API key during the push process. By default Paket will use the environment variable NUGET_KEY. If this environment variable doesn’t exist it will lookup the URL in its configuration and use the provided key there. To add your key to the configuration you can do the following:

.paket/paket.exe config add-token https://nuget.org YOUR_NUGET_KEY

Notice that there is no slash at the end of the NuGet URL. Replace YOUR_NUGET_KEY with your API key. Keep in mind that this won’t be encrypted and will be readable on your system. Choose which API key you want to use and push the package.

.paket/paket.exe push ./nupkg-output/FmgSoftware.LearnPaketLib.1.0.0.0.nupkg

Pushing this package on your end will obviously fail on the official NuGet repository as I have already pushed it myself. You might want to try it out with a private repository.16

DotNet CLI, NuGet CLI and others

We had a look at Paket in this article. But perhaps you’re aware that there are other tools to manage your NuGet dependencies. You can use the tools provided by your IDE (e.g. Visual Studio and Rider have plugins), the DotNet Core CLI or the NuGet CLI. In my opinion at the moment all those tools have different shortcomings compared to Paket. I might go into details of those shortcomings in a later article.