20

In my Vagrantfile, I can specify the URL of a box:

config.vm.box_url = "http://example.com/my-box.pkg"

According to the more recent documentation, I should be able to create a JSON file that contains the URLs for different versions of the box. The documentation also says that I can use the URL of this JSON file when running vagrant box add. I was hoping to be able to use the URL of that JSON file for config.vm.box_url. However, that doesn't seem to work. When I try it, it treats it like a box file:

Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'my-box' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Adding box 'my-box' (v0) for provider: virtualbox
    default: Downloading: http://example.com/my-box.pkg.json
    default: Progress: 100% (Rate: 876k/s, Estimated time remaining: 0:00:01)
The box failed to unpackage properly. Please verify that the box
file you're trying to add is not corrupted and try again. The
output from attempting to unpackage (if any):

bsdtar.EXE: Unrecognized archive format: Illegal byte sequence
bsdtar.EXE: Error exit delayed from previous errors.

Is it possible to tell Vagrant to use a box metadata JSON file in my Vagrantfile? I'd rather not have to use Vagrant Cloud.

3
  • Did you ever find a solution for this? Commented Aug 1, 2014 at 14:40
  • @JimRubenstein Unfortunately, no. The suggestion from Nicholas might work but I'm pretty sure my server is already sending the correct content type headers for JSON. The answer from Chux might be accurate, but I'm not convinced yet as the documentation implies otherwise. Unfortunately, Vagrant's documentation is pretty terrible all around and doesn't give much context between the basic-tutorial and contribute-to-the-project-and-hang-out-on-irc levels.... at least for me anyway.
    – Brad
    Commented Aug 1, 2014 at 15:07
  • i'm testing something as we speak on publishing a box + metadata to see if i can mimmic the behavior of vagrant cloud, locally. i'll let you know how it turns out. Commented Aug 1, 2014 at 17:07

5 Answers 5

12

As of today (2016-07-12, vagrant 1.8.4), if you want to run your own catalog in a manual way (that is, manually updating the boxes and editing the metadata.json file), but still have it behave like an actual catalog, keep in mind the following things:

  • There is no need for the file to be named "metadata.json". It can be named anything, as long as it contains the expected values. I'm using metadata.json here to clarify the steps further below.

  • each metadata.json file can only contain one single box. It can have multiple versions, and each version can have multiple providers (virtualbox, vmware, libvirt). If you need to have more than one box (say, fedora and ubuntu) you need two different metadata files.

  • Vagrant expects the metadata.json file to have a type of application/json (as Nicholas Hinds mentioned above. If your webserver does not return it (or, returns text/plain), vagrant will assume it's an actual box file, and try to parse it (and fail miserably).

  • Hashicorp's Atlas (what used to be Vagrant Cloud) is the exception to this, as the redirects lead you to content served as text/html. My best guess for this is it has something to do with the redirects (more on this below).

  • The box file does not need to be on the same place as the metadata file. You can have your metadata file in a local webserver, and the box in Amazon S3, no problem with that.

So, as far as I got, I found the easiest way to get this working on a webserver and still have pretty normal functionality is to do this:

On your webhost, create a file and directory structure similar to this:

d wwwroot/
d wwwroot/boxes
d wwwroot/boxes/yourname
f wwwroot/boxes/yourname/.htaccess
d wwwroot/boxes/yourname/box1
f wwwroot/boxes/yourname/box1/metadata.json
f wwwroot/boxes/yourname/box1/box1-$version1-$provider.box
f wwwroot/boxes/yourname/box1/box1-$version2-$provider.box
f wwwroot/boxes/yourname/box1/box1-$version2-$otherprovider.box
d wwwroot/boxes/yourname/box2
f wwwroot/boxes/yourname/box2/metadata.json
f wwwroot/boxes/yourname/box2/box2-$version1-$provider.box
(... etc)

(this layout means that your "metadata.json" for box1 will have to have it's URLs pointing to something like http://yourhost/boxes/yourname/box1/box1-$version1-$provider.box)

On your .htaccess, make sure that metadata.json is set for Directory index. The rest is optional, for negative cache and hiding the actual contents:

Header unset Pragma
FileETag None
Header unset ETag
DirectoryIndex metadata.json
IndexIgnore *
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate, private"
Header set Pragma "no-cache"
Header set Expires "Wed, 11 Jan 1984 05:00:00 GMT"

On your environment, export the VAGRANT_SERVER_URL pointing to your webhost. Note no trailing slash!

export VAGRANT_SERVER_URL="http://yourhost/boxes"

With this in place (and all the files with the correct contents), you can go and add your box directly:

vagrant box add yourname/box1

Since metadata.json is the index file for the box1 directory, it should redirect the contents right to it, vagrant will pick it up, interpret the metadata and download the appropriate box.

1
  • Awesome answer, lots of attention to detail. What helped me was that you even explained that the MIME type has to be application/json Commented Nov 12, 2021 at 18:01
19

After reading your question again, It seems you're trying to do something a little different than I am - but I think our end goal is the same.

I don't want to utilize the Vagrant Cloud service for hosting my base boxes, but I want to be able to distribute a development environment to my dev team, and utilize the features of the metadata.json file to maintain a versioning system for the development environment, which will then be available to my development team simply by using the facilities built into vagrant.

The vagrant documentation is really sparse in this area at the time of this writing (8/5/2014), presumably because it's a relatively new feature but I'm sure the fact that VagrantCloud has a paid tier has something to do with it also.

To figure out how to utilize the metadata.json file to version and distribute boxes, I took a look at some of the VMs available on the VagrantCloud. After looking through those, and reading some of the vagrant code - it became pretty easy to figure out how to accomplish my goal.

  • Package your box as you normally would. In my case, I'm packaging only for virtual box, because that's what our developers will be using to run the Vm. I also package a Vagrantfile with my basebox which does some provisioning for the development environment (setting up shares to appropriate folders, some basic apache configs, error logging, etc)
  • Create a metadata.json file to describe your base box, mine looks similar to this:

    {
        "description": "long box description",
        "short_description": "short box description",
        "name": "company/developer-environment",
        "versions": [{
            "version": "1",
            "status": "active",
            "description_html": "<p>Dev Environment</p>",
            "description_markdown": "Dev Environment",
            "providers": [{
                "name": "virtualbox",
                "url": "http:\/\/vagrant.domain.local/dev/company-developer-environment-1.box"
            }]
        }]
    }
    

Once I created my metadata.json file, I uploaded it to a local server running on our internal network (vagrant.domain.local/metadata.json). Once I did that, all that was left was to test it out with vagrant:

# add the box to vagrant using the definition from metadata.json
# (the box is actually downloaded here, so it can take a minute...or 10)
$ vagrant box add http://vagrant.domain.local/dev/metadata.json

# init the box (this creates a .vagrant folder and a Vagrantfile in the cwd with the appropriate box name)
$ vagrant init company/developer-environment

# boot the box
$ vagrant up

Voila, a remotely hosted, shared and versioned, private box that doesn't require usage of the Vagrant Cloud.

As you create new versions of your box, you'll package it up, and edit the metadata.json file. From what I can tell, you can use whatever versioning scheme you want be it semantic versioning (1.0.0, 1.0.1, etc) or just simple whole numbers for versions (1, 2, 3, etc). When your box users vagrant up vagrant automatically checks your metadata.json file for a new version, and will prompt them to do vagrant box update to update the box.

You can also skip the vagrant box add <metadata.json url> and vagrant init bits by defining a base Vagrantfile with the box name and box url, like so:

# -*- mode: ruby -*-
# vi: set ft=ruby :

# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "company/developer-environment"
  config.vm.box_url = "https://vagrant.domain.local/dev/metadata.json"
end

You could distribute a Vagrantfile with those contents, and all users would just be able to vagrant up. Though, I'm unsure about how that works when the versions get updated.

7
  • That's perfect, thanks! However... how do I add that JSON URL to the Vagrantfile?
    – Brad
    Commented Aug 5, 2014 at 18:10
  • You don't need to. When you add the box Vagrant downloads the current box image and stores the info in ~/.vagrant.d/boxes/<your box name>. Inside that folder there is the metadata_url file that is referenced around the documentation, which holds the URL for your JSON file that defines your versions. Vagrant handles all that automatically, so all you have to do is vagrant box add <your metadata.json url>, then just vagrant init <boxname> && vagrant up, vagrant does the rest Commented Aug 5, 2014 at 18:16
  • I understand that, but I'm trying to make it as easy as possible for developers to get up and running. By adding a box URL in the Vagrantfile, no vagrant box add is needed. If I could set the URL of that JSON file in the Vagrantfile, that's one less step for a new dev just joining the team to get up and running. It works for boxes, but I can't figure out why it doesn't work for the JSON file.
    – Brad
    Commented Aug 5, 2014 at 18:19
  • 1
    ah, gotcha - i actually just found a solution to that by just farting around. you have to define config.vm.box AND config.vm.box_url where box is your box name, and box_url is the URL to your json file. Commented Aug 5, 2014 at 18:27
  • 1
    @JimRubenstein Fantastic answer - just like Goldilocks, not too short, not too long :) Commented Jul 2, 2015 at 11:40
9

Vagrant requires that box metadata URLs are served up with the application/json content type. The error you are getting indicates that vagrant has interpreted your URL as a regular box.

Ensure your HTTP server is setting the Content-Type header appropriately. Most HTTP servers will automatically set the Content-Type header to application/json if your file has the extension .json

1
  • 1
    I don't know why your reply is not the answer because this is exactly what I needed to do to make local provisioning work with Vagrant.
    – Gaurav
    Commented Dec 2, 2014 at 9:51
4

I think you got their directives mixed up ..

The following is taken from the vagrant website:


BOX FILE

The actual box file is the required portion for Vagrant. It is recommended you always use a metadata file alongside a box file, but direct box files are supported for legacy reasons in Vagrant.

Box files are compressed using tar, tar.gz, or zip. The contents of the archive can be anything, and is specific to each provider. Vagrant core itself only unpacks the boxes for use later.

Within the archive, Vagrant does expect a single file: "metadata.json". This is a JSON file that is completely unrelated to the above "box metadata" component. This file must contain at least the "provider" key with the provider the box is for. For example, if your box was for VirtualBox, the metadata.json would look like this:

{
  "provider": "virtualbox"
}

If there is no metadata.json file or the file does not contain valid JSON with at least a "provider" key, then Vagrant will error when adding the box.


So, I think your box file format is probably wrong. Either it is not compressed with the recommended format, or you have not included a metadata.json file within the archive

1
  • For others with the same issue, the metadata.json path (in windows) is ..Users\username\vagrant.d\boxes\boxname\0\virtualbox\metadata.json
    – Nebojsac
    Commented Jan 8, 2016 at 13:38
1

You can try https://github.com/sparkoo/boxitory. It's simple one jar server. You point it to directory where you have your vagrant boxes and it creates compatible http interface for vagrant. Then you simply point it from your vagrantfile and you're done. You don't have to manually handle json files describing your boxes, adding new versions, providers, etc. It's all done for you for free. Just add new box file and Boxitory instantly returns it when requested.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .