Why can't chef resolve my cookbooks?

Question

Intro I am learning Chef to automate the server management at work. I downloaded chefdk 3.0 from here and now I am trying to build my first cookbook using chef.

Important I am using this in a Windows environment for testing purpose, I do expect it to fail since Windows does not have iptables, but I do not expect it to fail saying that it can't find the cookbook. I've tried using Windows cookbook and it works.

The problem I am able to create the cookbook and run it, but I am not able to reference dependencies from supermarket.

I have tried two alternatives:

Alternative 1

I used the following command to create the cookbook

chef generate cookbook learn_chef_httpd

(from this tutorial)

I was able to complete the tutorial and now I would like to test referencing another cookbook, so I chose simple_iptables

I added this line

cookbook 'simple_iptables', '~> 0.7.0'

To my Berksfile, as described in the Supermarket.

Then I added these lines to my default.rb file:

include_recipe 'simple_iptables'

# Allow HTTP, HTTPS
simple_iptables_rule "http" do
  rule [ "--proto tcp --dport 80",
         "--proto tcp --dport 443" ]
  jump "ACCEPT"
end

And I run the cookbook using:

chef-client --local-mode --runlist 'recipe[learn_chef_httpd]'

The problem is that Chef doesn't find the cookbook

Chef::Exceptions::CookbookNotFound
----------------------------------
Cookbook simple_iptables not found. If you're loading simple_iptables from anoth er cookbook, make sure you configure the dependency in your metadata

I tried adding it to the metadata:

depends 'simple_iptables', '~> 0.7.0'

But I still get an error:

Error Resolving Cookbooks for Run List:
Missing Cookbooks:

No such cookbook: simple_iptables

Alternative 2

I am still trying to make it work so I also tried making it "the berkshelf way", so I created a new cookbook.

berks cookbook test

And I added this line

cookbook 'simple_iptables', '~> 0.7.0'

To my Berksfile, as described in the Supermarket.

Then I added these lines to my default.rb file:

include_recipe 'simple_iptables'

# Allow HTTP, HTTPS
simple_iptables_rule "http" do
  rule [ "--proto tcp --dport 80",
         "--proto tcp --dport 443" ]
  jump "ACCEPT"
end

Executed berks install:

berks install

and ran it:

chef-client --local-mode --runlist 'recipe[test]'

The same error came back

Chef::Exceptions::CookbookNotFound
----------------------------------
Cookbook simple_iptables not found. If you're loading simple_iptables from anoth er cookbook, make sure you configure the dependency in your metadata

I tried adding it to the metadata:

depends 'simple_iptables', '~> 0.7.0'

But I still get an error:

Error Resolving Cookbooks for Run List:
Missing Cookbooks:

No such cookbook: simple_iptables

I looked at the ~/berkshelf folder and the cookbooks are there.

** Alternative 3 **

I started a CentOS 6.5 EC2 instance on Amazon, installed Ruby 2.1.3 and Chef. created a ~/chef-repo/cookbooks folder

I created a cookbook using berkshelf, ran

bundle install

added the reference/code as in the other alterantives then

berks install

and ran the same way I did last time.

I got the same issues.

What am I missing? What do I need to make it work?

Answer 1

Make sure you have configured your chef_repo_path as described in the docs.

Basically local-mode needs to know where to find your cookbooks, roles, environments, data bags, etc. Sadly, the documentation is not super clear on where/how you set chef_repo_path

Here's what I can tell from the code.

1. if client.rb is found and contains cookbook_path, chef_repo_path = "#{cookbook_path}/.."
2. if knife.rb is found and contains cookbook_path, chef_repo_path = "#{cookbook_path}/.."
3. Chef can try to divine the path. The code will search from pwd upward looking for a directory called cookbooks. If it finds it, then cookbooks/.. will be set as your chef_repo_path.
4. If all else fails, pwd will be used as your chef_repo_path

If you are using Berkshelf, your best bet is to do a berks vendor in order to get a single directory will ALL of the cookbooks you need. You can then point chef_repo_path to the parent of the cookbooks directory which holds your vendored cookbooks.

Mind you, that's from 10 minutes of digging in the source code, so I may not have it quite right.

Answer 2

While looking to run locally for testing I ran into the same "cannot resolve cookbook" error. Since I potentially want to run cookbooks without a Chef server but also without ChefDK installed for non-Chef developers to do a simple one time workstation setup, I put together the following workflow.

Your cookbook needs to live in a folder under a folder called cookbooks, this appears to be hardcoded "magic", but it seems to be able to live anywhere, eg userdir/projects/cookbooks/my_cookbook_name/.

If you have a Berksfile for the dependencies you need to get them locally before you can run chef-client -z -o my_cookbook_name so run berks vendor to pull the dependencies into a cookbooks directory in the current folder.

The berks vendor command pulls your other dependencies into the cookbooks directory so chef-zero/chef-solo can find them. Run the following in an Administrator PowerShell prompt, it may be Command Prompt compatible but I try not to encourage using cmd.exe.

cd /path/to/cookbooks/my_cookbook_name berks vendor . chef-client -z -o my_cookbook_name

The beauty of using the new chef-zero built into the client is you don't necessarily need the full ChefDK to test, though you will need berkshelf for the berks command on at least 1 machine as getting that installed without the ChefDK is a nightmare. Once you have berks you can run berks package and copy the file it creates to a machine with just the chef-client and extra it somewhere and then run the chef-client -z -o your_cookbook in the directory containing the cookbooks folder.

To get just chef use the omnibus installer and you have enough to bootstrap a node or run the recipe.

. { iwr -useb https://omnitruck.chef.io/install.ps1 } | iex; install -channel stable -project chef

If you need your berks and want a quick way to get the ChefDK, this will do the trick.

. { iwr -useb https://omnitruck.chef.io/install.ps1 } | iex; install -channel stable -project chefdk

Answer 3

After few hours of breaking my mind trying a combination of @kilian and @gratzy's answers from here, this helped,

cmd /c E: && cd /d E:\Items_CI Exploration\chef\chef-repo\cookbooks\ && chef-client --local-mode --runlist 'recipe[cisamplecookbooks::web]'

Yes, without the quotes. Also notice the "E:" added before. The number of '&' didn't make a difference though.

"E:\Items_CI Exploration\chef\chef-repo\cookbooks\" this is the path under which I had my cookbooks to be run. I guess last slash ("\") next to cookbooks is also important.